commit f874ac043dae8a2147ccbb428629888905a32603 Author: Colin Walters Date: Sun Oct 9 17:03:08 2011 -0400 Initial commit diff --git a/.gitignore b/.gitignore new file mode 100644 index 00000000..f5d07283 --- /dev/null +++ b/.gitignore @@ -0,0 +1,36 @@ +# Standard C/Automake goo +.deps +.libs +*.typelib +*.la +*.lo +*.o +*.pyc +*.stamp +*~ +Makefile +Makefile.in +aclocal.m4 +autom4te.cache +compile +config.guess +config.h +config.h.in +config.log +config.status +config.sub +configure +depcomp +gtk-doc.make +INSTALL +install-sh +libtool +ltmain.sh +missing +stamp-h1 +ylwrap +py-compile +config +m4 +po +ABOUT-NLS diff --git a/DESIGN b/DESIGN new file mode 100644 index 00000000..c17046a0 --- /dev/null +++ b/DESIGN @@ -0,0 +1,219 @@ +Hacktree + +This is a combined build system and runtime daemon that builds and +manages root filesystems. They both share an executable name +"hacktree", but are conceptually split up into two parts: + +hacktree-build +hacktree-root-manager + +== The recipe set == + +A recipe is similar to Bitbake's format, except just have metadata - +we don't allow arbitrary Python scripts. Also, we assume +autotools. Example: + +SUMMARY = "The basic file, shell and text manipulation utilities." +HOMEPAGE = "http://www.gnu.org/software/coreutils/" +BUGTRACKER = "http://debbugs.gnu.org/coreutils" +LICENSE = "GPLv3+" +LIC_FILES_CHKSUM = "file://COPYING;md5=d32239bcb673463ab874e80d47fae504\ + file://src/ls.c;startline=5;endline=16;md5=e1a509558876db58fb6667ba140137ad" +SRC_URI = "${GNU_MIRROR}/coreutils/${BP}.tar.gz \ + file://remove-usr-local-lib-from-m4.patch \ + " +DEPENDS = "gmp foo" + +Each recipe will output one or more artifacts. + +== The Root Repository == + +A root repository is simply a git repository (at first). + +Keep reading. + +== A Root == + +A root is a filesystem tree, stored in the repository using git. The +filesystem tree is built using a set of artifacts. Roots are actually +read-only bind-mounts on top of a real writable location, which isn't +"public". + +Each root is checkout of a branch in the repo. + +TODO can we modify git to add the concept of a "raw" object type only +used for blobs? The metadata for it could be stored in extended +attributes. Then we could simply hard link the object for regular +files to their checkout, and almost totally deduplicate. +http://thread.gmane.org/gmane.comp.version-control.git/102001/focus=102094 + +In GNOME, we will have a root per: + - major version (3.0, 3.2) + - "runtime", "sdk", and "devel" + - Build type (opt, debug) + - Architecture (ia32, x86_64) + +/gnome/root-3.2-runtime-opt-x86_64/{etc,bin,share,usr,lib} +/gnome/root-3.2-devel-debug-x86_64/{etc,bin,share,usr,lib} +/gnome/.real/root-3.2-runtime-opt-x86_64 +/gnome/.real/root-3.2-devel-debug-x86_64 + +A "runtime" root is what's necessary to run applications. A SDK root +is that, plus all the command line developer tools (gcc, gdb, make, +strace). And finally the "devel" root has all the API-unstable +headers not necessary for applications (NetworkManager.h etc.) + +Hmm, maybe we should punt developer tools into a Unix app framework. + +== Artifact == + +An artifact is a binary result of compiling a recipe (there may be +multiple). Think of an artifact as like a Linux distribution +"package", except there are no runtime dependencies, conflicts, or +pre/post scripts. It's basically just a gzipped tarball, and we +encode metadata in the filename. + +Example: + +gdk-pixbuf-runtime,o=master,r=3.2-opt-x86_64,b=opt,v=2.24.0-10-g1d39095,.tar.gz + +This is an artifact from the gdk-pixbuf component. Here's a decoding of the key/value pairs: + +o: The origin of the build - there are just "master" and "local" +r: The name of the root this artifact was compiled against +b: The name of the build flavor (known values are "opt" and "debug") +v: The output of "git describe". + +To build a root, we simply unpack the artifacts that compose it, and +run "git commit". + +hacktree will default to splitting up shared libraries' unversioned .so +link and header files into -devel, and the rest into -runtime. + +All binaries default to runtime. + +== Configuration Management == + +Have you ever been a system administrator on a zypper/yum system, done +an RPM update, which then drops .rpmnew files in your /etc/ that you +have to go and hunt for with "find" or something, and said to +yourself, "Wow, this system is awesome!!!" ? Right, that's what I +thought. + +Configuration (and systems) management is a tricky problem, and I +certainly don't have a magic bullet. However, one large conceptual +improvement I think is defaulting to "rebase" versus "merge". + +This means that we won't permit direct modification of /etc - instead, +you HAVE to write a script which accomplishes your goals. It's +recommended to make this script revision controlled. We just execute +/etc/gnomeos/config.d/* in alphabetical order, while the rootfs is +still mounted read-write. + +Hmm, probably we need to define it to operate on a shadow tree, which +we then diff? + +If the script fails, we can roll back the update, or drop to a shell +if interactive. + +== Local modifications == + +A key point of this whole endeavour is that we want developers to be +able to do local builds. This is surprisingly something not well +supported by the Debian/Fedora's tools at least. + +=== Updating a root with a new local artifact === + +Whenever you install a local artifact, if no "local" branch exists for +that root, it's created. + +Let's say we're debugging gdk-pixbuf, tracking down a memory +corruption bug. We've added a few printfs, and want to rerun things. +GCC optimization is screwing us, so we build it in debug mode (-O0). +The active root is root-3.2-opt. + +$ pwd +~/src/gdk-pixbufroot +$ echo $HACKTREE_ROOT +/gnome/root-3.2-opt + +$ hacktree make debug +