Initial commit
This commit is contained in:
Colin Walters committed
commit
f874ac043d
7 files changed
+305
No files matched your search
+36
@@ -0,0 +1,36 @@
|
||||
# Standard C/Automake goo
|
||||
.deps
|
||||
.libs
|
||||
*.typelib
|
||||
*.la
|
||||
*.lo
|
||||
*.o
|
||||
*.pyc
|
||||
*.stamp
|
||||
*~
|
||||
Makefile
|
||||
Makefile.in
|
||||
aclocal.m4
|
||||
autom4te.cache
|
||||
compile
|
||||
config.guess
|
||||
config.h
|
||||
config.h.in
|
||||
config.log
|
||||
config.status
|
||||
config.sub
|
||||
configure
|
||||
depcomp
|
||||
gtk-doc.make
|
||||
INSTALL
|
||||
install-sh
|
||||
libtool
|
||||
ltmain.sh
|
||||
missing
|
||||
stamp-h1
|
||||
ylwrap
|
||||
py-compile
|
||||
config
|
||||
m4
|
||||
po
|
||||
ABOUT-NLS
|
||||
@@ -0,0 +1,219 @@
|
||||
Hacktree
|
||||
|
||||
This is a combined build system and runtime daemon that builds and
|
||||
manages root filesystems. They both share an executable name
|
||||
"hacktree", but are conceptually split up into two parts:
|
||||
|
||||
hacktree-build
|
||||
hacktree-root-manager
|
||||
|
||||
== The recipe set ==
|
||||
|
||||
A recipe is similar to Bitbake's format, except just have metadata -
|
||||
we don't allow arbitrary Python scripts. Also, we assume
|
||||
autotools. Example:
|
||||
|
||||
SUMMARY = "The basic file, shell and text manipulation utilities."
|
||||
HOMEPAGE = "http://www.gnu.org/software/coreutils/"
|
||||
BUGTRACKER = "http://debbugs.gnu.org/coreutils"
|
||||
LICENSE = "GPLv3+"
|
||||
LIC_FILES_CHKSUM = "file://COPYING;md5=d32239bcb673463ab874e80d47fae504\
|
||||
file://src/ls.c;startline=5;endline=16;md5=e1a509558876db58fb6667ba140137ad"
|
||||
SRC_URI = "${GNU_MIRROR}/coreutils/${BP}.tar.gz \
|
||||
file://remove-usr-local-lib-from-m4.patch \
|
||||
"
|
||||
DEPENDS = "gmp foo"
|
||||
|
||||
Each recipe will output one or more artifacts.
|
||||
|
||||
== The Root Repository ==
|
||||
|
||||
A root repository is simply a git repository (at first).
|
||||
|
||||
Keep reading.
|
||||
|
||||
== A Root ==
|
||||
|
||||
A root is a filesystem tree, stored in the repository using git. The
|
||||
filesystem tree is built using a set of artifacts. Roots are actually
|
||||
read-only bind-mounts on top of a real writable location, which isn't
|
||||
"public".
|
||||
|
||||
Each root is checkout of a branch in the repo.
|
||||
|
||||
TODO can we modify git to add the concept of a "raw" object type only
|
||||
used for blobs? The metadata for it could be stored in extended
|
||||
attributes. Then we could simply hard link the object for regular
|
||||
files to their checkout, and almost totally deduplicate.
|
||||
http://thread.gmane.org/gmane.comp.version-control.git/102001/focus=102094
|
||||
|
||||
In GNOME, we will have a root per:
|
||||
- major version (3.0, 3.2)
|
||||
- "runtime", "sdk", and "devel"
|
||||
- Build type (opt, debug)
|
||||
- Architecture (ia32, x86_64)
|
||||
|
||||
/gnome/root-3.2-runtime-opt-x86_64/{etc,bin,share,usr,lib}
|
||||
/gnome/root-3.2-devel-debug-x86_64/{etc,bin,share,usr,lib}
|
||||
/gnome/.real/root-3.2-runtime-opt-x86_64
|
||||
/gnome/.real/root-3.2-devel-debug-x86_64
|
||||
|
||||
A "runtime" root is what's necessary to run applications. A SDK root
|
||||
is that, plus all the command line developer tools (gcc, gdb, make,
|
||||
strace). And finally the "devel" root has all the API-unstable
|
||||
headers not necessary for applications (NetworkManager.h etc.)
|
||||
|
||||
Hmm, maybe we should punt developer tools into a Unix app framework.
|
||||
|
||||
== Artifact ==
|
||||
|
||||
An artifact is a binary result of compiling a recipe (there may be
|
||||
multiple). Think of an artifact as like a Linux distribution
|
||||
"package", except there are no runtime dependencies, conflicts, or
|
||||
pre/post scripts. It's basically just a gzipped tarball, and we
|
||||
encode metadata in the filename.
|
||||
|
||||
Example:
|
||||
|
||||
gdk-pixbuf-runtime,o=master,r=3.2-opt-x86_64,b=opt,v=2.24.0-10-g1d39095,.tar.gz
|
||||
|
||||
This is an artifact from the gdk-pixbuf component. Here's a decoding of the key/value pairs:
|
||||
|
||||
o: The origin of the build - there are just "master" and "local"
|
||||
r: The name of the root this artifact was compiled against
|
||||
b: The name of the build flavor (known values are "opt" and "debug")
|
||||
v: The output of "git describe".
|
||||
|
||||
To build a root, we simply unpack the artifacts that compose it, and
|
||||
run "git commit".
|
||||
|
||||
hacktree will default to splitting up shared libraries' unversioned .so
|
||||
link and header files into -devel, and the rest into -runtime.
|
||||
|
||||
All binaries default to runtime.
|
||||
|
||||
== Configuration Management ==
|
||||
|
||||
Have you ever been a system administrator on a zypper/yum system, done
|
||||
an RPM update, which then drops .rpmnew files in your /etc/ that you
|
||||
have to go and hunt for with "find" or something, and said to
|
||||
yourself, "Wow, this system is awesome!!!" ? Right, that's what I
|
||||
thought.
|
||||
|
||||
Configuration (and systems) management is a tricky problem, and I
|
||||
certainly don't have a magic bullet. However, one large conceptual
|
||||
improvement I think is defaulting to "rebase" versus "merge".
|
||||
|
||||
This means that we won't permit direct modification of /etc - instead,
|
||||
you HAVE to write a script which accomplishes your goals. It's
|
||||
recommended to make this script revision controlled. We just execute
|
||||
/etc/gnomeos/config.d/* in alphabetical order, while the rootfs is
|
||||
still mounted read-write.
|
||||
|
||||
Hmm, probably we need to define it to operate on a shadow tree, which
|
||||
we then diff?
|
||||
|
||||
If the script fails, we can roll back the update, or drop to a shell
|
||||
if interactive.
|
||||
|
||||
== Local modifications ==
|
||||
|
||||
A key point of this whole endeavour is that we want developers to be
|
||||
able to do local builds. This is surprisingly something not well
|
||||
supported by the Debian/Fedora's tools at least.
|
||||
|
||||
=== Updating a root with a new local artifact ===
|
||||
|
||||
Whenever you install a local artifact, if no "local" branch exists for
|
||||
that root, it's created.
|
||||
|
||||
Let's say we're debugging gdk-pixbuf, tracking down a memory
|
||||
corruption bug. We've added a few printfs, and want to rerun things.
|
||||
GCC optimization is screwing us, so we build it in debug mode (-O0).
|
||||
The active root is root-3.2-opt.
|
||||
|
||||
$ pwd
|
||||
~/src/gdk-pixbufroot
|
||||
$ echo $HACKTREE_ROOT
|
||||
/gnome/root-3.2-opt
|
||||
<hack hack hack>
|
||||
$ hacktree make debug
|
||||
<time passes, hopefully not too much>
|
||||
$ ls gdk-pixbuf*.tar.gz
|
||||
gdk-pixbuf-runtime,o=master,r=3.2-opt,b=debug,v=2.24.0-10-g1d39095,.tar.gz
|
||||
gdk-pixbuf-devel,o=master,r=3.2-opt,b=debug,v=2.24.0-10-g1d39095,.tar.gz
|
||||
gdk-pixbuf-manifests,o=master,r=3.2-opt,b=debug,v=2.24.0-10-g1d39095,.tar.gz
|
||||
$ hacktree install gdk-pixbuf*,o=master,r=3.2-opt,b=debug,v=2.24.0-10-g1d39095,.tar.gz
|
||||
<policykit auth dialog>
|
||||
|
||||
Now here's where the cool stuff happens. hacktree takes
|
||||
/gnome/root-3.2-opt (the which is given in the r= above), and looks
|
||||
for the corresponding git branch (root-3.2-opt). Now hacktree notices
|
||||
there's no corresponding "local" branch, i.e. local-3.2-opt. One is
|
||||
created and checked out:
|
||||
|
||||
# pwd
|
||||
/gnome/repo.git
|
||||
# git branch local-3.2-opt root-3.2-opt
|
||||
# git clone --branch local-3.2-opt /gnome/repo.git /gnome/.real-local-3.2-opt
|
||||
|
||||
Now, the artifacts specified are overlaid:
|
||||
|
||||
# cd /gnome/.real-local-3.2-opt
|
||||
# tar xvf
|
||||
|
||||
Ok, now we need to remove old no longer shipped files from the root.
|
||||
Thus, we need a list of files corresponding to each original artifact,
|
||||
and to know which artifacts are in a root. Note above that one of the
|
||||
artifacts produced was "manifests". This contains files like:
|
||||
|
||||
/meta/manifests/gdk-pixbuf-runtime.list
|
||||
/meta/manifests/gdk-pixbuf-devel.list
|
||||
|
||||
Thus we diff the manifests, and clean up any leftover files.
|
||||
|
||||
# git commit -a -m "Install artifact gdk-pixbuf-runtime,o=master,r=3.2-opt,b=debug,v=2.24.0-10-g1d39095,.tar.gz"
|
||||
# git checkout
|
||||
|
||||
|
||||
== Updating roots from master ==
|
||||
|
||||
We need to split this into two phases - download, and apply. Luckily,
|
||||
git exposes exactly the operations we need, namely "git fetch" and
|
||||
"git checkout".
|
||||
|
||||
hacktree fetch
|
||||
|
||||
== Many roots on build master ==
|
||||
|
||||
builds.git
|
||||
Generated after every time an artifact is built.
|
||||
fastqa.git
|
||||
After each root is built, a very quick test suite is run in it;
|
||||
probably this is booting to GDM. If that works, the latest build
|
||||
is committed here. Hopefully we can get fastqa under 2 minutes.
|
||||
dailyqa.git
|
||||
Much more extensive tests, let's say they take 24 hours.
|
||||
|
||||
|
||||
== TODO figure out ==
|
||||
|
||||
* devel/runtime split for things with binaries (buildapi)
|
||||
* xserver uses cpp (ugh)
|
||||
-> try to move artifact split into upstream git?
|
||||
-> $#@$!$ fix it
|
||||
* Language packs?
|
||||
-> System extension
|
||||
* what about optional random ui tools that are built with tracker?
|
||||
-> don't build them by default (--disable-ui-tools), make them an app if cared
|
||||
|
||||
|
||||
== RPM Compatibility ==
|
||||
|
||||
We should be able to install LSB rpms. This implies providing "rpm".
|
||||
The tricky part here is since the OS itself is not assembled via RPMs,
|
||||
we need to fake up a database of "provides" as if we were. Even
|
||||
harder would be maintaining binary compatibilty with any arbitrary
|
||||
%post scripts that may be run.
|
||||
|
||||
|
||||
@@ -0,0 +1,9 @@
|
||||
bin_SCRIPTS += gbuild
|
||||
|
||||
SUBSTITUTIONS = sed -e s,@libdir\@,$(libdir), -e s,@datarootdir\@,$(datarootdir), -e s,@PYTHON\@,$(PYTHON),
|
||||
|
||||
gbuild: src/gbuild.in Makefile
|
||||
$(AM_V_GEN) $(SUBSTITUTIONS) $< > $@.tmp && mv $@.tmp $@
|
||||
@chmod a+x $@
|
||||
|
||||
CLEANFILES += gbuild
|
||||
+11
@@ -0,0 +1,11 @@
|
||||
NULL =
|
||||
BUILT_SOURCES =
|
||||
CLEANFILES =
|
||||
EXTRA_DIST =
|
||||
bin_SCRIPTS =
|
||||
libexec_PROGRAMS =
|
||||
noinst_LTLIBRARIES =
|
||||
noinst_PROGRAMS =
|
||||
|
||||
|
||||
include Makefile-gbuild.am
|
||||
Executable
+2
@@ -0,0 +1,2 @@
|
||||
#!/bin/sh
|
||||
exec autoreconf -vfi
|
||||
@@ -0,0 +1,25 @@
|
||||
AC_PREREQ([2.63])
|
||||
AC_INIT([hacktree], [0], [walters@verbum.org])
|
||||
AC_CONFIG_HEADER([config.h])
|
||||
AC_CONFIG_MACRO_DIR([m4])
|
||||
AC_CONFIG_AUX_DIR([build-aux])
|
||||
|
||||
AM_INIT_AUTOMAKE([1.11 -Wno-portability foreign no-define tar-ustar no-dist-gzip dist-xz])
|
||||
AM_MAINTAINER_MODE([enable])
|
||||
AM_SILENT_RULES([yes])
|
||||
|
||||
AC_PROG_CC
|
||||
AM_PROG_CC_C_O
|
||||
|
||||
# Initialize libtool
|
||||
LT_PREREQ([2.2])
|
||||
LT_INIT
|
||||
|
||||
PKG_PROG_PKG_CONFIG
|
||||
|
||||
PKG_CHECK_MODULES(GIO_UNIX, [gio-unix-2.0])
|
||||
|
||||
AC_CONFIG_FILES([
|
||||
Makefile
|
||||
])
|
||||
AC_OUTPUT
|
||||
@@ -0,0 +1,3 @@
|
||||
#!/bin/sh
|
||||
|
||||
echo test
|
||||
Reference in new issue
Block a user