Chroot tools (for packaging)

General talk about packaging procedures and packages.
Post Reply
Shador
Posts: 1295
Joined: 11. Jun 2009, 14:04
Location: Bavaria

Chroot tools (for packaging)

Post by Shador »

Disclaimer: If you really just want the essence, scroll down to the blue part.

:arrow: Background (not for very lazy people :P)
During our last and first official team meeting a point has been made that packages should be built with virtual machines. This is to ensure a clean environment, allow cross-package and testing in a basic and clean system. In a optimal scenario one would use snapshots to start clean every time.
That's actually a really good point although I'm not a big fan of VMs for packaging.

Why :?:
  1. Speed: VMs are slow(er) (at least for packaging and even more for some other special uses).
  2. Workload: Managing VMs and especially snapshots with them is a lot of work. And you can loose track with all that stuff like hard disks, configurations, snapshots, ...
  3. Storage: You need many hard disk and need to be able to create new ones on the fly (at least for snapshotting). Those hard disk are not necessarily very space efficient
  4. Isolation: VMs are almost a completely separate system, so without additional setup (--> workload) it can easily get quite painful to work with.
So much for the motivation and background.

:arrow: Alternative: Chroots (not for lazy people :P)
So...? If it's that bad you gotta have something else? Yes! Chroots are an alternative, which I've been using a while now for packaging 32-Bit on a 64-Bit system. It works pretty well.

Some cons for chroots:
  1. Isolation: The lower level of isolation doesn't require and allow you to boot a system from scratch. You're just using different binaries. Although this saves you booting and has other advantages, it could rarely cause problems for some very special packages. But as it's a less complete system, you usually notice that you need to switch to VMs or real hardware because something fails. I.e. usually you can say for most packages if it works in the chroot, it works on real hardware/VMs too.
  2. Flexibility: You can usually only package for the same arch, you're physically running. x86_64 is an exception as x86 code can run on a x86_64 cpu. As I'm only packaging for x86* that's fine with me.
:arrow: The Idea (not for slighly lazy people :P)
Once there was beautiful, shiny and very useful think-tank with an agency named NSA in a country called the US ... WTF?! ... Ok, you're right. No more fairy tales. ;)
Actually at some point I realized how perfectly useful a filesystem like btrfs can be here. As an enterprise-level filesystem it has features like subvolumes and snapshoting among othesr but these two are useful here. Subvolumes are much like separate filesystem sharing the same storage space and snapshots are copies of other subvolumes. The great thing about those snapshots is they share storage space for unmodified data and both snapshot and original subvolume are writable.

The actual idea is to create such a directory structure:

Code: Select all

i486/
    13.0/
    13.1/
    13.37/
x86_64/
    13.0/
    13.1/
    13.37/
Now those directories contain a Salix Core installation of that version and architecture used as chroot. Whenever I build a package, I create btrfs snapshot e.g. named i486/13.0-thunderbird_locale, install all build prerequisites there and build the packages. The creation is because of the usage of snapshots almost instantly and once I'm finished I can just drop the snapshot again.

:arrow: Getting to the point
Now to make that possible or a lot easier (primarily as in few work), I wrote some tools.
  • chroot-helper: Prepares are chroot, actually changes root and cleans it up afterwards (based on the switch32 script I posted somewhere else)
  • chroot-installer: Takes a Salix Installer ISO image and installs Salix to some arbitary directory for usage with chroots. As parts from the setup are reused, the installation (almost) completely resembles a normal installation (this was a requirement during development). Only the parts about bootloader, partitions setup, fstab, ... have been skipped as they don't make much sense when installling to a directory which might not even be a partition. So if you want your system to be actually bootable, you have to do that yourself.
  • pad: This tool helps you managing your btrfs pool with the subvolumes/chroots and the snapshots/tags. Apart you can easily perform actions on sets of chroots like commands or package installations.


Here's a package which you can download that includes all 3 tools + documentation (man page):
http://gaia.homelinux.org/salix/chroot- ... ch-1ab.txz
Requires btrfs-progs

Read the man page for detailed information. If somethings missing there let me know, so I can help you and fix it for the future. I recommend also checking out the btrfs command and the filesystem in general.


:arrow: Example
Apart from that here's an exemplary walkthrough:
If you have no (big enough) spare partition (otherwise use your device instead of /dev/loop2):

Code: Select all

truncate -s 30G test-btrfs.img
losetup -f test-btrfs.img
losetup -a  # get the loop device for that file, here: /dev/loop2
Let's setup the filesystem:

Code: Select all

mkfs.btrfs /dev/loop2
mount /dev/loop2 /mnt
mkdir /mnt/{i486,x86_64}
btrfs subvolume create /mnt/i486/13.37
btrfs subvolume create /mnt/x86_64/13.37
:idea: It's important that you use the btrfs commands on the second level as a normal directory is no subvolume and only subvolumes can be taken snapshots of.

Now we're going to use chroot-installer for installing Salix to those subvolumes and create a snapshot in case we mess up (don't use dashes in the names):

Code: Select all

chroot-installer --root=/mnt/i486/13.37 --iso=/home/shador/Downloads/salix-xfce-13.37.iso --kernel=smp --kb=de
btrfs subvolume snapshot /mnt/i486/13.37/ /mnt/i486/13.37_orig
chroot-installer --root=/mnt/x86_64/13.37 --iso=/home/shador/Downloads/salix64-xfce-13.37.iso --kernel= --kb=de
btrfs subvolume snapshot /mnt/x86_64/13.37/ /mnt/x86_64/13.37_orig
Just follow the onscreen instructions, which you probably know already. Later I'll explain how to create a user that matches your normal user, so you get no problems with permissions with a shared home.

Now change the config of pad with your favourite editor (POOL and CHROOT_USER should be enough to start with):

Code: Select all

vi /etc/pad/default.conf
pad l
If all was successful you should get an output like this from pad l:

Code: Select all

i486/13.37
i486/13.37_orig
x86_64/13.37
x86_64/13.37_orig
On my real chroot pool it looks like this:

Code: Select all

i486/13.0
i486/13.0.2a
i486/13.0.2a_orig
i486/13.1
i486/13.1.2
i486/13.1.2_orig
i486/13.37
i486/13.37_orig
x86_64/13.0
x86_64/13.0.2a
x86_64/13.0.2a_orig
x86_64/13.1
x86_64/13.1.2a
x86_64/13.1.2a_orig
x86_64/13.37
x86_64/13.37_orig
Whereas 13.1and 13.0 are actually just symlinks for convenience. Actually, stupid, I know. :D

Now we're going to prepare the systems:

Code: Select all

# changes the mirror, you wouldn't want mine as it's slow for everybody else
pad e - 13.37 % sed -e 's#http://salix.enialis.net#http://gaia.homelinux.org/salix/mirror#' -i /etc/slapt-get/slapt-getrc
# upgrades the system and install some always needed stuff
pad e - 13.37 % slapt-get --add-keys \;slapt-get -u \; slapt-get -upgrade \; slapt-get -i slk-pkgcheck
# that's for my user, your uid and user name are probably different (be sure to match at least the uid)
pad e - 13.37 $ useradd -g 100 -G users,lp,floppy,audio,video,cdrom,plugdev,power,netdev,scanner -M -N -u 1992 shador
# updates root's bashrc so the command prompt indicates the chroot
pad p - 13.37
Now add to the .bashrc of the user you specified as CHROOT_USER the same code:

Code: Select all

if [ "$CHROOT" == "true" ]; then
    if [ -z "$CHROOT_NAME" ]; then
        CHROOT_NAME=chroot
    fi
    PS1="($CHROOT_NAME) $PS1"
fi
As the /home path is shared with the chroot, you only need to change this once in your normal home.

Done! Now you can start packaging. :o 8-)

To build a package, I could now do this for example:

Code: Select all

cd ~/projects/salix/build/mktoc/
screen
pad c mktoc 13.37 %% chardet,python
slkbuild -X
# oops, missed setuptools
su
slapt-get -i setuptools
exit
slkbuild -X
:idea: Some final hints. Don't run two builds in the same directory at the same time (--> i486 and x86_64). That 'pad c mktoc ...' command should open two additional windows in screen. The second one only shows up after switching the screen. It could also take a while for it to show up, if a lot of packages are to install.
To clean up again issue:

Code: Select all

pad d --
or

Code: Select all

pad d mktoc 13.37
:arrow: Feedback/Questions/... :?: :?: :?:
At the very end I would be pleased about any kind of feedback or questions especially remaining bugs and feature request (or better patches :) ).
Image
User avatar
gapan
Salix Wizard
Posts: 6238
Joined: 6. Jun 2009, 17:40

Re: Chroot tools (for packaging)

Post by gapan »

Wow. This is very interesting. I'll give it a shot. Thanks a lot Shador! :)
Image
Image
User avatar
fredg
Posts: 232
Joined: 3. Oct 2009, 16:50
Location: Lyon, France
Contact:

Re: Chroot tools (for packaging)

Post by fredg »

Very interesting, I like the Btrfs part.
It can be compare to a light LXC, do you try it ?
http://www.salixos.org/forum/viewtopic.php?f=24&t=3386

++
User avatar
JRD
Salix Warrior
Posts: 950
Joined: 7. Jun 2009, 22:52
Location: Lyon, France

Re: Chroot tools (for packaging)

Post by JRD »

Very interresting...too bad I can't use it (I run Salix 32...so I cannot chroot in a 64bits env...only VM for me in this case)
Image
Shador
Posts: 1295
Joined: 11. Jun 2009, 14:04
Location: Bavaria

Re: Chroot tools (for packaging)

Post by Shador »

JRD wrote:Very interresting...too bad I can't use it (I run Salix 32...so I cannot chroot in a 64bits env...only VM for me in this case)
Yes, that's a major downside. But it can still be used for managing snapshots. Apart from that I read sometime that you can setup your system as to invoke qemu transparently for non-native binaries. That allows you to run any arch supported by qemu on any system transparently like native applications.
fredg wrote:It can be compare to a light LXC, do you try it ?
No, I saw it on svn and had a look at the installer code. But as I really wanted to match a standard intallation I decided to make my own installer.
Still, I guess, given appropriate modifications to the chroot-helper, pad can be used with lxc too if better isolation is required for example. But as that doesn't seem necessary for packaging and chroots are easier, more commonly known and more familiar to me, I went that road.
Image
User avatar
tsuren
Posts: 206
Joined: 7. Jun 2009, 17:32
Location: France

Re: Chroot tools (for packaging)

Post by tsuren »

This is really interesting. I will give a try for sure.
'Tommorow is like today, just happens tomorrow.'
User avatar
thenktor
Salix Wizard
Posts: 2426
Joined: 6. Jun 2009, 14:47
Location: Franconia
Contact:

Re: Chroot tools (for packaging)

Post by thenktor »

Yes, I think it's cool. Especially because I can use this over ssh :)
Image
burnCDDA (burns audio CDs)
geBIERt (German beer blog)
Post Reply