HOWTO Create A Build Host

From Gentoo Linux Wiki

[edit] Introduction

The purpose of this document is to describe a process for creating a gentoo build host on your local network.

CAVEAT: This process has worked on my system, your mileage may vary. Corrections are appreciated!

If your network has several gentoo installations this might be useful to you. In particular, if you have one high-powered system and several not-so-powerful systems this can be an effective way to keep all of them up to date while minimizing the time you sit waiting for compiles.

For this document we consider a network with two systems:

• buildHost - A dual AMD system with lots of memory and "enough" disk
• someClient - A low-end (i686) "support" system in the network

This document is written with the assumption that you have one client of the build host and, therefore, one architecture to support. However, the process described is designed to accomodate multiple clients and architectures.

[edit] Server Configuration

[edit] Create a filesystem dedicated to our builds

To keep things segregated I like to create dedicated filesystems for "odd" things like this. LVM makes it easy to do too. If you don't use LVM or don't want a clutter of small filesystems you can skip this section & substitute appropriately where you see /mnt/gentoo/i686 mentioned below. (Or, better yet, create a symlink from /mnt/gentoo/i686 to your build area.)

I use LVM so I create a logical volume for my build area.
10G is too much space but disk is cheap. Adjust as necessary.

lvcreate --name gentoo-i686 -L 10G buildHost
mke2fs -j /dev/buildHost/gentoo-i686
mkdir -p /mnt/gentoo/i686
echo '/dev/buildHost/gentoo-i686 /mnt/gentoo/i686 ext3 defaults 1 2' >> /etc/fstab
mount /mnt/gentoo/i686
df -k /mnt/gentoo/i686

If you don't see a 10G filesystem at /mnt/gentoo/i686, try again.

[edit] Install a basic gentoo system on that filesystem

The gentoo documentation is great. Follow the instructions to get stage3 configured in /mnt/gentoo/i686. Some things (such as network setup) can be safely skipped. Notice that our build area is called "i686" instead of "someClient". The idea here is that we will use one build area for all clients requiring i686 optimization rather than one build area per client. For instance, my network has three architectures: x86 (firewall), athlon-mp (build host), i686 (everything else).

Alternatively... You could just rsync someClient's filesystem to /mnt/gentoo/i686 since it is already installed and configured appropriately.

Configure /etc/make.conf in your /mnt/gentoo/i686 installation as though you were configuring the client itself. Specify whatever build optimizations you need as long as your build server can (cross)-compile to that target. The point of the build server is to build binary packages for use by the other clients in your network. To do that be sure to add

PKGDIR=/usr/portage/packages
FEATURES="buildpkg"

to make.conf. It is also wise to inclue

SYNC=rsync://buildHost/gentoo-portage

so that the builds pull from your build host's primary portage tree instead of the internet.

[edit] Configure NFS to make the build area's portage bits available

emerge nfs-utils
rc-update add nfs default
echo "/usr/portage/distfiles *.your.domain(sync,rw,no_root_squash)" >> /etc/exports
echo "/mnt/gentoo/i686/usr/portage/packages *.your.domain(sync,rw,no_root_squash)" >> /etc/exports
echo "/mnt/gentoo/i686/var/lib/portage      *.your.domain(sync,rw,no_root_squash)" >> /etc/exports
/etc/init.d/nfs start

The exports explained:

/usr/portage/distfiles 

If we build on the build host or the client we only have to download tarballs once.

/mnt/gentoo/i686/usr/portage/packages 

This is where build host builds will put binary packages. The client will mount this as /usr/portage/packages (pointed to by PKGDIR) so that it can download the binary packages built by the build host.

/mnt/gentoo/i686/var/lib/portage 

This is a hack so that the client can copy its /var/lib/portage/world file to the server. Other alternatives would be scp, rsync, etc... to make the client's world file available to the server. If you find a better approach be sure to modify the Maintenance and/or Maintain script(s) shown below.

[edit] Setup rsyncd to be used by someClient

If you don't have rsyncd installed on the server, do that now. http://www.gentoo.org/doc/en/rsync.xml is a good resource for setting up your rsyncd to make /usr/portage available to your clients. In short, you need an /etc/rsync/rsyncd.conf file something like this:

use chroot = no
max connections = 10
pid file = /var/run/rsyncd.pid
motd file = /etc/rsync/rsyncd.motd
transfer logging = yes
log format = %t %a %m %f %b
syslog facility = local3
timeout = 300

[gentoo-portage]
path = /usr/portage
comment = Gentoo Linux Portage tree mirror
exclude = distfiles/ packages/

Note that we exclude 'distfiles' and 'packages' because we'll be mounting them via NFS. Finally, start rsyncd:

rc-update add rsyncd default
/etc/init.d/rsyncd start

[edit] Create cron scripts to keep the build area up to date

There are some synchronization issues we have to deal with so I've chosen cron times carefully to avoid problems. If you adjust the times consider both the client and the server when you do.

echo "10 3 * * * root /usr/local/bin/emergeIt" >> /etc/crontab

Where /usr/local/bin/emergeIt looks something like this:

#!/bin/bash

# Update our portage tree.
# The build environments in /mnt/gentoo will use our copy
# rather than pulling a new one.
/usr/bin/emerge --sync

if [ -x /mnt/gentoo/Maintenance ]; then
  cd /mnt/gentoo
  ./Maintenance
fi

/mnt/gentoo/Maintenance will perform a build for each of our network architectures. I've left out x86 and athlon-mp for purposes of this document but you can see where they would fit into the 'for' loop easily.

#!/bin/bash

cd /mnt/gentoo

for arch in i686; do
  [ -f /tmp/NOBUILD-$arch ] && NOBUILD="NOBUILD" || NOBUILD=""
  mount -t nfs buildHost:/usr/portage/distfiles ${arch}/usr/portage/distfiles
  mount -t proc none ${arch}/proc
  cp Maintain ${arch}
  chroot ${arch} /Maintain ${arch} ${NOBUILD}
  rm -f ${arch}/Maintain
  umount ${arch}/usr/portage/distfiles
  umount ${arch}/proc
done

/mnt/gentoo/Maintain will "maintain" the build environment for a given architecture.

#!/bin/bash

ARCH="$1"
shift

# Collect any *.world files placed into /var/lib/portage
# by clients in our network.
sort -u /var/lib/portage/*.world > /var/lib/portage/world

# Ensure we have an updated portage tree.
# /etc/make.conf should be configured to rsync this from
# a local network host (possibly this build host)
emerge --sync

# Build everything required by any host in our network.
# This can take a very long time.
if [ "x$1" != "xNOBUILD" ]; then
  emerge --update --deep world
fi
# NOTE NOTE NOTE NOTE
# Please note that this script doesn't work, as simply adding a package to
# the world file doesn't force `emerge --update world` to build it.
# I'll post a fix when I find one.
# NOTE NOTE NOTE NOTE

[edit] Client Configuration

[edit] Configure portage to use the local server's rsyncd for 'emerge --sync'

 echo 'SYNC=rsync://buildHost/gentoo-portage' >> /etc/make.conf

[edit] Setup autofs to mount the server's build-area portage bits

emerge autofs
echo '/misc /etc/autofs/auto.misc --timeout=600' >> /etc/autofs/auto.master
echo 'distfiles       -fstype=nfs,bg,hard buildHost:/usr/portage/distfiles' >> /etc/autofs/auto.misc
echo 'packages        -fstype=nfs,bg,hard buildHost:/mnt/gentoo/i686/usr/portage/packages' >> /etc/autofs/auto.misc
echo 'var_lib_portage -fstype=nfs,bg,hard buildHost:/mnt/gentoo/i686/var/lib/portage' >> /etc/autofs/auto.misc
rc-update add portmap default
rc-update add autofs default
/etc/init.d/autofs start

Remove /usr/portage/packages and /usr/portage/distfiles
Don't just 'rm -rf' them now! Do it carefully!

ln -s /misc/distfiles /usr/portage/distfiles
ln -s /misc/packages /usr/portage/packages

df -k /usr/portage/distfiles /usr/portage/packages

Be sure you see NFS mounted filesystems from buildHost:/usr/portage/distfiles and buildHost:/mnt/gentoo/i686/usr/portage/packages

[edit] Setup cron to provide the client's 'world' file to the server

echo '40 2 * * * root /bin/cp /var/lib/portage/world /misc/var_lib_portage/`/bin/uname -n`.world' >> /etc/crontab

[edit] Setup cron to 'emerge --sync' nightly

echo '40 3 * * * root /usr/bin/emerge --sync --quiet' >> /etc/crontab

or, if you cannot use emerge,

echo '40 3 * * * root /usr/sbin/emerge-webrsync > /dev/null' >> /etc/crontab

[edit] Conclusion

That should do it! There are many places along the way where you can test. I highly recommend leaving out the crontab entries until you've manually tested everything.