picobsd - building small FreeBSD disk images

CONTENTS

Synopsis
Description
Environment
BUILDING PicoBSD
Using Alternate Source Trees
INSTALLING PicoBSD
     Floppy Install
     Hard Disk Install
     CDROM Install
     Booting From The Network
BOOTING PicoBSD
CONFIGURING a PicoBSD system
See Also
Authors
Bugs

picobsd [options] [config-name [site-name]]

The picobsd utility is a script which produces a minimal implementation of
.Fx (historically called PicoBSD) which typically fits on a small media such as a floppy disk, or can be downloaded as a single image file from some media such as CDROM, flash memory, or through etherboot.

The picobsd utility was originally created to build simple standalone systems such as firewalls or bridges, but because of the ability to cross-build images with different source trees than the one in the server, it can be extremely useful to developers to test their code without having to reinstall the system.

The boot media (historically a floppy disk, but also small CDROM or USB keys) contains a boot loader and a compressed kernel which includes a memory file system. Depending on the media, it might also contain a number of additional files, which can be updated at run time, and are used to override/update those in the memory file system.

The system loads the kernel in the normal way, uncompresses the memory file system and mounts it as root. It then updates the memory file system with files from the boot media (if present), and executes a specialized version of /etc/rc. The boot media (floppy, etc.) is required for loading only, and typically used read-only. After the boot phase, the system runs entirely from RAM.

The following options are available (but also check the picobsd script for more details). The most important options for common operations are -src , -init , -n -and -v .

--all_in_mfs
	Put the entire contents of the file system in the memory file system image which is contained in the kernel. This is the default behaviour, and is extremely useful as the kernel itself can be loaded, using etherboot or pxeboot(8),
-c -, -clean
	Clean the product of previous builds.
--cfg file
	Specify a file that contains additional config commands.
--floppy_size size
	Set the size of the disk image. Typical values for a floppy disk are 1440 or 2880, but other values can be used for other media (flash memories, CDROM, network booted kernels). Note that this option is overridden by the content of the config files (config in the image tree, or the one specified with -cfg )
--init
	When used together with the --src option, this initializes the <SRC_PATH/../usr> subtree as necessary to subsequently build picobsd images.
--iso	Generate an ISO image, picobsd.iso, in addition to the disk image picobsd.bin.
--modules
	Also build kernel modules. These are not stored on the picobsd image but are left available in the build directory.
-n	Make the script non-interactive, skipping the initial menu and proceeding with the build process without requiring user input.
--no_all_in_mfs
	Leaves files contained in the floppy.tree on the picobsd image, so they can be loaded separately from the kernel (and updated individually to customize the image).
--no_loader
	Omit /boot/loader, just rely on boot2 to load the kernel. This saves some space but may have problems with kernels > 4MB.
--objdir directory
	Specify a directory with the result of a previous buildworld. This saves the need for an -init call before creating an image.
--src SRC_PATH
	Use the source tree at SRC_PATH instead the one at /usr/src. This can be useful for cross-building picobsd images. When using this option, you must also create and initialize the subtree at <SRC_PATH/../usr> with the correct header files, libraries, and tools (such as the config(8) program) that are necessary for the cross-build (see the --init option). The source files are unmodified by the picobsd script. However the source tree is not completely read-only, because config(8) expects the kernel configuration file to be in one of its subdirectories, and also the process of initializing the usr subtree touches some parts of the source tree (this is a bug in the release build scripts which might go away with time).
-v	Make the script verbose, showing commands to be executed and waiting for user input before executing each of them. Useful for debugging. as a fully functional system.

As a result of extreme size limitations, the picobsd environment differs from the normal
.Fx in a number of ways:

• There are no dynamic libraries, and there is no directory /usr/lib. As a result, only static executables may be executed.
• In order to reduce the size of the executables, all executables on a specific floppy are joined together as a single executable built with crunchgen(1).
• Some programs are supplied in minimalistic versions, specifically ns, a cut-down version of netstat(1), and vm, a cut-down version of vmstat(8).

The picobsd sources reside in the hierarchy /usr/src/release/picobsd. In the following discussion, all relative path names are relative to this directory.

The supported build script is /usr/src/release/picobsd/build/picobsd which can be run from anywhere, and relies on the sysutils/makefs port to build a filesystem without requiring mdconfig or root privileges to mount a filesystem. When run in interactive mode (the default without the -n option), the script will let you configure the various parameters used to build the PicoBSD image. An image is configured using the files and directories described below. The base system contains a template, called bridge for historical reasons, that can be used as a base for building various kinds of network appliances.

You can define your own PicoBSD configuration, by creating a directory with a name of your choice (e.g. FOO) which contains some of the following files and directories. For more information on how to construct these files, look at one of the standard picobsd configurations as a reference.

PICOBSD
	The kernel configuration file (required). This is a mostly standard kernel configuration file, possibly stripped down by removing unnecessary drivers and options to reduce the kernel’s size. To be recognised as a picobsd kernel config file, the file must also contain the line beginning with "#PicoBSD" below, and a matching MD_ROOT_SIZE option: #marker def_sz init MFS_inodes floppy_inodes #PicoBSD 4200 init 8192 32768 options MD_ROOT_SIZE=4200 # same as def_sz This informs the script of the size of the memory file system and provides a few other details on how to build the image.
crunch.conf
	crunchgen(1) configuration (required). It contains the list of directories containing program sources, the list of binaries to be built, and the list of libraries that these programs use. See the crunchgen(1) manpage for the exact details on the syntax of this file. The following issues are particularly important when dealing with picobsd configurations: We can pass build options to those makefiles which understand that, in order to reduce the size of the programs. This is achieved with a line of the form     buildopts -DNO_PAM -DRELEASE_CRUNCH ... When providing the list of directories where source files are, it is convenient to list the following entry first:     srcdirs /usr/src/release/picobsd/tinyware so that -specific versions of the programs will be found there. The string "@__CWD__@" is replaced with the full pathname of the directory where the picobsd configuration resides (i.e., the one where we find PICOBSD, crunch.conf, and so on). This can be useful to refer source code that resides within a configuration, e.g.     srcdirs @__CWD__@/src
config
	Shell variables, sourced by the picobsd script (optional). The most important variables here are:

MY_DEVS
	(Not used in .Fx 5.0 where we have devfs(5)). Should be set to the list of devices to be created in the /dev directory of the image (it is really the argument passed to MAKEDEV(8), so refer to that manpage for the names).
fd_size
	Size (in kilobytes) of the picobsd image. By default, fd_size is set to 1440 which produces an image suitable for a standard floppy. If you plan to store the image on a CDROM (e.g. using the "El Torito" floppy emulation), you can set fd_size equal to 2880. If you are planning to dump the image onto a hard disk (either in a partition or on the whole disk), you are not restricted to one of the standard floppy sizes. Using a large image size per se does not waste RAM at runtime, because only the files that are actually loaded from the image contribute to the memory usage.
import_files
	Contains a list of files to be imported in the floppy tree. Absolute names refer to the standard file system, relative names refer to the root of the source tree being used (i.e. SRC_PATH/..). You can normally use this option if you want to import files such as shared libraries, or databases, without having to replicate them first in your configuration under the floppy.tree/ directory.

floppy.tree.exclude
	List of files from the standard floppy tree which we do not want to be copied (optional).
floppy.tree/
	Local additions to the standard floppy tree (optional). The content of this subtree will be copied as-is into the floppy image.
floppy.tree.<site-name>
	Same as above, but site-specific (optional).

More information on the build process can be found in the comments in the picobsd script.

The build script can be instructed to use an alternate source tree using the --src SRC_PATH option. The tree that you specify must contain full sources for the kernel and for all programs that you want to include in your image. As an example, to cross-build the bridge floppy using RELENG_4 sources, you can do the following:

cd <some_empty_directory>
mkdir FOO
(cd FOO; cvs -d<my_repository> co -rRELENG_4 src)
picobsd --src FOO/src --init    # this is needed only once
picobsd --src FOO/src -n -v bridge

If the build is successful, the directory build_dir-bridge/ will contain a kernel that can be downloaded with etherboot, a floppy image called picobsd.bin, plus the products of the compilation in other directories. If you want to modify the source tree in FOO/src, a new image can be produced by simply running

    picobsd --src FOO/src -n -v bridge

whereas if the change affects include files or libraries you first need to update them, e.g. by re-running

    picobsd --src FOO/src --init # this is needed only once

as you would normally do for any change of this kind.

Historically, picobsd is run from a floppy disk, where it can be installed with a simple

    dd if=picobsd.bin of=/dev/rfd0

and the floppy is ready to boot.

The same process can be used to store the image on a hard disk (entire volume or one of the slices):

dd if=picobsd.bin of=/dev/ada2
dd if=picobsd.bin of=/dev/ada2s3
dd if=picobsd.bin of=/dev/ada2 oseek=NN

The first form will install the image on the entire disk, and it should work in the same way as for a floppy.

The second form will install the image on slice number 3 (which should be large enough to store the contents of the image). However, the process will only have success if the partition does not contain a valid disklabel, otherwise the kernel will likely prevent overwriting the label. In this case you can use the third form, replacing NN with the actual start of the partition (which you can determine using gpart(8)). Note that after saving the image to the slice, it will not yet be recognised. You have to use the disklabel(8) command to properly initialize the label (do not ask why!). One way to do this is

disklabel -w ada0s2 auto
disklabel -e ada0s2

and from the editor enter a line corresponding to the actual partition, e.g. if the image has 2.88MB (5760 sectors) you need to enter the following line for the partition:

    a: 5760 0 4.2BSD 512 4096

At this point the partition is bootable. Note that the image size can be smaller than the slice size (indicated as partition "c:").

picobsd can produce an ISO image named picobsd.iso, which does not use "El Torito" emulation, so it has no size restrictions. Installing means just burning a media with the file.

Yet another way to use picobsd is to boot the image off the network. For this purpose you should use the uncompressed kernel which is available as a byproduct of the compilation. Refer to the documentation for network booting for more details, the picobsd kernel is bootable as a standard
.Fx kernel.

To boot picobsd, insert the floppy and reset the machine. The boot procedure is similar to the standard
.Fx boot. Booting from a floppy is normally rather slow (in the order of 1-2 minutes), things are much faster if you store your image on a hard disk, Compact Flash, or CDROM.

You can also use etherboot to load the preloaded, uncompressed kernel image which is a byproduct of the picobsd build. In this case the load time is a matter of a few seconds, even on a 10Mbit/s ethernet.

After booting, picobsd loads the root file system from the memory file system, starts /sbin/init, and passes control to a first startup script, /etc/rc. The latter populates the /etc and /root directories with the default files, then tries to identify the boot device (floppy, hard disk partition) and possibly override the contents of the root file system with files read from the boot device. This allows you to store local configuration on the same media. After this phase the boot device is no longer used, unless the user specifically does it.

After this, control is transferred to a second script, /etc/rc1 (which can be overridden from the boot device). This script tries to associate a hostname to the system by using the MAC address of the first ethernet interface as a key, and /etc/hosts as a lookup table. Then control is passed to the main user configuration script, /etc/rc.conf, which is supposed to override the value of a number of configuration variables which have been pre-set in /etc/rc.conf.defaults. You can use the hostname variable to create different configurations from the same file. After taking control back, /etc/rc1 completes the initializations, and as part of this it configures network interfaces and optionally calls the firewall configuration script, /etc/rc.firewall, where the user can store his own firewall configuration.

Note that by default picobsd runs entirely from main memory, and has no swap space, unless you explicitly request it. The boot device is also not used anymore after /etc/rc1 takes control, again, unless you explicitly request it.

The operation of a picobsd system can be configured through a few files which are read at boot time, very much like a standard
.Fx system. There are, however, some minor differences to reduce the number of files to store and/or customize, thus saving space. Among the files to configure we have the following:

/etc/hosts
	Traditionally, this file contains the IP-to-hostname mappings. In addition to this, the picobsd version of this file also contains a mapping between Ethernet (MAC) addresses and hostnames, as follows: #ethertable start of the ethernet->hostname mapping # mac_address hostname # 00:12:34:56:78:9a pinco # 12:34:56:* pallino # * this-matches-all where the line containing "#ethertable" marks the start of the table. If the MAC address is not found, the script will prompt you to enter a hostname and IP address for the system, and this information will be stored in the /etc/hosts file (in memory) so you can simply store them on disk later. Note that you can use wildcards in the address part, so a line like the last one in the example will match any MAC address and avoid the request.
/etc/rc.conf
	This file contains a number of variables which control the operation of the system, such as interface configuration, router setup, network service startup, etc. For the exact list and meaning of these variables see /etc/rc.conf.defaults. It is worth mentioning that some of the variables let you overwrite the contents of some files in /etc. This option is available at the moment for /etc/host.conf and /etc/resolv.conf, whose contents are generally very short and suitable for this type of updating. In case you use these variables, remember to use newlines as appropriate, e.g. host_conf="# this goes into /etc/host.conf hosts bind" Although not mandatory, in this file you should only set the variables indicated in /etc/rc.conf.defaults, and avoid starting services which depend on having the network running. This can be done at a later time: if you set firewall_enable ="YES", the /etc/rc.firewall script will be run after configuring the network interfaces, so you can set up your firewall and safely start network services or enable things such as routing and bridging.
/etc/rc.firewall
	This script can be used to configure the ipfw(4) firewall. On entry, the fwcmd variable is set to the pathname of the firewall command, firewall_type contains the value set in /etc/rc.conf, and hostname contains the name assigned to the host.

There is a small script called update which can be used to edit and/or save to disk a copy of the files you have modified after booting. The script takes one or more absolute pathnames, runs the editor on the files passed as arguments, and then saves a compressed copy of the files on the disk (mounting and unmounting the latter around the operation).

If invoked without arguments, update edits and saves rc.conf, rc.firewall, and master.passwd.

If one of the arguments is /etc (the directory name alone), then the command saves to disk (without editing) all the files in the directory for which a copy already exists on disk (e.g. as a result of a previous update).

crunchgen(1), mdconfig(8), nanobsd(8), swapon(8)

.An -nosplit
.An Andrzej Bialecki Aq abial@FreeBSD.org , with subsequent work on the scripts by
.An Luigi Rizzo Aq luigi@iet.unipi.it and others. Man page and Makefiles created by
.An Greg Lehey Aq grog@lemis.com .

Documentation is still incomplete.

Visit the GSP FreeBSD Man Page Interface.
Output converted with manServer 1.07.