|Ports directory||If the PORTSDIR environment variable is set when the initial profile is created, it establishes the initial value. Otherwise, synth will extract the value of PORTSDIR from /usr/share/mk and use that. If neither source points to a valid path, /usr/dports and /usr/ports will be successfully tried. If those dont exist either, synth will exit with an error message.|
|Packages directory||This is the location of the local repository metadata and all the packages that are built by synth. It clearly must have lots of available disk space. The default value is /var/synth/live_packages but it should be altered if the system has a /var partition with limited space.|
This indicates where all the ports source distribution files are stored, and
the initial value is determined by scanning the existing configuration. The
defaults are defined by ports and are
.Fx ) and /usr/distfiles for (
.Dx ). Most users will want to keep the initial value as that directory should already contain distfiles.
|Port options directory||This is the directory where all the selected options for ports are cached. The initial value comes from a system scan, so chances are it has the correct value. However, if the user would like a separate configuration area for port options, they would create the new directory and set this value accordingly (but the user would have to ensure the new location is passed to the port when configuring port options later in that case).|
|Build logs directory||A log is produced for every port built (subsequent builds of the same port will overwrite previous logs). This item dictates where those logs are stored, which is /var/log/synth by default. If the /var partition is limited, the user will definitely want to change this location as the uncompressed logs can potentially consume gigabyte of space.|
|Build base directory||This is the mount point for all the builders. When tmpfs is used, this location requires very little space as it contains empty directories that serve as mount points. If tmpfs is not used, then this location should have access to a lot available diskspace as it will provide the work area and virtual /usr/local for each builder. The default location of /usr/obj/synth-live should be fine for most users (it generally has lots of space and tmpfs is the default build mode).|
|System root directory||Most users will use the base system as the basis for the build environment, and thus the default setting of / is correct. Advanced users may use DESTDIR with installworld to create an alternative build base (e.g. i386 cross-build on amd64 architecture or a modified base that serves as a fallout test for an upcoming feature). Those would be the users that would modify this value.|
|Compiler cache directory|
|If ccache is properly installed on the system, set this value to the location of the cache. All the builders will leverage it. It is disabled by default. It can be disabled again by entering any invalid path as a new value, e.g. "none".|
|Num. concurrent builders||The represents the number of ports that can be simultaneously built on the system. The selected value is influenced by the number of physical cores and hyperthreads the system has, the amount of memory on the system, the amount of available swap, and if that swap is a solid-state drive or not. Generally memory is the limiting resource when tmpfs is used, so the default value for the number of builders is generally 75% of the number of CPUs of the system. The user is free to balance jobs versus builders as well.|
|Max. jobs per builder||If memory is constrained, its often a better performance tradeoff to reduce the number of builders and increase the number of jobs per builder. The default value varies depending on the system, but it will never exceed 5.|
|Use tmpfs for work area||If the initial scan shows there is at least 1.25 Gb per builder, this item will default to "true". Building on tmpfs is a big speed spot and its recommended if the memory resources allow it. This item is directed at the work area for building the port|
|Use tmpfs for /usr/local||Before a port can start building, all the build and library dependencies have to be installed into the localbase. Extracting, installing and later deinstalling is a lot of work for a disk, so using tmpfs here is another performance boost. It is recommended that users with adequate memory leave this enabled.|
|Display using ncurses||During the concurrent building, the default display is constructed using ncurses. It shows a lot of good information and it should be used. The display will fall back to a text mode if the terminal doesnt support it. This setting forces text mode unconditionally, but few users will want or need to use this option.|
|Fetch prebuilt packages||When active, this option will scan the external repository for suitable prebuilt packages to satisfy build requirements. To be considered suitable, the ABI, options, and dependencies of the remote package must all match requirements. If a package is located, it will be fetched and placed in the packages directory and later incorporated into the local repository. This option is not active by default, meaning that synth will normally build everything from source and ignore external repositories.|
To create alternative profiles, press the ">" key at the prompt. A new menu will appear that provides options to switch to another existing profile or create a new one that can be tailored and named. Any newly created profile becomes the new default profile, but this can be easily changed by repeating the profile selection process.
Pressing the Alphabetic key associated with each configuration item brings up a prompt to change the value. Directory entries must be valid paths except in the case of ccache. The boolean values are modified with a single keypress ( "T" or "F" ) and the numeric items require positive integers. After changing the values, the old menu returns and altered values are marked with an asterisk. Simply entering the "Enter" key ( carriage return ) saves the changes while pressing the "Escape" key will discard them.
If more than one profile exists, an option to delete profiles will appear on the menu. By pressing the "<" key, a new menu will appear that lists all the inactive profiles. Entering the number associated with one of those profiles will immediately delete the profile. The deletion cannot be undone.
The upgrade-system command automates the full upgrade of the system upon which synth is installed. It is expected that the ports tree has been updated to the latest version before the command is executed. Using the ports tree as a reference, synth will query pkg(8) to determine what is currently installed and which software has newer versions available. It will rebuild those packages and any package that depends libraries they contain or that have a runtime dependency on the rebuild packages. In addition, any port that has changed with respect to its options, dependencies, or ABI will be rebuilt as well. After analysis, the concurrent builder will begin building the set of packages that have been identified as requiring a rebuild. When that phase is complete, the local repository will be rebuilt without waiting for confirmation. Finally, synth will command pkg(8) to upgrade the system using the local repository, effectively bringing it completely up to date.
The prepare-system command has the same functionality of the upgrade-system command with the exception that the packages will not be installed. Once the local repository is fully rebuilt, the command exits.
The rebuild-repository command will perform a full sanity check on all packages present in the packages directory and remove the ones which fail the check. The local Synth repository will be created (or recreated) using the remaining packages. This step is also done as part of several other commands, so it mainly exists for scripting purposes.
The status command with no arguments performs a dry-run of the upgrade-system command. It will not delete obsolete packages, nor rebuild the local repository. It will list all the ports that will be rebuild along with a total, and it also logs the same information to /tmp/synth_status_results.txt since the full list is often longer than the terminal height.
The just-build command takes a list (one or more) of port origins ( an origin is the combination of a category and the port directory, e.g. lang/python27 ) and it will builds the packages of those ports if necessary. If analysis determines the packages are already up to date, nothing is done.
This is the first of several commands that take arguments. The arguments are either an unlimited number of port origins, e.g.synth just-build editors/joe editors/nano editors/libreofficeor they are limited to one, a path to a file, e.g.
synth just-build /tmp/build.listA file that is equivalent to the first example would contain:
editors/joe editors/nano editors/libreofficeWhen the building task is complete, synth exits.
The build command differs from the just-build command by asking the user if they wish to rebuild the local repository when the building is complete. The answer is usually "N" ( for "no" ) because rebuilding the repository can take a few minutes to complete, and it only makes sense to do when all the building is complete. Should the user opt to rebuild the repository, synth will then ask if the user wishes to install the packages that come from the ports list on the system.
The install command is similar to the build command except that it will not ask permission to rebuild the repository or install the resulting packages. It will just do it all in sequence.
The force command is similar to the build command except that any packages related to the ports list will be deleted first, regardless if they are up to date or not. This results in that every port on the given list will build. As with the build command, synth will request permission to rebuild the repository and possibly install the packages when the building phase is complete.
The test command will pre-delete any existing packages related to the ports list before rebuilding them with extra DEVELOPER_MODE tests and settings. When the building is complete, it will just exit without offering to rebuild the repository.
This version of the status command provides a dry-run and reports on what would happen if the just-build command was executed with the same arguments. The output and logging is similar to the solo status command, but the build scope is limited to the given ports list.
The everything command is not meant for most users. It will attempt to build every port in the ports tree, which can take several days depending on the power and resources of the building machine. Even downloading the tens of gigabytes worth of distfiles consumes significant time!
The status-everything command is just a dry-run for the everything command. Since synth works through increment building ( that is, it will not build a port if an up to date package already exists for it and all the packages dependencies are also in place ) , the resultant build list can be much shorter than the full list of ports in the tree. Only those rare users that wish to build the entire set of ports would use this command.
This is a useful command that will first scan the entire ports tree to gather a complete list of distfiles used by the ports tree, and then it will scan the contents of the configured distfiles directory. All source distribution files that cannot be matched to at least one port in the tree will be removed. This can result in gigabytes of recovered disk space if significant time has passed since the distfiles directory was last cleaned.
PORTSDIR The PORTSDIR environment variable is only checked on the very first signficant command of synth ( meaning everything except help and version ) when the default profile is created. It is also checked when adding new profiles. The variable dictates the default location of the ports tree with a higher priority than the standard locations. SYNTHPROFILE When this environment variable is set to name of an existing synth profile, that profile will be loaded when synth is launch rather than the active profile. It is a temporary override; removing the variable will allow the default profile to load again. WHYFAIL When this environment variable is defined to any value, the sanity check routines will provide more detail on dependency and option check failures on a separate output line. ENTERAFTER This environment variable only applies with the test command, and then only when a single port is specified. If both of those conditions are satisfied, and the variable has the value of:
then an interactive build will be attempted. First, all the ports prerequisites are built normally and synth will return to text mode. If everything built successfully, the specified port will build up to and including the point specified by ENTERAFTER. At that point, control will be given to the user by providing them with a tcsh shell with the current directory set at the root of the builder. When the user is done, they should terminate the shell with the exit command to signal synth to clean up and terminate. This is a developer tool to troubleshoot port build problems and it is not intended for the average user.
extract patch configure build stage install deinstall
LOCK This environment variable only applies with the test command. When it is defined to any value, the builders localbase is remounted read-only prior to the configure phase and restored to read/write permissions after building is complete. This is a diagnostic tool aimed at pinpointing file system violations as a port is not supposed to write to localbase prior to staging. Note that the localbase tmpfs setting will considered as False in this mode, so building may be noticeably slower than normal.
The following files may be used or created:
synth.ini This is the configuration file. It is automatically generated and manipulated using the interactive configuration menu. There is no need to touch this file other than to remove unwanted profiles as currently that is not possible from the program itself. It is normally located at /usr/local/etc/synth/synth.ini <profile>-make.conf This is an optional, user-provided file. If it exists, the builders /etc/make.conf will be appended with the contents of this file. For the default profile, the file would normally be located at /usr/local/etc/synth/LiveSystem-make.conf <profile>-environment This is an optional, user-provided file. It contains a list of name-value pairs joins with an equals sign (e.g. HTTP_PROXY=http://proxyserver:8888), one pair per line. If the file exists, these variables will be defined in the builder environment. For the default profile, the file would normally be located at /usr/local/etc/synth/LiveSystem-environment <profile>-private.key This is one of two files required to support the signing of the repository by a local RSA key. It is the output of the openssl genrsa command; see the EXAMPLE section for how to generate it. The file permissions are required to be set at 400 as well. For the default profile, the file is normally located at /usr/local/etc/synth/LiveSystem-private.key <profile>-public.key This is the second of two files required to support the signing of the repository by a local RSA key. synth must confirm the public counterpart of the signing key is available because its path is required for the generated pkg repository configuration file. For the default profile, the file is normally located at /usr/local/etc/synth/LiveSystem-public.key <profile>-signing_command This is one of two files required to support the signing of the repository by an external signing server. It consists of a single line containing the command; see the EXAMPLE section. For the default profile, the file is normally located at /usr/local/etc/synth/LiveSystem-signing_command <profile>-fingerprint This is the second of two files required to support the signing of the repository by an external signing server. It consists of a single line containing the SHA 256 hash of the public key that is the counterpart of the key that signed the repository. For the default profile, the file is normally located at /usr/local/etc/synth/LiveSystem-fingerprint synth_status_results.txt Whenever a status command is run, a list of ports that would build with the actual command is produced at /tmp/synth_status_results.txt synth_prefetch_list.txt Whenever a status command is run, and the option to use suitable prebuilt packages has been selected, a list of packages that will be fetched as a result will be written to /tmp/synth_prefetch_list.txt port logs Every single attempt at building a port results in a build log. The name of the log is always in the form of [category]___[port].log so that subsequent builds will overwrite previous attempts. They are created in the directory specified by the configuration. hook_run_start If this file exists, and it is executable ( or it is a symbolic link that leads to an executable file ) then it will be executed by synth at the start of the build. The same requirement exists for all hooks. This hook, and the next five hooks all define the following common environment variables: PROFILE, DIR_PACKAGES, DIR_REPOSITORY, DIR_PORTS, DIR_OPTIONS, DIR_DISTFILES, DIR_LOGS, DIR_BUILDBASE. Uniquely, this hook also defines PORTS_QUEUED which is the number of ports that synth will attempt to build. This file must be colated with synth.ini, so that would be /usr/local/etc/synth/hook_run_start for most installations. hook_run_end This hook is fired when the build completes. In addition to the common definitions, it defines PORTS_BUILT, PORTS_FAILED, PORTS_IGNORED, and PORTS_SKIPPED in the environment, which represents the results of the build. hook_pkg_success This hook is fired whenever a builder successfully packages a port. In addition to the common definitions, it defines RESULT=success, ORIGIN, and PKGNAME in the environment. hook_pkg_failure This hook is fired whenever a builder fails to package a port. In addition to the common definitions, it defines RESULT=failure, ORIGIN, and PKGNAME in the environment. hook_pkg_ignored This hook is fired prior to the start of a build when synth prescans the queue for ports that are to be ignored according to the ports tree. Once the build has started, this hook is never fired again. In addition to the common definitions, it defines RESULT=ignored, ORIGIN, and PKGNAME in the environment. hook_pkg_skipped This hook is fired whenever a package gets ignored or fails which in turns cascades into a number of ports getting removed from the build queue due to the dependencys failure. This may fire before the build starts due to ignored prescanning, or any time during the build. In addition to the common definitions, it defines RESULT=skipped, ORIGIN, and PKGNAME in the environment.
During development, the interrupt signal ( SIGINT ) was captured and handled. It turns out this handling extended to the builders, so any event resulting in a SIGINT in the build would lock up the builder permanently. Theres no apparent workaround for this given the design, so pressing Control-C is no longer handled. It will stop the build, but it will leave everything in bad state. The next significant call to synth will attempt to clean that up though. If a user wants to stop the build, the best approach is to type the Control-Q combination. The program will shut down as soon as it can. During the building phase no new builds will start after the shutdown begins, but the active builders will continue until completion.
At this time, synth can only be run by the root user.
A common question is "What does the Impulse indicator on the display mean?" It is the package build rate (packages/hour) over the last 500 seconds. For the first roughly ten minutes of a bulk run, the "Impulse" rate and the "Pkg/Hour" rate will be identical. After that, the "Impulse" build rate will differ from the rate averaged over the entire elapsed time, often by a great amount. Its just a way to gauge the package production rate over the previous few minutes.
Synth is written in Ada, and thus an Ada compiler or cross-compiler is required to build it. For
.Fx , that means synth is currently limited to amd64 and i386. It is feasible to port GNAT to the ARMv7 architecture as this has already been done ( See lang/gnatdroid-armv7 port ) but its not a trivial task to do it. Other architectures would be challenging, although not impossible, to support.
To activate the signing of default profiles repository with a local RSA key, generate the two key files:% cd /usr/local/etc/synth % openssl genrsa -out LiveSystem-private.key 2048 % chmod 0400 LiveSystem-private.key % openssl rsa -in LiveSystem-private.key -out \ LiveSystem-public.key -pubout
To activate the signing of default profiles repository with an external signing server, two files need to be created. Its assumed the signing server has already been configured as described by the pkg-rep(8) man page, and that the keys are located in the repo.pub and repo.key files. On the signing server which is 192.168.0.44 in this example:% sha256 -q repo.pub > LiveSystem-fingerprint % echo "/usr/bin/ssh 192.168.0.44 /usr/local/bin/sign.sh" \ > LiveSystem-signing_command
Then move both files to /usr/local/etc/synth directory on the build server. The build server root user needs to have a passphrase-free SSH key access to the signing server in order for Synth to sign the repository.
synth was conceived long before coding began in December of 2015. It was designed to be system-agnostic from the beginning, and in particular support for
.Dx was added simulaneously. It was hoped that it would attract users of the older PortMaster and PortUpgrade ports management tools by providing them with a superior and well-maintained alternative.
synth was written entirely by
.An John Marino <marino@FreeBSD.org> and released under the Internet Software Consortium license.