Continuous Build
We use both Buildbot and Hydra to perform continuous building and testing of Octave.
Buildbot
We are using Buildbot to build the current development version of Octave on multiple systems in a number of different configurations. Current status of the builds my be found at buildbot.octave.org:8010/waterfall.
Systems and Configurations
The following systems and configurations are currently covered:
| Builder ID | Component | Hg Version | Build System | Host System | Compiler | Build Options | Frequency |
|---|---|---|---|---|---|---|---|
| clang-3.7-debian | Octave | default | Debian Testing | Debian | Clang 3.7 | Disable GraphicsMagick | Any Change |
| clang-3.8-debian | Octave | default | Debian Testing | Debian | Clang 3.8 | Disable GraphicsMagick | Any Change |
| clang-3.9-debian | Octave | default | Debian Testing | Debian | Clang 3.9 | Any Change | |
| gcc-4.9-debian | Octave | default | Debian Testing | Debian | GCC 4.9 | Disable GraphicsMagick | Any Change |
| gcc-5-debian | Octave | default | Debian Testing | Debian | GCC 5 | Any Change | |
| gcc-6-debian | Octave | default | Debian Testing | Debian | GCC 6 | Any Change | |
| gcc-6-lto-debian | Octave | default | Debian Testing | Debian | GCC 6 | Enable link time optimization | Any Change |
| no-extras-debian | Octave | default | Debian Testing | Debian | GCC (OS default version) | Disable all optional dependencies | Any Change |
| w32-on-debian | mxe-octave | default | Debian Testing | Windows | GCC (mxe-octave default) | Windows 32 | Daily |
| w64-32-on-debian | mxe-octave | default | Debian Testing | Windows | GCC (mxe-octave default) | Windows 64 | Daily |
| w64-64-on-debian | mxe-octave | default | Debian Testing | Windows | GCC (mxe-octave default) | Windows 64, 64-bit indexing | Daily |
Build Slave Configuration
To run a build slave for Octave, you must do the following:
- Contact the Octave Maintainers mailing list to let us know that you wish to provide a system to use as a build slave.
- Install buildbot. Packages exist for most distributions. See the buildbot docs for other options. You should create a separate user account with no special privileges that will run buildbot.
- Create a configuration file (see below).
- Run buildbot on the slave system, preferably by starting it automatically when your system boots. It should be running with the buildbot user ID.
You may also want to set up ccache to work with buildbot (strongly recommended to speed up builds). If you create a directory ~/buildbot/bin, it will be added to the execution PATH when buildbot runs commands on the slave. This directory can have symbolic links like the following:
lrwxrwxrwx 1 buildbot buildbot 15 Aug 26 11:39 gcc -> /usr/bin/ccache lrwxrwxrwx 1 buildbot buildbot 15 Aug 26 11:40 cc -> /usr/bin/ccache lrwxrwxrwx 1 buildbot buildbot 15 Aug 26 11:40 c++ -> /usr/bin/ccache lrwxrwxrwx 1 buildbot buildbot 15 Aug 31 23:46 gfortran -> /usr/bin/ccache
They should point to the actual location of ccache if it is not in /usr/bin.
Your system may be behind a firewall. It does not have to have a distinct public IP address.
Sample Slave Configuration File
You must edit the settings for basedir, slavename, and password. The basedir should be the absolute and fully expanded name of the directory containing the configuration file. For example, if the home directory for the buildbot user is /var/lib/buildbot and your slavename is set to 'debian-x86_64' , then basedir might be '/var/lib/buildbot/slaves/debian-x86_64'. The password should not be anything valuable. You'll be asked to provide the slavename and password so that they may be added to the master configuration on buildbot.octave.org. Do not post the password to the octave-maintainers mailing list.
import os
from buildslave.bot import BuildSlave
from twisted.application import service
basedir = '/PATH/TO/DIRECTORY/CONTAINING/THIS/CONFIGURATION/FILE'
rotateLength = 10000000
maxRotatedFiles = 10
# if this is a relocatable tac file, get the directory containing the TAC
if basedir == '.':
import os.path
basedir = os.path.abspath(os.path.dirname(__file__))
# note: this line is matched against to check that this is a buildslave
# directory; do not edit it.
application = service.Application('buildslave')
try:
from twisted.python.logfile import LogFile
from twisted.python.log import ILogObserver, FileLogObserver
logfile = LogFile.fromFullPath(os.path.join(basedir, "twistd.log"), rotateLength=rotateLength,
maxRotatedFiles=maxRotatedFiles)
application.setComponent(ILogObserver, FileLogObserver(logfile).emit)
except ImportError:
# probably not yet twisted 8.2.0 and beyond, can't set log yet
pass
buildmaster_host = 'buildbot.octave.org'
port = 9989
slavename = 'SLAVE-NAME'
passwd = 'PASSWORD'
keepalive = 600
usepty = 0
umask = None
maxdelay = 300
allow_shutdown = None
s = BuildSlave(buildmaster_host, port, slavename, passwd, basedir,
keepalive, usepty, umask=umask, maxdelay=maxdelay,
allow_shutdown=allow_shutdown)
s.setServiceParent(application)
Space Requirements
Building Octave takes a significant amount of disk space. With debugging symbols, you may need several GB for each build, plus room for ccache (possibly 50GB) if you use it. If you use a cache size that is larger than the default, you'll need to specify that in the .ccache/ccache.conf file using a line like
max_size = 50G
If the directory containing the build and ccache directories doesn't have sufficient space, then these directory names may point to a separate partition that does have enough space available.
Starting the Slave
With everything in place, you can run the slave server with the following commands (assuming that your basedir is '/var/lib/buildbot/slaves/debian-x86_64').
cd ~buildbot/slaves sudo -u buildbot buildslave start debian-x86_64
If successful, buildbot will respond with
... FIXME ...
Hydra
There is a build server continuously building the latest tip of the development branch of Octave. The build server is running Hydra, which is based on the Nix functional package management system. This is the official continuous build system of the GNU Project. The Hydra build of Octave is currently maintained by User:Mtmiller.
Octave Job Overview
The octave-default jobset overview page is the main entry point to the Hydra build of Octave. It shows the status of the most recent builds ("evaluations" in Hydra terminology).
Latest Build Logs
The full build logs for the latest builds are available at the following links, for quick reference:
Snapshot Source Tarball
A nice product of the Hydra continuous build of Octave is that there is always a source snapshot available for download. This source snapshot is always built from the current tip of the development branch in whatever state it is in. Therefore, it may not compile or run properly and should definitely not be used for anything other than evaluation. This can be useful to evaluate the current state of Octave's development branch without needing to clone the Mercurial repository.
Coverage Report
Another product of the Hydra continuous build is the code coverage report. The latest coverage report is always built from the current tip of the development branch. This report gives an analysis of the code coverage of the current Octave test suite. This can be used to find unexercised code paths and help improve the coverage of Octave's test suite (see Projects#Tests).