The Shoes of the Fisherman's Wife Are Some Jive-Ass Slippers

Fri, 02 May 2008

Building RPMs in SCons

Here's a recipe I've developed for building a RPM in SCons with out too much mucking around. My big issue with RPM is the whole /usr/src/redhat thing which is silly from a permissions point of view, and also prevent multiple users on the same machine from building at the same time.

import string, os

# Sources for RPM

arch = 'i386'

sources = [
    'foo.spec',
    'foo-upstream.tar.gz',
    'bippity.patch',
    'flippity.patch',
]

# Create RPM build environment

env = Environment()

env.Append(
    ENV = {'HOME': os.environ['HOME']},
    TARFLAGS = '-z')

# Build tar file of sources

tarfile = env.Tar(
    'foo.tar.gz',
    sources)

# Build RPM from tarfile with included spec

rpm_defines = {
    '_topdir': Dir('#build').abspath,
}

rpm = env.Command(
    'rpm_dummy',
    tarfile,
    'rpmbuild %s -tb %s' % (
        string.join(['--define "%s %s" ' % (i[0], i[1])
                     for i in rpm_defines.items()], ' '),
        tarfile[0]))

# Take care of creating and removing various temporary directories
# required by RPM.

env.AddPreAction(rpm, Delete('#build'))

env.AddPreAction(rpm, [Mkdir('#build/SPECS'),
                       Mkdir('#build/BUILD'),
                       Mkdir('#build/RPMS/%s' % arch)])

env.AddPostAction(rpm, [Delete('#build/BUILD'),
                        Delete('#build/SPECS')])

# Clean up after ourselves

env.Clean(rpm, ['#build/BUILD', '#build/SRPMS', '#build/RPMS'])

I like the use of pre and post actions here to create the directory structure expected by RPM and the cleanup of it afterwards. Running scons -c will delete all the generated RPM files as well as the tar file of sources.

Oh yeah, installing and using ccache is absolutely essential for debugging RPM files.

Tue, 08 May 2007

SCons Builder for Ragel

Here's a SCons builder for the Ragel State Machine Compiler, based on the one for SWIG included in the SCons distribution.

#            tastes like -*- python -*-

"""Tool-specific initialisation for ragel."""

import SCons.Util
import SCons.Tool
import SCons.Action

def generate(env):

    c_file, cxx_file = SCons.Tool.createCFileBuilders(env)
    c_file.suffix['.rl'] = '.c'

    c_file.add_action('.rl', SCons.Action.Action('$RAGELCOM', '$RAGELCOMSTR'))

    env['RAGEL'] = 'ragel'
    env['RAGELFLAGS'] = SCons.Util.CLVar('')

    env['RLCODEGEN'] = 'rlcodegen'
    env['RLCODEGENFLAGS'] = SCons.Util.CLVar('')

    env['RAGELCOM'] = \
        '$RAGEL $RAGELFLAGS $SOURCE | $RLCODEGEN $RLCODEGENFLAGS -o $TARGET'

def exists(env):
    return env.Detect(['ragel', 'rlcodegen'])

Fri, 12 Nov 2004

I wish GNU make could do this

proforma$ scons --debug=explain
scons: rebuilding `src/HP_CPU/libHPCPUProvider.so' because:
            `src/HP_SMBIOS/SMBIOS.os' changed
            `src/Common/Version.os' changed
            `src/EzBMC/libEzBMC.a' changed

Fri, 21 May 2004

Building RPMs with SCons

One of the features of SCons is a guaranteed repeatable build regardless of the environment variables set at the time scons is invoked. As such, only the following environment variables are set in the default construction environment:

PATH=/usr/local/bin:/bin:/usr/bin
PWD=/net/proforma/data/wbem/hp.com-wbem-providers/trunk
SHLVL=1
_=/usr/bin/printenv

base_env = Environment(
    CPPPATH = [Dir('include')],
    CCFLAGS = '-W -Wall -ggdb')

base_env.Append(ENV = {"HOME": os.environ['HOME']})

Tue, 20 Apr 2004

More SCons coolness

People who write shared libraries for x86 often forget to compile everything in the library with -fPIC. On x86 it seems to not matter, but other architectures, ia64 for example, the library will not link.

Scons detects this situation and refuses to build a library containing non-PIC objects. From the manual page:

On some platforms building a shared object requires additional compiler options (e.g. -fPIC for gcc) in addition to those needed to build a normal (static) object, but on some platforms there is no differ- ence between a shared object and a normal (static) one. When there is a difference, SCons will only allow shared objects to be linked into a shared library, and will use a different suffix for shared objects.

Fix the problem, not the symptom! posted at: 16:19 | path: /software/scons | permanent link to this entry

Tue, 30 Mar 2004

SCons rocks like Spock in a box!

I've been using SCons for a project at work lately and have decided that it is not just a good tool but rather an amazingly good tool. For a good description of the shortcomings of make, see my favourite paper by Peter Miller, Recursive Make Considered Harmful. (Interestingly enough, it turns out that SCons uses Peter's Aegis software configuration management system which also contains a make replacement, called cook).

SCons has several extremely compelling advantages over regular and GNU make:

• Automatic calculation of header file dependencies.

  Most projects ignore header file dependencies, preferring to either just ignore them or make a half-assed broken attempt to represent them in make. This can be bad news for developers who have to have enough experience with the project to know when to run make clean after changing a critical header file. Incorrect or out of date hand-coded header file dependencies are slightly worse. You get the illusion of targets being rebuilt after header file changes but still have to know when to rebuild from scratch.

  There are a couple of solutions to this available. IDEs like Visual C++ automatically calculate header dependencies and rebuilt targets appropriately which is great for Windows programmers. In the autoconf world, header file dependencies are usually built around the creation of lots of little poos in your build directory containing the output of gcc -M. These files are included in your Makefile and you're off to the races. I don't think this is an ideal solution, especially if you aren't using gcc.

• Built in autoconf, automake and libtool-like functionality.

  I think the whole m4 based templating idea is a nice hack gone horribly wrong. While autoconf and friends serve the noble function of making software packages more portable across wierdly different architectures, they are quite slow compared to the speed of compilation, and this makes the test/debug/fix cycle for writing autoconf tests frustating and time-consuming.

  Disclaimer: builtin autotools functionality may not be completely equivalent to using the actual tools. I have had a brief play with creating runtime tests in SCons but I'm not sure how well this would scale to a large project with many runtime configuration tests and --with style options.

• The ability to create separate build environments for building different targets.

  This is something really special. Traditional autoconf based Makefiles have one set of environment variables - CFLAGS, CPPFLAGS, LDFLAGS, LIBS are the usual names for these guys. However, it is usually the case that every C file is built with the same flags. Creating multiple sets of flags is rather difficult. In Scons, environments can be copied and tweaked in a couple of lines of code.

• Automatic rebuilding of targets if the build command changes.

  This is actually quite a subtle feature and is something that you might not realise is a problem in make. If I make a change in a makefile, which targets need to be rebuilt? Make cannot determine this and so you either have to do make clean && make, or do a partial rebuild by removing selected objects or executables and rebuilding. SCons remembers the commands used to build a target and if this changes, then the target is rebuilt. If I write some comments, rename variables or otherwise make changes that don't affect the build commands, nothing is rebuilt.

SCons has some other advantages which aren't quite as eye-popping as the above list, but still are worthy features to have.

• It's written in Python which means your variables, tests and any other associated build stuff can be written in a fully-fledged scripting language, instead of make's environment which is pretty much a system for assigning values to variables. Indentation trolls are not welcome. (-:

• Running scons dir where dir is a subdirectory in your project will build all targets under that directory. Neat!

• Unecessary rebuilds are avoided by using checksums to determine whether a file has changed, rather than the timestamp.

• It works with Windows, not that there's anything wrong with that.

I guess to be fair I should add some negatives as well in a pathetic attempt to seem balanced.

• Scons is written in Python, so it requires some knowledge of the language to make full use of it. Fortunately Python is pretty easy to learn and the syntax is very intuitive.

• Scons is a very different environment from regular make. There is somewhat of a learning curve moving from one environment to another.
• I'll probably run into some more limitations as I continue to use the tool.