Using Ccache with Docker


Ccache is a wonderful program that acts as a front-end for compiling C and C++ files. If ccache detects that the same compilation is being done again, it returns the compilation output from its cache. The speed increases that are obtainable from ccache are significant, and it’s a very useful program to know about if you are compiling a lot of C or C++ code. This article describes how to use ccache and Docker together to get the isolation of containers with the speed of a shared compilation cache.

To combine ccache and Docker we’re going to use a slightly advanced concept out of Docker called a data volume. Data volumes are a method of decoupling data storage from the code that’s accessing it as well as decoupling the lifetime of the container performing the compilation from the lifetime of the cache. This turns out to be an essential property of our integration. There’s not much point in creating a cache of compiler output if it’s thrown away every time the container running the compilation exits!

Let’s start off by creating our data volume in Docker. In the example we have some storage set aside on the host filesystem at /mnt/ccache for storing cached data.

Create the data volume called ccache using the following command:

$ docker create -v /mnt/ccache:/ccache --name ccache debian

Now let’s create a container that “mounts” the data container created above using the --volumes-from command line option:

$ docker run -e CCACHE_DIR=/ccache --volumes-from ccache -it debian

We also pass through the CCACHE_DIR environment variable so that ccache can find our shared cache. The default cache location is $HOME/.ccache which will be discarded when the container exits.

Let’s give ccache a quick spin by building a small C program to see whether it works:

root@827d38c7cc20:/# apt-get update ; apt-get install -y ccache

root@827d38c7cc20:/# cat > foo.c << __EOF__
int main(int argc, char **argv)
{
    return 0;
}
__EOF__

root@827d38c7cc20:/# PATH=/usr/lib/ccache:$PATH gcc -o foo.o -c foo.c

We can use the -s option of ccache to display statistics about the cache:

root@827d38c7cc20:/# ccache -s
cache directory                     /ccache
cache hit (direct)                     0
cache hit (preprocessed)               0
cache miss                             1
files in cache                         2
cache size                             8 Kbytes
max cache size                       1.0 Gbytes

We can see from the output above that ccache did see the compilation and since we have a totally empty cache, a cache miss occured. Running the compile command again would result in a cache hit and a (very small in this case) performance increase. If the cache hit and miss statistics are all zeros then there is probably something wrong with your path or the CCACHE_DIR environment variable is not being set correctly.

If you are doing a lot of compilations or have an especially large project then the default cache size of 1GB may not be large enough. Disk is cheap these days so bumping that value up to a couple of tens of gigabytes isn’t going to hurt anyone. Edit the /mnt/ccache/ccache.conf file on the host to do this.

Related Posts

A Pattern for Unit Testing Completion Blocks in Objective-C

Exposing a HTTP endpoint for Docker

Installing Extra Packages on Boot2docker

Building Debian Packages with Ccache