GNU Make – Install and maintain multiple tools and versions

The idea is to use the power of GNU Make to simplify installing and maintaining multiple tools with multiple versions.

The point on keeping multiple versions is to be able to:

1 – Reproduce results from papers;

2 – Be able to reproduce your own results from the past.

The way I organize it is by using a root directory named “tools” with a subdirectory per tool. Each tool have a subdirectory per version. The tree looks like:

/tools
├── diamond
│   ├── 0.9.25
│   ├── 0.9.26
│   ├── 0.9.27
│   └── 0.9.28
└── picard
    ├── 2.21.1
    └── 2.21.2
    └── 2.21.3

In the /tools I have a Makefile that contains:

SHELL=/bin/bash
.ONESHELL:

.PHONY: all help

all: diamond/0.9.25/diamond \
    diamond/0.9.26/diamond \
    diamond/0.9.27/diamond \
    diamond/0.9.28/diamond \
    diamond/0.9.29/diamond \
    picard/2.21.1/picard.jar \
    picard/2.21.2/picard.jar \
    picard/2.21.3/picard.jar

## diamond.................. diamond/%/diamond
diamond/%/diamond:
    @mkdir -p diamond/$* && \
    cd diamond/$* && \
    wget -c https://github.com/bbuchfink/diamond/releases/download/v$*/diamond-linux64.tar.gz -O diamond_$*.tar.gz && \
    tar xf diamond_$*.tar.gz && \
    rm diamond_$*.tar.gz

## picard................... picard/%/picard.jar
picard/%/picard.jar:
    @mkdir -p picard/$* && \
    cd picard/$* && \
    wget -c https://github.com/broadinstitute/picard/releases/download/$*/picard.jar

help: Makefile
    @sed -n "s/^## //p" $<

If you type “make help“, the output looks like:

make help
diamond.................. diamond/%/diamond
picard................... picard/%/picard.jar

If you type “make” alone, all versions will be installed. To speed it up, one can use “make -j 4” to run 4 installations in parallel.

If you need only a specific version (i.e. picard v. 2.21.2) you can use:

make picard/2.21.2/picard.jar

This strategy along with Environment Modules has been proven useful – at least for me 🙂

More on Environment Modules in a future post.

References:

GNU Make – https://www.gnu.org/software/make/

Environment Modules – http://modules.sourceforge.net/

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s