Skip to content

Commit

Permalink
Add Docker files for bulding Caffe Docker images.
Browse files Browse the repository at this point in the history
These are provided in two flavours (both with CPU and GPU versions).
Firstly, runtime images can be used as direct replacements of the Caffe executable.
Development images are also included to allow for building Caffe source locally.
  • Loading branch information
Evan Lezar committed Feb 12, 2016
1 parent 33f2445 commit 3141521
Show file tree
Hide file tree
Showing 12 changed files with 510 additions and 0 deletions.
46 changes: 46 additions & 0 deletions docker/Makefile
Original file line number Diff line number Diff line change
@@ -0,0 +1,46 @@
# A makefile to build the docker images for caffe.

all: docker_files devel runtime

.PHONY: runtime devel

runtime: devel cpu_runtime gpu_runtime
devel: cpu_devel gpu_devel


cpu_runtime: runtime/cpu/Dockerfile
docker build -t caffe runtime/cpu

gpu_runtime: runtime/gpu/Dockerfile
docker build -t caffe:gpu runtime/gpu

gpu_devel: devel/gpu/Dockerfile
docker build -t caffe_devel:gpu devel/gpu

cpu_devel: devel/cpu/Dockerfile
docker build -t caffe_devel:cpu devel/cpu

docker_files: devel_files runtime_files

devel_files: devel/cpu/Dockerfile devel/gpu/Dockerfile
runtime_files: runtime/cpu/Dockerfile runtime/gpu/Dockerfile


from_gpu = "FROM nvidia/cuda:7.5-cudnn4-devel"
from_cpu = "FROM ubuntu:14.04"

define create_docker_file
@echo creating $@
@echo $(if $(strip $(findstring gpu,$@)),$(from_gpu),$(from_cpu)) > $@
@echo "MAINTAINER team-sl@zalando.de" >> $@
@echo "" >> $@
@cat $^ >> $@
endef


devel/%/Dockerfile: templates/devel.template
$(create_docker_file)

runtime/%/Dockerfile: templates/devel.template templates/%_build.template templates/runtime.template
$(create_docker_file)

78 changes: 78 additions & 0 deletions docker/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,78 @@
# Caffe runtime Dockerfiles.

The `runtime` subfolder contains docker files for generating both CPU and GPU runtimes for Caffe. The images can be built using make, or by running:

```
docker build -t caffe runtime/gpu
```
for example.

Note that the GPU runtime requires a CUDA 7.5 capable driver to be installed on the system and [nvidia-docker|https://github.com/NVIDIA/nvidia-docker] for running the Docker containers.

# Running Caffe using the docker image

In order to test the Caffe image, run:
```
docker run -ti caffe --version
```
which should show a message like:
```
libdc1394 error: Failed to initialize libdc1394
caffe version 1.0.0-rc3
```

One can also run the caffe tests using:
```
docker run -ti --entrypoint=/bin/bash caffe -c "cd /opt/caffe/build; make runtest"
```

In order to get the most out of the caffe image, some more advanced `docker run` could be used. For example, running:
```
docker run -ti -v $(pwd):/workspace caffe train --solver=example_solver.prototxt
```
will train a network defined in the `example_solver.prototxt` file in the current directory (`$(pwd)` is maped to the container volume '/workspace' using the `-v` Docker flag).

Note that docker runs all commands as root by default, and thus any output files (e.g. snapshots) generated will be owned by the root user. In order to ensure that the current user is used instead, the following command can be used:
```
docker run -ti -v $(pwd):/workspace -u $(id -u):$(id -g) caffe train --solver=example_solver.prototxt
```
where the `-u` Docker command line option runs the commands in the container as the specified user, and the shell command `id` is used to determine the user and group ID of the current user.


# Caffe development Dockerfile.

The files contained here allow for the Docker images to be built which contain
the development environment for Caffe.

In order to use GPU computing with docker, nvidia-docker (https://github.com/NVIDIA/nvidia-docker) is recommended. The Docker image uses the NVIDIA CUDA
image as a starting point to allow for GPU accelleration within Docker.

# Usage

First ensure that the docker image is built:

```make docker_devel```

This will create a docker image with the tag ```caffe:devel``` which can be
used with Docker as per usual.

A utility script is also provided to start a container based on this image.
This container can be used to build and run caffe.

To use the script run:

```./start_caffe_docker.sh bash```

This should show the following output:

```
Using nvidia-docker
[ NVIDIA ] =INFO= Driver version: 352.63
[ NVIDIA ] =INFO= CUDA image version: 7.5
elezar@caffe_devel:~/caffe$
```
Where this the NVIDIA docker wrapper is used in this case, and ```elezar``` is
the username of the user running docker.

The caffe source folder is mounted as a volume in the container at ```~/caffe```.
45 changes: 45 additions & 0 deletions docker/devel/cpu/Dockerfile
Original file line number Diff line number Diff line change
@@ -0,0 +1,45 @@
FROM ubuntu:14.04
MAINTAINER team-sl@zalando.de

RUN apt-get update \
&& DEBIAN_FRONTEND=noninteractive apt-get install -y \
libgflags-dev \
libgoogle-glog-dev \
libopencv-dev \
libleveldb-dev \
libsnappy-dev \
liblmdb-dev \
libhdf5-serial-dev \
libprotobuf-dev \
protobuf-compiler \
libatlas-base-dev \
python-dev \
python-pip \
python-numpy \
python-scipy \
gfortran

# Use --no-install-recommends for the boost libs.
RUN apt-get install -y --no-install-recommends \
libboost-all-dev

# Install utilities used for the build.
RUN apt-get install -y \
build-essential \
cmake \
cmake-curses-gui \
git \
ca-certificates \
bc \
wget

# Install the python requirements.
RUN wget https://github.com/raw/BVLC/caffe/master/python/requirements.txt -O /tmp/requirements.txt && \
cat /tmp/requirements.txt && \
cat /tmp/requirements.txt | xargs -n1 pip install

# Clean up the apt-get cache and remove unused packages.
RUN apt-get clean \
&& rm -rf /var/lib/apt/lists/*

CMD bash
45 changes: 45 additions & 0 deletions docker/devel/gpu/Dockerfile
Original file line number Diff line number Diff line change
@@ -0,0 +1,45 @@
FROM nvidia/cuda:7.5-cudnn4-devel
MAINTAINER team-sl@zalando.de

RUN apt-get update \
&& DEBIAN_FRONTEND=noninteractive apt-get install -y \
libgflags-dev \
libgoogle-glog-dev \
libopencv-dev \
libleveldb-dev \
libsnappy-dev \
liblmdb-dev \
libhdf5-serial-dev \
libprotobuf-dev \
protobuf-compiler \
libatlas-base-dev \
python-dev \
python-pip \
python-numpy \
python-scipy \
gfortran

# Use --no-install-recommends for the boost libs.
RUN apt-get install -y --no-install-recommends \
libboost-all-dev

# Install utilities used for the build.
RUN apt-get install -y \
build-essential \
cmake \
cmake-curses-gui \
git \
ca-certificates \
bc \
wget

# Install the python requirements.
RUN wget https://github.com/raw/BVLC/caffe/master/python/requirements.txt -O /tmp/requirements.txt && \
cat /tmp/requirements.txt && \
cat /tmp/requirements.txt | xargs -n1 pip install

# Clean up the apt-get cache and remove unused packages.
RUN apt-get clean \
&& rm -rf /var/lib/apt/lists/*

CMD bash
74 changes: 74 additions & 0 deletions docker/runtime/cpu/Dockerfile
Original file line number Diff line number Diff line change
@@ -0,0 +1,74 @@
FROM ubuntu:14.04
MAINTAINER team-sl@zalando.de

RUN apt-get update \
&& DEBIAN_FRONTEND=noninteractive apt-get install -y \
libgflags-dev \
libgoogle-glog-dev \
libopencv-dev \
libleveldb-dev \
libsnappy-dev \
liblmdb-dev \
libhdf5-serial-dev \
libprotobuf-dev \
protobuf-compiler \
libatlas-base-dev \
python-dev \
python-pip \
python-numpy \
python-scipy \
gfortran

# Use --no-install-recommends for the boost libs.
RUN apt-get install -y --no-install-recommends \
libboost-all-dev

# Install utilities used for the build.
RUN apt-get install -y \
build-essential \
cmake \
cmake-curses-gui \
git \
ca-certificates \
bc \
wget

# Install the python requirements.
RUN wget https://github.com/raw/BVLC/caffe/master/python/requirements.txt -O /tmp/requirements.txt && \
cat /tmp/requirements.txt && \
cat /tmp/requirements.txt | xargs -n1 pip install

# Clean up the apt-get cache and remove unused packages.
RUN apt-get clean \
&& rm -rf /var/lib/apt/lists/*

CMD bash
# Clone Caffe repo and move into it
RUN cd /opt && git clone https://github.com/BVLC/caffe.git && cd caffe && \
# Get the python prerequsites for this repository
cat python/requirements.txt | xargs -n1 pip install && \
# Create a build directory
mkdir build && cd build && \
# Run cmake
cmake -DCPU_ONLY=1 .. && \
# # Make
make -j"$(nproc)" all && \
make -j"$(nproc)" pycaffe
# Also build some other utility targets.
RUN cd /opt/caffe/build && make -j"$(nproc)" test.testbin


# Add to Python path
ENV CAFFE_ROOT=/opt/caffe

ENV PYTHONPATH=$CAFFE_ROOT/python:$PYTHONPATH
ENV PATH=$CAFFE_ROOT/build/tools:$PATH

# Set /workspace as working directory
WORKDIR /workspace

# Set the entrypoint
ENTRYPOINT ["caffe"]

# And the default command line options.
CMD ["--help"]
74 changes: 74 additions & 0 deletions docker/runtime/gpu/Dockerfile
Original file line number Diff line number Diff line change
@@ -0,0 +1,74 @@
FROM nvidia/cuda:7.5-cudnn4-devel
MAINTAINER team-sl@zalando.de

RUN apt-get update \
&& DEBIAN_FRONTEND=noninteractive apt-get install -y \
libgflags-dev \
libgoogle-glog-dev \
libopencv-dev \
libleveldb-dev \
libsnappy-dev \
liblmdb-dev \
libhdf5-serial-dev \
libprotobuf-dev \
protobuf-compiler \
libatlas-base-dev \
python-dev \
python-pip \
python-numpy \
python-scipy \
gfortran

# Use --no-install-recommends for the boost libs.
RUN apt-get install -y --no-install-recommends \
libboost-all-dev

# Install utilities used for the build.
RUN apt-get install -y \
build-essential \
cmake \
cmake-curses-gui \
git \
ca-certificates \
bc \
wget

# Install the python requirements.
RUN wget https://github.com/raw/BVLC/caffe/master/python/requirements.txt -O /tmp/requirements.txt && \
cat /tmp/requirements.txt && \
cat /tmp/requirements.txt | xargs -n1 pip install

# Clean up the apt-get cache and remove unused packages.
RUN apt-get clean \
&& rm -rf /var/lib/apt/lists/*

CMD bash
# Clone Caffe repo and move into it
RUN cd /opt && git clone https://github.com/BVLC/caffe.git && cd caffe && \
# Get the python prerequsites for this repository
cat python/requirements.txt | xargs -n1 pip install && \
# Create a build directory
mkdir build && cd build && \
# Run cmake
cmake -DUSE_CUDNN=1 .. && \
# # Make
make -j"$(nproc)" all && \
make -j"$(nproc)" pycaffe
# Also build some other utility targets.
RUN cd /opt/caffe/build && make -j"$(nproc)" test.testbin


# Add to Python path
ENV CAFFE_ROOT=/opt/caffe

ENV PYTHONPATH=$CAFFE_ROOT/python:$PYTHONPATH
ENV PATH=$CAFFE_ROOT/build/tools:$PATH

# Set /workspace as working directory
WORKDIR /workspace

# Set the entrypoint
ENTRYPOINT ["caffe"]

# And the default command line options.
CMD ["--help"]
Loading

0 comments on commit 3141521

Please sign in to comment.