-
Notifications
You must be signed in to change notification settings - Fork 18.7k
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Add Docker files for bulding Caffe Docker images.
These are provided in two flavours (both with CPU and GPU versions). Firstly, runtime images can be used as direct replacements of the Caffe executable. Development images are also included to allow for building Caffe source locally.
- Loading branch information
Evan Lezar
committed
Feb 12, 2016
1 parent
33f2445
commit 3141521
Showing
12 changed files
with
510 additions
and
0 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,46 @@ | ||
# A makefile to build the docker images for caffe. | ||
|
||
all: docker_files devel runtime | ||
|
||
.PHONY: runtime devel | ||
|
||
runtime: devel cpu_runtime gpu_runtime | ||
devel: cpu_devel gpu_devel | ||
|
||
|
||
cpu_runtime: runtime/cpu/Dockerfile | ||
docker build -t caffe runtime/cpu | ||
|
||
gpu_runtime: runtime/gpu/Dockerfile | ||
docker build -t caffe:gpu runtime/gpu | ||
|
||
gpu_devel: devel/gpu/Dockerfile | ||
docker build -t caffe_devel:gpu devel/gpu | ||
|
||
cpu_devel: devel/cpu/Dockerfile | ||
docker build -t caffe_devel:cpu devel/cpu | ||
|
||
docker_files: devel_files runtime_files | ||
|
||
devel_files: devel/cpu/Dockerfile devel/gpu/Dockerfile | ||
runtime_files: runtime/cpu/Dockerfile runtime/gpu/Dockerfile | ||
|
||
|
||
from_gpu = "FROM nvidia/cuda:7.5-cudnn4-devel" | ||
from_cpu = "FROM ubuntu:14.04" | ||
|
||
define create_docker_file | ||
@echo creating $@ | ||
@echo $(if $(strip $(findstring gpu,$@)),$(from_gpu),$(from_cpu)) > $@ | ||
@echo "MAINTAINER team-sl@zalando.de" >> $@ | ||
@echo "" >> $@ | ||
@cat $^ >> $@ | ||
endef | ||
|
||
|
||
devel/%/Dockerfile: templates/devel.template | ||
$(create_docker_file) | ||
|
||
runtime/%/Dockerfile: templates/devel.template templates/%_build.template templates/runtime.template | ||
$(create_docker_file) | ||
|
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,78 @@ | ||
# Caffe runtime Dockerfiles. | ||
|
||
The `runtime` subfolder contains docker files for generating both CPU and GPU runtimes for Caffe. The images can be built using make, or by running: | ||
|
||
``` | ||
docker build -t caffe runtime/gpu | ||
``` | ||
for example. | ||
|
||
Note that the GPU runtime requires a CUDA 7.5 capable driver to be installed on the system and [nvidia-docker|https://github.com/NVIDIA/nvidia-docker] for running the Docker containers. | ||
|
||
# Running Caffe using the docker image | ||
|
||
In order to test the Caffe image, run: | ||
``` | ||
docker run -ti caffe --version | ||
``` | ||
which should show a message like: | ||
``` | ||
libdc1394 error: Failed to initialize libdc1394 | ||
caffe version 1.0.0-rc3 | ||
``` | ||
|
||
One can also run the caffe tests using: | ||
``` | ||
docker run -ti --entrypoint=/bin/bash caffe -c "cd /opt/caffe/build; make runtest" | ||
``` | ||
|
||
In order to get the most out of the caffe image, some more advanced `docker run` could be used. For example, running: | ||
``` | ||
docker run -ti -v $(pwd):/workspace caffe train --solver=example_solver.prototxt | ||
``` | ||
will train a network defined in the `example_solver.prototxt` file in the current directory (`$(pwd)` is maped to the container volume '/workspace' using the `-v` Docker flag). | ||
|
||
Note that docker runs all commands as root by default, and thus any output files (e.g. snapshots) generated will be owned by the root user. In order to ensure that the current user is used instead, the following command can be used: | ||
``` | ||
docker run -ti -v $(pwd):/workspace -u $(id -u):$(id -g) caffe train --solver=example_solver.prototxt | ||
``` | ||
where the `-u` Docker command line option runs the commands in the container as the specified user, and the shell command `id` is used to determine the user and group ID of the current user. | ||
|
||
|
||
# Caffe development Dockerfile. | ||
|
||
The files contained here allow for the Docker images to be built which contain | ||
the development environment for Caffe. | ||
|
||
In order to use GPU computing with docker, nvidia-docker (https://github.com/NVIDIA/nvidia-docker) is recommended. The Docker image uses the NVIDIA CUDA | ||
image as a starting point to allow for GPU accelleration within Docker. | ||
|
||
# Usage | ||
|
||
First ensure that the docker image is built: | ||
|
||
```make docker_devel``` | ||
|
||
This will create a docker image with the tag ```caffe:devel``` which can be | ||
used with Docker as per usual. | ||
|
||
A utility script is also provided to start a container based on this image. | ||
This container can be used to build and run caffe. | ||
|
||
To use the script run: | ||
|
||
```./start_caffe_docker.sh bash``` | ||
|
||
This should show the following output: | ||
|
||
``` | ||
Using nvidia-docker | ||
[ NVIDIA ] =INFO= Driver version: 352.63 | ||
[ NVIDIA ] =INFO= CUDA image version: 7.5 | ||
elezar@caffe_devel:~/caffe$ | ||
``` | ||
Where this the NVIDIA docker wrapper is used in this case, and ```elezar``` is | ||
the username of the user running docker. | ||
|
||
The caffe source folder is mounted as a volume in the container at ```~/caffe```. |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,45 @@ | ||
FROM ubuntu:14.04 | ||
MAINTAINER team-sl@zalando.de | ||
|
||
RUN apt-get update \ | ||
&& DEBIAN_FRONTEND=noninteractive apt-get install -y \ | ||
libgflags-dev \ | ||
libgoogle-glog-dev \ | ||
libopencv-dev \ | ||
libleveldb-dev \ | ||
libsnappy-dev \ | ||
liblmdb-dev \ | ||
libhdf5-serial-dev \ | ||
libprotobuf-dev \ | ||
protobuf-compiler \ | ||
libatlas-base-dev \ | ||
python-dev \ | ||
python-pip \ | ||
python-numpy \ | ||
python-scipy \ | ||
gfortran | ||
|
||
# Use --no-install-recommends for the boost libs. | ||
RUN apt-get install -y --no-install-recommends \ | ||
libboost-all-dev | ||
|
||
# Install utilities used for the build. | ||
RUN apt-get install -y \ | ||
build-essential \ | ||
cmake \ | ||
cmake-curses-gui \ | ||
git \ | ||
ca-certificates \ | ||
bc \ | ||
wget | ||
|
||
# Install the python requirements. | ||
RUN wget https://github.com/raw/BVLC/caffe/master/python/requirements.txt -O /tmp/requirements.txt && \ | ||
cat /tmp/requirements.txt && \ | ||
cat /tmp/requirements.txt | xargs -n1 pip install | ||
|
||
# Clean up the apt-get cache and remove unused packages. | ||
RUN apt-get clean \ | ||
&& rm -rf /var/lib/apt/lists/* | ||
|
||
CMD bash |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,45 @@ | ||
FROM nvidia/cuda:7.5-cudnn4-devel | ||
MAINTAINER team-sl@zalando.de | ||
|
||
RUN apt-get update \ | ||
&& DEBIAN_FRONTEND=noninteractive apt-get install -y \ | ||
libgflags-dev \ | ||
libgoogle-glog-dev \ | ||
libopencv-dev \ | ||
libleveldb-dev \ | ||
libsnappy-dev \ | ||
liblmdb-dev \ | ||
libhdf5-serial-dev \ | ||
libprotobuf-dev \ | ||
protobuf-compiler \ | ||
libatlas-base-dev \ | ||
python-dev \ | ||
python-pip \ | ||
python-numpy \ | ||
python-scipy \ | ||
gfortran | ||
|
||
# Use --no-install-recommends for the boost libs. | ||
RUN apt-get install -y --no-install-recommends \ | ||
libboost-all-dev | ||
|
||
# Install utilities used for the build. | ||
RUN apt-get install -y \ | ||
build-essential \ | ||
cmake \ | ||
cmake-curses-gui \ | ||
git \ | ||
ca-certificates \ | ||
bc \ | ||
wget | ||
|
||
# Install the python requirements. | ||
RUN wget https://github.com/raw/BVLC/caffe/master/python/requirements.txt -O /tmp/requirements.txt && \ | ||
cat /tmp/requirements.txt && \ | ||
cat /tmp/requirements.txt | xargs -n1 pip install | ||
|
||
# Clean up the apt-get cache and remove unused packages. | ||
RUN apt-get clean \ | ||
&& rm -rf /var/lib/apt/lists/* | ||
|
||
CMD bash |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,74 @@ | ||
FROM ubuntu:14.04 | ||
MAINTAINER team-sl@zalando.de | ||
|
||
RUN apt-get update \ | ||
&& DEBIAN_FRONTEND=noninteractive apt-get install -y \ | ||
libgflags-dev \ | ||
libgoogle-glog-dev \ | ||
libopencv-dev \ | ||
libleveldb-dev \ | ||
libsnappy-dev \ | ||
liblmdb-dev \ | ||
libhdf5-serial-dev \ | ||
libprotobuf-dev \ | ||
protobuf-compiler \ | ||
libatlas-base-dev \ | ||
python-dev \ | ||
python-pip \ | ||
python-numpy \ | ||
python-scipy \ | ||
gfortran | ||
|
||
# Use --no-install-recommends for the boost libs. | ||
RUN apt-get install -y --no-install-recommends \ | ||
libboost-all-dev | ||
|
||
# Install utilities used for the build. | ||
RUN apt-get install -y \ | ||
build-essential \ | ||
cmake \ | ||
cmake-curses-gui \ | ||
git \ | ||
ca-certificates \ | ||
bc \ | ||
wget | ||
|
||
# Install the python requirements. | ||
RUN wget https://github.com/raw/BVLC/caffe/master/python/requirements.txt -O /tmp/requirements.txt && \ | ||
cat /tmp/requirements.txt && \ | ||
cat /tmp/requirements.txt | xargs -n1 pip install | ||
|
||
# Clean up the apt-get cache and remove unused packages. | ||
RUN apt-get clean \ | ||
&& rm -rf /var/lib/apt/lists/* | ||
|
||
CMD bash | ||
# Clone Caffe repo and move into it | ||
RUN cd /opt && git clone https://github.com/BVLC/caffe.git && cd caffe && \ | ||
# Get the python prerequsites for this repository | ||
cat python/requirements.txt | xargs -n1 pip install && \ | ||
# Create a build directory | ||
mkdir build && cd build && \ | ||
# Run cmake | ||
cmake -DCPU_ONLY=1 .. && \ | ||
# # Make | ||
make -j"$(nproc)" all && \ | ||
make -j"$(nproc)" pycaffe | ||
# Also build some other utility targets. | ||
RUN cd /opt/caffe/build && make -j"$(nproc)" test.testbin | ||
|
||
|
||
# Add to Python path | ||
ENV CAFFE_ROOT=/opt/caffe | ||
|
||
ENV PYTHONPATH=$CAFFE_ROOT/python:$PYTHONPATH | ||
ENV PATH=$CAFFE_ROOT/build/tools:$PATH | ||
|
||
# Set /workspace as working directory | ||
WORKDIR /workspace | ||
|
||
# Set the entrypoint | ||
ENTRYPOINT ["caffe"] | ||
|
||
# And the default command line options. | ||
CMD ["--help"] |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,74 @@ | ||
FROM nvidia/cuda:7.5-cudnn4-devel | ||
MAINTAINER team-sl@zalando.de | ||
|
||
RUN apt-get update \ | ||
&& DEBIAN_FRONTEND=noninteractive apt-get install -y \ | ||
libgflags-dev \ | ||
libgoogle-glog-dev \ | ||
libopencv-dev \ | ||
libleveldb-dev \ | ||
libsnappy-dev \ | ||
liblmdb-dev \ | ||
libhdf5-serial-dev \ | ||
libprotobuf-dev \ | ||
protobuf-compiler \ | ||
libatlas-base-dev \ | ||
python-dev \ | ||
python-pip \ | ||
python-numpy \ | ||
python-scipy \ | ||
gfortran | ||
|
||
# Use --no-install-recommends for the boost libs. | ||
RUN apt-get install -y --no-install-recommends \ | ||
libboost-all-dev | ||
|
||
# Install utilities used for the build. | ||
RUN apt-get install -y \ | ||
build-essential \ | ||
cmake \ | ||
cmake-curses-gui \ | ||
git \ | ||
ca-certificates \ | ||
bc \ | ||
wget | ||
|
||
# Install the python requirements. | ||
RUN wget https://github.com/raw/BVLC/caffe/master/python/requirements.txt -O /tmp/requirements.txt && \ | ||
cat /tmp/requirements.txt && \ | ||
cat /tmp/requirements.txt | xargs -n1 pip install | ||
|
||
# Clean up the apt-get cache and remove unused packages. | ||
RUN apt-get clean \ | ||
&& rm -rf /var/lib/apt/lists/* | ||
|
||
CMD bash | ||
# Clone Caffe repo and move into it | ||
RUN cd /opt && git clone https://github.com/BVLC/caffe.git && cd caffe && \ | ||
# Get the python prerequsites for this repository | ||
cat python/requirements.txt | xargs -n1 pip install && \ | ||
# Create a build directory | ||
mkdir build && cd build && \ | ||
# Run cmake | ||
cmake -DUSE_CUDNN=1 .. && \ | ||
# # Make | ||
make -j"$(nproc)" all && \ | ||
make -j"$(nproc)" pycaffe | ||
# Also build some other utility targets. | ||
RUN cd /opt/caffe/build && make -j"$(nproc)" test.testbin | ||
|
||
|
||
# Add to Python path | ||
ENV CAFFE_ROOT=/opt/caffe | ||
|
||
ENV PYTHONPATH=$CAFFE_ROOT/python:$PYTHONPATH | ||
ENV PATH=$CAFFE_ROOT/build/tools:$PATH | ||
|
||
# Set /workspace as working directory | ||
WORKDIR /workspace | ||
|
||
# Set the entrypoint | ||
ENTRYPOINT ["caffe"] | ||
|
||
# And the default command line options. | ||
CMD ["--help"] |
Oops, something went wrong.