Getting started with caffe
Caffe is a library written in C++, to facilitate the experimentation with and use of Convolutional Neural Networks (CNN). Caffe has been developed by Berkeley Vision and Learning Center (BVLC).
Caffe is actually an abbreviation referring to "Convolutional Architectures for Fast Feature Extraction". This acronym encapsulates an important scope of the library. Caffe in the form of a library offers a general programming framework/architecture which can be used to perform efficient training and testing of CNNs. "Efficiency" is a major hallmark of caffe, and stands as a major design objective of Caffe.
Caffe is an open-source library released under BSD 2 Clause license.
Caffe is maintained on GitHub
Caffe can be used to :
- Efficiently train and test multiple CNN architectures, specifically any architecture that can be represented as a directed acyclic graph (DAG).
- Utilize multiple GPUs (upto 4) for training and testing. It is recommended that all the GPUs should be of the same type. Otherwise, performance is limited by the limits of the slowest GPU in the system. For example, in case of TitanX and GTX 980, the performance will be limited by the latter. Mixing multiple architectures is not supported, e.g. Kepler and Fermi 3.
Caffe has been written following efficient Object Oriented Programming (OOP) principles.
A good starting point to begin an introduction to caffe is to get a bird's eye view of how caffe works through its fundamental objects.
Enable multithreading with Caffe
Caffe can run on multiple cores. One way is to enable multithreading with Caffe to use OpenBLAS instead of the default ATLAS. To do so, you can follow these three steps:
sudo apt-get install -y libopenblas-dev
- Before compiling Caffe, edit
BLAS := atlasby
BLAS := open
- After compiling Caffe, running
export OPENBLAS_NUM_THREADS=4will cause Caffe to use 4 cores.
Installation and setup
Below are detailed instructions to install Caffe, pycaffe as well as its dependencies, on Ubuntu 14.04 x64 or 14.10 x64.
Execute the following script, e.g. "bash compile_caffe_ubuntu_14.sh" (~30 to 60 minutes on a new Ubuntu).
At the end, you need to run "source ~/.bash_profile" manually or start a new shell to be able to do 'python import caffe'.
Regularization loss (weight decay) in Caffe
In the solver file, we can set a global regularization loss using the
In many cases we want different weight decay rates for different layers. This can be done by setting the
decay_mult option for each layer in the network definition file, where
decay_mult is the multiplier on the global weight decay rate, so the actual weight decay rate applied for one layer is
For example, the following defines a convolutional layer with NO weight decay regardless of the options in the solver file.
See this thread for more information.