Simplifies image manipulation for a broad range of languages and platforms
Ten years on from its first 1.0 release, OpenCV (Short for Open Computer Vision) is a C++ library that lets you create software with real-time computer vision. Originally developed in Russia for Intel, it’s fully open source and includes machine learning. It’s BSD licensed so can be used freely.
This is a very large library, with over 200 MB in various DLLs though you can also statically link to it if you prefer. It has several hundred algorithms in C++ with C, Python, Java and MATLAB interfaces and runs on Windows, Linux, Android, iOS and Mac OS. The earlier version 1.0 was C based though since version 2.0 it has been C++ based. The most recent version is 3.2 which was released in December 2016.
What’s in OpenCV?
OpenCV can be summed up as having virtually everything you need to capture images or video (streamed or loaded from disk), stored in memory and then manipulated.
Manipulation hardly begins to describe what you can do with the image. If you want to change individual pixels, calculate histograms, do face recognition, motion tracking, augmented reality, gesture recognition and more then OpenCV is a good place. Wikipedia provides a more detailed list of what the OpenCV library can do.
The list of OpenCV users includes people from major companies (IBM, Microsoft, Intel, SONY, Siemens, and Google), and research centers (such as Stanford, MIT, CMU, Cambridge, and INRIA).
There are binaries for Windows/Mac OSX but on Linux you have to download the tar file, extract then run make on it and then sudo make install. There are plenty of places showing how to do this and the OpenCV tutorials are particularly good. It’s a long enough build for a coffee break. On my PC (Ubuntu 16.04 LTS) it installed without a hitch.
You should also install Intel IPP which is available as a free download. This is integrated with OpenCV and speeds up many operations. There’s a function call that you should make
cvGetModuleInfo( 0, &libraries, &modules );
This provides a text description of the installed libraries and modules. If you are writing OpenCv applications then you can use this to verify that the target PCs has Intel IPP on it and prompt for download and install if not.
Working with OpenCV
It’s a huge library but you need to understand a few fundamentals before you use it.
The Mat class is the main data structure (cvMAT type in OpenCV 1.0) that manages the image in memory and includes both the header and the image data. It allocate and deallocate memory automatically unless you are sharing the data buffer which needs a bit more care. This is a powerful class and can use different data types such as vector<>, Mat<> or scalar values.
Though the library consists of many methods the HighGui module lets you create and use windows for displaying and remembering images (no OS paint callbacks needed). You can even add trackbars, handle simple mouse events, read/write images to/from disk or memory and read video from camera and write it to a file.
Use of GPUs
OpenCL integrates quite nicely with CUDA, the NVidia GPU technology. There appears to be an OCL module for OpenCL, the alternative GPU technology but as this StackOverflow answer suggests the module is not needed in OpenCV 3.*.
OpenCV comes with its own comprehensive machine learning library with Bayes Classifier, K-Nearest Neighbors, SVM, Decision Trees, Boosting, Random trees, neural networks and logistical regression. There’s also a Deep Neural Network module dnn (listed in the extra modules)though it’s designed only for forward pass computations (i. e. network testing). Network training is not supported.
If you are interested in creating computer vision applications then OpenCV should absolutely be your first port of call. Although it’s programmed in C++, the developers have mainly avoided using templates instead using polymorphism and runtime dispatching instead. Some operations such as pixel access make use of small templates classes and functions.
Although C++ is the primary language to develop OpenCV applications, Python is gaining in popularity and the OpenCV 3.2 documentation includes a number of Python specific tutorials. Given the acceleration of numpy (needed for OpenCV) in Intel Python, it would seem an ideal