The video capture class allows you to process video files into individual frames. You can then use different operators/functions such as read, get, and set to retrieve or manipulate the frames.

Video Capture

Cameras introduce radial and tangential distortion and an import pre-processing step in computer vision. To remove the distortion we need to find the intrinsic and extrinsic parameters of the camera. OpenCv provides functions such as CalibrateCamera, getOptimalNewCameraMatrix, undistort, etc.

Camera Calibration

An affine transformation represents a relationship between two images through rotation, translation, and scaling. OpenCV has a few functions such as affineTransform2D. affineTransform3D which create 2x3 matrix transformation matrices between two coordinates systems in the same dimension.

Affine Transform

OpenCV provides many object tracking algorithms such as:
- BOOSTING Tracker
- MIL Tracker
- Kernalized Correlation Filter Tracker
- Discriminative Correlation Filter Tracker
- MedianFlow Tracker
- TLD Tracker
- MOSSE Tracker
- GOTURN Tracker

Object Tracking in OpenCV

An important pre-processing step in computer vision is converting to different color schemes such as gray or HSV. OpenCV makes the seamless using cv2.cvtColor where you can input you image and use flags such as COLOR_BGR2RGB to convert it between color schemes.

Converting image in OpenCV

Given a set of features in an image, a popular CV problem is to find those same features in the other images. OpenCV has a brute-force matcher technique that takes the descriptor in one image and matches it with the others by using a distance calculation (type of distance used can be entered as a parameter but default L2 norm).

Feature Matching

OpenCV implements the Lucas-Kanade optical flow which does a least squares fit on the motion of 9 points (3x3 neighborhood of pixels). The algorithm is represented on little movement between frames since the motion is being represented in 2D. The function is called cv2.calcOpticalFlowPyrLK() and takes in the previous and next frame as well as the points from previous. It then returns the points for the next frame along with a status value telling you whether or not the point has been dropped (not found in the next frame).

Optical Flow in OpenCV

Smoothing and blurring is image processing step that reduces noise and detail in an image. It is important for edge detection and thresholding. OpenCV has few different options such as:
- Average blur
- Guassian blur
- Median filtering
- Bilateral filtering

Smoothing & Blurring in OpenCV

OpenCV is the largest open-source computer vision library that gives access to thousands of state of the art algorithms with detailed documentation. It has classes/functions for face detection, object identification, extraction of 3D models, tracking of movement, stitching frames, etc. Can be used in C++, Python, Java, and MATLAB. 

University of Michigan - Ann Arbor

Computer vision has tons of applications but the methods used have become relatively standardized so it's often useful to use established libraries/tools. Some examples are:
- OpenCV
- Scipy ndimage
- TensorFlow
- PyTorch Vision
- MATLAB Computer Vision Toolbox

Computer Vision Libraries

OpenCV

SciPy ndimage is an n-dimensional image that has a number of classes and functions for image processing and analysis. It can perform classification, feature extraction, registration, image i/o, etc.

SciPy Ndimage

Provides algorithms, functions, and apps for designing and testing computer vision, 3D vision, and video processing systems. Can be used to train deep and machine learning models and can leverage multicore or GPU systems.

Learn Before

Related

Learn After