Camera Calibration
Camera calibration is the estimation of the parameters of a camera, parameters about the camera required to determine an accurate relationship between a 3D point in the real world and its corresponding 2D projection (pixel) in the image captured by that calibrated camera.
The major purpose of camera calibration is to remove the distortions in the image and thereby establish a relation between image pixels and real world dimensions. In order to remove the distortion we need to find the intrinsic parameters in the intrinsic matrix K and the distortion parameters.
Image Distortion
A distortion can be radial or tangential. Calibration helps to undistort an image.
Radial distortion: Radial distortion occurs when the light rays bend more at the edges of the lens than the optical center of the lens. It essentially makes straight lines appear as slightly curved within an image.
x-distorted = x(1 + k1*r² + k2*r⁴ + k3*r⁶)
y-distorted = y(1 + k1*r² + k2*r⁴ + k3*r⁶)
x, y — undistorted pixels that are in image coordinate system.
k1, k2, k3 — radial distortion coefficients of the lens.
Tangential distortion: This form of distortion occurs when the lens of the camera being utilized is not perfectly aligned i.e. parallel with the image plane. This makes the image to be extended a little while longer or tilted, it makes the objects appear farther away or even closer than they actually are.
x-distorted = x + [2 * p1 * x * y + p2 * (r² + 2 * x²)]
y-distorted = y + [p1 * (r² + 2 *y²) + 2 * p2 * x * y]
x, y — undistorted pixels that are in image coordinate system.
p1, p2 — tangential distortion coefficients of the lens.
This distortion can be captured by five numbers called Distortion Coefficients, whose values reflect the amount of radial and tangential distortion in an image.
We need intrinsic and extrinsic parameters of camera to find distortion coefficient parameters (k1, k2, k3, p1, p2). The intrinsic parameters are camera-specific (same parameters for same camera) that are focal length (fx, fy) and optical centers (cx, cy).
Intrinsic parameters depend only on camera characteristics while extrinsic parameters depend on camera position.
The matrix containing this parameters is referred to as the camera matrix.
Mapping of 3D world coordinate to 2D Image coordinate
Calibration maps a 3D point (in the world) with [X, Y, Z] coordinates to a 2D Pixel with [X, Y] coordinates.
With calibrated camera, we can transformation world coordinates to pixel coordinates going through camera coordinates.
- Extrinsic calibration converts World Coordinates to Camera Coordinates. The extrinsic parameters are called R (rotation matrix) and T (translation matrix).
- Intrinsic calibration converts Camera Coordinates to Pixel Coordinates. It requires inner values for the camera such as focal length, optical center. The intrinsic parameter is a matrix we call K.
In the process of camera calibration, we have two formulas to get a point O from the world to the pixel space:
- World to Camera Conversion
The essential matrix is the part of the basic fundamental matrix that is only related to external parameters.
- Camera to Image Conversion
- World to Image Conversion
However the matrices doesn’t match. Due to this world matrix needs to be modified from [X Y Z] to [X Y Z 1]. This “1” is called a homogeneous coordinate.
P = [R|T] K
If we have the 2D coordinates, then using calibration parameters, we can map to 3D and vice versa using the following equation:
In this article, we learnt about camera calibration, different types of distortions introduced in am image, the parameters needed to calibrate camera — distortion coefficients, camera intrinsic parameters as well as the use of calibrated camera to undistort the images as well as to map 3D world coordinates to 2D pixel coordinates.
