Opencv C++ tutorial : Smoothing, blur, noise reduction / canceling

Smooth or blur, gaussian blur, and noise-canceling, This tutorial will learn OpenCV blur, GaussianBlur, median blur functions in C++. Additionally, the advanced technique for noise reduction fastNlMeansDenoising family will be introduced with a code example for each method.  


You can use blurring of the image to hide identity or reduce the noise of the image.  Blur can be a very useful operation and it is a very common operation as well. For example, the anonymization of pedestrians, face or is one possible target for blue operation. The blur is the most common task to perform over the image to reduce noise. The noise reduction is more task for Gaussian blur than for simple blur operation. The various blur operations are very common for image processing on mobile devices. 
The more important is the robustness issues of the data in pre-processing for machine learning. Sometimes, by blurring the images of the dataset can have a positive effect on the robustness of the achieved detector. This is for longer discussion because the chosen algorithm maters at the first place. The dataset blurring can bring surprisingly effect on the result and needs to be evaluated against the same ML algorithm.

To summarize this, Blur is used for example in the following situations. 

• Anonymization 
• Noise filtering 
• Image preparation for machine learning  processing 
• Image effects and processing 

Some basic mathematical for simple blur in OpenCV

Imagine just grayscale images, where the intensity is from 0 to 255. Our image has for simplicity just one row and 10 columns. Snake pixel. 

[0, 0, 0, 0, 0, 255, 255, 255, 255, 255]

There is a sharp transfer between the value 0 to 255. How to achieve this transfer much smoother that step from 0 to 255. The mean value of all 10 elements is first + second + another + .. + last divided by the number of all (10). We do not want to set all to the mean value 127 rounded down.

[127, 127, 127, 127, 127, 127, 127, 127, 127, 127]

What if we use the mean value of some selection. That can work. The mean value of three neighborhood elements that set the element in the middle.
[(0, (0, 0), (0), 0, 255), 255, 255, 255, 255]

This is mean of 0 + 0 +255 in blue bracket divided by 3. The fifth element is now 85. As our defined filter that calculates mean ower the image move one position to the right the sixth element is calculated as 0 + 255 + 255 divided by 3, This is 510 /3 =170. The next of our row will be the same as three times 255 divided three times is 255. Wait, what is our goal? Smooth the transition from 0 to 255. Great, goal is achieved and the edge is 0 85 170 255. This is a much smoother transition.

[(0, 0, 0),( 0, 85, 170), 255, 255, 255, 255]
This can be applied on images pixels as well, just more rows
[0, 0, 0,0, 85, 170, 255, 255, 255, 255;
0, 0, 0,0, 85, 170, 255, 255, 255, 255;
0, 0, 0,0, 85, 170, 255, 255, 255, 255]

This principle is called a normalized box filter. The most basic principle. The concrete pixel is the mean of the neighborhood. The window over is mean calculated is often called the kernel. Lets smooth the same row by mean of 5 neighborhood elements.

[(0, 0, 0, 0, 0), 255, 255, 255, 255, 255]
0
[0, (0, 0, 0, 0, 255), 255, 255, 255, 255]
255/5 = 51
[0, 0, (0, 0, 0, 255, 255), 255, 255, 255]
(2*255)/5 =102
(3*255)/5 =153
(4*255)/5 =204
(5*255)/5 =255

The filter of 3 neighborhood elements produces edge as 0 85 170 255. The filter over 5 neighborhood produce transfer edge 0 0 51 102 153 204 255 255 Much smoother transfer, right? This product image without the sharp edges. Let's say with fewer details as well. The great feature is that image is less noisy as well.The small detail, This simple math is performed in 2D blocks. 

Opencv blur

The OpenCV blur function has simple parameters as blur(input image, output image, size of the kernel). The size of the kernel is of type Size(rows,cols).
blur(loadedPictureHere, processedPicture, Size(5 ,5));

Gaussian blur 

The Gaussian blur is the same mechanism as a blur, with one exception. The kernel represents the Gaussian distribution. Each pixel of the array over the calculated one is weighted by this distribution to produce output.

Opencv GaussianBlur

The OpenCV gaussian blur function has simple parameters as GaussianBlur(input image, output image, size of the kernel, the standard deviation in x, the standard deviation in y). The size of the kernel is of type Size(rows,cols). The standard deviation is from the statistics of the neighborhood pixel.
GaussianBlur(loadedPictureHere, processedPicture, Size(5 ,5),0,0);

Median blur 

It is doing exactly the operation represented by a median. It takes array over some value. Sort the array from the lower to higher element. Replace the value of the element which is in the middle of the sorted array. 

OpenCV median blur

The OpenCV median blur function has simple parameters as MedianBlur(input image, output image, size of the kernel). The size of the kernel is of type int.
medianBlur(loadedPictureHere, processedPicture, 5);

Code of blur, Gaussian blur,  and median blur

// Prepare new mat container 

    Mat loadedPictureHere; 

    //imread will read the image from file

    loadedPictureHere = imread("C:/Users/Vlada/Desktop/DNN/x64/Release/noise.png");

    Mat processedPicture;

    loadedPictureHere.copyTo(processedPicture);

    // blur is contro by size of the block Size(x,y) of moving windows

    blur(loadedPictureHere, processedPicture, Size(5 ,5)); <<<<< Changing kernel

    // prepare bigger Mat to display image and process image side by side (this Mat is bigger)

    Mat output = Mat((loadedPictureHere.cols * 3), loadedPictureHere.rows *2 ,

  loadedPictureHere.type());

    // copy small images process and not processed into the bigger one. 

    loadedPictureHere.copyTo(output(Rect(0, 0,  loadedPictureHere.cols,

  loadedPictureHere.rows)));

    processedPicture.copyTo(output(Rect(loadedPictureHere.cols, 0,

  loadedPictureHere.cols, loadedPictureHere.rows)));

    namedWindow("Step 1", WINDOW_AUTOSIZE);// Create a window for display.

    imshow("Step 1", output);   // Display loaded image 

    waitKey(25);

-----------------------------------------------------------------------------------

blur(loadedPictureHere, processedPicture, Size(5 ,5));

GaussianBlur(loadedPictureHere, processedPicture, Size(5 ,5),0,0);

medianBlur(loadedPictureHere, processedPicture, 5);

blur(loadedPictureHere, processedPicture, Size(10,10)); 

GaussianBlur(loadedPictureHere, processedPicture, Size(11 ,11),0,0);

medianBlur(loadedPictureHere, processedPicture, 11);

blur(loadedPictureHere, processedPicture, Size(15,15));

GaussianBlur(loadedPictureHere, processedPicture, Size(15,15),0,0);
medianBlur(loadedPictureHere, processedPicture, 15);

Opencv advanced noise reduction fastNlMeansDenoising

This method fastNlMeansDenoising is based on an article Here. This method performs a search for average most similar pixels for one that should be denoise. These pixels can be far from the one that should be replaced. "The most similar pixels to given have no reason to be close at all". The search for such a pixel is not simple calculation over the neighborhood of such a pixel as in case of a blur, gaussian blur or median blur. 

fastNlMeansDenoising opencv denoising 

This function is to denoise the single grayscale image. The first parameter is input image and the second is processed image. 

fastNlMeansDenoising(loadedPictureHere, processedPicture,30, 7, 21);
1 parameter input image
2 output image
3 The recommended value is 7, Size of template path to compute weights
4 The recommended value is 21, Size of windows used to compute weight average, This influence performance
5 smaller value preserves details but also preserves some noise

fastNlMeansDenoisingColored opencv denoising 

This function should be used for colored images

fastNlMeansDenoisingColored(loadedPictureHere, processedPicture,30, 7, 3,10);
1 parameter input image

2 output image
3 The recommended value is 7, Size of template path to compute weights
4 The recommended value is 21, Size of windows used to compute weight average, This influence
performance
5 smaller value preserves details but also preserves some noise for gray
6 smaller value preserves details but also preserves some noise for colors. The 10 is the recommended value to remove color noise. 

fastNlMeansDenoisingColoredMulti OpenCV denoising 

This will perform the denoise for one image but find the best pixels value in the buffer of images. It is like the iPhone takes 3 images and combine this sequence into one perfect picture. 

vector buffer(5);

Mat image;

buffer[0] = image;

buffer[1] = image;

buffer[2] = image;

buffer[3] = image;

buffer[4] = image;

Mat output;

fastNlMeansDenoisingColoredMulti(buffer, output, 5, imgs_count, 12, 48);

1 parameter input image buffer
2 output image
3 index of the image in the buffer to be denoised
4 Temporal windows size is number of surrounding images. imgs_count
XXX This two can be skipped
5 smaller value preserves details but also preserves some noise for gray
6 smaller value preserves details but also preserves some noise for colors. The 10 is the recommended value to remove color noise.
XXX
7 The recommended value is 7, Size of template path to compute weights
8 The recommended value is 21, Size of windows used to compute weight average, This influence
performance

Tutorial code for basic blur and fastNlMeansDenoising

#include <iostream>

#include <opencv2/opencv.hpp>

#include <opencv2/highgui.hpp>

#include <opencv2/video.hpp>

#include <opencv2/imgproc.hpp>

#include <opencv2/core.hpp>

#include <opencv2/imgcodecs.hpp>

#include <opencv2/photo.hpp>

using namespace cv;

using namespace std;

int main()

{

    // Prepare new mat container 

    Mat loadedPictureHere; 

    //imread will read the image from file

    loadedPictureHere = imread("C:/Users/Vlada/Desktop/DNN/x64/Release/noise.png");

    Mat processedPicture;

    loadedPictureHere.copyTo(processedPicture);

    // You can use one of these 

    //blur(loadedPictureHere, processedPicture, Size(10,10)); 

    //GaussianBlur(loadedPictureHere, processedPicture, Size(11 ,11),0,0);

    //medianBlur(loadedPictureHere, processedPicture, 11);

    medianBlur(loadedPictureHere, processedPicture, 15);

    // prepare bigger Mat to display image and process image side by side (this Mat      //is bigger)

    Mat output = Mat((loadedPictureHere.cols * 3), loadedPictureHere.rows *2 ,
                 loadedPictureHere.type());

    // copy small images process and not processed into the bigger one. 

    loadedPictureHere.copyTo(output(Rect(0, 0,  loadedPictureHere.cols,
                      loadedPictureHere.rows)));

    processedPicture.copyTo(output(Rect(loadedPictureHere.cols, 0,
                      loadedPictureHere.cols, loadedPictureHere.rows)));

    namedWindow("Step 1", WINDOW_AUTOSIZE);// Create a window for display.

    imshow("Step 1", output);   // Display loaded image 

    waitKey(25);

    fastNlMeansDenoising(loadedPictureHere, processedPicture,30, 7, 21);

    //fastNlMeansDenoisingColored(loadedPictureHere, processedPicture, 7, 100); 

    // copy small images process and not processed into the bigger one. 

    loadedPictureHere.copyTo(output(Rect(0, 0, loadedPictureHere.cols,
                       loadedPictureHere.rows)));

    processedPicture.copyTo(output(Rect(loadedPictureHere.cols, 0,
                       loadedPictureHere.cols, loadedPictureHere.rows)));

    namedWindow("Step 2", WINDOW_AUTOSIZE);// Create a window for display.

    imshow("Step 2", output);   // Display loaded image 

    waitKey(25);

return 0; 

}