Introduction
In the previous tutorial we saw how to apply thresholding to images. Now we start a series to introduce the concept of filters. In the several upcoming tutorials we will see how to apply linear and non-linear filters to an image. First, we will see the most used linear filters, which OpenCV supports in the form of ready-to-use functions. We have seen some linear filters in this tutorial already.
In this tutorial we will see the basics of the linear filters and how to create and apply them. I will introduce you to two OpenCV functions, which we can use to apply basic linear filters. We will learn how to apply box filter and also how to define our own filter and apply it. Firstly, let’s start with some descriptions.
Linear filters theory
We saw image processing operation, which we apply to a single pixel and which depend only on that pixel’s value. They go by the name point operations. Filters on the other hand use the values of more than one pixel. Usually, they use the values of the neighboring pixels. The linear filters add the values of the neighboring pixels to the currently processed pixel. They use the so-called weighted summation. Each neighbor pixel contributes by some amount of its value. Usually, we normalize the contributions of each pixels so their sum is equal to one. This way we get smooth results. You can also use contributions, greater than one, but then the results will be different and not smooth.
In order to weight the summation of all the neighboring pixels, we apply a matrix with the contributions’ values. The name of that matrix is kernel. Usually, it looks something like this.
This is call box filter, because it comes in the shape of a box. We can use also other shapes. The above one is normalized, because we divide each value in the matrix to the total sum of all the values in the matrix. We can have other values, different the ones. See below two other kernel matrices, which we can use. First one is divided by 16 since the total sum of the matrix values is 16. The second one is a little bit different, because it has negative values. Its name is difference filter and we can use it for edge detection and image sharpening. The first filter is smoothing filter and we use it to remove noise and blur the image.
Creating linear filters using OpenCV functionс
In a previous tutorial we already saw how to blur an image using one OpenCV function. Now we will see other OpenCV functions. Let us look at the first example
Apply normalized box filter
// Apply normalized box filter
void applyNormalizedBoxFilter()
{
// Path to the input image
std::string l_pathToInputImage{ "../Resources/stormrider.jpg" };
// Create an object to hold the image data of the first image
Mat l_image;
// Read the image date from a file with no change to color scheme
l_image = imread(l_pathToInputImage, IMREAD_UNCHANGED);
// Check if we have read the first image data correctly
if (!l_image.data)
{
std::cout << "No image data \n";
return;
}
// Create the output image matrices
Mat l_outputImage;
// Create the kernel
constexpr int c_kernelSize = 3;
constexpr float c_kernelDiv = static_cast<float>(c_kernelSize * c_kernelSize);
Mat l_kernel = Mat::ones(Size{ c_kernelSize, c_kernelSize }, CV_32F) / c_kernelDiv;
// Apply the filter
filter2D(l_image, l_outputImage, -1, l_kernel);
// Display the input image
namedWindow("Input", WINDOW_NORMAL);
cv::imshow("Input", l_image);
// Display the result image
namedWindow("Result", WINDOW_NORMAL);
cv::imshow("Result", l_outputImage);
}As in every other example, we first load an image, which we will process. We create one output matrix, to hold the result. We can apply the point operations on the same image and get the result there straight. For the filters however, we need to use another medium image, where we hold the result. We don’t want to interfere with the results from the previous operations. As I mentioned the linear filters use neighboring pixels and if we use the changed pixel values, we will get wrong results.
The kernel
Next, we create one kernel, which we will apply on the image. We use the method ones() of the Mat object. It will create a matrix filled with ones. We have defined the size of that matrix. Then we define CV_32F to be the type of the matrix values, meaning we will operate with floats. Finally, we divide each matrix value to c_kernelDiv in order to normalize the matrix.
The OpenCV function
Following is the call to the filter2D() OpenCV function. As a first argument it takes the input image matrix. The second argument is the image matrix, where we will keep the result. Next argument is the desired depth of the output matrix. I use -1 because I want the output matrix to have the same depth as the input one. Here is more information about the possible depths. If you have problems reading the table, here it is an example: If you have CV_8U image (what we have in our case) you can use -1, CV_16S, CV_32F or CV_64F for the depth. The fourth argument is the kernel matrix.
There are three more arguments which I leave with their default values. Anchor is the point from the kernel matrix, which we want to put on top of the currently processed pixel. The default value uses the center point. This means that we will put the center kernel value on top of our currently processed pixel. And we will take each surrounding pixel. We can put the anchor on the top left pixel and this way we will take only values from the right and bottom pixels.
The delta is a value which we can add to the processed pixel before storing it into the output matrix. As a borderType we can select one of these values or better leave the default one.
The result
Lastly, we display the result. You can see that we have applied blur to the output image. We can change the size of the kernel in order to increase the blur. Try on your side with different values. Always use odd numbers. Here is the result on my side.
Apply random linear filter
This example is similar to the above one but here instead of using ones I will add some other numbers. Here is the code.
// Apply random linear filter
void applyRandomLinearFilter()
{
// Path to the input image
std::string l_pathToInputImage{ "../Resources/stormrider.jpg" };
// Create an object to hold the image data of the first image
Mat l_image;
// Read the image date from a file with no change to color scheme
l_image = imread(l_pathToInputImage, IMREAD_UNCHANGED);
// Check if we have read the first image data correctly
if (!l_image.data)
{
std::cout << "No image data \n";
return;
}
// Create the output image matrices
Mat l_outputImage;
// Create the kernel
constexpr float c_kernelDiv = static_cast<float>(1.0f/24.0f);
Mat l_kernel = c_kernelDiv * (Mat_<float>(3, 3) <<
1, 4, 1,
2, 8, 2,
1, 4, 1);
// Apply the filter
filter2D(l_image, l_outputImage, -1, l_kernel);
// Display the input image
namedWindow("Input", WINDOW_NORMAL);
cv::imshow("Input", l_image);
// Display the result image
namedWindow("Result", WINDOW_NORMAL);
cv::imshow("Result", l_outputImage);
}As you can see everything is the same as the above example. The only difference is that I create one kernel with numbers, which I have selected randomly (kinda randomly). I am again dividing the matrix to the sum of the value in order to normalize it. You can try to remove the division and see what will happen. Or pick your numbers and see the results. You can even try with negative numbers. Here is the result on my side, it is quite similar to the previous one.
Apply normalized box filter using OpenCV ready function
In the last example I will show you another OpenCV function, which we can use in order to apply a normalized box filter. Here is the code.
// Apply normalized box filter using OpenCV ready function
void applyNormalizedBoxFilterOpenCVFunction()
{
// Path to the input image
std::string l_pathToInputImage{ "../Resources/stormrider.jpg" };
// Create an object to hold the image data of the first image
Mat l_image;
// Read the image date from a file with no change to color scheme
l_image = imread(l_pathToInputImage, IMREAD_UNCHANGED);
// Check if we have read the first image data correctly
if (!l_image.data)
{
std::cout << "No image data \n";
return;
}
// Create the output image matrices
Mat l_outputImage;
// Box sizes
constexpr int c_boxWidth = 3;
constexpr int c_boxHeight = 5;
// Apply the filter
boxFilter(l_image, l_outputImage, -1, Size{ c_boxWidth, c_boxHeight }, Point{ -1, -1 }, true);
// Display the input image
namedWindow("Input", WINDOW_NORMAL);
cv::imshow("Input", l_image);
// Display the result image
namedWindow("Result", WINDOW_NORMAL);
cv::imshow("Result", l_outputImage);
}As always, we start by loading an image from a file. Then we create the output image matrix. Next, we set the box width and height. You can use the same or different numbers.
We apply a box filter using the boxFilter() OpenCV function. This function takes the input and the output image matrices as the first two arguments. The third argument is the desired depth of the output matrix. I use -1 because I want the output matrix to have the same depth as the input one. Here is more information about the possible depths. Next argument is the kernel size. Then we set the anchor point of the kernel. I am setting it to {-1, -1} in order to use the center of the kernel matrix as an anchor. Following is the Boolean normalize value, which will make the function use normalized values (it will divide each value to their sum). For the last argument – borderType we can select one of these values.
After the function call, we display the result. I am suggestion you to test with normalize equal to true and to false in order to see the difference. See the result on my side. Again, OpenCV has blurred the image. The bigger the kernel is the more blurred the image will be.
Conclusion
In this tutorial we saw some basics of the filters. We have already seen some of then in this tutorial. Now, here we see how to make our own linear filters. Linear filters are useful for noise removing and blurring an image. In the following tutorials we will see more about the filters.
The code we saw is available in GitHub and as an archive below.
Next, we will see how to generate kernel matrices for some widely used filters.
Passionate developer, loving husband and caring father. As an introvert programming is my life. I work as a senior software engineer in a big international company. My hobbies include playing computer games (mostly World of Warcraft), watching TV series and hiking.
