Introduction
Previously we saw how to apply morphological operations like dilate, erode and Hit-or-Miss on images. Now we will see one more useful image-processing algorithm – creating of image pyramid. This is useful if we want to convert an image to a different size, let us say if we want to upscale (make it bigger) or downscale (make it smaller) it. For this purpose, we will use some already existing functions of OpenCV. Having a ready to use functions really makes our lives easier.
Description of the image pyramid creation
I will just give a brief overview about the creation of image pyramids. More information is available here and here.
Image pyramid is a collection of images, each one being twice smaller than the previous one. If you put them on top of each other, they will look like a pyramid. We achieve this by taking the original image, scaling it down to half its size and then repeating the same process for the newly created image. We can also upscale a given image twice its size and make more images from it. However, when we are upscaling an image we lose the quality of the original image, because we are missing an information on how to fill the new pixels.
We can use several kernels (structuring elements) in order to upscale or downscale an image. We will use one Gaussian kernel, which I believe OpenCV also uses.
Downscaling an image
In order to downscale an image we need to convolve the image with the above Gaussian kernel. Then we need remove each even-numbered row and column. This way the image size will be reduced twice its size. In OpenCV we have the function pyrDown()
Upscaling and image
The first step to upscale an image is to add even rows and columns to the original image. Then we convolve the image with the above kernel, but first we multiplied it by 4. This is the opposite of the downscaling process. There is a ready function in OpenCV – pyrUp()
Now after we know how to upscale and downscale an image and create an image pyramid by downscaling an image multiple times in theory, it is time to see it in practice. Let us write some code.
Source code for image pyramid generation
Upscaling an image
Below is the code and explanation of the algorithm to upscale an image.
// Upscale an image by a given factor
void upscaleImage(int upscaleFactor)
{
// Path to the input image
std::string l_pathToInputImage{ "../Resources/stormrider.jpg" };
// Create an object to hold the image data of the first image
Mat l_image;
// Read the image date from a file with no change to color scheme
l_image = imread(l_pathToInputImage, IMREAD_UNCHANGED);
// Check if we have read the first image data correctly
if (!l_image.data)
{
std::cout << "No image data \n";
return;
}
// Create the output image matrix
Mat l_outputImage = l_image.clone();
// Upscale the image several times
for (int i = 0; i < upscaleFactor / 2; ++i)
{
// Upscale the image. Upscaling can only be done by a factor of two
pyrUp(l_outputImage, l_outputImage, Size(l_outputImage.cols * 2, l_outputImage.rows * 2));
}
// Print the resolution values on top of the image
printText(l_image);
printText(l_outputImage);
// Display the input image
namedWindow("Input", WINDOW_AUTOSIZE);
cv::imshow("Input", l_image);
// Display the result image
namedWindow("Result", WINDOW_AUTOSIZE);
cv::imshow("Result", l_outputImage);
}As usual, we load the image from a file first. We clone the image into another one, which our code will process and modify. Next, there is one loop to process the image several times. The function, which will upscale the image, can only upscale it by a factor of two. If we want to scale it by a bigger factor, we will call the function several times. That is why we have a loop.
The OpenCV function
The OpenCV function which we use to upscale the image is pyrUp(). Check the link for more information. The first argument is the input image matrix. The second one is the output matrix, which will hold the upscaled image. Then we use the same image matrix because we want to keep upscaling the same image. We have put the original image into it. We upscale it and it gets bigger. Then we take the newly upscaled image and upscale it again. This can go on and on as long as we want. The third argument is the new size to which we want to scale the image. There are also other arguments, which I leave with their default values. At the end of the loop, the l_outputImage will be holding the upscaled image.
Lastly, we call one function to print a text with the resolution of the image on top of them. I have written this function myself. You can find it in the GitHub code or in the attached archive. You can go without it. At the end, we display the two images.
You can see the results below. Notice that the Input image is smaller than the Result image. You can also notice than the Result image is somehow blurry. As I already said, we have lost some quality of the image due to the upscaling process.
Downscaling an image
Below is the code and explanation of the algorithm to downscale an image.
// Downscale an image
void downscaleImage(int downscaleFactor)
{
// Path to the input image
std::string l_pathToInputImage{ "../Resources/love.png" };
// Create an object to hold the image data of the first image
Mat l_image;
// Read the image date from a file with no change to color scheme
l_image = imread(l_pathToInputImage, IMREAD_UNCHANGED);
// Check if we have read the first image data correctly
if (!l_image.data)
{
std::cout << "No image data \n";
return;
}
// Create the output image matrix
Mat l_outputImage = l_image.clone();
// Downscale the image several times
for (int i = 0; i < downscaleFactor / 2; ++i)
{
// Downscale the image. Downscaling can only be done by a factor of two
pyrDown(l_outputImage, l_outputImage, Size(l_outputImage.cols / 2, l_outputImage.rows / 2));
}
// Print the resolution values on top of the image
printText(l_image);
printText(l_outputImage);
// Display the input image
namedWindow("Input", WINDOW_AUTOSIZE);
cv::imshow("Input", l_image);
// Display the result image
namedWindow("Result", WINDOW_AUTOSIZE);
cv::imshow("Result", l_outputImage);
}Firstly, we load the image from a file. Again, we clone it into another one, which our code will process and modify. Next, there is one loop to process the image several times. The function, which will downscale the image, can only downscale it by a factor of two. If we want to scale it by a bigger factor, we will call the function several times.
The OpenCV function
The OpenCV function which we use to downscale the image is pyrDown().The first argument is the input image matrix. The second one is the output matrix, which will hold the downscaled image. We use the same image matrix because we want to keep downscaling the same image. The third argument is the new size to which we want to scale the image. There are also other arguments, which I leave with their default values. At the end of the loop, the l_outputImage will be holding the downscaled image.
Next, I am calling one function to print a text with the resolution of the image on top of them. At the end, we display the two images.
You can see the results below. Notice that the Input image is bigger than the Result image. You can also notice than the Result image is not blurry as it was with the upscaling. In case of downscaling, there is no lost in the quality.
Generating an image pyramid
We will see one more OpenCV function and its usage – buildPyramid(). It generates a set of downscaled images. Instead of having a loop to create many scaled images, this function will create them for us and will store them in a vector of matrices. Here is the source code.
// Create an image pyramid
void createImagePyramid(int level)
{
// Path to the input image
std::string l_pathToInputImage{ "../Resources/love.png" };
// Create an object to hold the image data of the first image
Mat l_image;
// Read the image date from a file with no change to color scheme
l_image = imread(l_pathToInputImage, IMREAD_UNCHANGED);
// Check if we have read the first image data correctly
if (!l_image.data)
{
std::cout << "No image data \n";
return;
}
// Create the output image matrix to store the scaled images
std::vector<Mat> l_pyramid;
// Build the image pyramid
buildPyramid(l_image, l_pyramid, level);
// Display the image pyramid one image per window
for (int i = 0; i < l_pyramid.size(); ++i)
{
Mat l_result = l_pyramid.at(i);
printText(l_result);
namedWindow("Result " + std::to_string(i), WINDOW_AUTOSIZE);
imshow("Result " + std::to_string(i), l_result);
}
}
Again, we load the image from a file. Then we create one vector of Mat objects to hold the downscaled images.
The OpenCV function
Next, we call the buildPyramid() OpenCV function. As a first argument, we provide the input image. The second argument is the vector of Mat objects. The third argument is how many levels of downscaling we want to apply. In my case, I will apply four.
Once the function has finished generating the image pyramid, it is time to display the result. In one “for” loop I am traversing the result vector, getting the next image matrix and display it in a separate window. That is it all. Following is the result that I am getting. Notice how it resemblance a pyramid.
Conclusion
In this tutorial, we learned about the image pyramids. We saw how we could generate those using OpenCV ready functions. I am leaving to you to try with different scale levels and different images.
The source code is again available in GitHub and as an archive below.
Next, we will see an OpenCV function to convert between color spaces.
Passionate developer, loving husband and caring father. As an introvert programming is my life. I work as a senior software engineer in a big international company. My hobbies include playing computer games (mostly World of Warcraft), watching TV series and hiking.
