Opencv tutorial, Combine two overlapping videos frames into one, MatchTemplate example


My favourite

  • Opencv tutorial people detection
  • Head people cascade download
  • Opencv tutorial optical flow
  • Opencv Video stabilization
  • Opencv car dataset download
  • Opencv tutorial Transparent mask
  • Opencv videowriter
  • Opencv FFMPEG
  • Opencv Canny edge and hough lines
  • Combine two overlapping frame by simple correlation

    Share this for more tutorials and computer vision post from me.. Thanks best Vladimir
    Merge Two mat video frames

    Today morning, I read this post combine two overlapping video frame on stackoverflow. 
    Problem is to merge 2 images together or two video stream. I am thinking about the most simple not so much dummy solution. I find really simple solution that can be extend to more complex one. 

    First of all, I took 2 images from phone like on picture (color images). Program selects Rectangles region from both source images and resize end extract this roi rectangles. Idea is find the "best" overlapping Rect regions by normalized correlation and combine this images on place with maximal correspondence.

    Merge Two mat video frames

    Merge Two mat video frames

    I know well solution by SIFT and SURF, but it is little bit laborious than this.

    It is not the best solution but one of the most simple. If your cameras are fixed in stable position between themselves. This is good solution i thing. I hold my phone in hand :)

    You can also use this simple approach on video. The speed depends only on number of rectangle candidate you want to compare.

    You can improve this by smart region to compare selection.

    overlapping regions by optical flow

    Also, I am thinking about idea to use optical flow by putting images from camera at same time to sequence behind each other. From the possible overlapping regions in one images extract good features to track and find them in region of second images.

    matchTemplate Example

    #include <Windows.h>
    #include <fstream>
    #include <iostream>
    #include "opencv2\highgui.hpp"
    #include "opencv2\imgproc.hpp"
    #include "opencv2/imgcodecs/imgcodecs.hpp"
    #include "opencv2/videoio/videoio.hpp"

    using namespace cv;

    using namespace std;

    int main(int argc, const char** argv)


    Mat OneCamInput;

    Mat SecondCamInput;

    // load and resize source images of video

    OneCamInput = imread("1.JPG");

    SecondCamInput = imread("2.JPG");

    resize(OneCamInput, OneCamInput, Size(800, 600));

    resize(SecondCamInput,SecondCamInput, Size(800, 600));

    // Show Both imput images

    imshow("input1", OneCamInput);


    imshow("input2", SecondCamInput);


    //Convert both to gray images

    cvtColor(OneCamInput, OneCamInput, COLOR_BGR2GRAY);

    cvtColor(SecondCamInput, SecondCamInput, COLOR_BGR2GRAY);

    // Prepare Mat for MatchTemplates

    Mat res(1, 1, CV_32F);

    // Prepare values for max correspondance

    float resMax = 0;

    Rect RectOver1;

    Rect RectOver2;

    int iRes;

    // For loop over over compared rectangles with different size

    for (int i = 20; i <= OneCamInput.cols / 4; i = i + 1) {

          // Extract rectangles from both source images

           Mat M1 = OneCamInput(Rect(OneCamInput.cols - i, 0, i, OneCamInput.rows));

           Mat M2 = SecondCamInput(Rect(0 , 0, i, SecondCamInput.rows));

           imshow("Overlap Rect1", M1);


           imshow("Overlap Rect2", M2);


          // Match how similar selected rectangles are

           matchTemplate(M1, M2, res, TM_CCOEFF_NORMED);

             // Convert Mat res(1, 1, CV_32F); to flow

             float resFloat = ((float *)([0];

    //Save max correspondence

           if (resFloat >= resMax) {

                 resMax = resFloat;

                 cout << res << endl;

                 iRes = i;

                 RectOver1 = Rect(OneCamInput.cols - i, 0, i, OneCamInput.rows);

                 RectOver2 = Rect(0, 0, i, SecondCamInput.rows);



    Mat HM;

    // Crop original images defined by border of max correspondence

    Mat On1Res = OneCamInput(Rect(0, 0, OneCamInput.cols -iRes -RectOver1.width/6 , OneCamInput.rows));

    Mat On2Res = SecondCamInput(Rect(0, 0, SecondCamInput.cols - iRes , OneCamInput.rows));

    //Horizontal merge of this two images together

    hconcat(On1Res, On2Res, HM);

    imshow("Result", HM);


    imwrite("Result.jpg", HM);


    No comments:

    Post a Comment


    Follow by Email

    Powered by Blogger.