Skip to main content

Face mouth eyes, facial landmarks detection tutorial for Opencv C++

This tutorial shows simple and useful code on how to detect face and face landmarks in OpenCV C++. It is a very simple task for 30 minutes of your attention. The expected result of this tutorial is visible in the following picture. The main loop capture video from a web camera. This captured video frame is used further to detect the position of the face. Once the face is detected the predefined facial landmarks mask is calculated to match the position of landmarks of your face. 
opencv facial landmarks
Opencv Face landmarks tutorial

The setup description for OpenCV face landmark tutorial

My environment is Windows 10 and Visual Studio 2019. The OpenCV library in this tutorial is 3.4b build with contribution modules. My project in Visual Studio is set up as follows. The include header files are set up as follows. 

opencv visual studio set up
 In my project, I am using opencv_world343.lib and opencv_world343.dll as follows.
opencv visual studio library setting


Opencv detector requirements

This tutorial requires two files. The first is lbfmodel.yaml for facemark detector. You need to search over google for lbfmodel.yaml. I download this directly from a browser in txt file extension and loaded into OpenCV like follows. 
facemark->loadModel("lbfmodel.yaml.txt");
The second needed resource is haar cascade haarcascade_frontalface_alt2 for face detection. This cascade is distributed with OpenCV itself. You can load this cascade as follows. 
faceDetector.load("haarcascade_frontalface_alt2.xml");

Facial landmarks high-level description

There are only two functions in the loop. The first is int main() and the second one is void process(Mat imgMat imgcol). The main function is used to initialized global variables. The global variables refer to the face detector and face landmarks detector. The second purpose of the main function is to capture video from the camera. The video is resized and converted to a gray video. The gray video and colored video are sent to the process function. The process function performs the finding of the face in the image of the gray video. Once the face is detected the markers in the face are calculated to the desired position in the face. The result is drawn over the colored image. It is really that simple. 

The code description facemark

I will point to the code by references (x) to describe what is going on here. The first thing is to include the right header files. The (1) face.hpp is used for the Facemask detector. The (2) objdetect.hpp is used for a face detector. The global (3) facemark and face detector are loaded in the main function in the following part of code. The VideoCapture cap(0); is used to get video from a web camera.
    facemark = FacemarkLBF::create();
    facemark->loadModel("lbfmodel.yaml.txt");
    faceDetector.load("haarcascade_frontalface_alt2.xml");
    cout << "Loaded model" << endl;
This part of the main function is used to capture video into img container. In the second row resize the video. The last row performs a conversion from img to imgbw and the result is imgbw as a gray image. Both images are sent into the function process to detect face and face markers.
            cap >> img;
            vector<Rect> faces;
            resize(img, img, Size(460460), 00, INTER_LINEAR_EXACT);
            cvtColor(img, imgbw, COLOR_BGR2GRAY);
            process(imgbw, img);
This part of the code in main is just display of the resuls.
            namedWindow("Live", WINDOW_AUTOSIZE);
            setMouseCallback("Live", CallBackF, 0);
            imshow("Live", img);
            waitKey(5);
The process function detect faces by faceDetector from img container into the vector that represents rectangles over the faces.
    vector<Rect> faces;
    faceDetector.detectMultiScale(img, faces);
    vector< vector<Point2f> > shapes;
The faces are drawn into an image in this piece of code by function rectangle. The next line  imFace = imgcol(faces[i]); take ROI image of the case into new mat container imFace. In resize, the imFace is transform into five-time bigger face than the originally detected.
        for (size_t i = 0; i < faces.size(); i++)
        {
            cv::rectangle(imgcol, faces[i], Scalar(25500));
            imFace = imgcol(faces[i]);
            resize(imFace, imFace, Size(imFace.cols * 5imFace.rows * 5));
            faces[i] = Rect(faces[i].x = 0faces[i].y = 0faces[i].width * 5,
  (faces[i].height) * 5);
        }
The last part is to use facemark->fit to fit the markers in shapes over the imFace. imFace is fifth time bigger image of the detected face. It can be done for more detected faces than one, but I am taking just one imFace in this case. The inner for loop draw all the shapes like circles. The result is displayed by show after all these loops.
if (facemark->fit(imFace, faces, shapes))
        {
            for (unsigned long i = 0; i < faces.size(); i++) {
                for (unsigned long k = 0; k < shapes[i].size(); k++) {
                    cv::circle(imFace, shapes[i][k], 5cv::Scalar(00255), 
FILLED);
                }
            }
        }

Youtube description of opencv facemark the tutorial code

Tutorial Code C++ facemark 

#include <opencv2/opencv.hpp>
#include <opencv2/dnn/dnn.hpp>
//(1) include face header
#include "opencv2/face.hpp"
#include "opencv2/imgproc.hpp"
#include "opencv2/imgcodecs.hpp"
#include "opencv2/highgui.hpp"
//(2) include face header
#include "opencv2/objdetect.hpp"
#include <iostream>

using namespace cv;
using namespace std;
using namespace ml;

using namespace cv::face;

//(3) Global variables
Ptr<Facemark> facemark;
CascadeClassifier faceDetector;


void process(Mat imgMat imgcol) {

    vector<Rect> faces;
    faceDetector.detectMultiScale(img, faces);
    vector< vector<Point2f> > shapes;

    Mat imFace;
    if (faces.size() != 0) {

        for (size_t i = 0; i < faces.size(); i++)
        {
            cv::rectangle(imgcol, faces[i], Scalar(25500));
            imFace = imgcol(faces[i]);
            resize(imFace, imFace, Size(imFace.cols * 5imFace.rows * 5));
            faces[i] = Rect(faces[i].x = 0faces[i].y = 0faces[i].width * 5,
  (faces[i].height) * 5);
        }


        vector < Rect > measures;

        if (facemark->fit(imFace, faces, shapes))
        {
            for (unsigned long i = 0; i < faces.size(); i++) {

                for (unsigned long k = 0; k < shapes[i].size(); k++) {
                    cv::circle(imFace, shapes[i][k], 5cv::Scalar(00255),
 FILLED);
                }
            }
        }
        namedWindow("Detected_shape");
        imshow("Detected_shape", imFace);
        waitKey(5);
    }
    else {
        cout << "Faces not detected." << endl;
    }
}


int main()
{

    facemark = FacemarkLBF::create();
    facemark->loadModel("lbfmodel.yaml.txt");
    faceDetector.load("haarcascade_frontalface_alt2.xml");
    cout << "Loaded model" << endl;
    VideoCapture cap(0);

    int initialized = 0;
    for (;;)
    {
        if (!cap.isOpened()) {
            cout << "Video Capture Fail" << endl;
            break;
        }
        else {
            Mat img;
            Mat imgbw;
            cap >> img;  
            resize(img, img, Size(460460), 00, INTER_LINEAR_EXACT);
            cvtColor(img, imgbw, COLOR_BGR2GRAY);           

            process(imgbw, img);

            namedWindow("Live", WINDOW_AUTOSIZE);
            setMouseCallback("Live", CallBackF, 0);
            imshow("Live", img);
            waitKey(5);
        }
    }
}


Comments