Opencv 4 Linux application in visual studio 2017- Ubuntu Linux subsystem on windows 10
This tutorial is a step-by-step guide for the development of the Opencv 4 Linux application in Visual Studio 2017 under Windows. The motivation is simple. I would like to work in Visual Studio 2017, which is simply great. I personally love it. It is not the only motivation. I am a Windows user, but the market is full of Linux-based cloud machines and docker. So, I need to target Linux. The OpenCV running on Linux is faster than my 5-year experience(consider any possible optimization options). Another motivating point is the Windows file system. Write an application for Linux that accesses the Windows file system.
Again, I am a Windows person with lots of Linux experience, and lots of datasets, images, and video stored on Windows machines. I would like to use this data without any transfer or copying of my datasets. Just directly open the c:/myPositiveImages image directory in the Linux app as /mnt/c/myPositiveImages. Yes, it is possible. I am doing this to prepare my datasets stored in the Windows file system and prepare the proper inputs for the Yolo darknet. Yolo is not so great on Windows and is hard to compile. Data, and images prepared in Windows are not compatible with most of the Linux apps and mainly Yolo input. You can say some argument like. Prepare your dataset on Windows and transfer it to Linux and read it by Yolo, Learn your detector and enjoy deep neural networks. No this doesn't work. Yolo has a problem reading datasets prepared on Windows. This is why Visual Studio 2017 and Linux subsystem, where data from Windows are processed by a Linux app. This is only one motivation from the many possible. I simply like Visual Studio, but my target is Linux cloud, Linux containers,s or IoT devices like raspberry pi.
What will I learn in this OpenCV tutorial? (win-Linux tutorial)
- Prepare an environment, where Visual Studio communicates with remote or local Linux
- Install OpenCV from the GIT
- Build Linux Opencv application from Windows, linker, and compiler settings
- Use Windows file system in Linux Opencv app
Skills needed for Visual Studio connected to Linux
Hopefully, you can follow this tutorial without too many problems. Anyway, this is advanced staff and some knowledge is beneficial for the smooth process of configuration of the development environment.-Windows
-Do not be afraid of Linux and the command line
-Linux basics
-Do not be afraid to read step by step this text
-Try to figure out what went wrong if something fails
-Certificates, ssh
-Visu Studio
-Opencv
Environment on windows
1)Windows 10, just 10. This is not working in Windows 7. Linux subsystem can be turned on and Linux easily installed from the Windows store, but just on W10. Open PowerShell as Administrator and run the following command
Enable-WindowsOptionalFeature -Online -FeatureName Microsoft-Windows-Subsystem-Linux
The Windows subsystem Linux can be turned on as well in control panels (It is harder and harder to find control panels in W10). Just google it if you do not have PowerShell.
2)The second part of preparation installs Linux from the Windows store. Just find and install Ubuntu 18.04.
Enable-WindowsOptionalFeature -Online -FeatureName Microsoft-Windows-Subsystem-Linux
The Windows subsystem Linux can be turned on as well in control panels (It is harder and harder to find control panels in W10). Just google it if you do not have PowerShell.
2)The second part of preparation installs Linux from the Windows store. Just find and install Ubuntu 18.04.
3)Install Visual Studio 2017 from scratch or modify it. Include all C++ packages and make sure to install Linux development with C++. Mark the option on the following menu in the red circle.
Summary, 1) The windows subsystem Linux is enabled. 2) Ubuntu is installed from the Windows store. 3) Visual Studio is installed with Linux development C++.
Prepare Linux Environment to communicate with Visual Studio
sudo apt-get update update your package database
sudo apt-get upgrade upgrade packages in your linux
The system is prepared for the installation of necessary additional packages. The process is visible on the following image and all commands listed below the image. The image is attached to see the correct response to your commands.
sudo apt install -y build-essential Install essential packages to build Linux apps.
sudo apt install -y gdb Install GDB debugger program
sudo apt install -y gdbserver Install GDB server
The build-essential package contains the following programs. make, libc6, gcc, g++, dpkg-dev. Basic package to build(C++ compiler g++, C compiler gcc, c library and header files, etc) and link your program. The gdb is a debugger and gdb server is a way to communicate with this debugger and see the information directly in Visual Studio.
The visual studio communicates with Linux through SSH and a shared file system. In the same way, Putty communicates with your Linux instance in a cloud machine. We need an OpenSSH server and configured ssh for PasswordAuthentification.
sudo apt install -y openssh-server Install openssh server
sudo nano /etc/ssh/sshd_config sshd configuration file and modify as follows:
Set PasswordAuthentication to Yes “PasswordAuthentication” setting and make sure it’s set to “yes”:
sudo ssh-keygen -A Generate new host keys for SSH
You can expect this as the response "ssh-keygen: generating new host keys: DSA"
sudo service ssh start Start ssh service
You can expect this as the response "Starting OpenBSD Secure Shell server sshd".
The SSH configuration and key generation is the most problematic part. The problem's solution is not trivial for many people. Hopefully, now everything is prepared to communicate with Linux from Visual Studio.
Install and build opencv 4 in Linux from GIT
This part is easy for a little bit experienced person. The following text describes the process of downloading latest GIT repository with Opencv 4. Build the OpenCV library by CMAKE and install the OpenCV shared library into the system.
I have one /home/opencv directory. Here just issue following commands:
git clone https://github.com/opencv/opencv.git
git clone https://github.com/opencv/opencv_contrib.git
git clone https://github.com/opencv/opencv_contrib.git
This will create two folders opencv and opencv_contrib with download source code from the GitHub repository. By following commands, everything that you need to build OpenCV. Even the packages are already installed.
[we have this] sudo apt-get install build-essential
[additional required] sudo apt-get install cmake git libgtk2.0-dev pkg-config libavcodec-dev libavformat-dev libswscale-dev
[optional] sudo apt-get install python-dev python-numpy libtbb2 libtbb-dev libjpeg-dev libpng-dev libtiff-dev libjasper-dev libdc1394-22-dev
[additional required] sudo apt-get install cmake git libgtk2.0-dev pkg-config libavcodec-dev libavformat-dev libswscale-dev
[optional] sudo apt-get install python-dev python-numpy libtbb2 libtbb-dev libjpeg-dev libpng-dev libtiff-dev libjasper-dev libdc1394-22-dev
Next to opencv and opencv_contrib create the new directory build by mkdir and enter the directory.
mkdir build
cd build
cd build
The next step runs the configuration by cmake. Cmake creates your make file, all the parameters for compiling the opencv project, and links with installed packages.
cmake -D CMAKE_BUILD_TYPE=Release -D -D CMAKE_INSTALL_PREFIX=/usr/local ..
The cmake finishes with all the configuration you need to build opencv by make. This takes a bit of time. The final command that installs your library under the system directory /usr/local/lib as on the picture below commands.
sudo make install
Connect Visual Studio with Linux by cross-platform connection manager
To perform this configuration successfully the Prepare Linux Environment to communicate with the Visual Studio chapter needs to be set up correctly. The first is to create a new project in Visual Studio 2017. I picked up Console Application (Linux).
Now is the time to connect my project with a remote system. This means with my localhost Linux subsystem under Windows 10. In main Visual Studio, menu go to Tools-Options-Cross Platform. You will perform the operation under Add- and fille Connect to Remote System menu.
Host name - probably localhost or ip
Port - You are running OpenSSH with most probably port 22
user name - Your user name to Linux
Authentification type -Password
Password -Your password for the specified user (This option is enabled by sshd_config “PasswordAuthentication” setting and make sure it’s set to “yes” described in the prepare Linux environment section ).
Tools/options/crossplatform
The Remote IntelliSense is a very useful feature. Update button for selected already defined remote machine download header file into the local cache. When you are writing the code for Linux and using Linux or opencv headers Visual Studio knows the input parameters, and format, and marks them as known. This is a huge feature. Otherwise, most of your code will be marked as unknown.
#include <cstdio>
#include <iostream>
int main()
{
printf("hello from OpencvLinux!\n");
return 0;
}
Configure Linux Opencv 4 projects in Visual Studio
This is not an easy step. The project configuration is complex and usually done by CMAKE. Cmake prepares all the compiler options, C++ language standards and linking dependencies options for you. Visual Studio is used g++compiler and linker. I configure a simple project in pure Linux, run cmake configuration, and transfer all the compiler and linker options into Visual Studio. First, Look at the full compiler and linker Configuration in the following sections. The explanation follows under compiler and linker full options.
Opencv visual studio g++ compiler configuration
This is the full compiler options
"g++" -W"switch" -W"no-deprecated-declarations" -W"empty-body" -W"conversion" -W"return-type" -W"parentheses" -W"no-pointer-sign" -W"no-format" -W"uninitialized" -W"unreachable-code" -W"unused-function" -W"unused-value" -W"unused-variable" -std=c++11 -w -fno-strict-aliasing -I "/usr/local/include/opencv4/opnecv2" -g0 "g++" -O2 "3600000" -fno-threadsafe-statics -D "NDEBUG" -W"switch" -W"no-deprecated-declarations" -W"empty-body" -W"conversion" -W"return-type" -W"parentheses" -W"no-format" -W"uninitialized" -W"unreachable-code" -W"unused-function" -W"unused-value" -W"unused-variable" -fno-rtti -fno-omit-frame-pointer -std=c11 -fno-exceptions "1" -o "C:\Users\Vlada\source\repos\OpencvLinux\OpencvLinux\obj\x64\Release\%(filename).o"
Opencv visual studio g++ linker configuration
This is the full linker options
Opencv 4 compiler and linker options
The first thing is to add all in the property page of the C/C++ project all g++, include the directory, C++11 standard in case of OpenCV 4, and other things from the picture below. The windows below include all the options and you can mimic all configurations in the same window rather than in separate tabs.
The Linker OpenCV 4 configuration part is more tricky
Library dependencies -l"opencv_core" -l"opencv_imgcodecs" -l"opencv_imgproc" -l"opencv_video" -l"opencv_videoio" -l"pthread" -l"jpeg" -l"gcc_s" -l"stdc++" -l"tbb" -l"rt" -l"dl" -l"m" -l"z" -l"png16" -l"c"
You are just filling the following values into the Library dependencies option without -l
opencv_core
opencv_imgcodecs
opencv_imgproc
opencv_video
opencv_videoio
pthread
jpeg
gcc_s
stdc++
tbb
rt
dl
m
z
png16
c
/lib/x86_64-linux-gnu/
/usr/local/lib
/usr/lib/x86_64-linux-gnu/
/usr/local/lib/
/usr/lib/gcc/x86_64-linux-gnu/7
(you can have different hard to say), You can compile your C++ project directly in Linux by Cmake and look what shared libraries are used by your program. By command, LDD OpencvLinux is possible list your shared dependencies library with version and location. Build a simple opencv 4 project by cmake and include all in Visual Studio.
All linker configuration can be mimic by follow all the setting in this table.
Opencv Yolo darknet program loading image from windows
The following program is stupid, not parametric externally, but reads images from the Windows file system and performs some actions. I will rewrite this program and explain in more detail in the following tutorials how to prepare data for Yolo darknet. The scope of this article is a little bit different.
Yolo darknet dataset preparation program description
The program reads images from the positive image library /mnt/c/npos" (C:/npos) and the negative image library /mnt/c/neg (C:/neg). The positive image is inserted into a negative one on a random location. You can see an inserted person in the center of the image. This result is stored as input of Yolo darknet. 1.jpg sample with 1.txt file 1 0.490885 0.811523 0.031684 0.0651042 with class, and person normalized location in the image.
One last output is txt file of all the input images for train.txt and test.txt.
/mnt/c/result/0.jpg
/mnt/c/result/1.jpg
/mnt/c/result/2.jpg
/mnt/c/result/3.jpg
/mnt/c/result/4.jpg
/mnt/c/result/5.jpg
/mnt/c/result/6.jpg
/mnt/c/result/7.jpg
/mnt/c/result/8.jpg
/mnt/c/result/9.jpg
Sample code in progress
#include "opencv2/imgcodecs.hpp"
#include "opencv2/imgcodecs/imgcodecs.hpp"
#include "opencv2/core/core.hpp"
#include "opencv2/highgui/highgui.hpp"
#include "opencv2/imgproc/imgproc.hpp"
#include "opencv2/videoio/videoio.hpp"
#include "opencv2/video/video.hpp"
#include <opencv2/imgproc.hpp>
#include <opencv2/highgui.hpp>
#include <opencv2/videoio/videoio.hpp>
#include <opencv2/imgcodecs/imgcodecs.hpp>
#include <vector>
#include <cstdio>
#include <iostream>
#include <fstream>
#include <sstream>
#include <vector>
#include <string>
#include <time.h>
#include <ctime>
#include <random>
#include <iostream>
#include <sys/types.h>
#include <dirent.h>
using namespace std;
using namespace cv;
void read_directory(const std::string &name, vector<String> &v)
{
cout << "read dir" << endl;
DIR *dirp = opendir(name.c_str());
struct dirent *dp;
while ((dp = readdir(dirp)) != NULL)
{
v.push_back(dp->d_name);
cout << dp->d_name << endl;
}
closedir(dirp);
cout << "closing" << endl;
}
int main()
{
vector<String> neg;
read_directory("/mnt/c/neg", neg);
vector<String> pos;
read_directory("/mnt/c/npos", pos);
cout << "dir read ends" << endl;
long lowest = 0;
if (pos.size() > neg.size())
{
lowest = neg.size();
}
else
{
lowest = pos.size();
}
cout << lowest << endl;
std::random_device rd;
std::mt19937 gen(rd());
std::ofstream outtest("/mnt/c/test.txt");
std::ofstream outtrain("/mnt/c/train.txt");
cout << "step 2" << endl;
for (int i = 2; i < lowest; i++)
{
cout << pos.at(i) << endl;
cout << neg.at(i) << endl;
}
long iter = 2;
for (int i = 3; i < lowest; i++)
{
cout << "xxxxxxxxxxxxxxxxx" << endl;
cout << pos.at(i) << endl;
string pom = "/mnt/c/npos/" + pos.at(i);
Mat po = imread(pom, IMREAD_COLOR);
string nem = "/mnt/c/neg/" + neg.at(i);
cout << nem << endl;
Mat ne = imread(nem, IMREAD_COLOR);
cout << "xxxxxxxxxxxxxxxxx" << endl;
cout << "after reading" << endl;
std::uniform_int_distribution<> dis(1, ne.cols - po.cols + 1);
std::uniform_int_distribution<> dis2(1, ne.rows - po.rows + 1);
int a = dis(gen);
int b = dis2(gen);
cout << a << endl;
cout << b << endl;
cout << po.cols << endl;
cout << po.rows << endl;
po.copyTo(ne(Rect(a, b, po.cols, po.rows)));
cout << "writing result" << endl;
string base = "/mnt/c/result/";
stringstream builder;
builder << base << iter << ".jpg";
imwrite(builder.str(), ne);
stringstream builder2;
string base2 = "/mnt/c/result/";
builder2 << base << iter << ".txt";
std::ofstream outfile(builder2.str());
stringstream builder4;
builder4 << base2 << iter << ".jpg" << endl;
outtest << builder4.str();
outtrain << builder4.str();
//outfile << base2 << " " << a << " " << b << " " << a+po.cols << " "<< b+po.rows << std::endl;
outfile << 1 << " " << (float)a / ne.cols << " " << (float)b
/ ne.rows << " " << (float)po.cols / ne.cols << " " <<
(float)po.rows / ne.rows << std::endl;
outfile.close();
iter = iter + 1;
}
outtest.close();
outtrain.close();
cout << "all is done" << endl;
return 0;
}
Common problems with opencv like an undefined reference to cv::imwrite
1) Restart SSH if Visual Studio stop communicates with Linux subsystem:sudo service ssh restart
Symptoms:
undefined reference to cv::imwrite(std::basic_string
undefined reference to `cv::imread(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, int)' undefined reference to
2)Problems with a linker, by cmake all is configured ok, by making file and own g++ setting like in the case of Visual Studio the problem can be C++11 standard in opnecv 4 (std::__cxx11), including files but most probably the settings of the shared libraries.
undefined reference to cv::imwrite(std::basic_string
undefined reference to `cv::imread(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, int)' undefined reference to
cv::imwrite(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, cv::_InputArray const&, std::vector<int, std::allocator<int> > const&)
