Opencv Linux application in windows tutorial - Visual studio and linux subsystem
This tutorial is step by step guide for the development of Opencv 4 Linux application in Visual Studio 2017 under windows. The motivation is simple. I would like to work in Visual Studio 2017, which is simply great. I personally love it. It is not the only motivation. I am a windows user, but the market is full of Linux based cloud machines and docker. So, I need to target Linux. The opencv running on Linux is faster from my 5-year experience(consider any possible optimization options). Another motivating point is a windows file system. Write an application for Linux that accesses the windows file system.
Again, I am windows person with lots of Linux experience, lots of datasets, images, and video stored on windows machines. I would like to use this data without any transfer or copy of my datasets. Just directly open c:/myPositiveImages image directory in Linux app as /mnt/c/myPositiveImages. Yes, it is possible. I am doing this to prepare my datasets stored in the windows file system and prepare the proper inputs for Yolo darknet. Yolo is not so great on windows and hard to compile. Data, images prepared in windows are not compatible with most of the Linux apps and mainly Yolo input. You can say some argument like. Prepare your dataset on windows and transfer to the Linux and read by Yolo, learn your detector and enjoy deep neural networks. No this doesn't work. Yolo has a problem to read dataset prepared on windows. This is why Visual Studio 2017 and Linux subsystem, where data from windows are processed by a Linux app. This is only one motivation from the many possible. I simply like Visual Studio, but my target is linux cloud, linux container or IoT devices like rasbery pi.
What will I learn in this opencv tutorial? (win-Linux tutorial)
- Prepare environment, where Visual Studio communicates with remote or local Linux
- Install opencv from the GIT
- Build Linux Opencv application from windows, linker and compiler settings
- Use windows file system in Linux Opencv app
Skills needed for visual studio connected to Linux
Hopefully, you can follow this tutorial without too many problems. Anyway, this is advance staff and some knowledge is beneficial for the smooth process of configuration the development environment.
-Windows
-Do not afraid of Linux and command line
-Linux basics
-Do not afraid to read step by step this text
-Try to figure out what went wrong if something fails
-Certificates, ssh
-Visu Studio
-Opencv
Environment on windows
1)Windows 10, just 10. This is not working in windows 7. Linux subsystem can be turned on and Linux easily installed from windows store, but just on W10. Open PowerShell as Administrator and run following command
Enable-WindowsOptionalFeature -Online -FeatureName Microsoft-Windows-Subsystem-Linux
The Windows subsystem Linux can be turned on as well in control panels (It is harder and harder to find control panels in W10). Just google it if you do not have PowerShell.
2)The second part of preparation installs Linux from the windows store. Just find and install Ubuntu 18.04.
3)Install Visual studio 2017 from scratch or modify. Insclude all C++ packages and make sure to install
Linux development with C++. Mark the option on the following menu in red circle.
Summary, 1) The windows subsystem linux is enabled. 2) Ubuntu is installed from windows store. 3) Visual Studio is installed with Linux development C++.
Prepare Linux Environment to communicate with Visual Studio
Now, it is the time to start your Linux. Find the Ubuntu icon and start. This will open the Ubuntu command line for you. Let's start with the upgrade the package database and install updates by following two commands.
sudo apt-get update update your package database
sudo apt-get upgrade upgrade packages in your linux
The system is prepared for installation of necessary additional packages. The process is visible on the following image and all commands listed below the image. The image is attached to see the correct response for your commands.
sudo apt install -y build-essential Install essential packages to build Linux apps.
sudo apt install -y gdb Install GDB debugger program
sudo apt install -y gdbserver Install GDB server
Build-essential package contains following programs. make, libc6, gcc, g++, dpkg-dev. Basic package to build(C++ compiler g++, C compiler gcc, c library and header files, etc) and link your program. The gdb is debugger and gdbserver is way how to communicate with this debugger and see the information directly in Visual Studio.
The visual studio communicates with Linux through SSH and shared file system. The same way how Putty communicates with your Linux instance in cloud machine. We need OpenSSH-server and configure ssh for PasswordAuthentification.
sudo apt install -y openssh-server Install openssh server
sudo nano /etc/ssh/sshd_config sshd configuration file and modify as follows:
Set
PasswordAuthentication to Yes “PasswordAuthentication” setting and make sure it’s set to “yes”:
sudo ssh-keygen -A Generate new host keys for SSH
You can expect this as the response "ssh-keygen: generating new host keys: DSA"
sudo service ssh start Start ssh service
You can expect this as the response "Starting OpenBSD Secure Shell server sshd".
The ssh configuration, key generation is the most problematic part. The problem's solution is not trivial for many people. Hopefully, now everything is prepared to communicate with Linux from Visual Studio.
Install and build opencv 4 in Linux from GIT
This part is easy for little bit experienced person. Infollowing text is described process of download latest GIT repository with Opencv 4. Build the opencv library by CMAKE and install Opencv shared library into the system.
I have one /home/opencv directory. Here just issue following commands:
git clone https://github.com/opencv/opencv.git
git clone https://github.com/opencv/opencv_contrib.git
This will create two folders opencv and opencv_contrib with download source code from github repository. By following commands is installed everything what you need to build opencv. Even the packages already installed.
[we have this] sudo apt-get install build-essential
[additional required] sudo apt-get install cmake git libgtk2.0-dev pkg-config libavcodec-dev libavformat-dev libswscale-dev
[optional] sudo apt-get install python-dev python-numpy libtbb2 libtbb-dev libjpeg-dev libpng-dev libtiff-dev libjasper-dev libdc1394-22-dev
Next to opencv and opencv_contrib create the new directory build by mkdir and enter the directory.
mkdir build
cd build
The next step runs the configuration by cmake. Cmake creates your make file, all the parameters for compiling opencv project and link with installed packages.
cmake -D CMAKE_BUILD_TYPE=Release -D -D CMAKE_INSTALL_PREFIX=/usr/local ..
Do not forgot the .. double dots. to compile with opencv_contrib specify the path to contrib modules by parameter -D OPENCV_EXTRA_MODULES_PATH="PATH/
opencv_contrib/modules/"
The cmake finish with all configuration you need to build opencv by make. This takes a bit of time. The final command that install your library under system directory /usr/local/lib as on picture below commands.
make -j7
sudo make install
Connect Visual Studio with Linux by cross-platform connection manager
To perform this configuration successfully the Prepare Linux Environment to communicate with Visual Studio chapter needs to be set up correctly. The first this is to create new project in Visual Studio 2017. I pick up Console Application (Linux).
Now is the time to connect my project with a remote system. This means with my localhost Linux subsystem under windows 10. In main Visual studio, menu go to Tools-Options-Cross Platform. You will perform the operation under Add- and fille Connect to Remote System menu.
Host name - probably localhost or ip
Port - You are running OpenSSH with most probably port 22
user name - Your user name to Linux
Authentification type -Password
Password -Your password for the specified user (This option is enabled by sshd_config “PasswordAuthentication” setting and make sure it’s set to “yes” described in prepare Linux environment section ).
Tools/options/crossplatform
The Remote IntelliSence is a very useful feature. Update button for selected already defined remote machine download header file into the local cache. When you are writing the code for Linux and using Linux or opencv headers visual studio knows the input parameters, format and mark them as known. This is a huge feature. Otherwise, most of your code will be marked as unknown.
Now you can add some source code and just verify your connection by print something to console. Maybe some header like #include <cstdio> or #incude <iostream> are required.
#include <cstdio>
#include <iostream>
printf("hello from OpencvLinux!\n");
return 0;
Configure Linux Opencv 4 projects in Visual Studio
This is not an easy step. The project configuration is complex and usually done by CMAKE. Cmake prepares all the compiler options, C++ language standard and linking dependencies options for you. In visual studio is used g++compiler and linker. I configure a simple project in pure Linux, run cmake configuration and transfer all the compiler and linker option into Visual Studio. First, Look at full compiler and linker Configuration in the following sections. The explanation follows under compiler and linker full options.
Opencv visual studio g++ compiler configuration
This is the full compiler options
"g++" -W"switch" -W"no-deprecated-declarations" -W"empty-body" -W"conversion" -W"return-type" -W"parentheses" -W"no-pointer-sign" -W"no-format" -W"uninitialized" -W"unreachable-code" -W"unused-function" -W"unused-value" -W"unused-variable" -std=c++11 -w -fno-strict-aliasing -I "/usr/local/include/opencv4/opnecv2" -g0 "g++" -O2 "3600000" -fno-threadsafe-statics -D "NDEBUG" -W"switch" -W"no-deprecated-declarations" -W"empty-body" -W"conversion" -W"return-type" -W"parentheses" -W"no-format" -W"uninitialized" -W"unreachable-code" -W"unused-function" -W"unused-value" -W"unused-variable" -fno-rtti -fno-omit-frame-pointer -std=c11 -fno-exceptions "1" -o "C:\Users\Vlada\source\repos\OpencvLinux\OpencvLinux\obj\x64\Release\%(filename).o"
Opencv visual studio g++ linker configuration
This is the full linker options
-o"C:\Users\Vlada\source\repos\OpencvLinux\OpencvLinux\bin\x64\Release\OpencvLinux.out" -Wl,-rpath-link="/lib/x86_64-linux-gnu/" -Wl,-rpath-link="/usr/local/lib" -Wl,-rpath-link="/usr/lib/x86_64-linux-gnu/" -Wl,-rpath-link="/usr/local/lib/" -Wl,-rpath-link="/usr/lib/gcc/x86_64-linux-gnu/7" "3600000" -Wl,-z,norelro -Wl,--no-undefined "g++" -l"opencv_core" -l"opencv_imgcodecs" -l"opencv_imgproc" -l"opencv_video" -l"opencv_videoio" -l"pthread" -l"jpeg" -l"gcc_s" -l"stdc++" -l"tbb" -l"rt" -l"dl" -l"m" -l"z" -l"png16" -l"c"
Opencv 4 compiler and linker options
The first thing is to add in property page of C/C++ project all g++, include directory, C++11 standard in case of opencv 4 and other things from the picture below. The windows below include all the option and you can mimic this all configuration in a same window other than in separate tabs.
Linker opencv 4 configuration part is more tricky
Library dependencies -l"opencv_core" -l"opencv_imgcodecs" -l"opencv_imgproc" -l"opencv_video" -l"opencv_videoio" -l"pthread" -l"jpeg" -l"gcc_s" -l"stdc++" -l"tbb" -l"rt" -l"dl" -l"m" -l"z" -l"png16" -l"c"
You are just filling following values into Library dependencies option without -l
opencv_core
opencv_imgcodecs
opencv_imgproc
opencv_video
opencv_videoio
pthread
jpeg
gcc_s
stdc++
tbb
rt
dl
m
z
png16
c
The
shared library search path option is important to locate library specified in previous step. My shared libraries like c,z,tbb,stdc++ .... are located in this shared lib locations:
/lib/x86_64-linux-gnu/
/usr/local/lib
/usr/lib/x86_64-linux-gnu/
/usr/local/lib/
/usr/lib/gcc/x86_64-linux-gnu/7
(you can have different hard to say), You can compile your c++ project directly in Linux by Cmake and look what shared libraries are used by your program. By command, LDD OpencvLinux is possible list your shared dependencies library with version and location. Build a simple opencv 4 project by cmake and include all in visual studio.
All linker configuration can be mimic by follow all the setting in this table.
Opencv Yolo darknet program loading image from windows
The following program is stupid, not parametric externally, but reads images from windows file system and perform some actions. I will rewrite this program and explain in more details in following tutorials how to prepare data for yolo darknet. The scope of this article is little bit different.
Yolo darknet dataset preparation program description
Program reads images from positive image library /mnt/c/npos" (C:/npos) and negative image library /mnt/c/neg (C:/neg). The positive image is inserted into a negative one on random location. You can see an inserted person in the center of the image. This result is stored as input of yolo darknet. 1.jpg sample with 1.txt file 1 0.490885 0.811523 0.031684 0.0651042 with class, and person normalized location in the image.
One last output is txt file of all the input image for train.txt and test.txt.
/mnt/c/result/0.jpg
/mnt/c/result/1.jpg
/mnt/c/result/2.jpg
/mnt/c/result/3.jpg
/mnt/c/result/4.jpg
/mnt/c/result/5.jpg
/mnt/c/result/6.jpg
/mnt/c/result/7.jpg
/mnt/c/result/8.jpg
/mnt/c/result/9.jpg
Sample code in progress
#include "opencv2/imgcodecs.hpp"
#include "opencv2/imgcodecs/imgcodecs.hpp"
#include "opencv2/core/core.hpp"
#include "opencv2/highgui/highgui.hpp"
#include "opencv2/imgproc/imgproc.hpp"
#include "opencv2/videoio/videoio.hpp"
#include "opencv2/video/video.hpp"
#include <opencv2/imgproc.hpp>
#include <opencv2/highgui.hpp>
#include <opencv2/videoio/videoio.hpp>
#include <opencv2/imgcodecs/imgcodecs.hpp>
#include <vector>
#include <cstdio>
#include <iostream>
#include <fstream>
#include <sstream>
#include <vector>
#include <string>
#include <time.h>
#include <ctime>
#include <random>
#include <iostream>
#include <sys/types.h>
#include <dirent.h>
using namespace std;
using namespace cv;
void read_directory(const std::string& name, vector<String> &v)
{
cout << "read dir" << endl;
DIR* dirp = opendir(name.c_str());
struct dirent * dp;
while ((dp = readdir(dirp)) != NULL) {
v.push_back(dp->d_name);
cout << dp->d_name << endl;
}
closedir(dirp);
cout << "closing" << endl;
}
int main()
{
vector<String> neg;
read_directory("/mnt/c/neg", neg);
vector<String>pos;
read_directory("/mnt/c/npos", pos);
cout << "dir read ends" << endl;
long lowest = 0;
if (pos.size() > neg.size()) {
lowest = neg.size();
}
else {
lowest = pos.size();
}
cout << lowest << endl;
std::random_device rd;
std::mt19937 gen(rd());
std::ofstream outtest("/mnt/c/test.txt");
std::ofstream outtrain("/mnt/c/train.txt");
cout << "step 2" << endl;
for (int i = 2; i < lowest; i++) {
cout << pos.at(i)<< endl;
cout << neg.at(i) << endl;
}
long iter = 2;
for (int i = 3; i < lowest;i++) {
cout << "xxxxxxxxxxxxxxxxx" << endl;
cout << pos.at(i) << endl;
string pom = "/mnt/c/npos/" + pos.at(i);
Mat po = imread(pom, IMREAD_COLOR);
string nem = "/mnt/c/neg/" + neg.at(i);
cout << nem << endl;
Mat ne = imread(nem, IMREAD_COLOR);
cout << "xxxxxxxxxxxxxxxxx" << endl;
cout << "after reading" << endl;
std::uniform_int_distribution<> dis(1, ne.cols - po.cols + 1);
std::uniform_int_distribution<> dis2(1, ne.rows - po.rows + 1);
int a = dis(gen);
int b = dis2(gen);
cout << a << endl;
cout << b << endl;
cout << po.cols << endl;
cout << po.rows << endl;
po.copyTo(ne(Rect(a, b, po.cols, po.rows))); cout << "writing result" << endl;
string base = "/mnt/c/result/";
stringstream builder;
builder << base << iter << ".jpg";
imwrite(builder.str(), ne);
stringstream builder2;
string base2 = "/mnt/c/result/";
builder2 << base << iter << ".txt";
std::ofstream outfile(builder2.str());
stringstream builder4;
builder4 << base2 << iter << ".jpg" << endl;
outtest << builder4.str();
outtrain << builder4.str();
//outfile << base2 << " " << a << " " << b << " " << a+po.cols << " "<< b+po.rows << std::endl;
outfile << 1 << " " << (float)a / ne.cols << " " << (float)b / ne.rows << " " << (float)po.cols / ne.cols << " " << (float)po.rows / ne.rows << std::endl;
outfile.close();
iter = iter + 1;
}
outtest.close();
outtrain.close();
cout << "all is done" << endl;
return 0;
}
Future tutorial
In future work, I will rewrite the program into a more useful utility. The purpose is to prepare yolo darknet dataset with all inputs from images located in the windows filesystem. Are you interested?
Common problems with opencv like an undefined reference to cv::imwrite
1) Restart SSH if Visual studio stop communicates with linux subsystem:
sudo service ssh restart
2)Problems with a linker, by cmake all is configured ok, by make file and own g++ setting like in case of Visual Studio the problem can be C++11 standard in opnecv 4 (std::__cxx11), include files but most probably the settings of the shared libraries.
Symptoms:
undefined reference to cv::imwrite(std::basic_string
undefined reference to `cv::imread(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, int)' undefined reference to
cv::imwrite(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, cv::_InputArray const&, std::vector<int, std::allocator<int> > const&)
