Goals
In this tutorial, you will learn:
- Note
- Besides giving instructions to run OpenCV.js in Node.js, another objective of this tutorial is to introduce users to the basics of emscripten APIs, like Module and File System and also Node.js.
Minimal example
Create a file example1.js with the following content:
Module = {
onRuntimeInitialized() {
console.log(
cv.getBuildInformation())
}
}
cv = require(
'./opencv.js')
Execute it
- Save the file as
example1.js.
- Make sure the file
opencv.js is in the same folder.
- Make sure Node.js is installed on your system.
The following command should print OpenCV build information:
What just happened?
- In the first statement:, by defining a global variable named 'Module', emscripten will call
Module.onRuntimeInitialized() when the library is ready to use. Our program is in that method and uses the global variable cv just like in the browser.
- The statement **"cv = require('./opencv.js')"** requires the file
opencv.js and assign the return value to the global variable cv. require() which is a Node.js API, is used to load modules and files. In this case we load the file opencv.js form the current folder, and, as said previously emscripten will call Module.onRuntimeInitialized() when its ready.
- See emscripten Module API for more details.
Working with images
OpenCV.js doesn't support image formats so we can't load png or jpeg images directly. In the browser it uses the HTML DOM (like HTMLCanvasElement and HTMLImageElement to decode and decode images). In node.js we will need to use a library for this.
In this example we use jimp, which supports common image formats and is pretty easy to use.
Example setup
Execute the following commands to create a new node.js package and install jimp dependency:
The example
const Jimp = require('jimp');
async function onRuntimeInitialized(){
var jimpSrc = await Jimp.read('./lena.jpg');
var src =
cv.matFromImageData(jimpSrc.bitmap);
let anchor =
new cv.Point(-1, -1);
cv.dilate(src, dst, M, anchor, 1,
cv.BORDER_CONSTANT,
cv.morphologyDefaultBorderValue());
new Jimp({
width: dst.cols,
height: dst.rows,
data: Buffer.from(dst.data)
})
src.delete();
dst.delete();
}
Module = {
onRuntimeInitialized
};
cv = require(
'./opencv.js');
Execute it
- Save the file as
exampleNodeJimp.js.
- Make sure a sample image
lena.jpg exists in the current directory.
The following command should generate the file output.png:
1 node exampleNodeJimp.js
Emulating HTML DOM and canvas
As you might already seen, the rest of the examples use functions like cv.imread(), cv.imshow() to read and write images. Unfortunately as mentioned they won't work on Node.js since there is no HTML DOM.
In this section, you will learn how to use jsdom and node-canvas to emulate the HTML DOM on Node.js so those functions work.
Example setup
As before, we create a Node.js project and install the dependencies we need:
4 npm install canvas jsdom
The example
const { Canvas, createCanvas, Image, ImageData, loadImage } = require('canvas');
const { JSDOM } = require('jsdom');
const { writeFileSync, existsSync, mkdirSync } = require("fs");
installDOM();
await loadOpenCV();
const image = await loadImage('./lena.jpg');
const src =
cv.imread(image);
const dst =
new cv.
Mat();
const anchor =
new cv.Point(-1, -1);
cv.dilate(src, dst, M, anchor, 1,
cv.BORDER_CONSTANT,
cv.morphologyDefaultBorderValue());
const canvas = createCanvas(300, 300);
writeFileSync('output.jpg', canvas.toBuffer('image/jpeg'));
src.delete();
dst.delete();
})();
function loadOpenCV() {
return new Promise(resolve => {
global.Module = {
onRuntimeInitialized: resolve
};
global.cv = require('./opencv.js');
});
}
function installDOM() {
const dom = new JSDOM();
global.document = dom.window.document;
global.Image = Image;
global.HTMLCanvasElement = Canvas;
global.ImageData = ImageData;
global.HTMLImageElement = Image;
}
Execute it
- Save the file as
exampleNodeCanvas.js.
- Make sure a sample image
lena.jpg exists in the current directory.
The following command should generate the file output.jpg:
1 node exampleNodeCanvas.js
Dealing with files
In this tutorial you will learn how to configure emscripten so it uses the local filesystem for file operations instead of using memory. Also it tries to describe how files are supported by emscripten applications
Accessing the emscripten filesystem is often needed in OpenCV applications for example to load machine learning models such as the ones used in Load Caffe framework models and How to run deep networks in browser.
Example setup
Before the example, is worth consider first how files are handled in emscripten applications such as OpenCV.js. Remember that OpenCV library is written in C++ and the file opencv.js is just that C++ code being translated to JavaScript or WebAssembly by emscripten C++ compiler.
These C++ sources use standard APIs to access the filesystem and the implementation often ends up in system calls that read a file in the hard drive. Since JavaScript applications in the browser don't have access to the local filesystem, emscripten emulates a standard filesystem so compiled C++ code works out of the box.
In the browser, this filesystem is emulated in memory while in Node.js there's also the possibility of using the local filesystem directly. This is often preferable since there's no need of copy file's content in memory. This section is explains how to do do just that, this is, configuring emscripten so files are accessed directly from our local filesystem and relative paths match files relative to the current local directory as expected.
The example
The following is an adaptation of Face Detection using Haar Cascades.
const { Canvas, createCanvas, Image, ImageData, loadImage } = require('canvas');
const { JSDOM } = require('jsdom');
const { writeFileSync, readFileSync } = require('fs');
await loadOpenCV();
const image = await loadImage('lena.jpg');
const src =
cv.imread(image);
cv.cvtColor(src, gray,
cv.COLOR_RGBA2GRAY, 0);
let faces =
new cv.RectVector();
let eyes =
new cv.RectVector();
faceCascade.
load(
'./haarcascade_frontalface_default.xml');
eyeCascade.load('./haarcascade_eye.xml');
let mSize =
new cv.Size(0, 0);
faceCascade.detectMultiScale(gray, faces, 1.1, 3, 0, mSize, mSize);
for (let i = 0; i < faces.size(); ++i) {
let roiGray = gray.roi(faces.get(i));
let roiSrc = src.roi(faces.get(i));
let point1 =
new cv.Point(faces.get(i).x, faces.get(i).y);
let point2 =
new cv.Point(faces.get(i).x + faces.get(i).width, faces.get(i).y + faces.get(i).height);
cv.rectangle(src, point1, point2, [255, 0, 0, 255]);
eyeCascade.detectMultiScale(roiGray, eyes);
for (let j = 0; j < eyes.size(); ++j) {
let point1 =
new cv.Point(eyes.get(j).x, eyes.get(j).y);
let point2 =
new cv.Point(eyes.get(j).x + eyes.get(j).width, eyes.get(j).y + eyes.get(j).height);
cv.rectangle(roiSrc, point1, point2, [0, 0, 255, 255]);
}
roiGray.delete();
roiSrc.delete();
}
const canvas = createCanvas(image.width, image.height);
writeFileSync('output3.jpg', canvas.toBuffer('image/jpeg'));
src.delete(); gray.delete(); faceCascade.delete(); eyeCascade.delete(); faces.delete(); eyes.delete()
})();
function loadOpenCV(rootDir = '/work', localRootDir = process.cwd()) {
if(global.Module && global.Module.onRuntimeInitialized && global.cv && global.cv.imread) {
return Promise.resolve()
}
return new Promise(resolve => {
installDOM()
global.Module = {
onRuntimeInitialized() {
resolve()
},
preRun() {
const FS = global.Module.FS
if(!FS.analyzePath(rootDir).exists) {
FS.mkdir(rootDir);
}
if(!existsSync(localRootDir)) {
mkdirSync(localRootDir, { recursive: true});
}
FS.mount(FS.filesystems.NODEFS, { root: localRootDir}, rootDir);
}
};
global.cv = require('./opencv.js')
});
}
function installDOM(){
const dom = new JSDOM();
global.document = dom.window.document;
global.Image = Image;
global.HTMLCanvasElement = Canvas;
global.ImageData = ImageData;
global.HTMLImageElement = Image;
}
Execute it
- Save the file as
exampleNodeCanvasData.js.
- Make sure the files
aarcascade_frontalface_default.xml and haarcascade_eye.xml are present in project's directory. They can be obtained from OpenCV sources.
- Make sure a sample image file
lena.jpg exists in project's directory. It should display people's faces for this example to make sense. The following image is known to work:
image
The following command should generate the file output3.jpg:
1 node exampleNodeCanvasData.js