JavaScript Module - Chroma Filter¶
This web application consists on a WebRTC video communication in mirror (loopback) with a chroma filter element.
注解
This tutorial has been configurated for using https. Follow these instructions for securing your application.
For the impatient: running this example¶
First of all, you should install Kurento Media Server to run this demo. Please
visit the installation guide for further
information. In addition, the built-in module kms-chroma-6.0
should be also
installed:
sudo apt-get install kms-chroma-6.0
Be sure to have installed Node.js and Bower in your system. In an Ubuntu machine, you can install both as follows:
curl -sL https://deb.nodesource.com/setup_4.x | sudo bash -
sudo apt-get install -y nodejs
sudo npm install -g bower
Due to Same-origin policy, this demo has to be served by an HTTP server. A very simple way of doing this is by means of an HTTP Node.js server which can be installed using npm :
sudo npm install http-server -g
You also need the source code of this demo. You can clone it from GitHub. Then start the HTTP server:
git clone https://github.com/Kurento/kurento-tutorial-js.git
cd kurento-tutorial-js/kurento-chroma
git checkout 6.6.1
bower install
http-server -p 8443 -S -C keys/server.crt -K keys/server.key
Finally, access the application connecting to the URL https://localhost:8443/ through a WebRTC capable browser (Chrome, Firefox).
注解
These instructions work only if Kurento Media Server is up and running in the same machine
as the tutorial. Kurento Media Server must use WebSockets over SSL/TLS (WSS), so make sure
you check this too.
It is possible to locate the KMS in other machine simple adding the parameter ws_uri
to the URL:
https://localhost:8443/index.html?ws_uri=wss://kms_host:kms_port/kurento
Notice that the Kurento Media Server must connected using a Secure WebSocket (i.e., the KMS URI starts with wss://). For this reason, the support for secure WebSocket must be enabled in the Kurento Media Server you are using to run this tutorial. For further information about securing applications, please visit the following page.
Understanding this example¶
This application uses computer vision and augmented reality techniques to detect a chroma in a WebRTC stream based on color tracking.
The interface of the application (an HTML web page) is composed by two HTML5 video tags: one for the video camera stream (the local client-side stream) and other for the mirror (the remote stream). The video camera stream is sent to Kurento Media Server, which processes and sends it back to the client as a remote stream. To implement this, we need to create a Media Pipeline composed by the following Media Element s:
The complete source code of this demo can be found in GitHub.
This example is a modified version of the Magic Mirror tutorial. In this case, this demo uses a Chroma instead of FaceOverlay filter.
In order to perform chroma detection, there must be a color calibration stage. To accomplish this step, at the beginning of the demo, a little square appears in upper left of the video, as follows:
In the first second of the demo, a calibration process is done, by detecting the color inside that square. When the calibration is finished, the square disappears and the chroma is substituted with the configured image. Take into account that this process requires lighting condition. Otherwise the chroma substitution will not be perfect. This behavior can be seen in the upper right corner of the following screenshot:
注解
Modules can have options. For configure these options, you need get the constructor to them. In Javascript and Node, you have to use kurentoClient.getComplexType(‘qualifiedName’) . There is an example in the code.
The media pipeline of this demo is is implemented in the JavaScript logic as follows:
...
kurentoClient.register('kurento-module-chroma')
const WindowParam = kurentoClient.getComplexType('chroma.WindowParam')
...
kurentoClient(args.ws_uri, function(error, client) {
if (error) return onError(error);
client.create('MediaPipeline', function(error, _pipeline) {
if (error) return onError(error);
pipeline = _pipeline;
console.log("Got MediaPipeline");
pipeline.create('WebRtcEndpoint', function(error, webRtc) {
if (error) return onError(error);
setIceCandidateCallbacks(webRtcPeer, webRtc, onError)
webRtc.processOffer(sdpOffer, function(error, sdpAnswer) {
if (error) return onError(error);
console.log("SDP answer obtained. Processing...");
webRtc.gatherCandidates(onError);
webRtcPeer.processAnswer(sdpAnswer);
});
console.log("Got WebRtcEndpoint");
var options =
{
window: WindowParam({
topRightCornerX: 5,
topRightCornerY: 5,
width: 30,
height: 30
})
}
pipeline.create('chroma.ChromaFilter', options, function(error, filter) {
if (error) return onError(error);
console.log("Got Filter");
filter.setBackground(args.bg_uri, function(error) {
if (error) return onError(error);
console.log("Set Image");
});
client.connect(webRtc, filter, webRtc, function(error) {
if (error) return onError(error);
console.log("WebRtcEndpoint --> filter --> WebRtcEndpoint");
});
});
});
});
});
注解
The TURN and STUN servers to be used can be configured simple adding
the parameter ice_servers
to the application URL, as follows:
https://localhost:8443/index.html?ice_servers=[{"urls":"stun:stun1.example.net"},{"urls":"stun:stun2.example.net"}]
https://localhost:8443/index.html?ice_servers=[{"urls":"turn:turn.example.org","username":"user","credential":"myPassword"}]
Dependencies¶
The dependencies of this demo has to be obtained using Bower. The definition of these dependencies are defined in the bower.json file, as follows:
"dependencies": {
"kurento-client": "6.6.0",
"kurento-utils": "6.6.2"
"kurento-module-pointerdetector": "6.6.0"
}
To get these dependencies, just run the following shell command:
bower install
注解
We are in active development. You can find the latest versions at Bower.