Java - Hello world¶
This web application has been designed to introduce the principles of programming with Kurento for Java developers. It consists on a WebRTC video communication in mirror (loopback). This tutorial assumes you have basic knowledge of JavaScript, HTML and WebRTC. We also recommend reading the Introducing Kurento section before starting this tutorial.
注解
This tutorial has been configured to use https. Follow the instructions to secure your application.
For the impatient: running this example¶
You need to have installed the Kurento Media Server before running this example. Read the installation guide for further information.
To launch the application, you need to clone the GitHub project where this demo is hosted, and then run the main class:
git clone https://github.com/Kurento/kurento-tutorial-java.git
cd kurento-tutorial-java/kurento-hello-world
git checkout 6.6.2
mvn compile exec:java
Access the application connecting to the URL https://localhost:8443/ in a WebRTC capable browser (Chrome, Firefox).
注解
These instructions work only if Kurento Media Server is up and running in the same machine
as the tutorial. However, it is possible to connect to a remote KMS in other machine, simply adding
the flag kms.url
to the JVM executing the demo. As we’ll be using maven, you should execute
the following command
mvn compile exec:java -Dkms.url=ws://kms_host:kms_port/kurento
Understanding this example¶
Kurento provides developers a Kurento Java Client to control the Kurento Media Server. This client library can be used in any kind of Java application: Server Side Web, Desktop, Android, etc. It is compatible with any framework like Java EE, Spring, Play, Vert.x, Swing and JavaFX.
This hello world demo is one of the simplest web applications you can create with Kurento. The following picture shows a screenshot of this demo running:
The interface of the application (an HTML web page) is composed by two HTML5 video tags: one showing the local stream (as captured by the device webcam) and the other showing the remote stream sent by the media server back to the client.
The logic of the application is quite simple: the local stream is sent to the Kurento Media Server, which sends it back to the client without modifications. To implement this behavior, we need to create a Media Pipeline composed by a single Media Element, i.e. a WebRtcEndpoint, which holds the capability of exchanging full-duplex (bidirectional) WebRTC media flows. This media element is connected to itself so that the media it receives (from browser) is sent back (to browser). This media pipeline is illustrated in the following picture:
This is a web application, and therefore it follows a client-server architecture. At the client-side, the logic is implemented in JavaScript. At the server-side, we use a Spring-Boot based server application consuming the Kurento Java Client API, to control Kurento Media Server capabilities. All in all, the high level architecture of this demo is three-tier. To communicate these entities, two WebSockets are used. First, a WebSocket is created between client and application server to implement a custom signaling protocol. Second, another WebSocket is used to perform the communication between the Kurento Java Client and the Kurento Media Server. This communication takes place using the Kurento Protocol. For further information on it, please see this page of the documentation.
The diagram below shows a complete sequence diagram, of the interactions with the application interface to: i) JavaScript logic; ii) Application server logic (which uses the Kurento Java Client); iii) Kurento Media Server.
The following sections analyze in depth the server (Java) and client-side (JavaScript) code of this application. The complete source code can be found in GitHub.
Application Server Logic¶
This demo has been developed using Java in the server-side, based on the Spring Boot framework, which embeds a Tomcat web server within the generated maven artifact, and thus simplifies the development and deployment process.
注解
You can use whatever Java server side technology you prefer to build web applications with Kurento. For example, a pure Java EE application, SIP Servlets, Play, Vert.x, etc. Here we chose Spring Boot for convenience.
In the following, figure you can see a class diagram of the server side code:
The main class of this demo is HelloWorldApp. As you can see, the KurentoClient is instantiated in this class as a Spring Bean. This bean is used to create Kurento Media Pipelines, which are used to add media capabilities to the application. In this instantiation we see that we need to specify to the client library the location of the Kurento Media Server. In this example, we assume it is located at localhost, listening in port 8888. If you reproduce this example, you’ll need to insert the specific location of your Kurento Media Server instance there.
Once the Kurento Client has been instantiated, you are ready for communicating with Kurento Media Server and controlling its multimedia capabilities.
@EnableWebSocket
@SpringBootApplication
public class HelloWorldApp implements WebSocketConfigurer {
@Bean
public HelloWorldHandler handler() {
return new HelloWorldHandler();
}
@Bean
public KurentoClient kurentoClient() {
return KurentoClient.create();
}
@Override
public void registerWebSocketHandlers(WebSocketHandlerRegistry registry) {
registry.addHandler(handler(), "/helloworld");
}
public static void main(String[] args) throws Exception {
new SpringApplication(HelloWorldApp.class).run(args);
}
}
This web application follows a Single Page Application architecture
(SPA), and uses a WebSocket to communicate client with
application server by means of requests and responses. Specifically, the main
app class implements the interface WebSocketConfigurer
to register a
WebSocketHanlder
to process WebSocket requests in the path /helloworld
.
HelloWorldHandler
class implements TextWebSocketHandler
to handle text WebSocket requests.
The central piece of this class is the method handleTextMessage
. This
method implements the actions for requests, returning responses through the
WebSocket. In other words, it implements the server part of the signaling
protocol depicted in the previous sequence diagram.
public class HelloWorldHandler extends TextWebSocketHandler {
private final Logger log = LoggerFactory.getLogger(HelloWorldHandler.class);
private static final Gson gson = new GsonBuilder().create();
@Autowired
private KurentoClient kurento;
private final ConcurrentHashMap<String, UserSession> users = new ConcurrentHashMap<String, UserSession>();
@Override
public void handleTextMessage(WebSocketSession session, TextMessage message) throws Exception {
JsonObject jsonMessage = gson.fromJson(message.getPayload(), JsonObject.class);
log.debug("Incoming message: {}", jsonMessage);
switch (jsonMessage.get("id").getAsString()) {
case "start":
start(session, jsonMessage);
break;
case "stop": {
UserSession user = users.remove(session.getId());
if (user != null) {
user.release();
}
break;
}
case "onIceCandidate": {
JsonObject jsonCandidate = jsonMessage.get("candidate").getAsJsonObject();
UserSession user = users.get(session.getId());
if (user != null) {
IceCandidate candidate = new IceCandidate(jsonCandidate.get("candidate").getAsString(),
jsonCandidate.get("sdpMid").getAsString(), jsonCandidate.get("sdpMLineIndex").getAsInt());
user.addCandidate(candidate);
}
break;
}
default:
sendError(session, "Invalid message with id " + jsonMessage.get("id").getAsString());
break;
}
}
private void start(final WebSocketSession session, JsonObject jsonMessage) {
try {
// 1. Media logic (webRtcEndpoint in loopback)
MediaPipeline pipeline = kurento.createMediaPipeline();
WebRtcEndpoint webRtcEndpoint = new WebRtcEndpoint.Builder(pipeline).build();
webRtcEndpoint.connect(webRtcEndpoint);
// 2. Store user session
UserSession user = new UserSession();
user.setMediaPipeline(pipeline);
user.setWebRtcEndpoint(webRtcEndpoint);
users.put(session.getId(), user);
// 3. SDP negotiation
String sdpOffer = jsonMessage.get("sdpOffer").getAsString();
String sdpAnswer = webRtcEndpoint.processOffer(sdpOffer);
JsonObject response = new JsonObject();
response.addProperty("id", "startResponse");
response.addProperty("sdpAnswer", sdpAnswer);
synchronized (session) {
session.sendMessage(new TextMessage(response.toString()));
}
// 4. Gather ICE candidates
webRtcEndpoint.addIceCandidateFoundListener(new EventListener<IceCandidateFoundEvent>() {
@Override
public void onEvent(IceCandidateFoundEvent event) {
JsonObject response = new JsonObject();
response.addProperty("id", "iceCandidate");
response.add("candidate", JsonUtils.toJsonObject(event.getCandidate()));
try {
synchronized (session) {
session.sendMessage(new TextMessage(response.toString()));
}
} catch (IOException e) {
log.error(e.getMessage());
}
}
});
webRtcEndpoint.gatherCandidates();
} catch (Throwable t) {
sendError(session, t.getMessage());
}
}
private void sendError(WebSocketSession session, String message) {
try {
JsonObject response = new JsonObject();
response.addProperty("id", "error");
response.addProperty("message", message);
session.sendMessage(new TextMessage(response.toString()));
} catch (IOException e) {
log.error("Exception sending message", e);
}
}
}
The start
method performs the following actions:
- Configure media processing logic: This is the part in which the application configures how Kurento has to process the media. In other words, the media pipeline is created here. To that aim, the object KurentoClient is used to create a MediaPipeline object. Using it, the media elements we need are created and connected. In this case, we only instantiate one WebRtcEndpoint for receiving the WebRTC stream and sending it back to the client.
- Store user session: In order to release the resources in the Kurento Media Server, we store the user session (i.e. Media Pipeline and WebRtcEndpoint) to be able to perform a release process when the stop method is called.
- WebRTC SDP negotiation: In WebRTC, SDP (Session Description protocol) is used for negotiating media exchanges between peers. Such negotiation is based on the SDP offer and answer exchange mechanism. This negotiation is finished in the third part of the method processRequest, using the SDP offer obtained from the browser client and returning a SDP answer generated by WebRtcEndpoint.
- Gather ICE candidates: As of version 6, Kurento fully supports the Trickle ICE protocol. For that reason, WebRtcEndpoint can receive ICE candidates asynchronously. To handle this, each WebRtcEndpoint offers a listener (addOnIceGatheringDoneListener) that receives an event when the ICE gathering process is done.
Client-Side Logic¶
Let’s move now to the client-side of the application. To call the previously
created WebSocket service in the server-side, we use the JavaScript class
WebSocket
. We use a specific Kurento JavaScript library called
kurento-utils.js to simplify the WebRTC interaction with the server. This
library depends on adapter.js, which is a JavaScript WebRTC utility
maintained by Google that abstracts away browser differences. Finally
jquery.js is also needed in this application.
These libraries are linked in the
index.html
web page, and are used in the
index.js.
In the following snippet we can see the creation of the WebSocket (variable
ws
) in the path /helloworld
. Then, the onmessage
listener of the
WebSocket is used to implement the JSON signaling protocol in the client-side.
Notice that there are three incoming messages to client: startResponse
,
error
, and iceCandidate
. Convenient actions are taken to implement each
step in the communication. For example, in functions start
the function
WebRtcPeer.WebRtcPeerSendrecv
of kurento-utils.js is used to start a
WebRTC communication.
var ws = new WebSocket('ws://' + location.host + '/helloworld');
ws.onmessage = function(message) {
var parsedMessage = JSON.parse(message.data);
console.info('Received message: ' + message.data);
switch (parsedMessage.id) {
case 'startResponse':
startResponse(parsedMessage);
break;
case 'error':
if (state == I_AM_STARTING) {
setState(I_CAN_START);
}
onError('Error message from server: ' + parsedMessage.message);
break;
case 'iceCandidate':
webRtcPeer.addIceCandidate(parsedMessage.candidate, function(error) {
if (error)
return console.error('Error adding candidate: ' + error);
});
break;
default:
if (state == I_AM_STARTING) {
setState(I_CAN_START);
}
onError('Unrecognized message', parsedMessage);
}
}
function start() {
console.log('Starting video call ...');
// Disable start button
setState(I_AM_STARTING);
showSpinner(videoInput, videoOutput);
console.log('Creating WebRtcPeer and generating local sdp offer ...');
var options = {
localVideo : videoInput,
remoteVideo : videoOutput,
onicecandidate : onIceCandidate
}
webRtcPeer = new kurentoUtils.WebRtcPeer.WebRtcPeerSendrecv(options,
function(error) {
if (error)
return console.error(error);
webRtcPeer.generateOffer(onOffer);
});
}
function onOffer(error, offerSdp) {
if (error)
return console.error('Error generating the offer');
console.info('Invoking SDP offer callback function ' + location.host);
var message = {
id : 'start',
sdpOffer : offerSdp
}
sendMessage(message);
}
function onIceCandidate(candidate) {
console.log('Local candidate' + JSON.stringify(candidate));
var message = {
id : 'onIceCandidate',
candidate : candidate
};
sendMessage(message);
}
function startResponse(message) {
setState(I_CAN_STOP);
console.log('SDP answer received from server. Processing ...');
webRtcPeer.processAnswer(message.sdpAnswer, function(error) {
if (error)
return console.error(error);
});
}
function stop() {
console.log('Stopping video call ...');
setState(I_CAN_START);
if (webRtcPeer) {
webRtcPeer.dispose();
webRtcPeer = null;
var message = {
id : 'stop'
}
sendMessage(message);
}
hideSpinner(videoInput, videoOutput);
}
function sendMessage(message) {
var jsonMessage = JSON.stringify(message);
console.log('Senging message: ' + jsonMessage);
ws.send(jsonMessage);
}
Dependencies¶
This Java Spring application is implemented using Maven. The relevant part of the pom.xml is where Kurento dependencies are declared. As the following snippet shows, we need two dependencies: the Kurento Client Java dependency (kurento-client) and the JavaScript Kurento utility library (kurento-utils) for the client-side. Other client libraries are managed with webjars:
<dependencies>
<dependency>
<groupId>org.kurento</groupId>
<artifactId>kurento-client</artifactId>
</dependency>
<dependency>
<groupId>org.kurento</groupId>
<artifactId>kurento-utils-js</artifactId>
</dependency>
<dependency>
<groupId>org.webjars</groupId>
<artifactId>webjars-locator</artifactId>
</dependency>
<dependency>
<groupId>org.webjars.bower</groupId>
<artifactId>bootstrap</artifactId>
</dependency>
<dependency>
<groupId>org.webjars.bower</groupId>
<artifactId>demo-console</artifactId>
</dependency>
<dependency>
<groupId>org.webjars.bower</groupId>
<artifactId>adapter.js</artifactId>
</dependency>
<dependency>
<groupId>org.webjars.bower</groupId>
<artifactId>jquery</artifactId>
</dependency>
<dependency>
<groupId>org.webjars.bower</groupId>
<artifactId>ekko-lightbox</artifactId>
</dependency>
</dependencies>
注解
We are in active development. You can find the latest version of Kurento Java Client at Maven Central.
Kurento Java Client has a minimum requirement of Java 7. Hence, you need to include the following properties in your pom:
<maven.compiler.target>1.7</maven.compiler.target>
<maven.compiler.source>1.7</maven.compiler.source>