Java - Hello world

This web application has been designed to introduce the principles of programming with Kurento for Java developers. It consists on a WebRTC video communication in mirror (loopback). This tutorial assumes you have basic knowledge of JavaScript, HTML and WebRTC. We also recommend reading the Introducing Kurento section before starting this tutorial.

注解

This tutorial has been configured to use https. Follow the instructions to secure your application.

For the impatient: running this example

You need to have installed the Kurento Media Server before running this example. Read the installation guide for further information.

To launch the application, you need to clone the GitHub project where this demo is hosted, and then run the main class:

git clone https://github.com/Kurento/kurento-tutorial-java.git
cd kurento-tutorial-java/kurento-hello-world
git checkout 6.6.2
mvn compile exec:java

Access the application connecting to the URL https://localhost:8443/ in a WebRTC capable browser (Chrome, Firefox).

注解

These instructions work only if Kurento Media Server is up and running in the same machine as the tutorial. However, it is possible to connect to a remote KMS in other machine, simply adding the flag kms.url to the JVM executing the demo. As we’ll be using maven, you should execute the following command

mvn compile exec:java -Dkms.url=ws://kms_host:kms_port/kurento

Understanding this example

Kurento provides developers a Kurento Java Client to control the Kurento Media Server. This client library can be used in any kind of Java application: Server Side Web, Desktop, Android, etc. It is compatible with any framework like Java EE, Spring, Play, Vert.x, Swing and JavaFX.

This hello world demo is one of the simplest web applications you can create with Kurento. The following picture shows a screenshot of this demo running:

Kurento Hello World Screenshot: WebRTC in loopback

Kurento Hello World Screenshot: WebRTC in loopback

The interface of the application (an HTML web page) is composed by two HTML5 video tags: one showing the local stream (as captured by the device webcam) and the other showing the remote stream sent by the media server back to the client.

The logic of the application is quite simple: the local stream is sent to the Kurento Media Server, which sends it back to the client without modifications. To implement this behavior, we need to create a Media Pipeline composed by a single Media Element, i.e. a WebRtcEndpoint, which holds the capability of exchanging full-duplex (bidirectional) WebRTC media flows. This media element is connected to itself so that the media it receives (from browser) is sent back (to browser). This media pipeline is illustrated in the following picture:

Kurento Hello World Media Pipeline in context

Kurento Hello World Media Pipeline in context

This is a web application, and therefore it follows a client-server architecture. At the client-side, the logic is implemented in JavaScript. At the server-side, we use a Spring-Boot based server application consuming the Kurento Java Client API, to control Kurento Media Server capabilities. All in all, the high level architecture of this demo is three-tier. To communicate these entities, two WebSockets are used. First, a WebSocket is created between client and application server to implement a custom signaling protocol. Second, another WebSocket is used to perform the communication between the Kurento Java Client and the Kurento Media Server. This communication takes place using the Kurento Protocol. For further information on it, please see this page of the documentation.

The diagram below shows a complete sequence diagram, of the interactions with the application interface to: i) JavaScript logic; ii) Application server logic (which uses the Kurento Java Client); iii) Kurento Media Server.

Complete sequence diagram of Kurento Hello World (WebRTC in loopbak) demo

Complete sequence diagram of Kurento Hello World (WebRTC in loopbak) demo

The following sections analyze in depth the server (Java) and client-side (JavaScript) code of this application. The complete source code can be found in GitHub.

Application Server Logic

This demo has been developed using Java in the server-side, based on the Spring Boot framework, which embeds a Tomcat web server within the generated maven artifact, and thus simplifies the development and deployment process.

注解

You can use whatever Java server side technology you prefer to build web applications with Kurento. For example, a pure Java EE application, SIP Servlets, Play, Vert.x, etc. Here we chose Spring Boot for convenience.

In the following, figure you can see a class diagram of the server side code:

Server-side class diagram of the HelloWorld app

Server-side class diagram of the HelloWorld app

The main class of this demo is HelloWorldApp. As you can see, the KurentoClient is instantiated in this class as a Spring Bean. This bean is used to create Kurento Media Pipelines, which are used to add media capabilities to the application. In this instantiation we see that we need to specify to the client library the location of the Kurento Media Server. In this example, we assume it is located at localhost, listening in port 8888. If you reproduce this example, you’ll need to insert the specific location of your Kurento Media Server instance there.

Once the Kurento Client has been instantiated, you are ready for communicating with Kurento Media Server and controlling its multimedia capabilities.

@EnableWebSocket
@SpringBootApplication
public class HelloWorldApp implements WebSocketConfigurer {

   @Bean
   public HelloWorldHandler handler() {
      return new HelloWorldHandler();
   }

   @Bean
   public KurentoClient kurentoClient() {
      return KurentoClient.create();
   }

   @Override
   public void registerWebSocketHandlers(WebSocketHandlerRegistry registry) {
      registry.addHandler(handler(), "/helloworld");
   }

   public static void main(String[] args) throws Exception {
      new SpringApplication(HelloWorldApp.class).run(args);
   }
}

This web application follows a Single Page Application architecture (SPA), and uses a WebSocket to communicate client with application server by means of requests and responses. Specifically, the main app class implements the interface WebSocketConfigurer to register a WebSocketHanlder to process WebSocket requests in the path /helloworld.

HelloWorldHandler class implements TextWebSocketHandler to handle text WebSocket requests. The central piece of this class is the method handleTextMessage. This method implements the actions for requests, returning responses through the WebSocket. In other words, it implements the server part of the signaling protocol depicted in the previous sequence diagram.

public class HelloWorldHandler extends TextWebSocketHandler {

   private final Logger log = LoggerFactory.getLogger(HelloWorldHandler.class);
   private static final Gson gson = new GsonBuilder().create();

   @Autowired
   private KurentoClient kurento;

   private final ConcurrentHashMap<String, UserSession> users = new ConcurrentHashMap<String, UserSession>();

   @Override
   public void handleTextMessage(WebSocketSession session, TextMessage message) throws Exception {
      JsonObject jsonMessage = gson.fromJson(message.getPayload(), JsonObject.class);

      log.debug("Incoming message: {}", jsonMessage);

      switch (jsonMessage.get("id").getAsString()) {
      case "start":
         start(session, jsonMessage);
         break;
      case "stop": {
         UserSession user = users.remove(session.getId());
         if (user != null) {
            user.release();
         }
         break;
      }
      case "onIceCandidate": {
         JsonObject jsonCandidate = jsonMessage.get("candidate").getAsJsonObject();

         UserSession user = users.get(session.getId());
         if (user != null) {
            IceCandidate candidate = new IceCandidate(jsonCandidate.get("candidate").getAsString(),
                  jsonCandidate.get("sdpMid").getAsString(), jsonCandidate.get("sdpMLineIndex").getAsInt());
            user.addCandidate(candidate);
         }
         break;
      }
      default:
         sendError(session, "Invalid message with id " + jsonMessage.get("id").getAsString());
         break;
      }
   }

   private void start(final WebSocketSession session, JsonObject jsonMessage) {
      try {
         // 1. Media logic (webRtcEndpoint in loopback)
         MediaPipeline pipeline = kurento.createMediaPipeline();
         WebRtcEndpoint webRtcEndpoint = new WebRtcEndpoint.Builder(pipeline).build();
         webRtcEndpoint.connect(webRtcEndpoint);

         // 2. Store user session
         UserSession user = new UserSession();
         user.setMediaPipeline(pipeline);
         user.setWebRtcEndpoint(webRtcEndpoint);
         users.put(session.getId(), user);

         // 3. SDP negotiation
         String sdpOffer = jsonMessage.get("sdpOffer").getAsString();
         String sdpAnswer = webRtcEndpoint.processOffer(sdpOffer);

         JsonObject response = new JsonObject();
         response.addProperty("id", "startResponse");
         response.addProperty("sdpAnswer", sdpAnswer);

         synchronized (session) {
            session.sendMessage(new TextMessage(response.toString()));
         }

         // 4. Gather ICE candidates
         webRtcEndpoint.addIceCandidateFoundListener(new EventListener<IceCandidateFoundEvent>() {
            @Override
            public void onEvent(IceCandidateFoundEvent event) {
               JsonObject response = new JsonObject();
               response.addProperty("id", "iceCandidate");
               response.add("candidate", JsonUtils.toJsonObject(event.getCandidate()));
               try {
                  synchronized (session) {
                     session.sendMessage(new TextMessage(response.toString()));
                  }
               } catch (IOException e) {
                  log.error(e.getMessage());
               }
            }
         });
         webRtcEndpoint.gatherCandidates();

      } catch (Throwable t) {
         sendError(session, t.getMessage());
      }
   }

   private void sendError(WebSocketSession session, String message) {
      try {
         JsonObject response = new JsonObject();
         response.addProperty("id", "error");
         response.addProperty("message", message);
         session.sendMessage(new TextMessage(response.toString()));
      } catch (IOException e) {
         log.error("Exception sending message", e);
      }
   }
}

The start method performs the following actions:

  1. Configure media processing logic: This is the part in which the application configures how Kurento has to process the media. In other words, the media pipeline is created here. To that aim, the object KurentoClient is used to create a MediaPipeline object. Using it, the media elements we need are created and connected. In this case, we only instantiate one WebRtcEndpoint for receiving the WebRTC stream and sending it back to the client.
  2. Store user session: In order to release the resources in the Kurento Media Server, we store the user session (i.e. Media Pipeline and WebRtcEndpoint) to be able to perform a release process when the stop method is called.
  3. WebRTC SDP negotiation: In WebRTC, SDP (Session Description protocol) is used for negotiating media exchanges between peers. Such negotiation is based on the SDP offer and answer exchange mechanism. This negotiation is finished in the third part of the method processRequest, using the SDP offer obtained from the browser client and returning a SDP answer generated by WebRtcEndpoint.
  4. Gather ICE candidates: As of version 6, Kurento fully supports the Trickle ICE protocol. For that reason, WebRtcEndpoint can receive ICE candidates asynchronously. To handle this, each WebRtcEndpoint offers a listener (addOnIceGatheringDoneListener) that receives an event when the ICE gathering process is done.

Client-Side Logic

Let’s move now to the client-side of the application. To call the previously created WebSocket service in the server-side, we use the JavaScript class WebSocket. We use a specific Kurento JavaScript library called kurento-utils.js to simplify the WebRTC interaction with the server. This library depends on adapter.js, which is a JavaScript WebRTC utility maintained by Google that abstracts away browser differences. Finally jquery.js is also needed in this application.

These libraries are linked in the index.html web page, and are used in the index.js. In the following snippet we can see the creation of the WebSocket (variable ws) in the path /helloworld. Then, the onmessage listener of the WebSocket is used to implement the JSON signaling protocol in the client-side. Notice that there are three incoming messages to client: startResponse, error, and iceCandidate. Convenient actions are taken to implement each step in the communication. For example, in functions start the function WebRtcPeer.WebRtcPeerSendrecv of kurento-utils.js is used to start a WebRTC communication.

var ws = new WebSocket('ws://' + location.host + '/helloworld');

ws.onmessage = function(message) {
   var parsedMessage = JSON.parse(message.data);
   console.info('Received message: ' + message.data);

   switch (parsedMessage.id) {
   case 'startResponse':
      startResponse(parsedMessage);
      break;
   case 'error':
      if (state == I_AM_STARTING) {
         setState(I_CAN_START);
      }
      onError('Error message from server: ' + parsedMessage.message);
      break;
   case 'iceCandidate':
      webRtcPeer.addIceCandidate(parsedMessage.candidate, function(error) {
         if (error)
            return console.error('Error adding candidate: ' + error);
      });
      break;
   default:
      if (state == I_AM_STARTING) {
         setState(I_CAN_START);
      }
      onError('Unrecognized message', parsedMessage);
   }
}

function start() {
   console.log('Starting video call ...');

   // Disable start button
   setState(I_AM_STARTING);
   showSpinner(videoInput, videoOutput);

   console.log('Creating WebRtcPeer and generating local sdp offer ...');

   var options = {
      localVideo : videoInput,
      remoteVideo : videoOutput,
      onicecandidate : onIceCandidate
   }
   webRtcPeer = new kurentoUtils.WebRtcPeer.WebRtcPeerSendrecv(options,
         function(error) {
            if (error)
               return console.error(error);
            webRtcPeer.generateOffer(onOffer);
         });
}

function onOffer(error, offerSdp) {
   if (error)
      return console.error('Error generating the offer');
   console.info('Invoking SDP offer callback function ' + location.host);
   var message = {
      id : 'start',
      sdpOffer : offerSdp
   }
   sendMessage(message);
}

function onIceCandidate(candidate) {
   console.log('Local candidate' + JSON.stringify(candidate));

   var message = {
      id : 'onIceCandidate',
      candidate : candidate
   };
   sendMessage(message);
}

function startResponse(message) {
   setState(I_CAN_STOP);
   console.log('SDP answer received from server. Processing ...');

   webRtcPeer.processAnswer(message.sdpAnswer, function(error) {
      if (error)
         return console.error(error);
   });
}

function stop() {
   console.log('Stopping video call ...');
   setState(I_CAN_START);
   if (webRtcPeer) {
      webRtcPeer.dispose();
      webRtcPeer = null;

      var message = {
         id : 'stop'
      }
      sendMessage(message);
   }
   hideSpinner(videoInput, videoOutput);
}

function sendMessage(message) {
   var jsonMessage = JSON.stringify(message);
   console.log('Senging message: ' + jsonMessage);
   ws.send(jsonMessage);
}

Dependencies

This Java Spring application is implemented using Maven. The relevant part of the pom.xml is where Kurento dependencies are declared. As the following snippet shows, we need two dependencies: the Kurento Client Java dependency (kurento-client) and the JavaScript Kurento utility library (kurento-utils) for the client-side. Other client libraries are managed with webjars:

<dependencies>
   <dependency>
      <groupId>org.kurento</groupId>
      <artifactId>kurento-client</artifactId>
   </dependency>
   <dependency>
      <groupId>org.kurento</groupId>
      <artifactId>kurento-utils-js</artifactId>
   </dependency>
   <dependency>
      <groupId>org.webjars</groupId>
      <artifactId>webjars-locator</artifactId>
   </dependency>
   <dependency>
      <groupId>org.webjars.bower</groupId>
      <artifactId>bootstrap</artifactId>
   </dependency>
   <dependency>
      <groupId>org.webjars.bower</groupId>
      <artifactId>demo-console</artifactId>
   </dependency>
   <dependency>
      <groupId>org.webjars.bower</groupId>
      <artifactId>adapter.js</artifactId>
   </dependency>
   <dependency>
      <groupId>org.webjars.bower</groupId>
      <artifactId>jquery</artifactId>
   </dependency>
   <dependency>
      <groupId>org.webjars.bower</groupId>
      <artifactId>ekko-lightbox</artifactId>
   </dependency>
</dependencies>

注解

We are in active development. You can find the latest version of Kurento Java Client at Maven Central.

Kurento Java Client has a minimum requirement of Java 7. Hence, you need to include the following properties in your pom:

<maven.compiler.target>1.7</maven.compiler.target>
<maven.compiler.source>1.7</maven.compiler.source>