Create your own media streaming platform [Using Open source technologies] (Part- 2)

This article is for developers who wants to build their own live streaming platform, including media server and web or mobile application with open source technologies.

In this piece, we will focus on Back-end component of your application.
If you did not go through part one yet, please visit Part 1: Application introduction and setting up your own media server first.

We have two back-end components Media manager and Metadata service. Lets try to understand and implement them.

Back-end high level diagram

Media manager component

This component/application is basically a java spring boot web-socket server to which our front-end component can interact, Plus this application also uses Kurento Java Client to create media pipelines and endpoints.

Kurento Java Client is a Java SE layer which consumes the Kurento API and exposes its capabilities through a simple-to-use interface based on Java POJOs representing Media Elements and Media Pipelines.

Kurento media server has Media Elements, A Media Element is a functional unit performing a specific action on a media stream (Media Elements are capable of receiving and sending media from and to other elements).
Chain of these Media Elements is Media Pipeline.
Kurento also has many Endpoints, we will be using WebRtc Endpoint and Player Endpoint.

A WebRtc Endpoint is an input/output endpoint that provides media streaming for Real Time Communications (RTC) through the web. It implements WebRTC technology to communicate with browsers.
A Player Endpoint is an input endpoint that retrieves content from file system, HTTP URL and injects it into the Media Pipeline.

Lets start with creating a spring boot application (Maybe using spring initializr) and adding spring-boot-starter-websocket and kurento-client dependency in pom.xml

<dependency>
    <groupId>org.springframework.boot</groupId>
    <artifactId>spring-boot-starter-websocket</artifactId>
</dependency>
<dependency>
    <groupId>org.kurento</groupId>
    <artifactId>kurento-client</artifactId>
    <version>6.14.0</version>
</dependency>

Now we need to enable web socket and configure web socket,To do that we need to put @EnableWebSocket annotation and implement interface WebSocketConfigurer(org.springframework.web.socket.config.annotation.WebSocketConfigurer, To configure the WebSocket request handling), Also override registerWebSocketHandlers method and add a handler with ‘/djbot’ path.

Just for the sake of laziness lets do this in main class.

Here we are registering a WebSocketHandler,The callHandler method returns a class ‘CallHandler’ which extends TextWebSocketHandler (which is a convenient base class for WebSocketHandler implementation that process text messages only) and we are providing path as djbot.

So what we have done here is that whenever any kind of text message is transferred to our web-socket connection with path /djbot our CallHandler class (handleTextMessage method) will be able to listen that text message.

Once message is received we are using Gson library to convert JSON to java object.
Now you can see in the switch statement we are checking ‘id’ field in Json message with below mentioned string values. But first let me share some class object names that will be used in some of the methods.

viewers is a map to maintain current connected users.
playlist is a list of SystemTracks (We will get this list from Metadata service).
nominatedTracks, nominatedUITracks, nominators are some object that will be used in Vote for next track functionality.
currentTrackNumber is to maintain current track index in playlist list.
pipeline is to store djBot media pipeline
djBotUserSession is UserSession Object [Its a custom java class with WebSocketSession, WebRtcEndpoint, PlayerEndpoint type variables]

Do not worry, We will cover playlist related stuff in Metadata Service section.

• start: To spawn our DJ Bot, spawnDjBot method is called. Here we are creating a media pipeline using kurento.createMediaPipeline method. Now mainly here we want to create PlayerEndpoint which has capability to play local media or any http media, After creation we will set this playerEndpoint in djBotUserSession object to use it in any other method.

• nominateNextTrack: To vote for the next track, nominateNextTrack takes two arguments session and jsonMessage. session is added in nominators list just to make sure no one can vote more than one time and from jsonMessage we are getting votedTrackName and updating the nominatedTrack lists with +1 vote count. sendUpdatedNominatedTrackList method is than called which broadcasts the updated voted list to all the viewers/users.

• getPositionAndTrack: To get current Track details, Here we are just sending some track details to the session. (Whoever has asked for it, We will get more clarity in the next part of the series where we will cover the Front-end component).

• viewer: To connect as a user, After some basic checks we mainly want to create a new WebRtcEndpoint with our media pipeline object. As you can see here we are attaching some listeners one kind of tricky thing is that you can only send any messages once ConnectionState is CONNECTED, So as you can see after checking state we are sending current track details with nominated track list. The magic happens when do this (Flow of media is from left to right when using .connect() method, So medi will flow from playerEndpoint to webrtcEndpoint)

djBotUserSession.getPlayerEndpoint().connect(nextWebRtc);

Great, We have covered all front-end interactions, Let me show you some of the helper functions like getting playlist, filling up nominated list after track change etc.

Once CallHandler class is initialized, spawnDjBot is called to start the djBot using javax.annotation.PostConstruct

Inside CallHandler constructor, updatePlaylist method is called to fetch the playlist for the 1st time by calling Metadata rest service. Here we are also shuffling the playlist and updating nominated tracks list.

updateNextNominatedTracks is a simple function that clears up the lists and finds next random 5 tracks (That User will vote next track from), You can very much enhance the algorithm to cut out some of the fail case scenarios like once the track has been played it should not be shown in next 5 tracks etc…

changeTrack method is used whenever we need to change the current track like we have used it in spawnDjBot method (Whenever EndOfStreamEvent event has occurred or in case of any error), We have used this method recursively as you can see in the picture.
Now in this method we have called nextTrack method which changes the current track number on the basis of votes, It also triggers the updatePlaylist to get the latest shuffled playlist. After performing all these tasks it calls updateNextNominatedTracks. Also wanted to highlight here that inside changeTrack we are also calling getPositionAndTrackName and sendUpdatedNominatedTrackList method to share the current track details and updated nominated track list.

Metadata Service

This component/application is basically a java spring boot web application, Which has some rest controllers to provide playlist tracks, create playlist, load tracks in playlist etc.

There are many ways to perform above mentioned operations, We are not going to cover all the above operations in this series but will show you how to get presigned url and push data into aws s3.

List of SystemTrack is response from our metadata service, It contains some basic information along with two fields localPath and s3Url. localPath is nothing but path to the track (like /home/soundtracks/abc.mp3) and s3Url is pre-signedurl (Anyone who receives the presigned URL can then access the object. For example, if you have a video in your bucket and both the bucket and the object are private, you can share the video with others by generating a presigned URL.)of track that is present in aws s3.

You can build the aws s3 client using this class, It gets the necessary details from the application property file.

This class generates s3 presigned url with 7 days expiration time, uploads objects to s3, Here the s3 object key is in format
{artist name}-{track name} we need to store this in our database to fetch the itemor to generate s3 presigned url for this item.

Now both of our back-end components are ready to spawn, But have you wondered that how our Media manager would know the kurento media server details ? To be precise how KurentoClient will know the details.

We need to set kms.url property, Remember kms is kurento media server :)

Now as in Media manager we have dependency of Metadata service (To fetch playlist) lets start metadata service first on any port (Lets say 8081) and add medadata.service=http://localhost or {your server ip}:8081 in application property, You can use this property to call the metadata service maybe using RestTemplate. Assuming you have somehow managed to setup playlist by running mvn -U clean spring-boot:run from metadata service root dir or by creating a jar then doing java -jar metadata-service.jar etc.

Lets run Media manager using below command:

sudo nohup mvn -U clean spring-boot:run -Dspring-boot.run.jvmArguments="-Dkms.url=ws://your-elastic-ip:8888/kurento" > spring-log.txt &

In this command you can see we have passed kms.url, This will configure our KurentoClient. There are many good examples created by kurento team that you can check out in case of more clarity or for just exploring.

Great, now along with our media server, our back-end components are up and running, Ready to serve any client.

That’s all folks

for this part of the series, In the next part we will start with configuring a custom domain, setting up ssl certificate to media manager component and then we will create our front-end component using Ionic 5, Angular with Capacitor.