Introduction
Hello!
In this advanced WebRTC tutorial I will show you how to stream your camera to a HTML page using WebRTC, GStreamer and C++. We will be using boost to handle the signaling. By the end of this tutorial you should have a simple understanding on WebRTC GStreamer.
Requirements
Creating the Project
First we need a place to house our projects files, create a new directory like so:
mkdir webrtc-stream && cd webrtc-stream
First we need to create a build file in order to build the completed project, create a new file called “CMakeLists.txt” and populate it with the following:
cmake_minimum_required(VERSION 3.10)
# Set the project name and version
project(webrtc_server VERSION 1.0)
# Specify the C++ standard
set(CMAKE_CXX_STANDARD 14)
set(CMAKE_CXX_STANDARD_REQUIRED True)
# Find required packages
find_package(PkgConfig REQUIRED)
pkg_check_modules(GST REQUIRED gstreamer-1.0 gstreamer-webrtc-1.0 gstreamer-sdp-1.0)
find_package(Boost 1.65 REQUIRED COMPONENTS system filesystem json)
# Include directories
include_directories(${GST_INCLUDE_DIRS} ${Boost_INCLUDE_DIRS})
# Add the executable
add_executable(webrtc_server main.cpp)
# Link libraries
target_link_libraries(webrtc_server ${GST_LIBRARIES} Boost::system Boost::filesystem Boost::json)
# Set properties
set_target_properties(webrtc_server PROPERTIES
CXX_STANDARD 14
CXX_STANDARD_REQUIRED ON
)
# Specify additional directories for the linker
link_directories(${GST_LIBRARY_DIRS})
# Print project info
message(STATUS "Project: ${PROJECT_NAME}")
message(STATUS "Version: ${PROJECT_VERSION}")
message(STATUS "C++ Standard: ${CMAKE_CXX_STANDARD}")
message(STATUS "Boost Libraries: ${Boost_LIBRARIES}")
message(STATUS "GStreamer Libraries: ${GST_LIBRARIES}")
The above links all the required libraries together in order to build the code into an executable that can be executed.
Now we can get on to coding the project.
Coding the Project
Now we can start coding the source code for the project, create a new file called “main.cpp”, we will start by importing the necessary headers for GStreamer, WebRTC, Boost and standard libraries:
#define GST_USE_UNSTABLE_API
#include Next we will be define constants that will be used later, mainly the STUN server and port that the server will listen on:
#define STUN_SERVER "stun://stun.l.google.com:19302"
#define SERVER_PORT 8000
Now we will declare global variables for the GStreamer main loop and pipeline elements:
GMainLoop *loop;
GstElement *pipeline, *webrtcbin;
Next we will create the functions to handle each of the events. First one being a function that sends ICE candidates to the WebSocket client:
void send_ice_candidate_message(websocket::stream<:socket>& ws, guint mlineindex, gchar *candidate)
{
std::cout << "Sending ICE candidate: mlineindex=" << mlineindex << ", candidate=" << candidate << std::endl;
object ice_json;
ice_json["candidate"] = candidate;
ice_json["sdpMLineIndex"] = mlineindex;
object msg_json;
msg_json["type"] = "candidate";
msg_json["ice"] = ice_json;
std::string text = serialize(msg_json);
ws.write(net::buffer(text));
std::cout << "ICE candidate sent" << std::endl;
}
The next “on_answer_created” function handles the creation of a WebRTC answer and sends it back to the client:
void on_answer_created(GstPromise *promise, gpointer user_data)
{
std::cout << "Answer created" << std::endl;
websocket::stream<:socket>* ws = static_cast<:stream>*>(user_data);
GstWebRTCSessionDescription *answer = NULL;
const GstStructure *reply = gst_promise_get_reply(promise);
gst_structure_get(reply, "answer", GST_TYPE_WEBRTC_SESSION_DESCRIPTION, &answer, NULL);
GstPromise *local_promise = gst_promise_new();
g_signal_emit_by_name(webrtcbin, "set-local-description", answer, local_promise);
object sdp_json;
sdp_json["type"] = "answer";
sdp_json["sdp"] = gst_sdp_message_as_text(answer->sdp);
std::string text = serialize(sdp_json);
ws->write(net::buffer(text));
std::cout << "Local description set and answer sent: " << text << std::endl;
gst_webrtc_session_description_free(answer);
}
The next function is just a placeholder for handling negotiation events, this event is not needed in this example:
void on_negotiation_needed(GstElement *webrtc, gpointer user_data)
{
std::cout << "Negotiation needed" << std::endl;
}
The “on_set_remote_description” function sets the remote description and creates an answer:
void on_set_remote_description(GstPromise *promise, gpointer user_data)
{
std::cout << "Remote description set, creating answer" << std::endl;
websocket::stream<:socket>* ws = static_cast<:stream>*>(user_data);
GstPromise *answer_promise = gst_promise_new_with_change_func(on_answer_created, ws, NULL);
g_signal_emit_by_name(webrtcbin, "create-answer", NULL, answer_promise);
}
The “on_ice_candidate” function handles ICE candidate events and sends them to the WebSocket client:
void on_ice_candidate(GstElement *webrtc, guint mlineindex, gchar *candidate, gpointer user_data)
{
std::cout << "ICE candidate generated: mlineindex=" << mlineindex << ", candidate=" << candidate << std::endl;
websocket::stream<:socket>* ws = static_cast<:stream>*>(user_data);
send_ice_candidate_message(*ws, mlineindex, candidate);
}
The “handle_websocket_session” function manages the WebSocket connection, setting up the GStreamer pipeline and handling both SDP and ICE messages:
void handle_websocket_session(tcp::socket socket)
{
try
{
websocket::stream<:socket> ws{std::move(socket)};
ws.accept();
std::cout << "WebSocket connection accepted" << std::endl;
GstStateChangeReturn ret;
GError *error = NULL;
pipeline = gst_pipeline_new("pipeline");
GstElement *v4l2src = gst_element_factory_make("v4l2src", "source");
GstElement *videoconvert = gst_element_factory_make("videoconvert", "convert");
GstElement *queue = gst_element_factory_make("queue", "queue");
GstElement *vp8enc = gst_element_factory_make("vp8enc", "encoder");
GstElement *rtpvp8pay = gst_element_factory_make("rtpvp8pay", "pay");
webrtcbin = gst_element_factory_make("webrtcbin", "sendrecv");
if (!pipeline || !v4l2src || !videoconvert || !queue || !vp8enc || !rtpvp8pay || !webrtcbin)
{
g_printerr("Not all elements could be created.\n");
return;
}
g_object_set(v4l2src, "device", "/dev/video0", NULL);
g_object_set(vp8enc, "deadline", 1, NULL);
gst_bin_add_many(GST_BIN(pipeline), v4l2src, videoconvert, queue, vp8enc, rtpvp8pay, webrtcbin, NULL);
if (!gst_element_link_many(v4l2src, videoconvert, queue, vp8enc, rtpvp8pay, NULL))
{
g_printerr("Elements could not be linked.\n");
gst_object_unref(pipeline);
return;
}
GstPad *rtp_src_pad = gst_element_get_static_pad(rtpvp8pay, "src");
GstPad *webrtc_sink_pad = gst_element_get_request_pad(webrtcbin, "sink_%u");
gst_pad_link(rtp_src_pad, webrtc_sink_pad);
gst_object_unref(rtp_src_pad);
gst_object_unref(webrtc_sink_pad);
g_signal_connect(webrtcbin, "on-negotiation-needed", G_CALLBACK(on_negotiation_needed), &ws);
g_signal_connect(webrtcbin, "on-ice-candidate", G_CALLBACK(on_ice_candidate), &ws);
ret = gst_element_set_state(pipeline, GST_STATE_PLAYING);
if (ret == GST_STATE_CHANGE_FAILURE)
{
g_printerr("Unable to set the pipeline to the playing state.\n");
gst_object_unref(pipeline);
return;
}
std::cout << "GStreamer pipeline set to playing" << std::endl;
for (;;)
{
beast::flat_buffer buffer;
ws.read(buffer);
auto text = beast::buffers_to_string(buffer.data());
value jv = parse(text);
object obj = jv.as_object();
std::string type = obj["type"].as_string().c_str();
if (type == "offer")
{
std::cout << "Received offer: " << text << std::endl;
std::string sdp = obj["sdp"].as_string().c_str();
GstSDPMessage *sdp_message;
gst_sdp_message_new_from_text(sdp.c_str(), &sdp_message);
GstWebRTCSessionDescription *offer = gst_webrtc_session_description_new(GST_WEBRTC_SDP_TYPE_OFFER, sdp_message);
GstPromise *promise = gst_promise_new_with_change_func(on_set_remote_description, &ws, NULL);
g_signal_emit_by_name(webrtcbin, "set-remote-description", offer, promise);
gst_webrtc_session_description_free(offer);
std::cout << "Setting remote description" << std::endl;
}
else if (type == "candidate")
{
std::cout << "Received ICE candidate: " << text << std::endl;
object ice = obj["ice"].as_object();
std::string candidate = ice["candidate"].as_string().c_str();
guint sdpMLineIndex = ice["sdpMLineIndex"].as_int64();
g_signal_emit_by_name(webrtcbin, "add-ice-candidate", sdpMLineIndex, candidate.c_str());
std::cout << "Added ICE candidate" << std::endl;
}
}
}
catch (beast::system_error const& se)
{
if (se.code() != websocket::error::closed)
{
std::cerr << "Error: " << se.code().message() << std::endl;
}
}
catch (std::exception const& e)
{
std::cerr << "Exception: " << e.what() << std::endl;
}
}
The next “start_server” function initializes the server, sccepting TCP connections and spawning new threads to handle each connection:
void start_server()
{
try
{
net::io_context ioc{1};
tcp::acceptor acceptor{ioc, tcp::endpoint{tcp::v4(), SERVER_PORT}};
for (;;)
{
tcp::socket socket{ioc};
acceptor.accept(socket);
std::cout << "Accepted new TCP connection" << std::endl;
std::thread{handle_websocket_session, std::move(socket)}.detach();
}
}
catch (std::exception const& e)
{
std::cerr << "Exception: " << e.what() << std::endl;
}
}
Finally we just need to create the final main function to initialize GStreamer, start the server and run the main loop:
int main(int argc, char *argv[])
{
gst_init(&argc, &argv);
loop = g_main_loop_new(NULL, FALSE);
std::cout << "Starting WebRTC server" << std::endl;
std::thread server_thread(start_server);
g_main_loop_run(loop);
server_thread.join();
gst_element_set_state(pipeline, GST_STATE_NULL);
gst_object_unref(pipeline);
g_main_loop_unref(loop);
std::cout << "WebRTC server stopped" << std::endl;
return 0;
}
Done, now we can finally build the project!
Building the Project
To build the above source code into an executable first create a new directory called build:
mkdir build && cd build
Build the project:
cmake ..
make
If all goes well the project should be built successfully and you should have an executable.
Next we need to create a page to view the stream.
Creating the Frontend
Create a new directory called “public” and in it create a new html file called “index.html” and populate it with the following code:
html>
<html>
<head>
<title>WebRTC Streamtitle>
head>
<body>
<video id="video" autoplay playsinline muted>video>
<script>
const video = document.getElementById('video');
const signaling = new WebSocket('ws://localhost:8000/ws');
let pc = new RTCPeerConnection({
iceServers: [{urls: 'stun:stun.l.google.com:19302'}]
});
signaling.onmessage = async (event) => {
const data = JSON.parse(event.data);
console.log('Received signaling message:', data);
if (data.type === 'answer') {
console.log('Setting remote description with answer');
await pc.setRemoteDescription(new RTCSessionDescription(data));
} else if (data.type === 'candidate') {
console.log('Adding ICE candidate:', data.ice);
await pc.addIceCandidate(new RTCIceCandidate(data.ice));
}
};
pc.onicecandidate = (event) => {
if (event.candidate) {
console.log('Sending ICE candidate:', event.candidate);
signaling.send(JSON.stringify({
type: 'candidate',
ice: event.candidate
}));
}
};
pc.ontrack = (event) => {
console.log('Received track:', event);
if (event.track.kind === 'video') {
console.log('Attaching video track to video element');
video.srcObject = event.streams[0];
video.play().catch(error => {
console.error('Error playing video:', error);
});
video.load();
}
};
pc.oniceconnectionstatechange = () => {
console.log('ICE connection state:', pc.iceConnectionState);
};
pc.onicegatheringstatechange = () => {
console.log('ICE gathering state:', pc.iceGatheringState);
};
pc.onsignalingstatechange = () => {
console.log('Signaling state:', pc.signalingState);
};
async function start() {
pc.addTransceiver('video', {direction: 'recvonly'});
const offer = await pc.createOffer();
console.log('Created offer:', offer);
await pc.setLocalDescription(offer);
console.log('Set local description with offer');
signaling.send(JSON.stringify({type: 'offer', sdp: pc.localDescription.sdp}));
}
start();
script>
body>
html>
The above can be explained in my other WebRTC tutorials, but it simple communicates with the signalling server and when a remote stream is received plays the video in the video HTML element.
Done now we can actually run the project!
Running the Project
To run the project simple execute the following command:
./webrtc_server
To run the html page we will use a python module:
python3 -m http.server 9999
Navigate your browser to http://localhost:9999 and on load you should see your camera showing in the video element like so:
Done!
Considerations
In order to improve the above, I would like to implement the following:
Conclusion
In this tutorial I have shown you how to stream your camera using native C++, GStreamer and view the stream in a HTML page. I hope this tutorial has taught you something, I certainly had a lot of fun creating it.
As always you can find the source code for the project on my Github: https://github.com/ethand91/webrtc-gstreamer
Happy Coding!
Like my work? I post about a variety of topics, if you would like to see more please like and follow me. Also I love coffee.
If you are looking to learn Algorithm Patterns to ace the coding interview I recommend the [following course](https://algolab.so/p/algorithms-and-data-structure-video-course?affcode=1413380_bzrepgch
