News

WebRTC WHIP & WHEP Tutorial: Build a live Streaming App

stp2y0 Comments
WebRTC WHIP & WHEP Tutorial: Build a live Streaming App


This article was originally published on the Metered Blog: WebRTC WHIP & WHEP Tutorial: Build a live Streaming App

WHIP (WebRTC-HTTP Ingestion Protocol) and WHEP (WebRTC-HTTP Egress Protocol) are protocols that are designed to streamline signalling in WebRTC with the help of standard HTTP methods

  • Definition of WHIP: WHIP simplifies how client devices send media streams to the server.

    • It replaces the complex signalling mechanism that is required with simple HTTP GET requests, thus making it easier to ingest media into servers

  • Definition of WHEP: WHEP protocol is used for delivering media streams from servers to clients. It uses the HTTP protocol to handle signalling for media consumption thus enabling client devices to receive media streams without complex setups

Roles in Simplifying WebRTC Signalling

  • Ease of Implementation: WHEP and WHIP uses HTTP protocols thus these protocols reduce complexity that is associated with

  • Stateless Communication: This is because the HTTP is stateless protocol, the server does not need to maintain ongoing session information between requests.

  • Improved Compatibility: Since the HTTP has universal compatibility, using HTTP for signalling is the best for compatibility across platforms and devices

  • Fast development: Developers can implement WebRTC apps more efficiently, because they do not need to take into account intricate details of traditional signalling methodologies



How does WHIP work



How does WHIP handle Media Stream Ingestion

WHIP protocol has revolutionized how media streams can be sent to the servers from client devices by using the HTTP methods for signalling

Traditional to setup the WebRTC you need to setup complex signalling mechanism using web sockets or other protocols. With WHIP this process becomes simple by using the HTTP protocol for signalling and starting a WebRTC session

  • HTTP POST Request: Here the client devices sends an HTTP POST request with the SDP or the session description protocol offer in the body to the WHIP endpoint

  • Server Response: The Media server then processes the SDP offer and responds with 200 status code including the SDP answer in the request body

  • ICE Candidate Exchange: The WHIP protocol supports the ICE protocol by allowing the client to send additional HTTP PATCH requests whenever new ICE candidates become available 

  • Connection Establishment: Once the SDP exchange is complete then a peer to peer connection is established enabling the client to stream the media to the server



Benefits over traditional Ingestion Methods

  • Simplicity: By using the WHIP methods the WHIP reduces the need for persistent connections and signalling servers.

  • Ease of Implementation: Developers can use HTTP which has universal compatibility to speed up the development process

  • Scalability: The stateless HTTP requests allow the servers to handle multiple connection requests simultaneously thus easily managing a large number of connections.

  • Firewall and Proxy Friendly: HTTP is firewall friendly, almost all types of firewall allow HTTP traffic 

  • Cost Effective: The simplified signalling through HTTP reduces the costs associated with adding a signalling server

Image description



How does WHEP work

The WHEP protocol simplifies the process of delivering media from server to the client devices. 

Thus the WHEP protocol enables you to use HTTP to setup signalling for receiving media from server the client devices.



How does WHEP work in media streaming

  • HTTP GET Request: The client requests a media stream by sending an HTTP GET request to the servers WHEP endpoint

  • SDP Exchange: The server responds with the SDP offer in HTTP response, the client then sends back the SDP answer in the subsequent POST request

  • Media Reception: Once the connection is established the media stream is received over the established WebRTC connection. NOTE: Many times you need a TURN server to establish a WebRTC connection

  • Support for ICE: WHEP allows the exchange of ICE Candidates through additional HTTP patch requests thus enabling better connectivity 



Advantages in client side streaming

  • Simplified Client Implementation: Use of HTTP requests hence reducing the need for complex signalling mechanisms 

  • Improved Compatibility: Universal support for the HTTP protocol ensures improved compatibility across devices 

  • Enhanced Scalability: Because the HTTP is a stateless protocol, this improves the scalability of the servers and with small resources you can scale to a very large number of users

  • Better Network Traversal: Because you can do signalling with HTTP and do not need web sockets or other mechanisms, this improves NAT traversal for connectivity. Once the connection is established you do need TURN server for WebRTC

  • Reduced Latency: Signalling using HTTP can lead to faster connections thus enhancing user experience.

Image description



Synergy between WHIP and WHEP



Combining Both Protocols for Efficient End-to-End Communication:

By combining WHIP and WHEP the developers can create a comprehensive signalling solution for WebRTC

The combination simplifies the ingestion and delivery of media streams, ensuring a smoother implementation of WebRTC

  • Unified Signalling Approach: Using the HTTP for both ingestion and delivery creates a consistent signalling methodology

  • Reduced Complexity: Developers need to deal with only HTTP protocol, thus reducing the learning curve and maintenance of code

  • Enhanced Performance: Streamlining the code with a single protocol for signalling leads to quicker connection times and lower latency when you are transmitting media



Use-Cases Showcasing Improved Performance



Building The Live Streaming APP

Implementing the WHIP and WHEP in your WebRTC app is quite easy. In this section, we will be setting up a WHIP server and integrating it in your application using modern technologies like node and docker

Here we are going to use Metered.ca TURN server service for NAT traversal



Setting Up the WHIP Server

Pre-requisites and Environment Setup:

  • Node.js and NPM: Make sure you have the latest Node and nvm installed

  • Metered.ca Account: Create an free account on Metered TURN servers

  • Public IP Address: This is required for the server to be accessible on the internet. If you are using any cloud provider for your application you get a free public IP address with that

  • WebRTC Media Server: We need a media server such as GStreamer or Janus that has WHIP support



Step up Step Configuration Using Node.Js and GStreamer

  1. Install GStreamer with WHIP Support

  2. Set Up the WHIP Server with GStreamer

    1. Create a GStreamer pipeline that listens to the WHIP connections like so 
gst-launch-1.0 whipserversrc name=whip 
  ! queue ! videoconvert ! autovideosink
Enter fullscreen mode

Exit fullscreen mode

The above command starts a WHIP server that accepts incoming media streams and displays them 

  1. Configure the Server to Use Metered.ca TURN Server

    1. Modify the GStreamer pipeline to use TURN server. This is important because NAT routers and firewalls block connections 
gst-launch-1.0 whipserversrc name=whip ice-server="turn://YOUR_USERNAME:YOUR_CREDENTIAL@relay.metered.ca:80" 
  ! queue ! videoconvert ! autovideosink
Enter fullscreen mode

Exit fullscreen mode

  1. Set Up a Reverse Proxy with Node.JS (Optional):

    1. If you want to manage HTTP endpoints or add any other additional logic, you can set up a simple express js server 
const express = require('express');
const httpProxy = require('http-proxy');
const app = express();
const proxy = httpProxy.createProxyServer();

app.post('/whip', (req, res) => {
  proxy.web(req, res, { target: 'http://localhost:PORT_WHERE_GSTREAMER_IS_RUNNING' });
});

app.listen(3000, () => {
  console.log('Proxy server running on port 3000');
});
Enter fullscreen mode

Exit fullscreen mode



Implementing WHIP in your Application

Code Snippets for Integrating WHIP:

On the client side, you can capture media streams with the help of RTCPeerConnection and use HTTP requests to handle the signalling that is required to establish a connection 

  1. Capture Media Streams:

    1. You can capture media streams like 
const mediaStream = await navigator.mediaDevices.getUserMedia({ video: true, audio: true });
Enter fullscreen mode

Exit fullscreen mode

  1. Create an RTCPeerConnection:

You can create an RTCPeerConnection with the help of Metered TURN Servers

var myPeerConnection = new RTCPeerConnection({
  iceServers: [
      {
        urls: "stun:stun.relay.metered.ca:80",
      },
      {
        urls: "turn:global.relay.metered.ca:80",
        username: "your-username",
        credential: "your-credential",
      },
      {
        urls: "turn:global.relay.metered.ca:80?transport=tcp",
        username: "your-username",
        credential: "your-credential",
      },
      {
        urls: "turn:global.relay.metered.ca:443",
        username: "your-username",
        credential: "your-credential",
      },
      {
        urls: "turns:global.relay.metered.ca:443?transport=tcp",
        username: "your-username",
        credential: "your-credential",
      },
  ],
});
Enter fullscreen mode

Exit fullscreen mode

  1. Add Media Tracks to the Connection: 
mediaStream.getTracks().forEach((track) => {
  pc.addTrack(track, mediaStream);
});
Enter fullscreen mode

Exit fullscreen mode

  1. Create and Send SDP Offer to WHIP Server: 
const offer = await pc.createOffer();
await pc.setLocalDescription(offer);

const response = await fetch('http://YOUR_SERVER_IP:3000/whip', {
  method: 'POST',
  headers: { 'Content-Type': 'application/sdp' },
  body: pc.localDescription.sdp,
});

const sdpAnswer = await response.text();
await pc.setRemoteDescription({ type: 'answer', sdp: sdpAnswer });
Enter fullscreen mode

Exit fullscreen mode



Handling HTTP Requests and Responses for Signalling

  • Client Side:

    • HTTP POST Request: 

    • Expecting Response:

      • Status: 201 Created
      • Headers: Location header with resource URL 
      • Body: SDP answer as plain text

  • Server Side:

    • Receive SDP Offer: 

      • Read the SDP from the req.body
      • Create WebRTC endpoint and set the remote description 

    • Generate SDP Answer

      • An SDP answer from the server WebRTC endpoint
      • Send SDP answer back in the res.body
      • Using Metered.ca TURN server service

Image description



Using Metered.ca TURN server service



Purpose of TURN Server

Facilitates media traversal through NAT and firewall when direct peer to peer connection is not possible



Incorporate TURN Server in ICE Server Configuration

Here is a TURN Server credential and ICE Server

var myPeerConnection = new RTCPeerConnection({
  iceServers: [
      {
        urls: "stun:stun.relay.metered.ca:80",
      },
      {
        urls: "turn:global.relay.metered.ca:80",
        username: "e13b9bsdfdsfsdfb0676cc5b6",
        credential: "dedewdewfer+gq5iT",
      },
      {
        urls: "turn:global.relay.metered.ca:80?transport=tcp",
        username: "e13bdfdsfds6b0676cc5b6",
        credential: "dewfrefre+gq5iT",
      },
      {
        urls: "turn:global.relay.metered.ca:443",
        username: "e13b9fsdfdsfsd86b0676cc5b6",
        credential: "csdfwefeer+gq5iT",
      },
      {
        urls: "turns:global.relay.metered.ca:443?transport=tcp",
        username: "e13b9dsfsdfe6b0676cc5b6",
        credential: "sdfewtrererer+gq5iT",
      },
  ],
});
Enter fullscreen mode

Exit fullscreen mode



Deploying a WHEP client

Having a WHIP client allows your app to receive media streams from the server using HTTP signalling.



Integrating WHEP on the client side

  • Basic understanding of WebRTC API in Javascript 

  • Media server that supports WHEP GStreamer Janus or any other

  • Metered.ca TURN server credentials



Step by step WHEP integration in your App.

  1. Initialize the RTCPeerConnection

    1. Create a RTCPeerConnection instance with the ICE server that has metered.ca turn servers 
var myPeerConnection = new RTCPeerConnection({
  iceServers: [
      {
        urls: "stun:stun.relay.metered.ca:80",
      },
      {
        urls: "turn:global.relay.metered.ca:80",
        username: "e13b9bsdfdsfsdfb0676cc5b6",
        credential: "dedewdewfer+gq5iT",
      },
      {
        urls: "turn:global.relay.metered.ca:80?transport=tcp",
        username: "e13bdfdsfds6b0676cc5b6",
        credential: "dewfrefre+gq5iT",
      },
      {
        urls: "turn:global.relay.metered.ca:443",
        username: "e13b9fsdfdsfsd86b0676cc5b6",
        credential: "csdfwefeer+gq5iT",
      },
      {
        urls: "turns:global.relay.metered.ca:443?transport=tcp",
        username: "e13b9dsfsdfe6b0676cc5b6",
        credential: "sdfewtrererer+gq5iT",
      },
  ],
});
Enter fullscreen mode

Exit fullscreen mode

  1. Handle incoming Media Tracks

Setup an event listener to revive remote tracks from the server

pc.addEventListener('track', (event) => {
  const [remoteStream] = event.streams;
  // Attach the remote stream to a video element
  const remoteVideo = document.getElementById('remoteVideo');
  remoteVideo.srcObject = remoteStream;
});
Enter fullscreen mode

Exit fullscreen mode

  1. Send an HTTP GET Request to the WHEP server:

Send a GET request to the server

const whepServerEndpoint = 'http://YOUR_SERVER_IP:3000/whep'; // Replace with your server's WHEP endpoint

const response = await fetch(whepEndpoint, {
  method: 'GET',
  headers: {
    Accept: 'application/sdp',
  },
});

const sdpOffer = await response.text();
Enter fullscreen mode

Exit fullscreen mode

  1. Remote description with the received SDP offer 
await pc.setRemoteDescription({
  type: 'offer',
  sdp: sdpOffer,
});
Enter fullscreen mode

Exit fullscreen mode

  1. Create and send the SDP answer

Create an SDP answer and send it to the server through a HTTP POST request

const sdpAnswer = await pc.createAnswer();
await pc.setLocalDescription(sdpAnswer);

await fetch(whepEndpoint, {
  method: 'POST',
  headers: {
    'Content-Type': 'application/sdp',
  },
  body: pc.localDescription.sdp,
});
Enter fullscreen mode

Exit fullscreen mode

  1. Handle ICE Candidate Exchange (Optional):

If you need to send ICE candidates separately then handle the icecandidate event

pc.addEventListener('icecandidate', (event) => {
  if (event.candidate) {
    // Send the candidate to the server through HTTP PATCH request
    fetch(whepEndpoint, {
      method: 'PATCH',
      headers: {
        'Content-Type': 'application/trickle-ice-sdpfrag',
      },
      body: event.candidate.candidate,
    });
  }
});
Enter fullscreen mode

Exit fullscreen mode



Handle Media streams on the front-end

  1. create a video element in HTML 
<video id="remoteVideo" autoplay playsinline></video>
Enter fullscreen mode

Exit fullscreen mode

  1. Attach remote stream to the video element

when a track event is a fired attach the recieved stream to the video element

  1. Handling the Media Stream Event

    1. Connection state change 
pc.addEventListener('connectionstatechange', () => {
  if (pc.connectionState === 'connected') {
    console.log('WebRTC connection established');
  } else if (pc.connectionState === 'failed' || pc.connectionState === 'disconnected') {
    console.error('WebRTC connection failed or disconnected');
  }
});
Enter fullscreen mode

Exit fullscreen mode

b. Negotiation needed

pc.addEventListener('negotiationneeded', async () => {
  try {
    const offer = await pc.createOffer();
    await pc.setLocalDescription(offer);
    // Send new SDP offer to the server if needed
  } catch (error) {
    console.error('Error during renegotiation:', error);
  }
});
Enter fullscreen mode

Exit fullscreen mode

  1. Full example code 
async function startWHEPClient() {
 var pc = new RTCPeerConnection({
  iceServers: [
      {
        urls: "stun:stun.relay.metered.ca:80",
      },
      {
        urls: "turn:global.relay.metered.ca:80",
        username: "e13b9bsdfdsfsdfb0676cc5b6",
        credential: "dedewdewfer+gq5iT",
      },
      {
        urls: "turn:global.relay.metered.ca:80?transport=tcp",
        username: "e13bdfdsfds6b0676cc5b6",
        credential: "dewfrefre+gq5iT",
      },
      {
        urls: "turn:global.relay.metered.ca:443",
        username: "e13b9fsdfdsfsd86b0676cc5b6",
        credential: "csdfwefeer+gq5iT",
      },
      {
        urls: "turns:global.relay.metered.ca:443?transport=tcp",
        username: "e13b9dsfsdfe6b0676cc5b6",
        credential: "sdfewtrererer+gq5iT",
      },
  ],
});

  pc.addEventListener('track', (event) => {
    const [remoteStream] = event.streams;
    const remoteVideo = document.getElementById('remoteVideo');
    remoteVideo.srcObject = remoteStream;
  });

  pc.addEventListener('connectionstatechange', () => {
    if (pc.connectionState === 'connected') {
      console.log('WebRTC connection established');
    }
  });

  const whepEndpoint = 'http://YOUR_SERVER_IP:3000/whep';

  try {
    const response = await fetch(whepEndpoint, {
      method: 'GET',
      headers: {
        Accept: 'application/sdp',
      },
    });

    const sdpOffer = await response.text();

    await pc.setRemoteDescription({
      type: 'offer',
      sdp: sdpOffer,
    });

    const sdpAnswer = await pc.createAnswer();
    await pc.setLocalDescription(sdpAnswer);

    await fetch(whepEndpoint, {
      method: 'POST',
      headers: {
        'Content-Type': 'application/sdp',
      },
      body: pc.localDescription.sdp,
    });

  } catch (error) {
    console.error('Error occoured during the WHEP connection:', error);
  }
}

// Start the WHEP client
startWHEPClient();
Enter fullscreen mode

Exit fullscreen mode

Image description

Metered TURN servers

  1. API: TURN server management with powerful API. You can do things like Add/ Remove credentials via the API, Retrieve Per User / Credentials and User metrics via the API, Enable/ Disable credentials via the API, Retrive Usage data by date via the API.

  2. Global Geo-Location targeting: Automatically directs traffic to the nearest servers, for lowest possible latency and highest quality performance. less than 50 ms latency anywhere around the world

  3. Servers in all the Regions of the world: Toronto, Miami, San Francisco, Amsterdam, London, Frankfurt, Bangalore, Singapore,Sydney, Seoul, Dallas, New York

  4. Low Latency: less than 50 ms latency, anywhere across the world.

  5. Cost-Effective: pay-as-you-go pricing with bandwidth and volume discounts available.

  6. Easy Administration: Get usage logs, emails when accounts reach threshold limits, billing records and email and phone support.

  7. Standards Compliant: Conforms to RFCs 5389, 5769, 5780, 5766, 6062, 6156, 5245, 5768, 6336, 6544, 5928 over UDP, TCP, TLS, and DTLS.

  8. Multi‑Tenancy: Create multiple credentials and separate the usage by customer, or different apps. Get Usage logs, billing records and threshold alerts.

  9. Enterprise Reliability: 99.999% Uptime with SLA.

  10. Enterprise Scale: With no limit on concurrent traffic or total traffic. Metered TURN Servers provide Enterprise Scalability

  11. 5 GB/mo Free: Get 5 GB every month free TURN server usage with the Free Plan

  12. Runs on port 80 and 443

  13. Support TURNS + SSL to allow connections through deep packet inspection firewalls.

  14. Supports both TCP and UDP

  15. Free Unlimited STUN

You can consider some of our other articles:

  1. WebRTC Data Channels: A Guide

  2. Simple Peer Tutorial: Add TURN Server for Video, DataChannel

  3. Guide to Setting Up Your WebRTC TURN Server with Metered

  4. WebRTC vs HLS: Which is Best for You?



Source link
lol

By stp2y

Leave a Reply

Your email address will not be published. Required fields are marked *

No widgets found. Go to Widget page and add the widget in Offcanvas Sidebar Widget Area.