Behind the Scenes

Let’s take another look at the RTMP to WebRTC sample from Getting Started (TL;DR Version).

const norsk = await Norsk.connect();  

const input = await norsk.input.rtmpServer({ id: "rtmpInput" }); 
const output = await norsk.output.whep({ id: "webrtc", ...webRtcServerConfig }); 

output.subscribe([{ source: input, sourceSelector: selectAV }]); 
console.log(`WebRTC Player URL: ${output.playerUrl}`);

It’s easy to check what Norsk is up to. As well as the extensive OpenTelemetry instrumentation it provides, Norsk has a built-in real-time workflow visualizer. By default, it’s available on http://127.0.0.1:6791/visualiser (replace 127.0.0.1 with the address of your Norsk host if it isn’t running locally).

If you open up the visualizer while your RTMP to WebRTC application is running you’ll see this:

visualiser

It’s showing that Norsk is doing exactly what we asked - taking the audio and video from the RTMP source and passing it to a WebRTC output. Great!

Panic!!! That’ll Never Work!!!

Those of you familiar with media container formats may have already thrown your hands up in horror. The inbound RTMP audio will likely be AAC or MP3 format, whereas WebRTC requires Opus audio (a format that RTMP does not even support). Square peg meets round hole - not good.

If you didn’t happen to know that, it’s not a problem, because Norsk Media does.

A Little Further Behind the Scenes

Looking back at that visualiser screenshot, note the Show conversions checkbox. Below is what you’ll see if you click on that:

visualiser conversions

Panic Over

When you create a workflow, Norsk Media analyses it for any places where things might not line up. In this example, Norsk is well aware that the audio encoding will definitely need changing. WebRTC requires Opus - RTMP does not even support it. So before handing any audio to the WebRTC server for publication, it will first decode the audio received over RTMP, resample/reformat it as necessary, and then encode it as Opus.

It also knows that it might need to transcode the incoming video, so will check the video format received and, only if necessary, convert it into a WebRTC-compatible form. In this particular case, the video being sent over RTMP is using the H.264 codec which is also supported by WebRTC.

There is however a small tweak Norsk had to make to ensure the media flowed as expected. The video stream it received was MP4 Encapsulated whereas WebRTC wants AnnexB Encapsulation. Just changing the encapsulation is vastly less effort than a full decode and re-encode and also produces higher quality output than a transcode would - so that’s what Norsk did.

You are in good company (me for one!) if you weren’t fully across all the details from the previous paragraph (Surely an annex is something attached to a building??? What’s that got to do with video).

The great thing is that you don’t need to worry about any of that!

Norsk is a Media Expert, So You Don’t Have To be

With Norsk, you can focus on delivering your end-to-end media workflow and on your business logic. If you are interested, Norsk makes it easy to dive into the detail of what it is having to do to deliver the media flow you requested. Further, Norsk Media allows you to control those steps if you want. You could, for example, add an explicit Opus encode into your workflow to control the bitrate of the output.