RTP

Node.js

An input type that allows streaming video and audio to the Smelter server over RTP. It supports both streaming over UDP and TCP (smelter works as a TCP server).

Usage

rtpInputExample.tsx

import Smelter from "@swmansion/smelter-node";

async function run() {
  const smelter = new Smelter();
  await smelter.init();
  await smelter.registerInput("example", {
      type: "rtp_stream",
      port: 8001,
      transportProtocol: "tcp_server",
      video: { decoder: "ffmpeg_h264" },
      audio: { decoder: "opus" }
  });
  // At this point you can connect to 8001 TCP port
  // and start sending RTP traffic
}
void run();

Reference

Type definitions

type RegisterRtpInput = {
  type: "rtp_stream";
  port: string | number;
  transportProtocol?: "udp" | "tcp_server";
  video?: InputRtpVideoOptions;
  audio?: InputRtpAudioOptions;
  required?: bool;
  offsetMs?: number;
}

Parameters for registering an RTP stream as an input.

Note

At least one of video and audio has to be defined.

Properties

port

A port number or a port range in format START:END. If range is specified, a port from that range will be returned from registerInput.

• Type: string | number

---

transportProtocol

Transport protocol.

• Type: "udp" | "tcp_server"
• Supported values:
  • udp - UDP protocol.
  • tcp_server - TCP protocol where Smelter is the server side of the connection.

---

video

Parameters of a video included in the RTP stream.

• Type: InputRtpVideoOptions

---

audio

Parameters of an audio source included in the RTP stream.

• Type: InputRtpAudioOptions

---

required

Determines if the input stream is essential for output frame production. If set to true and the stream is delayed, Smelter will postpone output frames until the stream is received.

• Type: boolean
• Default value: false

---

offsetMs

Offset in milliseconds relative to the pipeline start (start request). If unspecified, the stream synchronizes based on the delivery time of the initial frames.

• Type: number

InputRtpVideoOptions

Type definitions

type InputRtpVideoOptions = {
  decoder: "ffmpeg_h264" | "vulkan_video";
}

Parameters of a video source included in the RTP stream.

Properties

decoder

Video decoder.

• Type: "ffmpeg_h264" | "vulkan_video"
• Supported values:

  • "ffmpeg_h264" - Use the software decoder based on ffmpeg.

  • "vulkan_video" (Required feature:vk-video ) - Use hardware decoder based on Vulkan Video.

    This should be faster and more scalable than the ffmpeg decoder, if the hardware and OS support it.

    This requires hardware that supports Vulkan Video. Another requirement is this program has to be compiled with the vk-video feature enabled (enabled by default on platforms which support Vulkan, i.e. non-Apple operating systems and not the web).

InputRtpAudioOptions

Type definitions

type InputRtpAudioOptions =
| {
    decoder: "opus";
    forwardErrorCorrection?: bool;
  }
| {
    decoder: "aac";
    audioSpecificConfig: string;
    rtpMode?: "low_bitrate" | "high_bitrate";
  }

Parameters of an audio source included in the RTP stream.

Properties (decoder: “opus”)

forwardErrorCorrection

Specifies whether the stream uses forward error correction. It’s specific for the Opus codec. For more information, visit RFC specification.

• Type: boolean
• Default value: false

Properties (decoder: “aac”)

audioSpecificConfig

Configuration encoded in the format described in RFC 3640.

• Type: string

For detailed instructions on obtaining this value, refer to the information provided in the table below:

Format/Protocol	Location of AAC Specific Config (ASC)	Notes
FFmpeg Streaming	SDP file	Use the -sdp_file FILENAME option when streaming to a smelter to generate an SDP file containing the ASC.
MP4 Files	Inside the esds box	The ASC is embedded as part of the esds box, not the entire box. Applies to regular MP4 and fragmented MP4s (used in HLS playlists with MP4 files).
FLV Files / RTMP	Inside the AACAUDIODATA tag	The ASC is contained within the AACAUDIODATA tag.

rtpMode

Specifies the RFC 3640 mode that should be used when depacketizing this stream. For more information, visit RFC specification

• Type: "low_bitrate" | "high_bitrate"
• Default value: "high_bitrate"