Skip to content
Smelter logo Smelter logo Smelter

WHIP client

Node.js Browser (Client)

An output that lets you connect to a WHIP server endpoint to stream video and audio to it.

Usage

whipClientOutputExample.tsx
import Smelter from "@swmansion/smelter-node";
import { View } from "@swmansion/smelter";


async function run() {
  const smelter = new Smelter();
  await smelter.init();
  await smelter.registerOutput("example", <View />, {
    type: "whip_client",
    endpointUrl: "https://example.com/whip",
    bearerToken: "<TOKEN>",
    video: {
      encoderPreferences: [
        { type: "ffmpeg_h264" },
        { type: "any" },
      ],
      resolution: { width: 1920, height: 1080 },
    },
    audio: true,
  });
}
void run();

Reference

Type definitions 

type RegisterWhipClientOutput = {
  type: "whip_client";
  endpointUrl: string;
  bearerToken?: string;
  video?: VideoOptions;
  audio?: true | AudioOptions;
};

Parameters for an WHIP output stream from Smelter.

Properties

endpointUrl

The destination URL for sending media streams using the WHIP.

  • Type: string

bearerToken

Token used for authentication when connecting to the WHIP endpoint.

  • Type: string

video

Parameters of a video included in the WHIP stream.

audio

Parameters of an audio included in the WHIP stream.

When set to true, audio will be enabled without requiring explicit AudioOptions - they will be set automaticly based on negotation.

VideoOptions

Type definitions 

type VideoOptions = {
  resolution: {
    width: number;
    height: number;
  };
  sendEosWhen?: OutputEndCondition;
  encoderPreferences?: VideoEncoderOptions[];
};

Parameters for registering an output that sends composed video/audio as an WHIP stream.

Properties

resolution

Output resolution in pixels.

  • Type: { width: number; height: number;}

sendEosWhen

Condition for termination of the output stream based on the input streams states. If output includes both audio and video streams, then EOS needs to be sent for every type.

encoderPreferences

An ordered list of preferred video encoders. The first element in the list has the highest priority during WHIP negotiation.

Behavior:

  • If the list ends with “any”:

    • Smelter will first try the encoders explicitly listed (in order) and use the first one that is supported and negotiated in WHIP signaling.
    • If none of the listed encoders are supported, Smelter will fall back to any supported codec from the negotiated list that wasn’t already in the preferences.

  • If “any” is not included:

    • Only the encoders listed will be considered.
    • If none are supported, no fallback will occur.

VideoEncoderOptions

Type definitions 

type VideoEncoderOptions =
  | {
      type: "ffmpeg_h264";
      preset?:
        | "ultrafast"
        | "superfast"
        | "veryfast"
        | "faster"
        | "fast"
        | "medium"
        | "slow"
        | "slower"
        | "veryslow"
        | "placebo";
      pixelFormat?: "yuv420p" | "yuv422p" | "yuv444p";
      ffmpegOptions?: Record<string, string>;
    }
  | {
      type: "ffmpeg_vp8";
      ffmpegOptions?: Record<string, string>;
    }
  | {
      type: "ffmpeg_vp9";
      pixelFormat?: "yuv420p" | "yuv422p" | "yuv444p";
      ffmpegOptions?: Record<string, string>;
    }
  | {
      type: "vulkan_h264";
      bitrate?: { averageBitrate: number; maxBitrate: number; } | number;
    }
  | {
      type: "any";
    };

Configuration for the video encoder, based on the selected codec.

Properties (type: “ffmpeg_h264”)

preset

Video output encoder preset. Visit FFmpeg docs to learn more.

  • Type: "ultrafast" | "superfast" | "veryfast" | "faster" | "fast" | "medium" | "slow" | "slower" | "veryslow" | "placebo"
  • Default value: fast
  • Supported values:
    • ultrafast
    • superfast
    • veryfast
    • faster
    • fast
    • medium
    • slow
    • slower
    • veryslow
    • placebo

pixelFormat

Encoder pixel format

  • Type: "yuv420p" | "yuv422p" | "yuv444p"
  • Default value: yuv420p
  • Supported values:
    • yuv420p
    • yuv422p
    • yuv444p

ffmpegOptions

Raw FFmpeg encoder options. Visit FFmpeg docs to learn more.

  • Type: Record<string, string>

Properties (type: “ffmpeg_vp8”)

ffmpegOptions

Raw FFmpeg encoder options. Visit FFmpeg docs to learn more.

  • Type: Record<string, string>

Properties (type: “ffmpeg_vp9”)

pixelFormat

Encoder pixel format

  • Type: "yuv420p" | "yuv422p" | "yuv444p"
  • Default value: yuv420p
  • Supported values:
    • yuv420p
    • yuv422p
    • yuv444p

ffmpegOptions

Raw FFmpeg encoder options. Visit FFmpeg docs to learn more.

  • Type: Record<string, string>

Properties (type: “vulkan_h264”) Required feature: vk-video

Hardware encoder. Requires GPU that supports Vulkan Video encoding.

bitrate

Encoding bitrate. If not provided, bitrate is calculated based on resolution and framerate. For example at 1080p 30 FPS the average bitrate is 5000 kbit/s and max bitrate is 6250 kbit/s.

Type definitions 

type VulkanH264EncoderBitrate = { averageBitrate: number; maxBitrate: number; } | number;

Properties

averageBitrate

Average bitrate measured in bits/second. Encoder will try to keep the bitrate around the provided average, but may temporarily increase it to the provided max bitrate.

maxBitrate

Max bitrate measured in bits/second.

AudioOptions

Type definitions 

type AudioOptions = {
  channels?: "mono" | "stereo";
  mixingStrategy?: "sum_clip" | "sum_scale";
  sendEosWhen?: OutputEndCondition;
  encoderPreferences?: AudioEncoderOptions[];
}

Parameters of an audio source included in the WHIP stream.

Properties

channels

Channels configuration

  • Type: "mono" | "stereo"
  • Default value: "stereo"
  • Supported values:
    • mono - Mono audio (single channel).
    • stereo - Stereo audio (two channels).

mixingStrategy

Specifies how audio should be mixed.

  • Type: "sum_clip" | "sum_scale"
  • Default value: "sum_clip"
  • Supported values:
    • sum_clip - First, the input samples are summed. If the result exceeds the i16 PCM range, it is clipped.
    • sum_scale - First, the input samples are summed. If the result exceeds the i16 PCM range, the summed samples are scaled down by a factor to fit within the range.

sendEosWhen

Condition for termination of the output stream based on the input streams states. If output includes both audio and video streams, then EOS needs to be sent for every type.

encoderPreferences

An ordered list of preferred audio encoders. The first element in the list has the highest priority during WHIP negotiation.

Behavior:

  • If the list ends with “any”:

    • Smelter will first try the encoders explicitly listed (in order) and use the first one that is supported and negotiated in WHIP signaling.
    • If none of the listed encoders are supported, Smelter will fall back to any supported codec from the negotiated list that wasn’t already in the preferences.

  • If “any” is not included:

    • Only the encoders listed will be considered.
    • If none are supported, no fallback will occur.

AudioEncoderOptions

Type definitions 

type AudioEncoderOptions = {
  type: "opus";
  preset?: "quality" | "voip" | "lowest_latency";
  sampleRate?: number;
  forwardErrorCorrection?: boolean;
} | {
  type: "any";
}

Configuration for the audio encoder, based on the selected codec.

Properties(type: “opus”)

preset

Audio output encoder preset.

  • Type: "quality" | "voip" | "lowest_latency"
  • Default value: voip
  • Supported values:
    • quality - Recommended for broadcast and high-fidelity applications requiring decoded audio to maintain maximum fidelity to the input signal.
    • voip - Recommended for VoIP and videoconferencing applications, prioritizing listening quality and speech intelligibility.
    • lowest_latency - Recommended only when achieving the lowest possible latency is the highest priority.

sampleRate

Sample rate for audio encoder.

  • Type: number
  • Default value: 48000
  • Supported values: 8000, 16000, 24000, 48000.

forwardErrorCorrection

When enabled, include in-band forward error correction data to protect against packet loss. For more information, visit RFC specification sections 2.1.7 and 4.2.5

  • Type: bool
  • Default value: false

Properties(type: “any”)

Specifies that any audio encoder supported by Smelter can be used. If "any" is not included in the encoderPreferences list, Smelter will only use the encoders explicitly listed in the preferences.

OutputEndCondition

Type definitions 

type OutputEndCondition =
  | { anyOf: string[]; }
  | { allOf: string[]; }
  | { anyInput: boolean; }
  | { allInputs: boolean; };

Defines when the output stream should end based on the state of the input streams. Only one of the nested fields can be set at a time.

By default, the input stream is considered finished/ended when:

  • TCP connection was dropped/closed.
  • RTCP Goodbye packet (BYE) was received.
  • MP4 track has ended.
  • Input was unregistered already (or never registered).

Properties

anyOf

List of the input streams. The output stream will terminate if any stream in the list finishes.

  • Type: string[]

allOf

List of the input streams. The output stream will terminate when all streams in the list finish.

  • Type: string[]

anyInput

Terminate the output stream if any of the input streams end, including streams added after the output was registered. Notably, the output stream will not terminate if no inputs were ever connected.

  • Type: boolean

allInputs

Terminate the output stream only when all input streams have finished. Notably, the output stream will terminate if no inputs were ever connected.

  • Type: boolean

We use cookies

We use cookies to persist your settings and analyze our traffic. Please select your preferences.

Cookie Preferences