Quick start

In this guide, you’ll learn how to create a basic Smelter setup using HTTP API. We will show you how to take two input videos and produce an output video with both of the inputs displayed side-by-side.

Start the Smelter server (learn more). Check out configuration page for available configuration options.

POST: /api/output/output_1/register
Content-Type: application/json

{
  "type": "rtp_stream",
  "transport_protocol": "udp",
  "port": 9004,
  "ip": "127.0.0.1",
  "video": {
    "resolution": { "width": 1280, "height": 720 },
    "encoder": { "type": "ffmpeg_h264" },
    "initial": {
      "root": { "type": "view" }
    }
  },
  "audio": {
    "encoder": { "type": "opus", "channels": "stereo" },
    "initial": {
      "inputs": []
    }
  }
}

POST: /api/start
Content-Type: application/json

{}

This step creates an output that renders just an empty View component. Resulting blank image is encoded and sent over UDP to local port 9004 via RTP protocol.

Sending start request will cause all registered outputs to start producing frames. All user-facing timestamp or offset are relative to this start call.

How to receive and display this stream?

Above example sends that over UDP, so you need to start listening on that port before registering the output.

GStreamer

To view the output stream with GStreamer run the following command:

gst-launch-1.0 rtpptdemux name=demux \
  udpsrc port=9004 ! "application/x-rtp" ! queue ! demux. \
  demux.src_96 ! "application/x-rtp,media=video,clock-rate=90000,encoding-name=H264" ! queue \
  ! rtph264depay ! decodebin ! videoconvert ! autovideosink \
  demux.src_97 ! "application/x-rtp,media=audio,clock-rate=48000,encoding-name=OPUS" ! queue \
  ! rtpopusdepay ! decodebin ! audioconvert ! autoaudiosink sync=false

FFmpeg (`ffplay`)

To view the output stream with FFmpeg:

Create the SDP file.

v=0
o=- 0 0 IN IP4 127.0.0.1
s=No Name
c=IN IP4 127.0.0.1
m=video 9004 RTP/AVP 96
a=rtpmap:96 H264/90000
a=fmtp:96 packetization-mode=1
a=rtcp-mux

Run the following command

ffplay -protocol_whitelist "file,rtp,udp" output.sdp

Note: FFmpeg does not support multiplexing audio and video on the same port, so this example will only work with video.

Or checkout other alternative protocols:

Option 1: WHIP output

With WHIP output you can send it e.g. to Twitch.

POST: /api/output/output_1/register
Content-Type: application/json

{
  "type": "whip",
  "endpoint_url": "https://g.webrtc.live-video.net:4443/v2/offer",
  "bearer_token": "<YOUR STREAM TOKEN>",
  "video": {
    "resolution": { "width": 1280, "height": 720 },
    "encoder": { "type": "ffmpeg_h264" },
    "initial": {
      "root": { "type": "view" }
    }
  },
  "audio": {
    "encoder": { "type": "opus", "channels": "stereo" },
    "initial": {
      "inputs": []
    }
  }
}

Option 2: MP4 output

With mp4 output you can record the output to file.

POST: /api/output/output_1/register
Content-Type: application/json

{
  "type": "mp4",
  "path": "output.mp4",
  "video": {
    "resolution": { "width": 1280, "height": 720 },
    "encoder": { "type": "ffmpeg_h264" },
    "initial": {
      "root": { "type": "view" }
    }
  },
  "audio": {
    "encoder": { "type": "aac", "channels": "stereo" },
    "initial": {
      "inputs": []
    }
  }
}

Note that mp4 must be gracefully terminated to write metadata, so if you just kill the application the file will be corrupted. When you want to finish recording, send unregister request:

POST: /api/output/output_1/unregister
Content-Type: application/json

Register 2 input streams.

For simplicity, in this example that will be two mp4 files, but the same logic would apply for any other input type.
```
POST: /api/input/input_1/register
Content-Type: application/json

{
  "type": "mp4",
  "path": "https://example.com/input1.mp4"
}
```
```
POST: /api/input/input_2/register
Content-Type: application/json

{
  "type": "mp4",
  "path": "https://example.com/input2.mp4"
}
```
Those 2 request will register input streams input_1 and input2. However, new streams are not used anywhere in the scene definition yet, so it will not change the image produced on an output stream from the previous step.

Update output to show one of the input streams.

POST: /api/output/output_1/update
Content-Type: application/json

{
  "video": {
    "root": {
      "type": "input_stream",
      "input_id": "input_1"
    }
  },
  "audio": {
    "inputs": [
      { "input_id": "input_1", "volume": 0.9 }
    ]
  }
}

Resulting output stream will show input_1 input stream and play the audio from that source with slightly lowered volume. (If input has a different resolution it won’t be automatically rescaled).

Update output to show both of the input streams.

Tiles component is a simple way to display element in a layout that uses available space efficiently (similar to stream layout in most video call apps).

POST: /api/output/output_1/update
Content-Type: application/json

{
  "video": {
    "root": {
      "type": "tiles",
      "background_color": "#4d4d4d",
      "children": [
        { "type": "input_stream", "input_id": "input_1" },
        { "type": "input_stream", "input_id": "input_2" }
      ]
    }
  },
  "audio": {
    "inputs": [
      { "input_id": "input_1", "volume": 0.9 },
      { "input_id": "input_2" }
    ]
  }
}

Resulting output stream will show both inputs side-by-side as shown below. Audio from both streams is mixed together, but input_1 audio has slightly lowered volume.

Quick start

GStreamer

FFmpeg (ffplay)

FFmpeg (`ffplay`)