Overview

This is the real-time video editing capability that Vidu S offers to enterprise customers: the model continuously receives the video stream you publish and outputs an edited video stream in real time. It is streaming by nature — frames are edited as they are pushed, and you can change the reference image or switch the editing scenario at any time without interrupting the video stream.

The four capabilities	Description
Style rendering `style_transfer`	Redraws the live picture in the style of the reference image. Commonly used for anime-style or artistic live streams.
Character replacement `subject_replacement`	Replaces the person in the picture with the character from the reference image, preserving the original motion, expressions, and camera movement.
Background replacement `background_replacement`	Replaces the background with the scene from the reference image, while preserving the subject and foreground composition.
Virtual try-on `virtual_tryon`	Dresses the person in the picture according to the reference image, while preserving the person’s posture and body shape.

Minimal integration flow

Create an editing stream

Create over HTTP and save live_id / rtc info

After obtaining an API Key, call the create endpoint to obtain all the credentials you need for the steps that follow.

curl -X POST "https://api.vidu.cn/live/s_editing/realtime" \
  -H "Authorization: vda_<your_api_key>" \
  -H "Content-Type: application/json" \
  -d '{
    "image_url": "https://example.com/reference.png",
    "editing_type": "style_transfer"
  }'

*image_url: the reference image. It accepts an http(s) URL, a base64 data URI (local file selection is converted automatically), or a platform upload URI. The server returns live (the live stream ID — use it as live_id in the signals and URLs below), client_secret, rtc (the AliRTC join credentials), and render_uid** (the user ID of the rendered stream — once you have joined the channel, subscribe to that user's video stream to get the composited result).*

Connect, join, and go live

Publish input over AliRTC and subscribe to the result

First establish the WebSocket control connection. A successful connection does not mean you are live:

# Browser: pass client_secret as a query parameter
wss://api.vidu.cn/live/ws/live/connect?live_id=<live_id>&conn_id=<conn_id>&client_secret=<client_secret>

# Server-side integration: Authorization header
wss://api.vidu.cn/live/ws/live/connect?live_id=<live_id>&conn_id=<conn_id>
Authorization: vda_<your_api_key>

Send the init signal immediately after the connection is established. The server will not say anything on its own; the client must send this frame.

{
  "type": 1,
  "live_id": "995200676634042368",
  "conn_id": "conn-xxx",
  "seq_id": 1,
  "payload": { "conn_init": { "version": 1 } }
}

The server returns conn_init_ack.success=true* once both the client and the rendering service are ready. If RTMP relay is enabled, the relay has started at that point. If it returns error_code: "NOT_READY", the rendering service is not ready yet and the client should resend the init signal a moment later. Retries are subject to a server-side budget; once that budget is exhausted the live stream ends and you have to create a new one.* Then join the RTC channel. Use the rtc credentials returned by the create call to invoke AliRTC's joinChannel, and publish your camera stream (your camera feed, or a local video file) as the editing input:

// 1. Join the channel; the second argument is the display name, and rtc.user_id works directly.
await aliRtc.joinChannel(rtc.token, rtc.user_id);
// 2. Publish the editing input (camera stream).
await aliRtc.publishLocalVideoStream(true);
// 3. Subscribe to the rendered result: enable automatic subscription to remote video,
//    then, once videoSubscribeStateChanged becomes subscribed(3),
//    use setRemoteViewConfig to bind the camera stream of render_uid to the playback container.
await aliRtc.setDefaultSubscribeAllRemoteVideoStreams(true);
aliRtc.on("videoSubscribeStateChanged", (userId, oldState, newState) => {
  if (userId === render_uid && newState === 3) {
    aliRtc.setRemoteViewConfig(videoEl, userId, 1); // 1 = camera stream
  }
});

Before playing the edited result, wait for the subscription state to become subscribed, then bind the remote video to your playback container.

Changing the reference image during a live stream

WebSocket switch_prompt signal

To change the reference image, send the switch_prompt signal during the live stream. The picture switches within a few seconds:

{
  "type": 13,
  "live_id": "995200676634042368",
  "conn_id": "conn-xxx",
  "seq_id": 2,
  "payload": {
    "switch_prompt": {
      "prompts": [{ "type": "image", "content": "https://example.com/new-reference.png" }],
      "editing_type": "virtual_tryon"
    }
  }
}

No acknowledgement is returned on success; on failure, a type=14 acknowledgement carrying error_code / error_msg.

Protocol overview

End-to-end flow

Call the "create live stream" endpoint to obtain live_id, the WebSocket credential client_secret, the RTC join credentials rtc, and the rendered-stream user ID render_uid.
Use client_secret to establish the WebSocket control connection, send the init signal, and wait for the server's acknowledgement (the live stream starts and billing begins once it succeeds).
Use the rtc credentials to join the real-time audio/video channel (Alibaba Cloud RTC, AliRTC), and publish your video stream (camera stream or local video file) inside the channel as the editing input.
Subscribe inside the channel to the render_uid user's video stream — that is the real-time edited result.
During the live stream, send the image-switch signal as needed to change the reference image or editing scenario. When finished, send the hangup signal; the live stream ends and billing stops.

Endpoint summary

Endpoint	Method and path	Purpose
POST	`/live/s_editing/realtime`	Create a real-time editing live stream (the only entry point; returns all credentials, including live_id)
WebSocket	`/live/ws/live/connect`	Control signals (init, image switch, hangup)
—	AliRTC `joinChannel`	Real-time media (publish the editing input / subscribe to the composited result)

General conventions

Item	Convention
Environment	China (mainland): `api.vidu.cn`. Overseas: `api.vidu.com`. The WebSocket domain allowlist is `api.vidu.cn` / `api.vidu.com`.
Authentication	HTTP requests carry `Authorization: vda_<your_api_key>` in the header (without the `Token` prefix). WebSocket connections use the same header; when a browser cannot set custom WebSocket headers, pass the `client_secret` returned by the create endpoint as a query parameter instead. API keys are available in the console.
Data format	`Content-Type: application/json`; Time fields are standardized as Unix timestamp (seconds) strings.
Request ID	Optional `X-Request-ID` header (a unique identifier of up to 36 characters; a UUID is recommended), used only for request tracing and troubleshooting.
Errors	Non-2xx responses use the `code` / `reason` / `message` / `metadata` structure; `metadata` may carry a `trace_id`. Insufficient account credits return `CreditInsufficient`; exceeding the concurrency limit returns 429.
Billing	Billed by actual live duration in seconds. Billing is measured from the moment `conn_init_ack.success=true` is received (the stream enters `on_live`) and ends at hangup. Establishing the connection alone or joining RTC alone is not billed. The image-switch signal is not billed separately.
Reconnection	If the connection drops unexpectedly, you can reconnect within the grace period using the same `live_id` and the same `conn_id`. After reconnecting, you must resend `conn_init`. If you do not reconnect within the grace period, the live stream ends and billing stops. Contact platform support for the exact grace period.