Vidu S1 API（Beta）

Vidu S1 API（Beta） | Vidu API

On this page

On this page

Vidu S1

VIDU S1 · A World-Leading Real-Time Interactive Model

Vidu S1 is a real-time interactive video generation model that supports voice-controlled custom digital characters, unlimited-duration video generation, and high-frame-rate output.

Core flow: Create a Live session (HTTP) → obtain the RTC join information and live_id → establish two connections in parallel: WebSocket sends the start signal (control) + AliRTC joins the channel to send and receive audio and video (media) → begin interaction after both connections are ready → send the hang-up signal and leave the channel when finished.

Concept	Description
Live (Session)	A complete digital character interaction; obtain live_id
RTC Channel	audio and video transmission channel carrying microphone, camera, and digital character audio and video
WebSocket Signaling	Control-command channel for starting, text, interruption, and hang-up
Persona	Digital character personality, speaking style, and other settings

Vidu S1 Digital Character Live Quick Start Vidu S1 Digital Character Quick Start

Step 1: Create a Session

Create API-Key, you can create a session.

Request Example

POST https://{host}/live/v1/lives
Authorization: Token vda_xxx
Content-Type: application/json

{
  "call_mode": "video",
  "avatar": {
    "persona": "You are a friendly customer service representative. Interact naturally with the user in real time.",
    "image_uri": "https://your-digital-character-image-url.png",
    "voice": ""
  }
}

Save live.id from the response. It will be used later for WebSocket, billing queries, and text messages.
Save rtc.token and rtc.user_id from the response. They will be used to joining AliRTC in Step 3.

Step 2: Establish the WebSocket Connection and Send the Start Signal

First, establish the connection.

wss://{host}/live/ws/live/connect?live_id={live_id}
Authorization: Token vda_xxx

Send conn_init immediately after the connection is established:

{
  "type": 1,
  "live_id": "123456789",
  "seq_id": 1,
  "payload": { "conn_init": { "version": 1 } }
}

When conn_init_ack.success=true is received, the control connection has been initialized.
If NOT_READY is received in video mode, wait 2–3 seconds and retry conn_init.

Step 3: Connect to RTC

The Alibaba Cloud RTC SDK is used for audio and video transmission. Download link:Alibaba Cloud RTC SDK Documentation

await aliRtc.joinChannel(rtc.token, rtc.user_id);
await aliRtc.publishLocalAudioStream(true);
if (callMode === 'video') {
  await aliRtc.publishLocalVideoStream(true);
}

Mode	App Publishing / Subscription
audio	Join live-audio-{liveID}, publish microphone audio, and subscribe to digital character audio at live-bot-....
video	Join live-user-{liveID}, publish microphone and camera streams, and subscribe to digital character audio and video at live-video-push-....

Note: To display the digital character, you must also subscribe to the remote audio and video and bind the remote video container after the subscription status changes to subscribed. See “RTC Channel-Joining Protocol: AliRTC joinChannel” for the complete subscription and rendering code.

API Overview

The table below lists all public interfaces. Locate the entry point by purpose, then see the relevant subsection for the detailed protocol. All interfaces require Authorization: Token vda_xxx

Method	Path	Purpose
POST	/live/v1/lives	Create a Live session (core entry point; returns RTC join and WS initialization information)
GET	/live/v1/lives/{live_id}	Query the status and billing of a single Live session
GET	/live/v1/lives	List Live Sessions
WebSocket	/live/ws/live/connect	App control signaling (start / text / interrupt / hang up)
—	AliRTC joinChannel	Real-time media streams (microphone / camera / digital character audio and video); requires AliRTC SDK integration
POST	/live/v1/voices/clone	Clone a voice to create a custom voice
GET	/live/v1/voices	Query the custom voice list
POST / PUT	/tools/v2/files/uploads	Upload an image and obtain a URI usable as avatar.image_uri

General Conventions

Item	Convention
{host}	{host} is the domain name without a protocol. The China environment uses api.vidu.cn, and the international environment uses api.vidu.com. Use https://{host} for HTTP requests and wss://{host} for WebSocket.
Authentication	All app-side HTTP and WebSocket interfaces require Authorization: Token vda_xxx。The API key is available in the Console
JSON Fields	Request and response fields use snake_case，for example call_mode、live_id、token_expire_at。
IDs and Time	live_id、character_id、created_at、token_expire_at and other int64 fields should be handled as strings on the frontend to avoid JavaScript number precision issues.
Media Transmission	App WebSocket carries control signaling only; the microphone, camera, and digital character audio and videoare all transmitted through AliRTC.

{
  "code": 400,
  "reason": "BAD_REQUEST",
  "message": "FieldInvalid",
  "metadata": {}
}

Note: Error codes and recommended handling are maintained centrally in Section 9, “Error Responses and Compatibility Notes.” This section only describes the HTTP error-response structure to avoid inconsistencies caused by maintaining error-code tables in multiple places.

Request:

POST https://{host}/live/v1/lives
Authorization: Token vda_xxx
Content-Type: application/json

Complete Request Body Example:

{
  "call_mode": "video",
  "character_id": "1",
  "avatar": {
    "persona": "You are Sweet Tina. My voice is like warm milk tea—sweet and comforting—but I am always clear and capable when solving problems!",
    "image_uri": "https://scene.cf.vidu.studio/media-asset/070302-ZFkgvJxBTM0ZQJoO.png",
    "voice": "Tina"
  },
  "moderation": "disabled",
  "extra_motion": true,
  "audio": {
    "enable_transcription": true
  },
  "vad": {
    "type": "semantic",
    "threshold": 0.5,
    "silence_duration_ms": 200,
    "idle_timeout_ms": 500
  },
  "llm": {
    "temperature": 0.7,
    "top_p": 0.8,
    "top_k": 20,
    "frequency_penalty": 1,
    "presence_penalty": 0.3,
    "seed": -1,
    "max_tokens": 50
  },
  "idle_timeout_seconds": 7200,
  "memory_retrieval": {
    "enabled": true,
    "endpoint": "https://api.example.com/memory/search",
    "authorization": "Bearer memory-api-token",
    "timeout_ms": 3000
  },
  "knowledge_retrieval": {
    "enabled": true,
    "endpoint": "https://api.example.com/knowledge/search",
    "authorization": "Bearer knowledge-api-token",
    "timeout_ms": 3000
  }
}

Basic Field Descriptions

Field	Type	Required	Description
call_mode	String	Yes	Interaction mode: audio for audio only; video for audio and video
avatar.persona	String	Yes	Digital character persona, up to 50,000 Chinese characters.
avatar.image_uri	String	Yes	digital character image image URL. Supports an image URL, Base64, or image_uri (see the Image Upload section). Formats: PNG / JPG / JPEG / WEBP; image size ≤ 50 MB; byte length after Base64 decoding < 20 MB
avatar.voice	String	No	Voice; default: Tina
avatar.greeting_instruction	string	No	Opening greeting prompt, ≤ 200 characters

audio: Speech-to-Text

Field	Type	Default	Description
audio.enable_transcription	bool	false	When true, returns user speech-to-text and digital character output audio-to-text results through WebSocket.

vad: Interruption Control

Field	Type	Default	Description
vad.type	String	server	server：filters backchannel responses / background noise; semantic: interrupts the current response as soon as the user starts speaking
vad.threshold	float	0.5	Noise suppression strength, range [0, 1.0]，lower values apply less suppression
vad.silence_duration_ms	int	400	Amount of silence after speech ends before triggering a response; range [200, 6000]ms。
vad.idle_timeout_ms	int	0	The model proactively starts a conversation; 0 or [500, 30000] ms; 0 means no proactive conversation

llm: Language Model Output Control

Field	Type	Default	Description
llm.temperature	float	0.7	Creativity, range [0.0, 2.0)
llm.top_p	float	0.8	Quality threshold, range (0.0, 1.0]
llm.top_k	int	20	range [0, 100]
llm.frequency_penalty	float	1.0	Content repetition; default 1.0 means no penalty；< 1 encourages repetition；> 1 penalizes repetition
llm.presence_penalty	float	0.3	Topic breadth, range [0, 2]
llm.seed	int	-1	Range: -1 or [0–2³¹]; default -1 means random
llm.max_tokens	int	50	Maximum model output length，range 1 ~ 64000

idle_timeout_seconds: Session Lifecycle

Field	Type	Default	Description
idle_timeout_seconds	int	7200	Automatic disconnection time when there is no input. Default: 7200 s; range [10 ~ 7200]s

memory_retrieval / knowledge_retrieval: External Memory and Knowledge Retrieval (Beta)

memory_retrieval and knowledge_retrieval are optional fields that allow the digital character to call external memory retrieval and external knowledge retrieval during a conversation. See the “External Memory API” and “External Knowledge API” sections for details.

Field	Type	Default	Description
*.enabled	bool	None	Required; whether to enable this capability
*.endpoint	String	None	Required when enabled; retrieval API URL
*.authorization	String	None	Required when enabled; forwarded as the external API Authorization header and validated as non-empty
*.timeout_ms	int	30000	Request timeout; maximum 30000ms

You will receive:

{
  "live": {
    "id": "123456789",
    "status": "waiting",
    "live_duration": 600,
    "call_mode": "video"
  },
  "rtc": {
    "app_id": "xxxx",
    "channel_id": "live-user-123456789",
    "user_id": "live-user-1001-123456789",
    "token": "base64-token...",
    "token_expire_at": "1750003600"
  }
}

Field	Type	Description
live.id	String	Room ID
live.status	String	waiting means the digital character is not ready
live.live_duration	Int	Maximum duration of this session; automatically disconnects on timeout; maximum 600 seconds
rtc.token	String	Credential for joining the video room; valid for 1 hour
rtc.user_id	String	Your username in the RTC channel
rtc.token_expire_at	String	Token expiration time

Note: Successful creation in video mode only means that the room and external stream have been created; it does not mean that the digital character rendering service has completed its callback connection. The app must still establish a WebSocket connection, send conn_init, and determine readiness from conn_init_ack.

Request

GET https://{host}/live/v1/lives/{live_id}
Authorization: Token vda_xxx

Parameter	Type	Description
live_id	String	Required; live_id must belong to the account associated with the current API key

Response Fields

Response Fields	Description
live.status	Session status：waiting、prepared、on_live、ending、ended
live.created_at	Creation time in Unix seconds.
live.start_time	Actual Start Time
live.end_time	End Time
live.trace_id	Trace ID
live.billed_seconds	Billable Seconds
live.credits_cost	Credits consumed by this session.

Query Live history for the current account. This is suitable for admin lists or troubleshooting historical sessions. Request

GET https://{host}/live/v1/lives?pager.page=0&pager.pagesz=10
Authorization: Token vda_xxx

Query Parameters	Type	Default	Description
pager.page	Int	0	Page number, starting from 0
pager.pagesz	Int	10	Items per page; maximum 100.
pager.page_token	String	-	next_page_token returned by the server

Response

{
  "total": 12,
  "lives": [
    {
      "id": "1234567890",
      "status": "ended",
      "call_mode": "video",
      "billed_seconds": 120,
      "credits_cost": 20
    }
  ],
  "next_page_token": ""
}

The App WebSocket is used to initialize sessions, send text, interrupt responses, and hang up. It does not transmit audio or video media.

wss://{host}/live/ws/live/connect?live_id={live_id}&conn_id={conn_id}
Authorization: Token vda_xxx

Query Parameters	Type	Default	Description
live_id	String	None	Required; live.id returned when creating the Live session
conn_id	String	None	Optional，Client connection ID; generated automatically by the server if omitted

{
  "type": 1,
  "live_id": "1234567890",
  "user_id": "10000",
  "conn_id": "app-conn-1",
  "seq_id": 1,
  "payload": {}
}

type	Name	Direction	Description
1	conn_init	App → live	Connection initialization; must be sent after the WS opens.
2	conn_init_ack	live → App	Initialization result; success=true indicates that the real-time control connection is ready.
5	call_hangup	App → live	Active Hang-Up
6	force_hangup	live → App	Server-Forced Hang-Up
7	audio_interrupted	App → live	User interrupts the current digital character output
99	text_msg	App → live	Send a Text Message

conn_init

{
  "type": 1,
  "live_id": "1234567890",
  "conn_id": "app-conn-1",
  "seq_id": 1,
  "payload": {
    "conn_init": { "version": 1 }
  }
}

{
  "type": 2,
  "live_id": "1234567890",
  "conn_id": "app-conn-1",
  "seq_id": 2,
  "payload": {
    "conn_init_ack": {
      "success": true,
      "error_code": "",
      "error_msg": "",
      "server_timestamp": 1710000000
    }
  }
}

{
  "type": 2,
  "live_id": "1234567890",
  "conn_id": "app-conn-1",
  "seq_id": 2,
  "payload": {
    "conn_init_ack": {
      "success": false,
      "error_code": "NOT_READY",
      "error_msg": "live sip endpoint not ready",
      "server_timestamp": 1710000000
    }
  }
}

Note: When conn_init_ack.success=false, see Section 9, “Error Responses and Compatibility Notes,” for the meaning of error_code and recommended handling. This section only provides request, success-response, and failure-response examples.

text_msg

{
  "type": 99,
  "live_id": "1234567890",
  "conn_id": "app-conn-1",
  "seq_id": 10,
  "payload": {
    "text_msg": {
      "msg_id": "client-msg-1",
      "content": "Hello, please introduce yourself.",
      "timestamp": 1710000000000
    }
  }
}

Field	Type	Default	Description
msg_id	string	None	Optional，Optional message ID generated by the client for log correlation.
content	string	None	Required; text content sent to the digital character.
timestamp	int64	0	Optional，Client send time in milliseconds.

audio_interrupted

{
  "type": 7,
  "live_id": "1234567890",
  "conn_id": "app-conn-1",
  "seq_id": 11,
  "payload": {}
}

call_hangup and force_hangup

{
  "type": 5,
  "live_id": "1234567890",
  "conn_id": "app-conn-1",
  "seq_id": 12,
  "payload": {
    "hangup": { "hangup_reason": "user_end" }
  }
}

{
  "type": 6,
  "live_id": "1234567890",
  "conn_id": "app-conn-1",
  "payload": {
    "hangup": { "hangup_reason": "timeout" }
  }
}

hangup_reason	Common Scenario
user_end	The user actively ends the session.
timeout	The session reaches its maximum duration.
audit_violation	A content-safety or risk-control policy is triggered.
credit_insufficient	Insufficient credits.
sip_closed / provider_closed	The service or digital character rendering service closes.
`sip_reconnect_timeout / client_reconnect_timeout`	The disconnection exceeds the reconnection grace period.
external	Closed by the backend or external control.

Note: In addition to listening for type=6, the client must listen for WebSocket close/error events as a fallback. After actively hanging up, close the WebSocket first, then call the RTC SDK’s leaveChannel().

Two connections must be established in parallel: the App WebSocket handles control, while AliRTC handles media. Both are required.

Connection	Content Carried	Typical Action
App WebSocket	Control Signaling	conn_init starts, text_msg provides text interaction, audio_interrupted interrupts, and call_hangup hangs up.
AliRTC	Real-Time Media Streams	Subscribe to digital character audio and video.

flowchart LR
  CreateLive["Create Live"] --> WsInit["App WS conn_init"]
  CreateLive --> RtcJoin["AliRTC joinChannel"]
  WsInit --> ControlReady["Control Ready"]
  RtcJoin --> MediaReady["Media Ready"]
  ControlReady --> Interact["Live Interaction"]
  MediaReady --> Interact

Additional AliRTC Notes

The client must integrate the AliRTC SDK separately. For web integration, refer to the Alibaba Cloud RTC Web SDK documentation and load the corresponding SDK on the page.
The rtc object returned by the Create Live API contains the channel-joining credentials; the client does not need to generate a token.
rtc.token: Authentication information passed to joinChannel.
rtc.user_id: The current app user’s identity in the RTC room, passed as the second parameter to joinChannel.
rtc.channel_id: The RTC room identifier created by the server, mainly used for logs, subscription callbacks, and troubleshooting. The client generally does not need to construct it.
rtc.token_expire_at: Token expiration time in Unix seconds.
channel: The RTC room corresponding to one Live session. Only users in the same channel can exchange media streams.
publish: Publishes the local microphone or camera to the RTC room so the server and digital character connection can hear or see the user.
subscribe: Receives audio or video from remote users in the RTC room. In this scenario, the remote streams are primarily the digital character’s audio and video.

Join the Channel and Publish Local Media

await aliRtc.setDefaultSubscribeAllRemoteAudioStreams(true);
await aliRtc.setDefaultSubscribeAllRemoteVideoStreams(true);
await aliRtc.joinChannel(rtc.token, rtc.user_id);
await aliRtc.publishLocalAudioStream(true);
if (callMode === 'video') {
  await aliRtc.publishLocalVideoStream(true);
}

Mode	channel_id	App Responsibilities
audio	live-audio-{live_id}	Publish microphone audio and subscribe to digital character audio.
video	live-user-{live_id}	Publish microphone and camera streams and subscribe to the digital character video stream.

Subscribe to and Render Digital Character audio and video

Prepare a <video autoplay playsinline> container for remote video and bind the remote view after the subscription status changes to subscribed.

&lt;video id="remoteVideo" autoplay playsinline&gt;&lt;/video&gt;

const remoteVideoElement = document.getElementById('remoteVideo');
aliRtc.on('remoteUserOnLineNotify', (userId) => {
  console.log('remote user online', userId);
});
aliRtc.on('videoSubscribeStateChanged', (userId, oldState, newState) => {
  if (newState === 'subscribed') {
    // streamType=1 indicates the remote camera stream. In video mode, digital character video generally uses this callback.
    aliRtc.setRemoteViewConfig(remoteVideoElement, userId, 1);
    remoteVideoElement.play().catch(() => {
      console.warn('remote video autoplay failed');
    });
  }
});
aliRtc.on('screenShareSubscribeStateChanged', (userId, oldState, newState) => {
  if (newState === 'subscribed') {
    // streamType=2 indicates the remote screen stream. If the server uses this stream type, the view must also be bound.
    aliRtc.setRemoteViewConfig(remoteVideoElement, userId, 2);
  }
});

Role ID Rules

Role	User ID Format
User	live-user-{creatorID}-{liveID}
Digital Character Audio	live-bot-{creatorID}-{liveID}
Digital Character Video Stream	`live-video-push-{creatorID}-{liveID}`

POST https://{host}/live/v1/voices/clone
Authorization: Token vda_xxx
Content-Type: application/json

{
  "audio_url": "https://example.com/reference.wav",
  "voice": "my_custom_voice",
  "text": "This is the transcript corresponding to the reference audio.",
  "language": "zh"
}

Field	Type	Default	Description
audio_url	String	None	Required; reference audio; supports a URL or data URI with a length of 1–2048. Formats: WAV (16-bit), MP3, M4A. Recommended duration: 10–20 seconds; maximum: 60 seconds. File size < 10 MB.
voice	String	None	Required; voice name, 1–64 characters; only digits, uppercase/lowercase letters, and underscores are allowed.
text	String	Empty	Reference audio transcript; maximum length 4096
language	String	Empty	Reference audio language; maximum length 16

Query Custom Voices: GET /live/v1/voices → \{ "voices": [\{ "voice": "my_custom_voice" \}] \}

Supported language values: zh (Chinese), en (English), de (German), it (Italian), pt (Portuguese), es (Spanish), ja (Japanese), ko (Korean), fr (French), ru (Russian), th (Thai), id (Indonesian), ar (Arabic), cs (Czech), da (Danish), nl (Dutch), fi (Finnish), he (Hebrew), hi (Hindi), is (Icelandic), ms (Malay), no (Norwegian), fa (Persian), pl (Polish), sv (Swedish), tl (Tagalog), tr (Turkish), ur (Urdu), and vi (Vietnamese). Chinese dialects: Dongbei, Shaanxi, Sichuan, Henan, Changsha, Tianjin, Hangzhou, Liaoning, Shenyang, and Anshan.

The uploaded image can be used as avatar.image_uri when creating a session. Requirements: aspect ratio must be between 1:4 and 4:1; file size must be under 10 MB; supported formats are PNG, WEBP, JPEG, and JPG.

Create an Upload URL

Response Example:

{
  "id": "your_resource_id",
  "put_url": "put_url_example",
  "expires_at": "2024-09-05T04:20:29.589251952Z"
}

Upload the Image Using the Upload URL

Upload the image via HTTP PUT using the returned put_url.

curl -v -H "Content-Type: image/png" -X PUT --data-binary '@upload.png' '{put_url_example}'
# Response header example：
< HTTP/2 200
< etag: "d035e206b3243107372249559495346c"

Complete the Upload

Send the finish request with the recorded etag to obtain the image URI.

Response:

{
  "uri": "ssupload:?id=your_resource_id"
}

The external memory API provided by the business must follow this HTTP JSON protocol.

Invocation

POST <memory_retrieval.endpoint>
Content-Type: application/json
Accept: application/json
Authorization: <memory_retrieval.authorization>

endpoint and authorization come from the memory_retrieval configuration in the CreateLive request.
The external service must determine the retrievable scope based on authentication information, endpoint bindings, or server-side context.
The request body does not include user_id, character_id, database filters, embedding vectors, or sensitive-information switches.

Request Example

{
  "live_id": "1234567890",
  "query": "The user’s preferred coding style, programming language, and script output style",
  "reason": "The user asked for a script in their previously preferred coding style, but the current context contains no relevant preference information.",
  "memory_types": ["preference", "style"],
  "time_hint": "Long-term preferences",
  "max_results": 5
}

Field	Type	Default	Description
live_id	string	None	Required; current session ID
query	string	None	Required; natural-language retrieval intent generated by the model
reason	string	None	Required; the model’s explanation of why memory retrieval is needed for the current response
memory_types	string[]	Empty	Desired memory types, including preference/profile/history/project/constraint/relationship/style/other
time_hint	string	Empty	Time-range hint
max_results	integer	5	Maximum number of results，range[1-10]

memory_types Values

preference: User preferences, such as diet, language, tools, and response style
profile: Basic user information, such as occupation, location, and role
history：Facts from prior conversations and historical choices
project: Projects and long-term tasks the user is working on
constraint: Explicit user constraints, such as not using Python or having a limited budget
relationship：Relationships, team members, and contacts
style：The user’s preferred output style, tone, and format
other：Other unspecified types

Response Example

The external API must return JSON. Returning only the memories array is recommended; has_more is not required.

{
  "memories": [
    {
      "id": "mem_101",
      "summary": "The user prefers concise code, avoids excessive abstraction, and uses clear variable names.",
      "type": "style",
      "confidence": 0.94,
      "updated_at": "2026-06-01T09:00:00+08:00",
      "source": "long_term_memory"
    },
    {
      "id": "mem_102",
      "summary": "The user generally prefers Go examples; Python may be used when explicitly requested.",
      "type": "preference",
      "confidence": 0.86,
      "updated_at": "2026-05-20T13:00:00+08:00",
      "source": "long_term_memory"
    }
  ]
}

memories[].summary：Core content provided to the model
memories[].type：Memory type, consistent with the memory_types values in the request.
memories[].confidence: Confidence; recommended range [0–1]
memories[].updated_at：Update time
memories[].source：Memory source，for example long_term_memory、explicit_user_statement
Empty result: When no results are found, return “"memories": []”.

Error Response

If the external API returns a non-2xx status, the system normalizes the tool result into an error output and submits it to the model. Response body size is limited; an oversized response is treated as response_too_large.

The external knowledge API provided by the business must follow this HTTP JSON protocol.

Invocation

POST <knowledge_retrieval.endpoint>
Content-Type: application/json
Accept: application/json
Authorization: <knowledge_retrieval.authorization>

endpoint and authorization come from the knowledge_retrieval configuration in the CreateLive request.
The request body does not include user_id, creator_id, character_id, collection_ids, document_ids, database filters, embedding vectors, permission switches, or sensitive-information switches.

Request Example

{
  "live_id": "1234567890",
  "query": "Troubleshooting steps when Vidu real-time digital character creation fails",
  "reason": "The user asked for a specific troubleshooting process, but the current context contains no product documentation.",
  "knowledge_types": ["procedure", "troubleshooting"],
  "time_hint": "Latest version",
  "max_results": 5
}

Field	Type	Default	Description
live_id	string	None	Required; conversation Live ID
query	string	None	Required; natural-language retrieval intent generated by the model
reason	string	None	Required; reason knowledge retrieval is needed
knowledge_types	string[]	Empty	Desired knowledge types
time_hint	string	Empty	Time or version hint
max_results	integer	5	Maximum number of results，[1-10]

knowledge_types Values

faq：Frequently asked questions and standard Q&A
document：Product documentation, instructions, and long-document excerpts
policy：Policies, rules, restrictions, and compliance notes
product：Product capabilities, parameters, and feature descriptions
procedure：Operating procedures, integration steps, and user guides
troubleshooting：Troubleshooting guides, error-code descriptions, and issue-isolation steps
other：Other unspecified types

Response Example

The external API must return JSON. Returning only the knowledge array is recommended.

{
  "knowledge": [
    {
      "id": "doc_001_chunk_03",
      "title": "Real-Time Digital Character Creation Troubleshooting Guide",
      "content": "If CreateLive returns a SIP creation failure, first check call_mode, avatar.image_uri, the SIP provider return code, and trace_id in the Live logs.",
      "type": "troubleshooting",
      "source": "ops_runbook",
      "updated_at": "2026-07-01T10:00:00+08:00",
      "confidence": 0.92,
      "url": "https://kb.example.com/docs/live-troubleshooting"
    }
  ]
}

knowledge[].content：Core content provided to the model
knowledge[].title / source / url：Optional; helps the model evaluate the source and cite supporting information when needed
knowledge[].type：Knowledge type
knowledge[].confidence：Confidence，range[0-1]。
knowledge[].updated_at：Update time
Empty result: When no results are found, return “"knowledge": []”.

Error Responses and Other Notes

HTTP Errors

When an HTTP API call fails, it generally returns the following standardized JSON error structure:

{
  "code": 400,
  "reason": "BAD_REQUEST",
  "message": "Invalid parameter",
  "metadata": {}
}

Source	Status / Error Code	Typical Cause	Recommended Client Handling
HTTP	400	Invalid parameters, missing fields, invalid call_mode, text exceeding the length limit, or a session state that does not permit further operations.	Correct the request according to the field descriptions. If the Live session has ended, create a new session.
HTTP	401	The API key is missing, incorrectly formatted, or invalid.	Check Authorization: Token vda_xxx and confirm that the API key belongs to the current environment.
HTTP	403	The requested Live session does not belong to the current API key/account.	Use the same API key that created the Live session; do not reuse live_id across accounts.
HTTP	404	The session does not exist, has expired, or live_id is incorrect.	Confirm live_id; if recovery is impossible, create a new session.
HTTP	500	An internal service dependency failed, such as the database, SIP/video provider, or AliRTC token service.	Retry after a short delay. If the failure persists, provide live_id, trace_id, and the request time for investigation.

WebSocket Business Errors

Source	Status / Error Code	Typical Cause	Recommended Client Handling
WS	NOT_READY	In video mode, the digital character rendering service has not completed its callback connection, so conn_init cannot succeed yet.	Do not reload the page immediately. Wait 2–3 seconds and retry conn_init; exponential backoff may be used.
WS	LIVE_CONN_INIT_FAILED	Initialization failed, possibly due to an abnormal session state or a server-side dependency failure.	Close the current WS and create a new Live session. If it still fails, provide live_id and logs for investigation.

Agent Skills

Skills available for Claw: Agent Skills

Recommended Persona Templates and Real-World Examples

More information: Persona Templates Persona templates and examples

Available Voice List

More information: Available Voice List Available Voice List

Vidu S1

VIDU S1 · A World-Leading Real-Time Interactive Model

Vidu S1 is a real-time interactive video generation model that supports voice-controlled custom digital characters, unlimited-duration video generation, and high-frame-rate output.

Concept	Description
Live (Session)	A complete digital character interaction; obtain live_id
RTC Channel	audio and video transmission channel carrying microphone, camera, and digital character audio and video
WebSocket Signaling	Control-command channel for starting, text, interruption, and hang-up
Persona	Digital character personality, speaking style, and other settings

Vidu S1 Digital Character Live Quick Start Vidu S1 Digital Character Quick Start

Step 1: Create a Session

Create API-Key, you can create a session.

Request Example

POST https://{host}/live/v1/lives
Authorization: Token vda_xxx
Content-Type: application/json

{
  "call_mode": "video",
  "avatar": {
    "persona": "You are a friendly customer service representative. Interact naturally with the user in real time.",
    "image_uri": "https://your-digital-character-image-url.png",
    "voice": ""
  }
}

Save live.id from the response. It will be used later for WebSocket, billing queries, and text messages.
Save rtc.token and rtc.user_id from the response. They will be used to joining AliRTC in Step 3.

Step 2: Establish the WebSocket Connection and Send the Start Signal

First, establish the connection.

wss://{host}/live/ws/live/connect?live_id={live_id}
Authorization: Token vda_xxx

Send conn_init immediately after the connection is established:

{
  "type": 1,
  "live_id": "123456789",
  "seq_id": 1,
  "payload": { "conn_init": { "version": 1 } }
}

When conn_init_ack.success=true is received, the control connection has been initialized.
If NOT_READY is received in video mode, wait 2–3 seconds and retry conn_init.

Step 3: Connect to RTC

The Alibaba Cloud RTC SDK is used for audio and video transmission. Download link:Alibaba Cloud RTC SDK Documentation

await aliRtc.joinChannel(rtc.token, rtc.user_id);
await aliRtc.publishLocalAudioStream(true);
if (callMode === 'video') {
  await aliRtc.publishLocalVideoStream(true);
}

Mode	App Publishing / Subscription
audio	Join live-audio-{liveID}, publish microphone audio, and subscribe to digital character audio at live-bot-....
video	Join live-user-{liveID}, publish microphone and camera streams, and subscribe to digital character audio and video at live-video-push-....

API Overview

The table below lists all public interfaces. Locate the entry point by purpose, then see the relevant subsection for the detailed protocol. All interfaces require Authorization: Token vda_xxx

Method	Path	Purpose
POST	/live/v1/lives	Create a Live session (core entry point; returns RTC join and WS initialization information)
GET	/live/v1/lives/{live_id}	Query the status and billing of a single Live session
GET	/live/v1/lives	List Live Sessions
WebSocket	/live/ws/live/connect	App control signaling (start / text / interrupt / hang up)
—	AliRTC joinChannel	Real-time media streams (microphone / camera / digital character audio and video); requires AliRTC SDK integration
POST	/live/v1/voices/clone	Clone a voice to create a custom voice
GET	/live/v1/voices	Query the custom voice list
POST / PUT	/tools/v2/files/uploads	Upload an image and obtain a URI usable as avatar.image_uri

General Conventions

Item	Convention
{host}	{host} is the domain name without a protocol. The China environment uses api.vidu.cn, and the international environment uses api.vidu.com. Use https://{host} for HTTP requests and wss://{host} for WebSocket.
Authentication	All app-side HTTP and WebSocket interfaces require Authorization: Token vda_xxx。The API key is available in the Console
JSON Fields	Request and response fields use snake_case，for example call_mode、live_id、token_expire_at。
IDs and Time	live_id、character_id、created_at、token_expire_at and other int64 fields should be handled as strings on the frontend to avoid JavaScript number precision issues.
Media Transmission	App WebSocket carries control signaling only; the microphone, camera, and digital character audio and videoare all transmitted through AliRTC.

{
  "code": 400,
  "reason": "BAD_REQUEST",
  "message": "FieldInvalid",
  "metadata": {}
}

Request:

POST https://{host}/live/v1/lives
Authorization: Token vda_xxx
Content-Type: application/json

Complete Request Body Example:

{
  "call_mode": "video",
  "character_id": "1",
  "avatar": {
    "persona": "You are Sweet Tina. My voice is like warm milk tea—sweet and comforting—but I am always clear and capable when solving problems!",
    "image_uri": "https://scene.cf.vidu.studio/media-asset/070302-ZFkgvJxBTM0ZQJoO.png",
    "voice": "Tina"
  },
  "moderation": "disabled",
  "extra_motion": true,
  "audio": {
    "enable_transcription": true
  },
  "vad": {
    "type": "semantic",
    "threshold": 0.5,
    "silence_duration_ms": 200,
    "idle_timeout_ms": 500
  },
  "llm": {
    "temperature": 0.7,
    "top_p": 0.8,
    "top_k": 20,
    "frequency_penalty": 1,
    "presence_penalty": 0.3,
    "seed": -1,
    "max_tokens": 50
  },
  "idle_timeout_seconds": 7200,
  "memory_retrieval": {
    "enabled": true,
    "endpoint": "https://api.example.com/memory/search",
    "authorization": "Bearer memory-api-token",
    "timeout_ms": 3000
  },
  "knowledge_retrieval": {
    "enabled": true,
    "endpoint": "https://api.example.com/knowledge/search",
    "authorization": "Bearer knowledge-api-token",
    "timeout_ms": 3000
  }
}

Basic Field Descriptions

Field	Type	Required	Description
call_mode	String	Yes	Interaction mode: audio for audio only; video for audio and video
avatar.persona	String	Yes	Digital character persona, up to 50,000 Chinese characters.
avatar.image_uri	String	Yes	digital character image image URL. Supports an image URL, Base64, or image_uri (see the Image Upload section). Formats: PNG / JPG / JPEG / WEBP; image size ≤ 50 MB; byte length after Base64 decoding < 20 MB
avatar.voice	String	No	Voice; default: Tina
avatar.greeting_instruction	string	No	Opening greeting prompt, ≤ 200 characters

audio: Speech-to-Text

Field	Type	Default	Description
audio.enable_transcription	bool	false	When true, returns user speech-to-text and digital character output audio-to-text results through WebSocket.

vad: Interruption Control

Field	Type	Default	Description
vad.type	String	server	server：filters backchannel responses / background noise; semantic: interrupts the current response as soon as the user starts speaking
vad.threshold	float	0.5	Noise suppression strength, range [0, 1.0]，lower values apply less suppression
vad.silence_duration_ms	int	400	Amount of silence after speech ends before triggering a response; range [200, 6000]ms。
vad.idle_timeout_ms	int	0	The model proactively starts a conversation; 0 or [500, 30000] ms; 0 means no proactive conversation

llm: Language Model Output Control

Field	Type	Default	Description
llm.temperature	float	0.7	Creativity, range [0.0, 2.0)
llm.top_p	float	0.8	Quality threshold, range (0.0, 1.0]
llm.top_k	int	20	range [0, 100]
llm.frequency_penalty	float	1.0	Content repetition; default 1.0 means no penalty；< 1 encourages repetition；> 1 penalizes repetition
llm.presence_penalty	float	0.3	Topic breadth, range [0, 2]
llm.seed	int	-1	Range: -1 or [0–2³¹]; default -1 means random
llm.max_tokens	int	50	Maximum model output length，range 1 ~ 64000

idle_timeout_seconds: Session Lifecycle

Field	Type	Default	Description
idle_timeout_seconds	int	7200	Automatic disconnection time when there is no input. Default: 7200 s; range [10 ~ 7200]s

memory_retrieval / knowledge_retrieval: External Memory and Knowledge Retrieval (Beta)

Field	Type	Default	Description
*.enabled	bool	None	Required; whether to enable this capability
*.endpoint	String	None	Required when enabled; retrieval API URL
*.authorization	String	None	Required when enabled; forwarded as the external API Authorization header and validated as non-empty
*.timeout_ms	int	30000	Request timeout; maximum 30000ms

You will receive:

{
  "live": {
    "id": "123456789",
    "status": "waiting",
    "live_duration": 600,
    "call_mode": "video"
  },
  "rtc": {
    "app_id": "xxxx",
    "channel_id": "live-user-123456789",
    "user_id": "live-user-1001-123456789",
    "token": "base64-token...",
    "token_expire_at": "1750003600"
  }
}

Field	Type	Description
live.id	String	Room ID
live.status	String	waiting means the digital character is not ready
live.live_duration	Int	Maximum duration of this session; automatically disconnects on timeout; maximum 600 seconds
rtc.token	String	Credential for joining the video room; valid for 1 hour
rtc.user_id	String	Your username in the RTC channel
rtc.token_expire_at	String	Token expiration time

Request

GET https://{host}/live/v1/lives/{live_id}
Authorization: Token vda_xxx

Parameter	Type	Description
live_id	String	Required; live_id must belong to the account associated with the current API key

Response Fields

Response Fields	Description
live.status	Session status：waiting、prepared、on_live、ending、ended
live.created_at	Creation time in Unix seconds.
live.start_time	Actual Start Time
live.end_time	End Time
live.trace_id	Trace ID
live.billed_seconds	Billable Seconds
live.credits_cost	Credits consumed by this session.

Query Live history for the current account. This is suitable for admin lists or troubleshooting historical sessions. Request

GET https://{host}/live/v1/lives?pager.page=0&pager.pagesz=10
Authorization: Token vda_xxx

Query Parameters	Type	Default	Description
pager.page	Int	0	Page number, starting from 0
pager.pagesz	Int	10	Items per page; maximum 100.
pager.page_token	String	-	next_page_token returned by the server

Response

{
  "total": 12,
  "lives": [
    {
      "id": "1234567890",
      "status": "ended",
      "call_mode": "video",
      "billed_seconds": 120,
      "credits_cost": 20
    }
  ],
  "next_page_token": ""
}

The App WebSocket is used to initialize sessions, send text, interrupt responses, and hang up. It does not transmit audio or video media.

wss://{host}/live/ws/live/connect?live_id={live_id}&conn_id={conn_id}
Authorization: Token vda_xxx

Query Parameters	Type	Default	Description
live_id	String	None	Required; live.id returned when creating the Live session
conn_id	String	None	Optional，Client connection ID; generated automatically by the server if omitted

{
  "type": 1,
  "live_id": "1234567890",
  "user_id": "10000",
  "conn_id": "app-conn-1",
  "seq_id": 1,
  "payload": {}
}

type	Name	Direction	Description
1	conn_init	App → live	Connection initialization; must be sent after the WS opens.
2	conn_init_ack	live → App	Initialization result; success=true indicates that the real-time control connection is ready.
5	call_hangup	App → live	Active Hang-Up
6	force_hangup	live → App	Server-Forced Hang-Up
7	audio_interrupted	App → live	User interrupts the current digital character output
99	text_msg	App → live	Send a Text Message

conn_init

{
  "type": 1,
  "live_id": "1234567890",
  "conn_id": "app-conn-1",
  "seq_id": 1,
  "payload": {
    "conn_init": { "version": 1 }
  }
}

{
  "type": 2,
  "live_id": "1234567890",
  "conn_id": "app-conn-1",
  "seq_id": 2,
  "payload": {
    "conn_init_ack": {
      "success": true,
      "error_code": "",
      "error_msg": "",
      "server_timestamp": 1710000000
    }
  }
}

{
  "type": 2,
  "live_id": "1234567890",
  "conn_id": "app-conn-1",
  "seq_id": 2,
  "payload": {
    "conn_init_ack": {
      "success": false,
      "error_code": "NOT_READY",
      "error_msg": "live sip endpoint not ready",
      "server_timestamp": 1710000000
    }
  }
}

text_msg

{
  "type": 99,
  "live_id": "1234567890",
  "conn_id": "app-conn-1",
  "seq_id": 10,
  "payload": {
    "text_msg": {
      "msg_id": "client-msg-1",
      "content": "Hello, please introduce yourself.",
      "timestamp": 1710000000000
    }
  }
}

Field	Type	Default	Description
msg_id	string	None	Optional，Optional message ID generated by the client for log correlation.
content	string	None	Required; text content sent to the digital character.
timestamp	int64	0	Optional，Client send time in milliseconds.

audio_interrupted

{
  "type": 7,
  "live_id": "1234567890",
  "conn_id": "app-conn-1",
  "seq_id": 11,
  "payload": {}
}

call_hangup and force_hangup

{
  "type": 5,
  "live_id": "1234567890",
  "conn_id": "app-conn-1",
  "seq_id": 12,
  "payload": {
    "hangup": { "hangup_reason": "user_end" }
  }
}

{
  "type": 6,
  "live_id": "1234567890",
  "conn_id": "app-conn-1",
  "payload": {
    "hangup": { "hangup_reason": "timeout" }
  }
}

hangup_reason	Common Scenario
user_end	The user actively ends the session.
timeout	The session reaches its maximum duration.
audit_violation	A content-safety or risk-control policy is triggered.
credit_insufficient	Insufficient credits.
sip_closed / provider_closed	The service or digital character rendering service closes.
`sip_reconnect_timeout / client_reconnect_timeout`	The disconnection exceeds the reconnection grace period.
external	Closed by the backend or external control.

Two connections must be established in parallel: the App WebSocket handles control, while AliRTC handles media. Both are required.

Connection	Content Carried	Typical Action
App WebSocket	Control Signaling	conn_init starts, text_msg provides text interaction, audio_interrupted interrupts, and call_hangup hangs up.
AliRTC	Real-Time Media Streams	Subscribe to digital character audio and video.

flowchart LR
  CreateLive["Create Live"] --> WsInit["App WS conn_init"]
  CreateLive --> RtcJoin["AliRTC joinChannel"]
  WsInit --> ControlReady["Control Ready"]
  RtcJoin --> MediaReady["Media Ready"]
  ControlReady --> Interact["Live Interaction"]
  MediaReady --> Interact

Additional AliRTC Notes

The client must integrate the AliRTC SDK separately. For web integration, refer to the Alibaba Cloud RTC Web SDK documentation and load the corresponding SDK on the page.
The rtc object returned by the Create Live API contains the channel-joining credentials; the client does not need to generate a token.
rtc.token: Authentication information passed to joinChannel.
rtc.user_id: The current app user’s identity in the RTC room, passed as the second parameter to joinChannel.
rtc.channel_id: The RTC room identifier created by the server, mainly used for logs, subscription callbacks, and troubleshooting. The client generally does not need to construct it.
rtc.token_expire_at: Token expiration time in Unix seconds.
channel: The RTC room corresponding to one Live session. Only users in the same channel can exchange media streams.
publish: Publishes the local microphone or camera to the RTC room so the server and digital character connection can hear or see the user.
subscribe: Receives audio or video from remote users in the RTC room. In this scenario, the remote streams are primarily the digital character’s audio and video.

Join the Channel and Publish Local Media

await aliRtc.setDefaultSubscribeAllRemoteAudioStreams(true);
await aliRtc.setDefaultSubscribeAllRemoteVideoStreams(true);
await aliRtc.joinChannel(rtc.token, rtc.user_id);
await aliRtc.publishLocalAudioStream(true);
if (callMode === 'video') {
  await aliRtc.publishLocalVideoStream(true);
}

Mode	channel_id	App Responsibilities
audio	live-audio-{live_id}	Publish microphone audio and subscribe to digital character audio.
video	live-user-{live_id}	Publish microphone and camera streams and subscribe to the digital character video stream.

Subscribe to and Render Digital Character audio and video

Prepare a <video autoplay playsinline> container for remote video and bind the remote view after the subscription status changes to subscribed.

&lt;video id="remoteVideo" autoplay playsinline&gt;&lt;/video&gt;

const remoteVideoElement = document.getElementById('remoteVideo');
aliRtc.on('remoteUserOnLineNotify', (userId) => {
  console.log('remote user online', userId);
});
aliRtc.on('videoSubscribeStateChanged', (userId, oldState, newState) => {
  if (newState === 'subscribed') {
    // streamType=1 indicates the remote camera stream. In video mode, digital character video generally uses this callback.
    aliRtc.setRemoteViewConfig(remoteVideoElement, userId, 1);
    remoteVideoElement.play().catch(() => {
      console.warn('remote video autoplay failed');
    });
  }
});
aliRtc.on('screenShareSubscribeStateChanged', (userId, oldState, newState) => {
  if (newState === 'subscribed') {
    // streamType=2 indicates the remote screen stream. If the server uses this stream type, the view must also be bound.
    aliRtc.setRemoteViewConfig(remoteVideoElement, userId, 2);
  }
});

Role ID Rules

Role	User ID Format
User	live-user-{creatorID}-{liveID}
Digital Character Audio	live-bot-{creatorID}-{liveID}
Digital Character Video Stream	`live-video-push-{creatorID}-{liveID}`

POST https://{host}/live/v1/voices/clone
Authorization: Token vda_xxx
Content-Type: application/json

{
  "audio_url": "https://example.com/reference.wav",
  "voice": "my_custom_voice",
  "text": "This is the transcript corresponding to the reference audio.",
  "language": "zh"
}

Field	Type	Default	Description
audio_url	String	None	Required; reference audio; supports a URL or data URI with a length of 1–2048. Formats: WAV (16-bit), MP3, M4A. Recommended duration: 10–20 seconds; maximum: 60 seconds. File size < 10 MB.
voice	String	None	Required; voice name, 1–64 characters; only digits, uppercase/lowercase letters, and underscores are allowed.
text	String	Empty	Reference audio transcript; maximum length 4096
language	String	Empty	Reference audio language; maximum length 16

Query Custom Voices: GET /live/v1/voices → \{ "voices": [\{ "voice": "my_custom_voice" \}] \}

Create an Upload URL

Response Example:

{
  "id": "your_resource_id",
  "put_url": "put_url_example",
  "expires_at": "2024-09-05T04:20:29.589251952Z"
}

Upload the Image Using the Upload URL

Upload the image via HTTP PUT using the returned put_url.

curl -v -H "Content-Type: image/png" -X PUT --data-binary '@upload.png' '{put_url_example}'
# Response header example：
< HTTP/2 200
< etag: "d035e206b3243107372249559495346c"

Complete the Upload

Send the finish request with the recorded etag to obtain the image URI.

Response:

{
  "uri": "ssupload:?id=your_resource_id"
}

The external memory API provided by the business must follow this HTTP JSON protocol.

Invocation

POST <memory_retrieval.endpoint>
Content-Type: application/json
Accept: application/json
Authorization: <memory_retrieval.authorization>

endpoint and authorization come from the memory_retrieval configuration in the CreateLive request.
The external service must determine the retrievable scope based on authentication information, endpoint bindings, or server-side context.
The request body does not include user_id, character_id, database filters, embedding vectors, or sensitive-information switches.

Request Example

{
  "live_id": "1234567890",
  "query": "The user’s preferred coding style, programming language, and script output style",
  "reason": "The user asked for a script in their previously preferred coding style, but the current context contains no relevant preference information.",
  "memory_types": ["preference", "style"],
  "time_hint": "Long-term preferences",
  "max_results": 5
}

Field	Type	Default	Description
live_id	string	None	Required; current session ID
query	string	None	Required; natural-language retrieval intent generated by the model
reason	string	None	Required; the model’s explanation of why memory retrieval is needed for the current response
memory_types	string[]	Empty	Desired memory types, including preference/profile/history/project/constraint/relationship/style/other
time_hint	string	Empty	Time-range hint
max_results	integer	5	Maximum number of results，range[1-10]

memory_types Values

preference: User preferences, such as diet, language, tools, and response style
profile: Basic user information, such as occupation, location, and role
history：Facts from prior conversations and historical choices
project: Projects and long-term tasks the user is working on
constraint: Explicit user constraints, such as not using Python or having a limited budget
relationship：Relationships, team members, and contacts
style：The user’s preferred output style, tone, and format
other：Other unspecified types

Response Example

The external API must return JSON. Returning only the memories array is recommended; has_more is not required.

{
  "memories": [
    {
      "id": "mem_101",
      "summary": "The user prefers concise code, avoids excessive abstraction, and uses clear variable names.",
      "type": "style",
      "confidence": 0.94,
      "updated_at": "2026-06-01T09:00:00+08:00",
      "source": "long_term_memory"
    },
    {
      "id": "mem_102",
      "summary": "The user generally prefers Go examples; Python may be used when explicitly requested.",
      "type": "preference",
      "confidence": 0.86,
      "updated_at": "2026-05-20T13:00:00+08:00",
      "source": "long_term_memory"
    }
  ]
}

memories[].summary：Core content provided to the model
memories[].type：Memory type, consistent with the memory_types values in the request.
memories[].confidence: Confidence; recommended range [0–1]
memories[].updated_at：Update time
memories[].source：Memory source，for example long_term_memory、explicit_user_statement
Empty result: When no results are found, return “"memories": []”.

Error Response

The external knowledge API provided by the business must follow this HTTP JSON protocol.

Invocation

POST <knowledge_retrieval.endpoint>
Content-Type: application/json
Accept: application/json
Authorization: <knowledge_retrieval.authorization>

endpoint and authorization come from the knowledge_retrieval configuration in the CreateLive request.
The request body does not include user_id, creator_id, character_id, collection_ids, document_ids, database filters, embedding vectors, permission switches, or sensitive-information switches.

Request Example

{
  "live_id": "1234567890",
  "query": "Troubleshooting steps when Vidu real-time digital character creation fails",
  "reason": "The user asked for a specific troubleshooting process, but the current context contains no product documentation.",
  "knowledge_types": ["procedure", "troubleshooting"],
  "time_hint": "Latest version",
  "max_results": 5
}

Field	Type	Default	Description
live_id	string	None	Required; conversation Live ID
query	string	None	Required; natural-language retrieval intent generated by the model
reason	string	None	Required; reason knowledge retrieval is needed
knowledge_types	string[]	Empty	Desired knowledge types
time_hint	string	Empty	Time or version hint
max_results	integer	5	Maximum number of results，[1-10]

knowledge_types Values

faq：Frequently asked questions and standard Q&A
document：Product documentation, instructions, and long-document excerpts
policy：Policies, rules, restrictions, and compliance notes
product：Product capabilities, parameters, and feature descriptions
procedure：Operating procedures, integration steps, and user guides
troubleshooting：Troubleshooting guides, error-code descriptions, and issue-isolation steps
other：Other unspecified types

Response Example

The external API must return JSON. Returning only the knowledge array is recommended.

{
  "knowledge": [
    {
      "id": "doc_001_chunk_03",
      "title": "Real-Time Digital Character Creation Troubleshooting Guide",
      "content": "If CreateLive returns a SIP creation failure, first check call_mode, avatar.image_uri, the SIP provider return code, and trace_id in the Live logs.",
      "type": "troubleshooting",
      "source": "ops_runbook",
      "updated_at": "2026-07-01T10:00:00+08:00",
      "confidence": 0.92,
      "url": "https://kb.example.com/docs/live-troubleshooting"
    }
  ]
}

knowledge[].content：Core content provided to the model
knowledge[].title / source / url：Optional; helps the model evaluate the source and cite supporting information when needed
knowledge[].type：Knowledge type
knowledge[].confidence：Confidence，range[0-1]。
knowledge[].updated_at：Update time
Empty result: When no results are found, return “"knowledge": []”.

Error Responses and Other Notes

HTTP Errors

When an HTTP API call fails, it generally returns the following standardized JSON error structure:

{
  "code": 400,
  "reason": "BAD_REQUEST",
  "message": "Invalid parameter",
  "metadata": {}
}

Source	Status / Error Code	Typical Cause	Recommended Client Handling
HTTP	400	Invalid parameters, missing fields, invalid call_mode, text exceeding the length limit, or a session state that does not permit further operations.	Correct the request according to the field descriptions. If the Live session has ended, create a new session.
HTTP	401	The API key is missing, incorrectly formatted, or invalid.	Check Authorization: Token vda_xxx and confirm that the API key belongs to the current environment.
HTTP	403	The requested Live session does not belong to the current API key/account.	Use the same API key that created the Live session; do not reuse live_id across accounts.
HTTP	404	The session does not exist, has expired, or live_id is incorrect.	Confirm live_id; if recovery is impossible, create a new session.
HTTP	500	An internal service dependency failed, such as the database, SIP/video provider, or AliRTC token service.	Retry after a short delay. If the failure persists, provide live_id, trace_id, and the request time for investigation.

WebSocket Business Errors

Source	Status / Error Code	Typical Cause	Recommended Client Handling
WS	NOT_READY	In video mode, the digital character rendering service has not completed its callback connection, so conn_init cannot succeed yet.	Do not reload the page immediately. Wait 2–3 seconds and retry conn_init; exponential backoff may be used.
WS	LIVE_CONN_INIT_FAILED	Initialization failed, possibly due to an abnormal session state or a server-side dependency failure.	Close the current WS and create a new Live session. If it still fails, provide live_id and logs for investigation.

Agent Skills

Skills available for Claw: Agent Skills

Recommended Persona Templates and Real-World Examples

More information: Persona Templates Persona templates and examples

Available Voice List

More information: Available Voice List Available Voice List