Detailed Parameters
1. Creating a Live Session
Creates a real-time character session and returns the information required for the subsequent WebSocket initialization and AliRTC channel join.
Request
Request body example
Response body example
Basic fields
modelStringvidu-s2 (latest model; currently in beta testing)
vidu-s1 (default; high stability; does not support prompt_operation)
resloutionStringcall_modeStringavatar.personaStringRequiredavatar.image_uriStringConditionalImage address of the character's appearance. Supports image URL and base64. Only one [single-person] image is supported; full-body / half-body and all styles are supported. Formats: PNG / JPG / JPEG / WEBP; image size ≤ 50MB; byte length after base64 decode < 20M, and a content-type prefix must be included, for example data:image/png;base64,{base64_encode}
avatar.idStringConditionalavatar.voiceStringavatar.greeting_disabledboolavatar.greeting_instructionstringavatar.action_promptstringavatar.farewell_enabledboolavatar.persona_enhanceboolenable_recordingboolidle_timeout_secondsintvoice_model
rtc
You can specify the RTC provider when creating a session. If the entire rtc object is omitted, the default provider artc is used, and the client's join method and the response structure are exactly the same as before the upgrade, so existing integrations need no changes.
rtc.providerStringAvailable values for rtc.provider:
| Value | Provider |
|---|---|
| artc | Alibaba Cloud RTC (default) |
| trtc | Tencent Cloud RTC |
| agora | Agora RTC |
| volcengine | Volcengine RTC |
The validity period of the join credentials is subject to rtc.token_expire_at in the response.
You can choose the real-time speech model provider when creating a session.
voice_model.providerStringvoice_model.qwenObjectvoice_model.doubaoObjectThe configuration fields of the two branches are identical:
voiceStringlanguageStringThe branch and provider must match: when provider=doubao, only the doubao branch may be passed. Cloned voices are bound to the provider: qwen_omni supports session_update, doubao does not yet.
audio
Once enabled, the text content of the user's input audio / the character's output audio is returned over the WebSocket.
audio.enable_transcriptionboolvad
Used to control interruption-related parameters, so that the content of a user's turn can be captured more accurately.
vad.typeStringvad.thresholdfloatvad.silence_duration_msintvad.idle_timeout_msintidle_motion
Used to control the small movements the character performs automatically while not interacting (the idle period). Written at creation time and effective then; cannot be hot-updated during the session.
idle_motion.enabledbool · truetrue; when explicitly false, the remaining fields are ignoredidle_motion.sourceString · defaultdefault: the platform's default small-movement library custom: use only the requested list mixed: merge both; for identical names the custom one is keptidle_motion.recommendString · randomrandom weighted random; intelligent recommends based on the persona and the last two dialogue turnsidle_motion.motionsArrayRequiredsource is custom / mixed, at most 50 entries; ignored when defaultidle_motion.motions[].contentString · Requiredidle_motion.motions[].nameString · Requiredmixed, deduplicated by the final nameidle_motion.motions[].weightfloat · 1idle_motion.motions[].descriptionString · emptyrecommend=intelligentidle_motion.scheduling.motion_intervalint · 10idle_motion.scheduling.rest_after_motionint · 5idle_motion.motion_countint · Response onlyRhythm: during the idle period, one movement is issued every motion_interval seconds, and after rest_after_motion seconds it settles back to rest, then the cycle repeats. The user speaking or the character speaking interrupts it and restarts the timer, without accumulation; no small movements are inserted while speaking. Out-of-range parameters always return 400 and the session is not created.
action
Used to control whether runtime action matching is performed on the user's speech, and whether the platform's default action library or a self-built action library is used. For the management APIs of self-built action libraries, see the "Action Library" section
action.enabledbool · falseaction.library_idString · "0"llm
Used to control the output details of the language model, so as to suit the needs of different users.
llm.temperaturefloatllm.top_pfloatllm.top_kintllm.frequency_penaltyfloatllm.presence_penaltyfloatllm.seedintllm.max_tokensintknowledge_retrieval
All knowledge_retrieval fields are optional; they enable knowledge retrieval during the conversation. The source of the retrieval service is determined by provider: self means the customer's own HTTP retrieval service (for the protocol see the "External Knowledge" section); vidu means the platform's built-in knowledge base is used, with no need to build your own service — the assets are maintained through the APIs in the "Knowledge Base Management" section.
knowledge_retrieval.enabledboolknowledge_retrieval.providerStringknowledge_retrieval.endpointStringknowledge_retrieval.authorizationStringknowledge_retrieval.timeout_msintknowledge_retrieval.tool_instructionStringknowledge_retrieval.knowledge_base_idStringConditionalknowledge_retrieval.force_searchboolRequest body example
The Query Live and List Live APIs do not echo knowledge_retrieval, so the echo cannot be used to judge whether the configuration took effect.
memory_retrieval
Enables memory retrieval during the conversation. The source of the retrieval service is determined by provider: self means the customer's own HTTP retrieval service (for the protocol see the "External Memory" section); vidu means the platform's built-in long-term memory is used, with no need to build your own service — the assets are maintained through the APIs in the "Memory Management" section.
memory_retrieval.enabledboolmemory_retrieval.providerStringmemory_retrieval.endpointStringmemory_retrieval.authorizationStringmemory_retrieval.timeout_msintmemory_retrieval.tool_instructionStringmemory_retrieval.user_idStringConditionalmemory_retrieval.agent_idStringmemory_retrieval.enable_writeboolRequest body example
The Query Live and List Live APIs do not echo memory_retrieval, so the echo cannot be used to judge whether the configuration took effect.
Response field descriptions
live.idStringlive.statusStringlive.live_durationIntlive.voice_modelObjectlive.idle_motionObjectlive.actionObjectrtc.tokenStringrtc.user_idStringrtc.token_expire_atStringrecording.enabledBoolIn video mode, a successful creation only means the room and the external stream have been created; it does not mean the character's rendering side has finished connecting back. The App still needs to establish the WebSocket and send conn_init, and use conn_init_ack to determine whether it is ready.
2. Querying a Live Session and Its Billing
Queries the current status, session time and billing fields of a single live.
Request example
Response fields
| Response field | Description |
|---|---|
live.status | Session status: waiting, prepared, on_live, ending, ended |
live.created_at | Creation time, Unix seconds |
live.start_time | Official start time, usually written after both ends are ready |
live.end_time | End time |
live.trace_id | The trace ID used for troubleshooting; it is recommended to include it when reporting issues |
live.billed_seconds | Billed seconds, usually counted from when the session is ready |
live.credits_cost | Credits consumed by this live |
live.voice_model | Echoes the real-time speech model configuration used when the session was created |
3. Querying the Live List
Queries live history under the current account — suitable for back-office lists or for investigating past sessions.
Request example
pager.pageIntpager.pageszIntpager.page_tokenStringResponse example
4. Querying the Recording File
Queries the download address of the recording file for the whole call. enable_recording: true must be passed when creating the session; the recording file is generated asynchronously after the session ends and can usually be queried within 2 minutes. The download address is valid for 24 hours and the file format is MP4.
Request example
Response example
Response fields
recording.enabledBoolrecording.video_urlStringrecording.error_codeStringrecording.error_messageStringerror_code enum
| Value | Description |
|---|---|
unspecified | No failure |
start_failed | Recording failed to start |
stop_failed | Recording failed to stop |
query_failed | Recording file query timed out |
transfer_failed | Recording file storage failed |
5. App WebSocket Control Protocol
Used to initialize the session, send text, interrupt and hang up. It does not transmit audio or video media.
Request example
live_idStringRequiredlive.id returned when the live was created.conn_idStringOptionalResponse example
| type | Name | Direction | Description |
|---|---|---|---|
| 1 | conn_init | App → live | Connection initialization; must be sent after the WS opens. |
| 2 | conn_init_ack | live → App | Initialization result. success=true means the real-time control link is ready. |
| 5 | call_hangup | App → live | The App hangs up on its own initiative. |
| 6 | force_hangup | live → App | The server forces a hangup, for example on timeout, risk control, insufficient credits or a shutdown on the service side. |
| 7 | audio_interrupted | App → live | The user interrupts the current AI output. |
| 99 | text_msg | App → live | Send a text message. |
conn_init
Request example
Response example
text_msg
The server currently does not return a separate business ack for text_msg; the character's reply is output through RTC audio/video, and some text responses may also be delivered over the WS.
Request example
| Field | Description |
|---|---|
msg_id | Message ID generated by the client, to make it easier to correlate client logs. |
content | The text content sent to the character. |
timestamp | Client send time, a millisecond timestamp. |
text_tts
Specified text is directly synthesized into audio for playback, without producing a dialogue reply: it does not enter the dialogue context, does not produce a reply, and does not trigger character actions.
Request example
| Field | Required | Description |
|---|---|---|
content | Yes | Text content. |
msg_id | No | Caller-defined identifier. |
text_action
Used only to trigger character actions; it does not produce a dialogue reply and does not interrupt the character's currently playing reply. Request fields are the same as text_tts.
Request example
text_dialog
Dialogue-only text; it enters the dialogue and produces a reply but does not trigger character actions. Request fields are the same as text_tts.
Request example
text_tts, text_action, and text_dialog have no success acknowledgment: successful acceptance returns no frame at all, consistent with the existing text_msg.
Support Across Voice Models
The support for the three new text signals (text_tts, text_action, text_dialog) across voice models is as follows:
Semantics \ voice_model.provider | qwen_omni | doubao |
|---|---|---|
text_msg | Supported | Not supported domestically; supported in the overseas version |
text_tts (direct-to-audio from specified text) | Not supported | Supported |
text_action (action text) | Supported | Supported |
text_dialog (dialogue-only text) | Supported | Not supported domestically; supported in the overseas version |
audio_interrupted
Used for the user to interrupt the current AI output. There is currently no separate ack; the client can handle it in combination with the subsequent audio stopping, changes in the character's response, or its own UI state.
Request example
call_hangup and force_hangup
Request example
| hangup_reason | Trigger scenario |
|---|---|
user_end | The user disconnected on their own initiative |
timeout | The session timed out |
audit_violation | The session triggered a risk-control disconnection |
credit_insufficient | Insufficient credits |
sip_closed | Closed on the SIP / provider side |
provider_closed | The provider closed the connection on its own initiative |
sip_reconnect_timeout | SIP disconnected and still had not recovered after the reconnection grace period |
client_reconnect_timeout | The App client went offline and still had not recovered after the reconnection grace period |
prepared_sip_disconnected | The video live was still in prepared and the App had never become ready / never started streaming when SIP disconnected — it ends directly |
owner_taken_over | The owner was taken over by another node during the SIP disconnection, and the local session closed itself |
owner_lease_lost | The owner lease renewal failed / was lost, and the local session closed itself |
ai_output_closed | The AI output channel closed, causing the session to close |
external | Redis live: close, a broadcast shutdown |
PS: In addition to listening for type=6, the client must also listen for WebSocket close / error events as a fallback. After hanging up on its own initiative, it is recommended to close the WebSocket first and then call the RTC SDK's leaveChannel().
session_update
Used to dynamically update part of the session configuration while the session is in progress, without hanging up and rebuilding the session. The server validates this update as a whole; if any field is invalid, the update fails, none of the fields take effect, and the original configuration remains unchanged.
Request example
Updatable fields and constraints (the types and ranges from session creation apply; within the knowledge_retrieval / memory_retrieval groups the whole group is replaced, while for the other fields you only pass the parts you need to update)
avatar.voiceStringavatar.personaStringvad.typeStringvad.thresholdfloatvad.silence_duration_msintknowledge_retrieval.enabledboolknowledge_retrieval.endpointStringknowledge_retrieval.authorizationStringknowledge_retrieval.timeout_msintmemory_retrieval.enabledboolmemory_retrieval.endpointStringmemory_retrieval.authorizationStringmemory_retrieval.timeout_msintllm.seedintllm.max_tokensintllm.frequency_penaltyfloatllm.presence_penaltyfloatllm.top_kintllm.top_pfloatllm.temperaturefloatResponse example
prompt_operation
Switches or clears the reference images of the character's picture (the R2V reference-generation capability) while the live is in progress, without interrupting the stream. The message is a JSON text frame with type=11; the outer structure is the same as for the other signals.
Switching reference images (switch): send 1–3 new reference images. Each reference image independently carries its address, a stable identifier (image_id), and an optional scene type and natural-language description.
op_typestringOptionalswitch (default, may be omitted) / removeimagesobject[]Requiredimage_id of any entry in the array is invalid, the entire operation fails.images[].image_uristringRequiredssupload:?id=...), base64 data URIs (data:image/*;base64,...) and http(s) URLs — the same formats as the reference image used when creating the live. Note: a single WebSocket message is capped at 16MB; watch the frame size when using base64.
images[].image_idstringRequiredremove) uses it to locate the image.images[].kindstringOptionalobject (handheld item) / garment (outfit change) / background (background change). Determined automatically from the reference image when not passed.images[].user_textstringOptionalAfter switching reference images, it takes a few seconds of rendering for the picture to take effect; the audio conversation is unaffected in the meantime. When images[].kind is not passed, it is determined automatically from that reference image; if a single reference image contains several possible editing targets at once — clothing, objects and background, for example — the automatic result may not match expectations, so passing kind explicitly is recommended.
Clearing reference images (remove): the character's picture returns to its original state. Two forms are supported — clear the most recent one: omit images to return to the state before the last reference-image switch; targeted clearing by image_id: pass 1–3 entries in images, where each entry needs only image_id (the identifier used in the earlier switch), so several images can be cleared at once.
Targeted clearing of two reference images by image_id:
Once the server has finished processing, it returns a type=12 acknowledgement:
Values of error_code on failure:
| error_code | Description |
|---|---|
PROMPT_OP_PARAM_INVALID | Invalid parameters (images missing or more than 3 entries, unsupported image_uri format, image_id empty / longer than 128 characters / repeated within one operation, and so on) |
PROMPT_OP_MODEL_NOT_SUPPORTED | The current live is not using the vidu-s2 model, so reference-image operations are not supported |
PROMPT_OP_LIVE_NOT_ACTIVE | The live is not in progress |
PROMPT_OP_IMAGE_TRANSFER_FAILED | Failed to fetch the reference image (address unreachable, etc.) |
PROMPT_OP_SIP_NOT_CONNECTED | The rendering link is not ready; please retry later |
PROMPT_OP_FAILED | Other internal errors |
Operations do not depend on acknowledgement order: each switch is an independent, complete operation and the client may resend it at any time; a switch can follow a remove. image_id is maintained by the caller: it must not be repeated within a single operation, and reusing the same image_id across operations is discouraged, so that targeted clearing maps precisely to the earlier reference image. A single message is capped at 16MB; when one switch carries several base64 reference images, the total counts toward that cap.
baseline_reset (type=15)
The client can trigger this at any point during the conversation to immediately reset the character's pose to the initial, silent picture. Request fields:
typeintegerRequired15.live_idstringRequiredconn_idstringOptionalseq_idintegerOptionalpayloadobjectOptionalRequest example:
Acknowledgment semantics: no acknowledgment is sent on success; an error frame (type=103) is sent only in the two cases below, with payload.error.source fixed to baseline_reset; other runtime exceptions do not produce an acknowledgment. Repeated triggers are idempotent (equivalent to resending the same closed-mouth pose), but sending them repeatedly at high frequency with no purpose is not recommended.
Failure acknowledgment example:
Error codes:
| Error Code | Trigger Condition |
|---|---|
not_supported | The current live session type does not support this capability (e.g., voice-only mode). |
live_not_active | The live session is not running (ended / unavailable). |
switch_avatar (type=16)
While a live session is running, the client can have the character play a set of transition videos and switch the character to a new one during playback. The message is a JSON text frame with type=16; the outer structure matches other signals.
Request fields:
typeintegerRequired16.live_idstringRequiredconn_idstringOptionalseq_idintegerOptionalpayload.switch_avatar.switch_videosstring[]Requiredpayload.switch_avatar.avatar_idintegerRequest example:
Acknowledgment semantics: no acknowledgment is sent on success (do not judge the execution result by whether the WS write succeeded); on failure, an error frame is sent.
Failure acknowledgment fields:
typeinteger103.live_idstringseq_idintegerseq_id as-is.payload.error.sourcestringswitch_avatar.payload.error.codestringpayload.error.messagestringError codes:
| Error Code | Trigger Condition |
|---|---|
SWITCH_AVATAR_PARAM_INVALID | Invalid input: switch_videos is empty or contains a non-http(s) address; avatar_id is not provided; or the specified character does not exist / is not usable / is not yet ready. |
SWITCH_AVATAR_LIVE_NOT_ACTIVE | The live session is not running (or has ended / is unavailable). |
SWITCH_AVATAR_MEDIA_TRANSFER_FAILED | Failed to process the transition video or the new character image. |
SWITCH_AVATAR_RENDER_NOT_CONNECTED | The rendering service is not ready; the switch cannot be performed right now — retry later. |
SWITCH_AVATAR_FAILED | Other failures. |
Notes:
switch_avatar(type=17)
While the live session is running, reports the client's microphone-mute state so that the Doubao (China mainland) language model does not auto-hang-up due to no input. The message is a JSON text frame with type=17; the outer structure matches the other signals.
Request fields:
typeintegerRequiredlive_idstringRequiredconn_idstringOptionalseq_idintegerOptionalpayloadstringRequiredRequest Example:
Two links must be established at the same time during integration: the App WebSocket handles control and AliRTC handles media. The two links run in parallel and both are indispensable.
| Link | What it carries | Typical actions |
|---|---|---|
| App WebSocket | Control signaling | conn_init to start, text_msg for text interaction, audio_interrupted to interrupt, call_hangup to hang up. |
| AliRTC | Real-time media streams | Publishes the App's microphone audio; in video mode publishes camera video; subscribes to the character's audio and video. |
switch_avatar(type=18)
While the live session is running, reports the client's microphone-mute state so that the Doubao (China mainland) language model does not auto-hang-up due to no input. The message is a JSON text frame with type=18; the outer structure matches the other signals.
Request fields:
typeintegerRequiredlive_idstringRequiredconn_idstringOptionalseq_idintegerOptionalpayloadstringRequiredRequest example:
- The new character takes effect once the transition video finishes playing; during the transition the picture plays through the video list in order.
- When using avatar_id, that character's action description is applied together with the switch.
6. RTC Join Protocol: AliRTC joinChannel
Two links must be established at the same time during integration: the App WebSocket handles control and AliRTC handles media. The two links run in parallel and both are indispensable.
| Link | What it carries | Typical actions |
|---|---|---|
| App WebSocket | Control signaling | conn_init to start, text_msg for text interaction, audio_interrupted to interrupt, call_hangup to hang up. |
| AliRTC | Real-time media streams | Publishes the App's microphone audio; in video mode publishes camera video; subscribes to the character's audio and video. |
AliRTC prerequisites
- The client needs to integrate the AliRTC SDK separately. For the web, refer to the Alibaba Cloud RTC Web SDK documentation and load the corresponding SDK on the page.
- The
rtcobject returned by the create-live API is the source of the join credentials; the client does not need to generate a token itself. rtc.token: the authentication information passed tojoinChannel.rtc.user_id: the current App user's identity in the RTC room, used as the second parameter ofjoinChannel.rtc.channel_id: the identifier of the RTC room created by the server, used mainly for logging, subscription callbacks and troubleshooting; the client usually does not need to assemble it itself.rtc.token_expire_at: the token expiration time, in Unix seconds; once expired, the live should be created again.
7. Voice Cloning
Creates a custom voice from reference audio. The voice returned by the API can be used as avatar.voice when creating a live.
audio_urlStringRequiredvoiceStringRequiredtextStringlanguageStringChinese dialects: Northeastern Mandarin (Dongbei), Shaanxi dialect (Shannxi), Sichuan dialect (Sichuan), Henan dialect (Henan), Changsha dialect (Changsha), Tianjin dialect (Tianjin), Hangzhou dialect (Hangzhou), Liaoning dialect (Liaoning), Shenyang dialect (Shenyang), Anshan dialect (Anshan)
providerStringQuery custom voices: GET /live/v1/voices → {"voices": [{"voice": "my_custom_voice"}]} Delete a custom voice: DELETE /live/v1/voices/{my_custom_voice} The voice (voice name) returned by cloning is filled into avatar.voice or into the corresponding provider branch's voice when creating a session, and must be used consistently with voice_model.provider, otherwise 400 is returned.
8. External Memory
The HTTP JSON protocol that a business-side external memory API must follow.
How it is called
Note: endpoint and authorization come from the memory_retrieval configuration in the CreateLive request.
Request example
live_idstringRequiredquerystringRequiredreasonstringRequiredmemory_typesstring[]time_hintstringmax_resultsintegermemory_types enum
| Type | Meaning |
|---|---|
preference | User preferences, for example food, language, tools, answer style |
profile | Basic user information, for example occupation, location, role |
history | Facts from past conversations, historical choices |
project | Projects the user is working on, long-term tasks |
constraint | Explicit user constraints, for example do not use Python, limited budget |
relationship | Interpersonal relationships, team members, contacts |
style | The output style, tone and format the user prefers |
other | Other, uncertain types |
Response example
The external API should return JSON. It is recommended to return only the memories array; there is no need to return has_more.
| Type | Meaning |
|---|---|
memories[].summary | The core content shown to the model; a short natural-language sentence is recommended — do not return large amounts of raw chat history |
memories[].type | Memory type; it is recommended to keep it consistent with the memory_types enum in the request. |
memories[].confidence | Confidence; the higher, the more trustworthy |
memories[].updated_at | Update time |
memories[].source | Memory source |
PS: When the result is empty, return it like this — "memories": [] — and do not return plain text (for example "nothing found").
Error responses
When the external API returns a non-2xx status, the system normalizes the tool result into an error output and submits it to the model. The system limits the response body size, and an oversized response is treated as response_too_large. It is still recommended to return structured JSON:
9. External Knowledge
How it is called
Note: endpoint and authorization come from the knowledge_retrieval configuration in the CreateLive request.
Request example
live_idstringRequiredquerystringRequiredreasonstringRequiredknowledge_typesstring[]time_hintstringmax_resultsintegerknowledge_types enum
| Type | Meaning |
|---|---|
faq | Frequently asked questions, standard Q&A |
document | Product documentation, explanatory documents, long-document excerpts |
policy | Policies, rules, restrictions, compliance notes |
product | Product capabilities, parameters, feature descriptions |
procedure | Operating procedures, integration steps, usage guides |
troubleshooting | Troubleshooting guides, error-code explanations, problem-localization steps |
other | Other, uncertain types |
Response example
The external API should return JSON. It is recommended to return only the knowledge array.
| Type | Meaning |
|---|---|
knowledge[].content | The core content shown to the model; a short natural-language sentence is recommended — do not return large amounts of raw chat history |
knowledge[].type | Knowledge type; it is recommended to keep it consistent with the knowledge_types enum in the request. |
knowledge[].confidence | Confidence; the higher, the more trustworthy |
knowledge[].updated_at | Update time |
knowledge[].source | Knowledge source |
knowledge[].title | Knowledge source |
knowledge[].url | Knowledge source |
When the result is empty, return it like this — "knowledge": [] — and do not return plain text (for example "nothing found").
Error responses
When the external API returns a non-2xx status, the system normalizes the tool result into an error output and submits it to the model. The system limits the response body size, and an oversized response is treated as response_too_large. It is still recommended to return structured JSON:
10. Memory Management
Asset management APIs for the platform's built-in long-term memory, for use with memory_retrieval.provider=vidu. Isolated by the account the API Key belongs to; cross-account access returns "resource does not exist". This group of APIs is not billed separately.
Memory object fields
idStringmemoryStringuser_idStringagent_idStringrun_idStringmetadataObjectcreated_at / updated_atStringscoreNumberCreating memories
Extracts memories from a set of conversation messages and stores them. The upstream may accept the request without extracting any new memory, in which case results is an empty array. messages (at least one, each element containing role and content) and user_id (length 1-64) are required; agent_id (up to 64), run_id and metadata are optional.
Searching memories
query (a non-empty natural-language query) and user_id are required; agent_id / run_id for narrowing the scope, filters and limit are optional. Returns {"results": [memory objects with a score]}.
Getting the memory list
user_id is required; agent_id / run_id, filters and page / page_size are optional. Returns results and total.
Deleting a user's memories
The query parameter user_id is required; it clears the memories within that user's scope. agent_id / run_id are optional, for deleting by an additional scope. On success an empty object {} is returned, and after deletion that user_id releases its memory user quota.
11. Knowledge Base Management
Asset management APIs for the platform's built-in knowledge base, for use with knowledge_retrieval.provider=vidu. Isolated by the account the API Key belongs to; by default each account may have at most 100 knowledge bases, and each knowledge base at most 1000 documents.
| Object | Returned fields |
|---|---|
| Knowledge base | id, name, status, error_msg, created_at, updated_at |
| Knowledge document | id, knowledge_base_id, file_uri, name, status, error_msg, created_at, updated_at |
Creating a knowledge base
Created synchronously, with indexing completed. name is optional and at most 256 characters after trimming leading and trailing whitespace. Measured to take about 2 seconds.
Listing / querying / deleting knowledge bases
The list query parameter pager.page starts from 0 (default 0), and pager.pagesz defaults to 10 with a maximum of 100. Returns total, items and next_page_token.
Adding a knowledge document
Added synchronously with ingestion completed; it can be retrieved immediately on success. file_uri is required — pass a file address reachable by the platform; name is optional, at most 256 characters, and is derived from the file address when omitted.
Listing / querying / deleting knowledge documents
The paging parameters are the same as for the knowledge base list. The file_uri returned when querying a single document is a temporarily accessible address (signed, with a validity period) — do not store it long-term.
Retrieval behavior and limits
- A retrieval failure, timeout or empty result does not interrupt the Live; the character continues the conversation.
- Automatic memory search during a session does not restrict run_id, so it can recall long-term memory across sessions; when enable_write writes back, live_id is automatically used as the run_id.
- At most 4 managed tool calls are processed at the same time; beyond that, a "tool busy" result is returned.
- Creating a Live only validates that the knowledge base exists and belongs to the current account; it does not validate whether it contains any documents.
12. Action Library
Self-built action libraries are bound with action.library_id when creating a session. They are isolated by account owner, and multiple API Keys under the same user share one set of libraries; each account may have at most 20 libraries, and each library at most 100 actions. All requests require Authorization: Token vda_xxx, and IDs are strings in JSON.
| Method | Path | Description |
|---|---|---|
| POST | /action-libraries | Create a library. name is required; exceeding the quota or a duplicate name returns 400 |
| GET | /action-libraries?pager.page=0&pager.pagesz=10 | List; returns total + items, in reverse chronological order of creation, containing only this account's non-deleted libraries |
| GET | /action-libraries/{id} | Library details. Another person's library or a non-existent one both return 404 |
| PATCH | /action-libraries/{id} | Change name / description; a duplicate name returns 400 |
| DELETE | /action-libraries/{id} | Delete the library; the actions in it are deleted along with it; sessions in progress are unaffected |
| POST | /action-libraries/{id}/actions | Add an action; the description is rewritten synchronously before it is stored, which commonly takes a few seconds |
| GET | /action-libraries/{id}/actions[/{action_id}] | List / details; you can view the rewritten description |
| PATCH | /action-libraries/{id}/actions/{action_id} | Changing name does not trigger a rewrite; changing description rewrites it first, and if the rewrite fails the original record is unchanged |
| DELETE | /action-libraries/{id}/actions/{action_id} | Delete a single action |
Field constraints
| Field | Required | Description |
|---|---|---|
| Library name | Yes | 1-50 characters, unique within the non-deleted scope of the same account |
| Library description | No | ≤ 200, may be empty |
| Action name | Yes | 1-20 characters, unique within the non-deleted scope of the library |
| Action description | Yes | 5-500 on the request side; after the server rewrites it, it is validated again as 5-500. If the rewrite fails it is not stored, and the same request can be resent to retry |
The library object additionally returns id, action_count (the number of non-deleted actions), and created_at / updated_at (ISO 8601); the action object additionally returns id, action_library_id and the same time fields.
13. Error Responses and Other Notes
| Source | Status / error code | Typical cause | Suggested client handling |
|---|---|---|---|
| HTTP | 400 | Invalid parameters, missing fields, invalid call_mode, text exceeding the length limit, or a session state that does not allow the operation to continue. | Correct the request according to the API field descriptions; if the live has already ended, create the session again. |
| HTTP | 401 | The API Key is missing, incorrectly formatted or invalid. | Check Authorization: Token vda_xxx; confirm that the API Key belongs to the current environment. |
| HTTP | 403 | You accessed a live that does not belong to the current API Key / current account. | Use the same API Key that created the live; do not reuse a live_id across accounts. |
| HTTP | 404 | The session does not exist, has expired, or the live_id was written incorrectly. | Confirm the live_id; when it cannot be recovered, create the session again. |
| HTTP | 500 | An internal service dependency is abnormal, for example DB, SIP / video provider, the AliRTC token, etc. | You may retry briefly; if it keeps failing, report it for investigation along with the live_id, trace_id and request time. |
| WS | NOT_READY | In video mode, the character's rendering side has not finished connecting back, so conn_init cannot succeed yet. | Do not rebuild the page immediately; wait 2 to 3 seconds and retry conn_init, optionally with exponential backoff. |
| WS | LIVE_CONN_INIT_FAILED | Initialization failed, possibly due to an abnormal session state or an abnormal server-side dependency. | Close the current WS and create the live again; if it still fails, report it for investigation along with the live_id and the logs. |
14. Integration Essentials
- WebSocket authentication goes through the Header; it is recommended that the server act as a proxy to avoid leaking the Vidu token.
- Alibaba Cloud RTC requires downloading and integrating the SDK separately — it is not just a matter of sending an HTTP request.
- In
videomode you may encounterNOT_READY; this is normal and retries must be implemented. - Credits are deducted automatically according to duration during the task, and the connection is dropped automatically when the balance reaches zero.
- A single session lasts at most 7200 seconds; the service closes it on its own initiative when the limit is reached.
- Maintain a heartbeat: the client must send some message within 15 seconds, otherwise the connection is judged dead and forcibly closed.
- In addition to listening for
conn_init_ack, the client must also listen for WebSocketclose/errorevents as a fallback. - The App WS only accepts text JSON signals; do not send binary audio or video over the App WS.
audio_start,audio_stopandemotion_signalare reserved in the protocol enum; the current integration does not require the client to send them.- Apart from
conn_init,text_msg,audio_interruptedandcall_hangupcurrently have no separate business ack. stream_urlis currently mainly a reserved field; real-time media playback should connect to AliRTC using thertcinformation.
15. Pricing Information
- 3 credits per 2 seconds (about CNY 0.094; 1 credit = CNY 0.03125)
- Billing starts from
on_live— that is, the momentconn_init_ackreturnssuccess: true - Pricing method: when the duration is not divisible by 2, the total duration is rounded up (a total duration of 11 seconds is billed as 12 seconds)
- If the minimum number of credits in the user's account balance is less than 45 credits (about CNY 1.4), a task cannot be started
- Voice cloning costs 899 credits each time (about CNY 28); the first ten clones are free for each customer
16. Additional Information
Skills that can be used for claw: agent skills Available voices: voice list