The Tuya Cloud provides real-time video access of WebRTC protocol for IoT devices that are capable of transmitting audios and videos. For the description of WebRTC protocol, see Build the backend services needed for a WebRTC app.
API description
The MQTT configuration for generating links for the users.
API address
POST /v1.0/open-hub/access/config
Request parameter
| Parameter | Type | Location | Description | Required |
|---|---|---|---|---|
| link_id | Int | BODY | The unique flag of the user connection with the maximum length of 8 bits. | Yes |
| uid | String | BODY | User ID. Not required in authorization code mode. | No |
| link_type | String | BODY | Connection mode that supports MQTT. | Yes |
| topics | String | BODY | IPC. MQTT topic type of WebRTC. | Yes |
Response parameter
| Parameter | Type | Description |
|---|---|---|
| result | MQTT |
MQTT login information |
| success | Boolean | Status of the response result |
| t | Long | Response |
MQTT| Parameter | Type | Description |
|---|---|---|
| url | String | The address to be connected |
| username | String | The username used for connection |
| password | String | The password used for connection |
| client_id | String | The client_id used for connection |
| source_topic | String | Subscribe Topic |
| sink_topic | String | Post Topic |
| expire_time | Integer | Valid duration |
Request example
{
"uid": "ay1564026880284v****",
"link_id": "123456",
"link_type": "MQTT",
"topics": "ipc"
}
**Response example**
```json
{
"result": {
"client_id": "cloud_7ef68bc84629ea3f51152760cdf2****",
"expire_time": 7200,
"password": "0426d6917bfd8b88f037c4a598a0****",
"sink_topic": {
"ipc": "/av/moto/moto_id/u/{device_id}"
},
"source_topic": {
"ipc": "/av/u/d09735be24f4b7eb3583b30bcaa2****"
},
"url": "ssl://m1-cn.wgine.com:8883",
"username": "cloud_d09735be24f4b7eb3583b30bcaa2****"
},
"success": true,
"t": 1600847208953
}
API address
GET /v1.0/users/{uId}/devices/{deviceId}/webrtc-configs
Request parameter
| Parameter | Type | Location | Description | Required |
|---|---|---|---|---|
| uid | String | URI | User ID | Yes |
| deviceId | String | URI | Device ID | Yes |
Response parameter
| Parameter | Type | Description |
|---|---|---|
| audio_attributes | AudioAttributes |
Attributes of talk |
| auth | String | Authorization information |
| id | String | Device ID |
| moto_id | String | Instance ID used for connection |
| p2p_config | P2PConfig |
Configuration information of connection service |
| skill | String | Skill |
| supports_webrtc | Boolean | Whether WebRTC is supported |
| vedio_clarity | Integer | Video resolution |
AudioAttributes| Parameter | Type | Description |
|---|---|---|
| call_mode | Integer | Talk mode. 1: one-way, 2: two-way |
| hardware_capability | Integer | Hardware capability. 1: MIC, 2: Speaker |
P2PConfig| Parameter | Type | Description |
|---|---|---|
| ices | Token |
P2P Token list |
Token| Parameter | Type | Description |
|---|---|---|
| urls | String | ICE service address |
| username | String | ICE service username |
| credential | String | ICE service password |
| ttl | Integer | The valid duration of ICE service. The unit is second. |
Response example
{
"result": {
"audio_attributes": {
"call_mode": [
1,
2
],
"hardware_capability": [
1,
2
]
},
"auth": "h85L4pljbuHFR0a/iTgViwA35xi3yTl3NyMsFQL5****",
"id": "6cf2b6d2b09a2f8597****",
"moto_id": "moto_cnpre002",
"p2p_config": {
"ices": [
{
"urls": "stun:49.234.141.77:3478"
},
{
"urls": "stun:tx1stun.tuyacn.com:3478"
},
{
"urls": "nat:tx1nat.tuyacn.com:3478"
},
{
"urls": "nat:tx2nat.tuyacn.com:3478"
},
{
"credential": "kb/EA2whGCcNSM5FjXV2dxAM1MU=",
"ttl": 36000,
"urls": "turn:49.234.141.77:3478",
"username": "1600883205:6cf2b6d2b09a2f8597****"
},
{
"credential": "kb/EA2whGCcNSM5FjXV2dxAM****",
"ttl": 36000,
"urls": "turn:tx1turn.tuyacn.com:3478",
"username": "1600883205:6cf2b6d2b09a2f8597****"
}
]
},
"skill": "{\"webrtc\":3,\"audios\":[{\"channels\":1,\"dataBit\":16,\"codecType\":101,\"sampleRate\":8000}],\"videos\":[{\"streamType\":2,\"profileId\":\"\",\"width\":1920,\"codecType\":2,\"sampleRate\":90000,\"height\":1080},{\"streamType\":4,\"width\":640,\"codecType\":2,\"height\":360}]}",
"supports_webrtc": true,
"vedio_clarity": 4
},
"success": true,
"t": 1600847205437
}
Response parameter
| Parameter | Type | Description |
|---|---|---|
| protocol | Integer | The protocol number of the MQTT message. webRTC is a live streaming service with the value of 302 |
| pv | String | The version of communication protocol |
| t | Integer | Unix timestamp in seconds |
| data | Data |
MQTT message frame |
Data| Parameter | Type | Description |
|---|---|---|
| header | Header |
MQTT message header |
| msg | Msg |
MQTT message body. It can be offer candidate answer disconnect. |
Header| Parameter | Type | Description |
|---|---|---|
| type | String | MQTT message type. It can be offer candidate answer disconnect. |
| from | String | Enter uid for the client, and enter device_id for the deviceclient |
| to | String | Enter its ID. For example: device_id or uid |
| sub_dev_id | String | Sub-device node_id that is only used by NVR device |
| sessionid | String | It is generated at random with a byte length of 32 upon every connection. The signalings of the same connection share the same sessionid. |
| moto_id | String | It is obtained through /v1.0/users/{uId}/devices/{deviceId}/webrtc-configs API. If the returned result lacks moto_id field, it is not supported. |
MsgIt currently supports three MQTT message types: offer, candidate, and disconnet. Different MQTT message types differ in msg format.
When type=offer, the Msg format is as follows.
| Parameter | Type | Description |
|---|---|---|
| mode | String | Connection mode: WebRTC |
| sdp | String | WebRTC offer generated in Web |
| stream_type | Interger | Stream type. It is sub-stream 1 by default. |
| auth | String | Get it through /v1.0/users/{uId}/devices/{deviceId}/webrtc-configs API. Its field is auth. |
Response example
{
"protocol":302,
"pv":"2.2",
"t":1600820048671,
"data":{
"header":{
"from":"AY1600819753305aHO5Sdj8pQMtLZ68XHMUpHKlRKJ87s",
"to":"6c9a943f2ea6929675ymcq",
"sessionid":"00b00036521743319b4d4c01f1705c48",
"moto_id":"moto_5f685396jK",
"type":"offer"
},
"msg":{
"sdp":"v=0\r\no=- 4529163812828363188 2 IN IP4 127.0.0.1\r\ns=-\r\nt=0 0\r\na=group:BUNDLE 0 1\r\na=msid-semantic: WMS 1VpYoJaai0xSYjWhYxPHqySybB3PaQ6Y3wXP\r\nm=audio 9 UDP/TLS/RTP/SAVPF 111 103 104 9 0 8 106 105 13 110 112 113 126\r\nc=IN IP4 0.0.0.0\r\na=rtcp:9 IN IP4 0.0.0.0\r\na=ice-ufrag:Q93I\r\na=ice-pwd:P58s/ZyBRNVnuIxcrcmEmRG5\r\na=ice-options:trickle\r\na=fingerprint:sha-256 E1:01:E0:B3:F1:97:7F:86:07:61:54:BE:42:5F:56:E8:84:58:76:E3:E4:22:94:F1:33:2A:A3:C2:FC:67:05:3E\r\na=setup:actpass\r\na=mid:0\r\na=extmap:1 urn:ietf:params:rtp-hdrext:ssrc-audio-level\r\na=extmap:2 http://www.webrtc.org/experiments/rtp-hdrext/abs-send-time\r\na=extmap:3 http://www.ietf.org/id/draft-holmer-rmcat-transport-wide-cc-extensions-01\r\na=extmap:4 urn:ietf:params:rtp-hdrext:sdes:mid\r\na=extmap:5 urn:ietf:params:rtp-hdrext:sdes:rtp-stream-id\r\na=extmap:6 urn:ietf:params:rtp-hdrext:sdes:repaired-rtp-stream-id\r\na=sendrecv\r\na=msid:1VpYoJaai0xSYjWhYxPHqySybB3PaQ6Y3wXP 1c7d25a4-9948-4165-bf4d-62fc39b8b528\r\na=rtcp-mux\r\na=rtpmap:111 opus/48000/2\r\na=rtcp-fb:111 transport-cc\r\na=fmtp:111 minptime=10;useinbandfec=1\r\na=rtpmap:103 ISAC/16000\r\na=rtpmap:104 ISAC/32000\r\na=rtpmap:9 G722/8000\r\na=rtpmap:0 PCMU/8000\r\na=rtpmap:8 PCMA/8000\r\na=rtpmap:106 CN/32000\r\na=rtpmap:105 CN/16000\r\na=rtpmap:13 CN/8000\r\na=rtpmap:110 telephone-event/48000\r\na=rtpmap:112 telephone-event/32000\r\na=rtpmap:113 telephone-event/16000\r\na=rtpmap:126 telephone-event/8000\r\na=ssrc:724809951 cname:7UznE7uyn6JBJ4PA\r\na=ssrc:724809951 msid:1VpYoJaai0xSYjWhYxPHqySybB3PaQ6Y3wXP 1c7d25a4-9948-4165-bf4d-62fc39b8b528\r\na=ssrc:724809951 mslabel:1VpYoJaai0xSYjWhYxPHqySybB3PaQ6Y3wXP\r\na=ssrc:724809951 label:1c7d25a4-9948-4165-bf4d-62fc39b8b528\r\nm=video 9 UDP/TLS/RTP/SAVPF 96 97 98 99 100 101 122 102 120 127 119 125 107 108 109 121 114 115 124 118 123\r\nc=IN IP4 0.0.0.0\r\na=rtcp:9 IN IP4 0.0.0.0\r\na=ice-ufrag:Q93I\r\na=ice-pwd:P58s/ZyBRNVnuIxcrcmEmRG5\r\na=ice-options:trickle\r\na=fingerprint:sha-256 E1:01:E0:B3:F1:97:7F:86:07:61:54:BE:42:5F:56:E8:84:58:76:E3:E4:22:94:F1:33:2A:A3:C2:FC:67:05:3E\r\na=setup:actpass\r\na=mid:1\r\na=extmap:14 urn:ietf:params:rtp-hdrext:toffset\r\na=extmap:2 http://www.webrtc.org/experiments/rtp-hdrext/abs-send-time\r\na=extmap:13 urn:3gpp:video-orientation\r\na=extmap:3 http://www.ietf.org/id/draft-holmer-rmcat-transport-wide-cc-extensions-01\r\na=extmap:12 http://www.webrtc.org/experiments/rtp-hdrext/playout-delay\r\na=extmap:11 http://www.webrtc.org/experiments/rtp-hdrext/video-content-type\r\na=extmap:7 http://www.webrtc.org/experiments/rtp-hdrext/video-timing\r\na=extmap:8 http://www.webrtc.org/experiments/rtp-hdrext/color-space\r\na=extmap:4 urn:ietf:params:rtp-hdrext:sdes:mid\r\na=extmap:5 urn:ietf:params:rtp-hdrext:sdes:rtp-stream-id\r\na=extmap:6 urn:ietf:params:rtp-hdrext:sdes:repaired-rtp-stream-id\r\na=recvonly\r\na=rtcp-mux\r\na=rtcp-rsize\r\na=rtpmap:96 VP8/90000\r\na=rtcp-fb:96 goog-remb\r\na=rtcp-fb:96 transport-cc\r\na=rtcp-fb:96 ccm fir\r\na=rtcp-fb:96 nack\r\na=rtcp-fb:96 nack pli\r\na=rtpmap:97 rtx/90000\r\na=fmtp:97 apt=96\r\na=rtpmap:98 VP9/90000\r\na=rtcp-fb:98 goog-remb\r\na=rtcp-fb:98 transport-cc\r\na=rtcp-fb:98 ccm fir\r\na=rtcp-fb:98 nack\r\na=rtcp-fb:98 nack pli\r\na=fmtp:98 profile-id=0\r\na=rtpmap:99 rtx/90000\r\na=fmtp:99 apt=98\r\na=rtpmap:100 VP9/90000\r\na=rtcp-fb:100 goog-remb\r\na=rtcp-fb:100 transport-cc\r\na=rtcp-fb:100 ccm fir\r\na=rtcp-fb:100 nack\r\na=rtcp-fb:100 nack pli\r\na=fmtp:100 profile-id=2\r\na=rtpmap:101 rtx/90000\r\na=fmtp:101 apt=100\r\na=rtpmap:122 VP9/90000\r\na=rtcp-fb:122 goog-remb\r\na=rtcp-fb:122 transport-cc\r\na=rtcp-fb:122 ccm fir\r\na=rtcp-fb:122 nack\r\na=rtcp-fb:122 nack pli\r\na=fmtp:122 profile-id=1\r\na=rtpmap:102 H264/90000\r\na=rtcp-fb:102 goog-remb\r\na=rtcp-fb:102 transport-cc\r\na=rtcp-fb:102 ccm fir\r\na=rtcp-fb:102 nack\r\na=rtcp-fb:102 nack pli\r\na=fmtp:102 level-asymmetry-allowed=1;packetization-mode=1;profile-level-id=42001f\r\na=rtpmap:120 rtx/90000\r\na=fmtp:120 apt=102\r\na=rtpmap:127 H264/90000\r\na=rtcp-fb:127 goog-remb\r\na=rtcp-fb:127 transport-cc\r\na=rtcp-fb:127 ccm fir\r\na=rtcp-fb:127 nack\r\na=rtcp-fb:127 nack pli\r\na=fmtp:127 level-asymmetry-allowed=1;packetization-mode=0;profile-level-id=42001f\r\na=rtpmap:119 rtx/90000\r\na=fmtp:119 apt=127\r\na=rtpmap:125 H264/90000\r\na=rtcp-fb:125 goog-remb\r\na=rtcp-fb:125 transport-cc\r\na=rtcp-fb:125 ccm fir\r\na=rtcp-fb:125 nack\r\na=rtcp-fb:125 nack pli\r\na=fmtp:125 level-asymmetry-allowed=1;packetization-mode=1;profile-level-id=42e01f\r\na=rtpmap:107 rtx/90000\r\na=fmtp:107 apt=125\r\na=rtpmap:108 H264/90000\r\na=rtcp-fb:108 goog-remb\r\na=rtcp-fb:108 transport-cc\r\na=rtcp-fb:108 ccm fir\r\na=rtcp-fb:108 nack\r\na=rtcp-fb:108 nack pli\r\na=fmtp:108 level-asymmetry-allowed=1;packetization-mode=0;profile-level-id=42e01f\r\na=rtpmap:109 rtx/90000\r\na=fmtp:109 apt=108\r\na=rtpmap:121 H264/90000\r\na=rtcp-fb:121 goog-remb\r\na=rtcp-fb:121 transport-cc\r\na=rtcp-fb:121 ccm fir\r\na=rtcp-fb:121 nack\r\na=rtcp-fb:121 nack pli\r\na=fmtp:121 level-asymmetry-allowed=1;packetization-mode=1;profile-level-id=4d0015\r\na=rtpmap:114 H264/90000\r\na=rtcp-fb:114 goog-remb\r\na=rtcp-fb:114 transport-cc\r\na=rtcp-fb:114 ccm fir\r\na=rtcp-fb:114 nack\r\na=rtcp-fb:114 nack pli\r\na=fmtp:114 level-asymmetry-allowed=1;packetization-mode=1;profile-level-id=640015\r\na=rtpmap:115 rtx/90000\r\na=fmtp:115 apt=114\r\na=rtpmap:124 red/90000\r\na=rtpmap:118 rtx/90000\r\na=fmtp:118 apt=124\r\na=rtpmap:123 ulpfec/90000\r\n",
"auth":"3iHAObTiJ+P1o/OeX8My208vis9Ar6JQygHSLrBxv5U=",
"mode":"webrtc",
"stream_type":1
}
}
}
When type=candidate, the Msg format is as follows.
| Parameter | Type | Description |
|---|---|---|
| mode | String | Connection mode: WebRTC |
| candidate | String | WebRTC candidate addresses collected by both parties |
Response example
{
"protocol":302,
"pv":"2.2",
"t":1600820048672,
"data":{
"header":{
"from":"AY1600819753305aHO5Sdj8pQMtLZ68XHMUpHKlRKJ87s",
"to":"6c9a943f2ea6929675ymcq",
"sessionid":"00b00036521743319b4d4c01f1705c48",
"moto_id":"moto_5f685396jK",
"type":"candidate"
},
"msg":{
"mode":"webrtc",
"candidate":"a=candidate:512512433 1 udp 2122260223 192.168.0.227 50828 typ host generation 0 ufrag Q93I network-id 1",
}
}
}
When type=disconnect, the Msg format is as follows.
| Parameter | Type | Description |
|---|---|---|
| mode | String | Connection mode, WebRTC |
Response example
{
"protocol":302,
"pv":"2.2",
"t":1600820048679,
"data":{
"header":{
"from":"AY1600819753305aHO5Sdj8pQMtLZ68XHMUpHKlRKJ87s",
"to":"6c9a943f2ea6929675ymcq",
"sessionid":"00b00036521743319b4d4c01f1705c48",
"moto_id":"moto_5f685396jK",
"type":"disconnect"
},
"msg":{
"mode":"webrtc"
}
}
}
Web front-end
Web back-end
Tuya Cloud
Tuya MQTT
Tuya IPC
Log in to the Cloud Development Platform and get Access ID and Access Secret in the project information.
Update Access ID and Access Secret to clientId and secret in the webrtc.json file respectively.
Update authorization mode and information.
easyauthEnter deviceId. Select an IPC on the Tuya Smart app and check the device ID.
Download the source code from GitHub. In the root directory of the source code, execute go get and then execute go build.
Execute ./webrtc-demo-go.
Open http://localhost:3333 with Chrome and click Call.
Q: How to receive Tuya MQTT service messages?
A: After you get the open platform configs, you need to use the string after /av/u/ in JSON field in result.source_topic.ipc as the from in MQTT Header. In this way, you can receive MQTT service messages properly.