Video Calling
ConnectyCube Video Calling P2P API is built on top of WebRTC protocol and based on top of WebRTC Mesh architecture.
Max people per P2P call is 4.
To get a difference between P2P calling and Conference calling please read our ConnectyCube Calling API comparison blog page.
Get started with SDK
Follow the Getting Started guide on how to connect ConnectyCube SDK and start building your first app.
Code samples
There are ready-to-go FREE code samples to help you better understand how to integrate video calling capabilities in your apps:
- Video Chat code sample for React Native src code
- Video Chat code sample for Cordova/PhoneGap - src code
- Video Chat code sample for browser - src code
Preparations
ConnectyCube Chat API is used as a signaling transport for Video Calling API, so in order to start using Video Calling API you need to connect user to Chat.
Create video session
In order to use Video Calling API you need to create a session object - choose your opponents with whom you will have a call and a type of session (VIDEO or AUDIO):
!> Important note: The calleesIds
array must contain the opponents ids only and exclude current user id.
!> In a case of low bandwidth network, you can try to limit the call bandwidth cap to get better quality vs stability results. It can be done by passing const additionalOptions = {bandwidth: 256};
or 128 value.
Access local media stream
In order to have a video chat session you need to get an access to the user’s devices (webcam / microphone):
This method lets the browser ask the user for permission to use devices. You should allow this dialog to access the stream. Otherwise, the browser can’t obtain access and will throw an error for getUserMedia
callback function.
For more information about possible audio/video constraints, here is a good code sample from WebRTC team how to work with getUserMedia constraints: https://webrtc.github.io/samples/src/content/getusermedia/resolution/
Call quality
Limit bandwidth
Despite WebRTC engine uses automatic quality adjustement based on available Internet bandwidth, sometimes it’s better to set the max available bandwidth cap which will result in a better and smoother user experience. For example, if you know you have a bad internet connection, you can limit the max available bandwidth to e.g. 256 Kbit/s.
This can be done either when initiate a call (see below ‘Initiate a call’ API documentation):
which will result in limiting the max vailable bandwidth for ALL participants
or/and during a call:
which will result in limiting the max available bandwidth for current user only.
HD video quality
If HD video quality is required - the following audio/video constraints are required to pass:
More info about all possible is are available here https://developer.mozilla.org/en-US/docs/Web/API/MediaDevices/getUserMedia
Attach local media stream
Then you should attach your local media stream to HTML video element.
For Web-like environments, including Cordova - use the following method:
Initiate a call
The extension is used to pass any extra parameters in the request to your opponents.
After this, your opponents will receive a callback call:
Or if your opponents are offline or did not answer the call request:
Accept a call
To accept a call the following code snippet is used:
After this, your opponents will get a confirmation in the following callback:
Also, both the caller and opponents will get a special callback with the remote stream:
From this point, you and your opponents should start seeing each other.
Receive a call in background
For mobile apps, it can be a situation when an opponent’s user app is either in closed (killed) or background (inactive) state.
In this case, to be able to still receive a call request, you can use Push Notifications. The flow should be as follows:
- a call initiator should send a push notification along with a call request
- when an opponent’s app is killed or in background state - an opponent will receive a push notification about an incoming call, and will be able to accept/reject the call. If accepted or pressed on a push notification - an app will be opened, a user should auto login and connect to chat and then will be able to join an incoming call.
Please refer to Push Notifications API guides regarding how to integrate Push Notifications in your app:
For even better integration - CallKit and VoIP push notifications can be used. Please check CallKit and VoIP push notifications
section on each platform Push Notifications API guide page.
Reject a call
After this, the caller will get a confirmation in the following callback:
Sometimes, it could a situation when you received a call request and want to reject, but the call session object has not arrived yet. It could be in a case when you integrated CallKit to receive call requests while an app is in background/killed state. To do a reject in this case, the following snippet can be used:
End a call
After this, the opponents will get a confirmation in the following callback:
Mute audio
Mute video
Switch video cameras
First of all you need to obtain all your device’s available cameras:
Then you can choose some deviceId
and switch the video stream to exact this device:
Switch audio output
For switching audio - use the same above flow for switching camera. Just replace a videoinput
to audioinput
in getMediaDevices
and video
to audio
in constraints
.
Screen Sharing
Request a desktop stream by calling getDisplayMedia
:
More info about what else options can be passed can be found here https://developer.mozilla.org/en-US/docs/Web/API/MediaDevices/getDisplayMedia
If the local stream already exists, the next call to getUserMedia or getDisplayMedia will update the tracks in the stream and preserve the track’s enabled state for the audio track.
Group video calls
Because of Mesh architecture we use for multipoint where every participant sends and receives its media to all other participants, current solution supports group calls with up to 4 people.
Also ConnectyCube provides an alternative solution for up to 12 people - Multiparty Video Conferencing API.
Monitor connection state
There is a callback function to track the session connection state:
The possible values of connectionState are those of an enum of type ConnectyCube.videochat.SessionConnectionState
:
- ConnectyCube.videochat.SessionConnectionState.UNDEFINED
- ConnectyCube.videochat.SessionConnectionState.CONNECTING
- ConnectyCube.videochat.SessionConnectionState.CONNECTED
- ConnectyCube.videochat.SessionConnectionState.DISCONNECTED
- ConnectyCube.videochat.SessionConnectionState.FAILED
- ConnectyCube.videochat.SessionConnectionState.CLOSED
- ConnectyCube.videochat.SessionConnectionState.COMPLETED
Tackling Network changes
If a user’s network environment changes (e.g., switching from Wi-Fi to mobile data), the existing call connection might no longer be valid. Normally, in a case of short network interruptions, the ConnectyCube SDK will automatically restore the call so you can see via onSessionConnectionStateChangedListener
callback with connectionState
changing to DISCONNECTED
and then again to CONNECTED
.
But not all cases are the same, and in some of them the connection needs to be manually refreshed due to various issues like NAT or firewall behavior changes or even longer network environment changes, e.g. when a user is offline for more than 30 seconds.
This is where ICE restart helps to re-establish the connection to find a new network path for communication.
The correct and recommended way for an application to handle all such ‘bad’ cases is to trigger an ICE restart when the connection state goes to either FAILED
or DISCONNECTED
for an extended period of time (e.g. > 30 seconds).
Following is the preliminary code snippet regarding how to work with ICE restart:
-
Firstly, we have to disable the call termination logic after the network is disconnected for > 30 seconds by increasing the
videochat.disconnectTimeInterval
value to e.g. 5 mins. -
Secondly, define a variable which can track the Internet connection state:
-
Thirdly, define a function that will perform ICE restart:
-
Fourthly, define a
ConnectyCube.videochat.onSessionConnectionStateChangedListener
callback and try to perform ICE restart if not automatic call restoration happened after 30 seconds: -
Finally, in a case a user got working Internet connection later, do ICE restart there:
After these changes the call connection should be restored to working state again.
Configuration
There are various calling related configs that can be changed.
alwaysRelayCalls
The alwaysRelayCalls
config sets the WebRTC RTCConfiguration.iceTransportPolicy
config. Setting it to true
means the calling media will be routed through TURN server all the time, and not directly P2P between users even if the network path allows it:
Recording
For the recording feature implementation you can use MediaStream Recording API
The recorder accepts a media stream and record it to file. Both local and remote streams can be recorded and saved to file.
There is also a good article about client-side recording implementation https://webrtchacks.com/jitsi-recording-getdisplaymedia-audio/
Continue calling in background
If you are developing dedicated apps for iOS and Android - it’s required to apply additional configure for the app to continue playing calling audio when it goes into the background.
On iOS: there is no way to continue a video call in background because of some OS restrictions. What is supported there is to continue with voice calling while an app is in background. Basically, the recommended to achieve this is to switch off device camera when an app goes to background and then switch camera on back when an app goes to foreground.
Furthermore, even voice background call are blocked by default on iOS. To unblock - you need to setup proper background mode capabilities in your project. Please find the Enabling Background Audio link with more information how to do it properly. Generally speaking, you need to enable Voice over IP
and Remote notifications
capabilities:
For Android, we also recommend to implement the same camera switch flow when go to background and then return to foreground.