Video Calling
ConnectyCube Video Calling P2P API is built on top of WebRTC protocol and based on top of WebRTC Mesh architecture.
Max people per P2P call is 4.
To get a difference between P2P calling and Conference calling please read our ConnectyCube Calling API comparison blog page.
Preparations
ConnectyCube Chat API is used as a signaling transport for Video Calling API, so in order to start using Video Calling API you need to connect user to Chat.
Connect WebRTC lib
Since “react-native-connectycube” version 3.34.0, the “react-native-webrtc” has been replaced from dependencies to peerDependencies to support autolinking. Install the “react-native-webrtc” and follow the Getting started. Configure your iOS and Android projects.
Until “react-native-connectycube” version 3.34.0, the “react-native-webrtc” is included to “react-native-connectycube”. Follow the Android Manual Linking and iOS Manual Linking guides.
Create video session
In order to use Video Calling API you need to create a call session object - choose your opponents with whom you will have a call and a type of session (VIDEO or AUDIO):
!> In a case of low bandwidth network, you can try to limit the call bandwidth cap to get better quality vs stability results. It can be done by passing const additionalOptions = {bandwidth: 256};
or 128 value.
Access local media stream
In order to have a video chat session you need to get an access to the user’s devices (webcam / microphone):
This method lets the browser ask the user for permission to use devices. You should allow this dialog to access the stream. Otherwise, the browser can’t obtain access and will throw an error for getUserMedia
callback function.
For more information about possible audio/video constraints, here is a good code sample from WebRTC team how to work with getUserMedia constraints: https://webrtc.github.io/samples/src/content/getusermedia/resolution/
Call quality
Limit bandwidth
Despite WebRTC engine uses automatic quality adjustement based on available Internet bandwidth, sometimes it’s better to set the max available bandwidth cap which will result in a better and smoother user experience. For example, if you know you have a bad internet connection, you can limit the max available bandwidth to e.g. 256 Kbit/s.
This can be done when initiate a call (see below ‘Initiate a call’ API documentation):
which will result in limiting the max vailable bandwidth for ALL participants.
Attach local media stream
Then you should attach your local media stream to a video element:
Initiate a call
The extension is used to pass any extra parameters in the request to your opponents.
After this, your opponents will receive a callback call:
Or if your opponents are offline or did not answer the call request:
Accept a call
To accept a call the following code snippet is used:
After this, you will get a confirmation in the following callback:
Also, both the caller and opponents will get a special callback with the remote stream:
From this point, you and your opponents should start seeing each other.
Receive a call in background
See CallKit section below.
Reject a call
After this, the caller will get a confirmation in the following callback:
Sometimes, it could a situation when you received a call request and want to reject, but the call sesison object has not arrived yet. It could be in a case when you integrated CallKit to receive call requests while an app is in background/killed state. To do a reject in this case, the following snippet can be used:
End a call
After this, the opponents will get a confirmation in the following callback:
Mute audio
Mute video
Switch video cameras
Switch audio output
- connect and install https://github.com/react-native-webrtc/react-native-incall-manager lib
- Use
InCallManager
class to switch audio output:
Also, pay attention to InCall Manager lib if you need to use one of the options below:
- Manage devices events like wired-headset plugged-in state, proximity sensors and expose functionalities to javascript.
- Automatically route audio to proper devices based on events and platform API.
- Toggle speaker or microphone on/off, toggle flashlight on/off
- Play ringtone/ringback/dtmftone
Screen Sharing
Request a desktop stream by calling getDisplayMedia
:
In React Native, the getDisplayMedia method does not accept constrains as a parameter due to react-native-webrtc limitations.
If the local stream already exists, the next call to getUserMedia or getDisplayMedia will update the tracks in the stream and preserve the track’s enabled state for the audio track.
Screen Sharing on Android
Follow this guide to configure Screen Sharing on Android and this to support Screen Capture on Android 10+. Call the getDisplayMedia
after starting a ForegroundService:
Screen Sharing on iOS
Preform the Screen Sharing integration for iOS guide from Jitsi. Use <ScreenCapturePickerView>
component from ‘react-native-webrtc’ to prepare:
Group video calls
Because of Mesh architecture we use for multipoint where every participant sends and receives its media to all other participants, current solution supports group calls with up to 4 people.
Also ConnectyCube provides an alternative solution for up to 12 people - Multiparty Video Conferencing API.
Monitor connection state
There is a callback function to track the session connection state:
The possible values of connectionState are those of an enum of type ConnectyCube.videochat.SessionConnectionState
:
- ConnectyCube.videochat.SessionConnectionState.UNDEFINED
- ConnectyCube.videochat.SessionConnectionState.CONNECTING
- ConnectyCube.videochat.SessionConnectionState.CONNECTED
- ConnectyCube.videochat.SessionConnectionState.DISCONNECTED
- ConnectyCube.videochat.SessionConnectionState.FAILED
- ConnectyCube.videochat.SessionConnectionState.CLOSED
- ConnectyCube.videochat.SessionConnectionState.COMPLETED
Tackling Network changes
If a user’s network environment changes (e.g., switching from Wi-Fi to mobile data), the existing call connection might no longer be valid. Normally, in a case of short network interruptions, the ConnectyCube SDK will automatically restore the call so you can see via onSessionConnectionStateChangedListener
callback with connectionState
changing to DISCONNECTED
and then again to CONNECTED
.
But not all cases are the same, and in some of them the connection needs to be manually refreshed due to various issues like NAT or firewall behavior changes or even longer network environment changes, e.g. when a user is offline for more than 30 seconds.
This is where ICE restart helps to re-establish the connection to find a new network path for communication.
The correct and recommended way for an application to handle all such ‘bad’ cases is to trigger an ICE restart when the connection state goes to either FAILED
or DISCONNECTED
for an extended period of time (e.g. > 30 seconds).
Following is the preliminary code snippet regarding how to work with ICE restart:
-
Firstly, we have to disable the call termination logic after the network is disconnected for > 30 seconds by increasing the
videochat.disconnectTimeInterval
value to e.g. 5 mins. -
Secondly, define a variable which can track the Internet connection state:
-
Thirdly, define a function that will perform ICE restart:
-
Fourthly, define a
ConnectyCube.videochat.onSessionConnectionStateChangedListener
callback and try to perform ICE restart if not automatic call restoration happened after 30 seconds: -
Finally, in a case a user got working Internet connection later, do ICE restart there:
After these changes the call connection should be restored to working state again.
Configuration
There are various calling related configs that can be changed.
alwaysRelayCalls
The alwaysRelayCalls
config sets the WebRTC RTCConfiguration.iceTransportPolicy
config. Setting it to true
means the calling media will be routed through TURN server all the time, and not directly P2P between users even if the network path allows it:
Echo cancelation issue on some Android devices
A possible solution can be found here https://github.com/react-native-webrtc/react-native-webrtc/issues/713
Recording
Not available as for now
Continue call in background
If you are developing dedicated apps for iOS and Android - it’s required to apply additional configuration for the app to keeping the call alive when it goes into the background.
**iOS: **
There is no way to continue a video call in background because of some OS restrictions. What is supported there is to continue with voice calling while an app is in background. Basically, the recommended to achieve this is to switch off device camera when an app goes to background and then switch camera on back when an app goes to foreground.
Furthermore, even voice background call are blocked by default on iOS. To unblock - you need to setup proper background mode capabilities in your project. Please find the Enabling Background Audio link with more information how to do it properly. Generally speaking, you need to enable Voice over IP
and Remote notifications
capabilities:
**Android: **
For Android, we also recommend to implement the same camera switch flow when go to background and then return to foreground.
To keep the call while in background, we recomment to use the react-native-background-actions library.
The flow is the following:
- when start a call or go background -> display notification
- when stop call or return to foreground -> hide notification
CallKit
A ready RNVideoChat code sample with CallKit integrated is available at GitHub. All the below code snippets will taken from it.
For mobile apps, it can be a situation when an opponent’s user app is either in closed (killed) or background (inactive) state.
In this case, to be able to still receive a call request, a flow called CallKit is used. It’s a mix of CallKit API + Push Notifications API + VOIP Push Notifications API.
The complete flow is similar to the following:
- a call initiator should send a push notification (for iOS it’s VOIP push notification) along with a call request
- when an opponent’s app is killed or in background state - an opponent will receive a push notification about an incoming call, so the CallKit incoming call screen will be displayed, where a user can accept/reject the call.
The following libs should be used to integrate CallKit functionality:
- CallKeep lib for accessing CallKit API
- react-native-notifications lib along with ConnectyCube Push Notifications API guide for both push notifications and VOIP push notifications integration.
Below we provide a detailed guide on additional steps that needs to be performed in order to integrate CallKit into a RN app.
Initiate a call
When initiate a call via session.call()
, additionally we need to send a push notification (standard for Android user and VOIP for iOS). This is required for an opponent(s) to be able to receive an incoming call request when an app is in background or killed state.
The following request will initiate a standard push notification for Android and a VOIP push notification for iOS:
We recommend to simply copy-past the entire src/services/pushnotifications-service.js file from RNVideoChat code sample into your app.
Receive call request in background/killed state
The goal of CallKit is to receive call request when an app is in background or killed state. For iOS we will use CallKit and for Android we will use standard capabilities.
Android
First of all, we need to setup callbacks to receive push notification - in background and in killed state (there is a dedicated doc regarding how to setup a callback to receive pushes in killed state https://developers.connectycube.com/reactnative/push-notifications/#receive-pushes-in-killed-state-android):
What we do is we simply open app (bringing the app to foreground) once a push re incoming call is received and display an incoming call screen. This is done via react-native-invoke-app lib.
Also, we have PermisisonsService
to check if a user granted a DrawOverlays
permission to make the switch to foreground possible:
We recommend to simply copy-past the entire src/services/permissions-service.js file from RNVideoChat code sample into your app.
iOS
For iOS we need to setup CallKit. For this a react-native-callkeep library will be used. We recommend to connect a version from github rather than froim npm: "react-native-callkeep": "github:react-native-webrtc/react-native-callkeep#master"
All the logic is presented in call-service.js
file:
Also, when perform any operations e.g. start call, accept, reject, stop etc, we need to report back to CallKit lib - to have both app UI and CallKit UI in sync:
For the callUUID
we will be using call’s session.ID
.
The last point is to do the needed changes at iOS native code.
When receive a VOIP push notification in background/killed state, we must immediately display an incoming CallKit screen. Otherwise, the app will be banned with an error. To do so, the following changes in AppDelegate.mm
should be done:
All the above code snippets can be found in a ready RNVideoChat code sample with CallKit integrated.