Multiparty Video Conferencing feature overview
ConnectyCube Multiparty Video Conferencing API is built on top of WebRTC protocol and based on top of WebRTC SFU architecture.
Max people per Conference call is 12.
Video Conferencing is available starting from Advanced plan.
To get a difference between P2P calling and Conference calling please read our ConnectyCube Calling API comparison blog page.
Features supported
- Video/Audio Conference with up to 12 people
- Join-Rejoin video room functionality (like Skype)
- Guest rooms (coming soon)
- Mute/Unmute audio/video streams
- Display bitrate
- Display mic level
- Switch video input device (camera)
- Switch audio input device (microphone)
- Switch audio output device (desktop platforms and chrome-based browsers)
- Screen sharing
- Simulcasting
Get started with SDK
Follow the Getting Started guide on how to connect ConnectyCube SDK and start building your first app.
Code sample
There is ready-to-go FREE Conference Calls Sample to help you better understand how to integrate video calling capabilities in your apps.
Preparations
Required preparations for supported platforms
iOS
Add the following entries to your Info.plist file, located in <project root>/ios/Runner/Info.plist
:
This entries allow your app to access the camera and microphone.
Android
Ensure the following permission is present in your Android Manifest file, located in <project root>/android/app/src/main/AndroidManifest.xml
:
If you need to use a Bluetooth device, please add:
The Flutter project template adds it, so it may already be there.
Also you will need to set your build settings to Java 8, because official WebRTC jar now uses static methods in EglBase
interface. Just add this to your app level build.gradle
:
If necessary, in the same build.gradle
you will need to increase minSdkVersion
of defaultConfig
up to 18
(currently default Flutter generator set it to 16
).
macOS
Add the following entries to your *.entitlements files, located in <project root>/macos/Runner
:
This entries allow your app to access the internet, microphone, and camera.
Windows
It does not require any special preparations.
Web
It does not require any special preparations.
Linux
It does not require any special preparations.
Create meeting
In order to have a conference call, a meeting object has to be created.
name
- the meeting name.- As for
attendees
- either ConnectyCube users ids or external emails can be provided. start_date
- the date when meeting starts.end_date
- the date when meeting ends.- Pass
withChat = true
if you want to have a chat connected to meeting. - Pass
record = true
if you want to have a meeting call recorded. Read more about Recording feature https://connectycube.com/2021/02/23/connectycube-releases-server-side-calls-recording-along-with-new-meetings-api/
Once meeting is created, you can use meeting.meetingId
as a conf room identifier in the below requests when join a call.
ConferenceClient setup
To manage Conference calls in flutter you should use ConferenceClient
. Please see code below to find out possible functionality.
Create call session
In order to use Conference Calling API you need to create a session object - set your current user and a type of session (VIDEO or AUDIO optional). ConferenceSession
creates via ConferenceClient
:
Add listeners
Also, there are a few callbacks in the callSession
to manage the connection state with user:
Join video room
Just join the room and for example, send invite to some opponent:
Subscribe/unsubscribe
The SDK automatically subscribes to publishers when get the onPublishersReceived event, but you may need to subscribe manually (for example after unsubscribing) to the active publisher:
To unsubscribe from publisher:
Leave
To leave current room session:
Mute audio
Switch audio output
For iOS/Android platforms use:
For Chrome-based browsers and Desktop platforms use:
Mute video
Switch video cameras
For iOS/Android platforms use:
For the Web platform and Desktop platforms use:
Get available cameras list
Get available Audio input devices list
Get available Audio output devices list
Use the custom media stream
Toggle the torch
Screen Sharing
The Screen Sharing feature allows you to share the screen from your device to other call members. Currently the Connectycube Flutter SDK supports the Screen Sharing feature for all supported platforms.
For switching to the screen sharing during the call use next code snippet:
Android specifics of targeting the targetSdkVersion
to the version 31 and above
After updating the targetSdkVersion
to the version 31 you can encounter an error:
To avoid it do the next changes and modifications in your project:
1. Connect the flutter_background
plugin to your project using:
2. Add to the file app_name/android/app/src/main/AndroidManifest.xml
to section manifest
next permissions:
3. Add to the file app_name/android/app/src/main/AndroidManifest.xml
to section application
next service:
4. Create the next function somewhere in your project:
and call it somewhere after the initialization of the app or before starting the screen sharing.
5. Call the function FlutterBackground.enableBackgroundExecution()
just before starting the screen
sharing and function FlutterBackground.disableBackgroundExecution()
after ending the screen sharing
or finishing the call.
IOS screen sharing using the Screen Broadcasting feature.
The Connectycube Flutter SDK supports two types of Screen sharing on the iOS platform. There are In-app screen sharing and Screen Broadcasting. The In-app screen sharing doesn’t require any additional preparation on the app side. But the Screen Broadcasting feature requires some.
All required features we already added to our P2P Calls sample.
Below is the step-by-step guide on adding it to your app. It contains the following steps:
- Add the
Broadcast Upload Extension
; - Add required files from our sample to your iOS project;
- Update project configuration files with your credentials;
Add the Broadcast Upload Extension
For creating the extension you need to add a new target to your application, selecting the
Broadcast Upload Extension
template. Fill in the desired name, change the language to Swift, make
sure Include UI Extension
(see screenshot) is not selected, as we don’t need custom UI for our case, then press
Finish. You will see that a new folder with the extension’s name was added to the project’s tree,
containing the SampleHandler.swift
class. Also, make sure to update the Deployment Info, for the
newly created extension, to iOS 14 or newer. To learn more about creating App Extensions check the
official documentation.
Add the required files from our sample to your own iOS project
After adding the extension you should add prepared files from our sample to your own project. Copy
next files from our Broadcast Extension
directory: Atomic.swift
,
Broadcast Extension.entitlements
(the name can be different according to your extension’s name),
DarwinNotificationCenter.swift
, SampleHandler.swift
(replace the automatically created file),
SampleUploader.swift
, SocketConnection.swift
. Then open your project in Xcode and link these files
with your iOS project using Xcode tools. For it, call the context menu of your extension directory, select
‘Add Files to “Runner”…’ (see screenshot) and select files copied to your extension directory before.
Update project configuration files
Do the following for your iOS project configuration files:
1. Add both the app and the extension to the same App Group. For it, add to both (app and extension)
*.entitlements
files next lines:
where the group.com.connectycube.flutter
is your App group. To learn about working with app groups,
see Adding an App to an App Group.
We recommend you create the app group in the Apple Developer Console before.
Next, add the App group id value to the
app’s Info.plist
of your app for the RTCAppGroupIdentifier
key:
where the group.com.connectycube.flutter
is your App group.
2. Add a new key RTCScreenSharingExtension
to the app’s Info.plist
with the extension’s Bundle Identifier
as the value:
where the com.connectycube.flutter.p2p-call-sample.app.Broadcast-Extension
is the Bundle ID of your
Broadcast Extension. Take it from the Xcode
3. Update SampleHandler.swift
’s appGroupIdentifier
constant with the App Group name your
app and extension are both registered to.
where the group.com.connectycube.flutter
is your app group.
4. Make sure voip
is added to UIBackgroundModes
, in the app’s Info.plist
, in order to work when
the app is in the background.
After performing mentioned actions you can switch to Screen sharing during the call using useIOSBroadcasting = true
:
Requesting desktop capture source
Desktop platforms require the capture source (Screen or Window) for screen sharing. We prepared a widget ready for using that requests the available sources from the system and provides them to a user for selection. After that, you can use it as the source for screen sharing.
In code it can look in a next way:
The default capture source (usually it is the default screen) will be captured if set null
as a capture source
for the desktop platform.
WebRTC Stats reporting
Stats reporting is an insanely powerful tool that can provide detailed info about a call.
There is info about the media, peer connection, codecs, certificates, etc. To enable stats report you should first set stats
reporting frequency using RTCConfig
.
Then you can subscribe to the stream with reports using the instance of the call session:
To disable fetching Stats reports set this parameter as 0.
Monitoring mic level and video bitrate using Stats
Also, we prepared the helpful manager CubeStatsReportsManager
for processing Stats reports and getting
some helpful information like the opponent’s mic level and video bitrate.
For its work, you just need to configure the RTCConfig
as described above. Then create the
instance of CubeStatsReportsManager
and initialize it with the call session.
After that you can subscribe on the interested data:
After finishing the call you should dispose of the manager for avoiding memory leaks. You can do it in
the onSessionClosed
callback:
Configurations
ConnectyCube Flutter SDK provides possibility to change some default parameters for call session.
Media stream configurations
Use instance of RTCMediaConfig
class to change some default media stream configs.
Call connection configurations
Signaling implementation
To implement regular calls with events such as call, reject, hang up there should be used some signaling mechanism.
Signaling
As signaling mechanism there can be used ConnectyCube system-messages with predefined custom properties.
Start Call
Just join the room and send an invitation start call message to opponents:
Reject Call
Send reject message when decline/busy call:
End call
Send end call message when hangup/answer_timeout call:
Get call events
Listen and parse all call events with systemMessagesManager:
Adding user to call
For adding user to current call you can send invite message with current roomId and participantIds:
And then on the receiver side when the new user successfully joins the room, he automatically subscribes to all active participants in current call (at the same time, other participants will receive onPublishersReceived and will be able to subscribe to that new user).
Retrieve meetings
Retrieve a meeting by id:
Retrieve a list of meetings:
or use the getMeetingsPaged
for requesting meeting using the pagination feature:
Edit meeting
A meeting creator can edit a meeting:
or use the method updateMeetingById
for updating only some fields of the meeting model:
Delete meeting
A meeting creator can delete a meeting:
Recording
Server-side recording is available. Read more about Recording feature https://connectycube.com/2021/02/23/connectycube-releases-server-side-calls-recording-along-with-new-meetings-api/
Simulcasting
Simulcast is a feature in video conferencing that enables the simultaneous broadcasting of multiple versions of the same video stream, each with different resolutions and bitrates. This feature is commonly used to optimize the video quality for all participants in a video call, especially when some participants have slower or less reliable internet connections.
In a video call, the video stream is sent from the sender’s device to the recipient’s device. If the recipient has a slow or unreliable internet connection, the video quality may suffer, and the video may lag or freeze. Simulcast allows the sender to send multiple versions of the video stream, each with different resolutions and bitrates.
Overall, simulcast is a useful feature in video conferencing that can help improve the quality and reliability of video calls, especially when participants have different network conditions.
Request preferred quality of incoming streams
Use the next code snippet to request the preferred quality of the remote stream for user:
Or there is a separate method for requesting the different qualities using a single request for the set of users:
Limit the bitrate of your own stream
The sender can limit the bitrate of the own video stream according to the app logic using the next code snippet:
Listen to the stream quality changes
When the sender limits the stream’s quality or the server limits the stream’s bitrate according to its internal logic the receivers will receive a notification about it. To listen to these events just subscribe to the next event broadcaster:
Temporal layers
During a conference call, video is typically transmitted in real-time between multiple participants. However, the quality of the video may suffer due to network bandwidth limitations, especially when there are multiple participants with varying network conditions.
Temporal layers allow for the efficient transmission of video by dividing the video frames into different layers. The lower layers contain basic information about the video, such as color and shape, while the higher layers contain more detailed information, such as facial expressions or small movements.
Request the layer of the stream
By default, the server sends the layer with the highest quality but you can request another one according to your app needs.
Use the next code snippet for requesting of the preferred layer:
Or use the next method for requesting different layers for a lot of users in a single request:
!> Pay attention: not all platforms support the layer feature in full. For example, iOS/Android platforms support three layers and the Chrome browser supports only one layer, etc.
Listen to the stream layer changes
After manually requesting the preferred layer the server sends the event the about success of changing the layer for user. To listen to these events just subscribe to the next event broadcaster: