Operating room WebRTC based communication system

• Direct feed from and into the OR. Video from medical devices 2D and 3D: medical cameras, microscopes, endoscopes, C-arms, surgical robots, and other medical video devices up to 4K@60Hz, both in 2D and 3D.

• Remote consultations (second opinion) and real-time audio/video collaboration for doctors.

• Medical education for medical universities and schools of surgery (incl. interactive).

• Advanced Surgical Skills.

• Symposiums and conferences.

Software audio/video streaming and collaboration library

Real-Time video streaming.
Audio.
Chat.
Data sharing (files, messages).
Drawing over the video (visible for all participants). Drawing over the video is supported from any remote device that has a touch screen. It’s implemented as JS inside the browser. The drawing browser sends to the server a set of points defining the curve and the server draws it inside the video frames.
Video overlays.
Echo cancelation, noise reduction.
In case of use SoC of sufficient power, then the filters could be added for video (improving contrast, clarity, etc.) if needed.
Full control from operation room (camera selection, start / stop video, mute / unmute audio, audio volume control, etc.).
Zero configuration – works through NAT and Firewalls.
The code has passed TÜV tests for use in healthcare facilities.

Technical information

Easy to embed

Interaction between the clients

We do not require external services.

An HTTP server, MQTT broker (for signaling and control), STUN / TURN, WebRTC server are implemented in one module.

Capabilities:

The server is easy to embed (runs on all Raspberry pi as well as docker).
Security declaration. Security is built- in MQTT and STUN/TURN – so all WebRTC connections share common security requirements. Security is built on the highest standards – TLS 1.3 and JWT. No plain text messages.
Communication between the server and clients has an additional security layer on top of TLS 1.3. Each message request is additionally signed with ED25519.
Audio streaming supports the new improved method of audio mixing. Each connected client will receive a mix of all other audio sources except his own – reducing the feedback and improving noise cancellation.
Data channels allow interaction between the clients so that each client can draw, point real-time over the live video stream.
Video quality is up to 4K@60Hz with hardware encoding H264/VP8/VP9.
Video pre-processing and quality enhancements (frame-rate, color-space adjusting, blur/sharpness).
Video/Audio streaming is based on modified WebRTC. It’s compatible with the Web and with all standards. Besides that, it adds additional new capabilities.
A single video streaming source can stream to multiple clients without the use of SFU.
Hardware video encoding for Android and Linux.
Newest video streaming standards including AV1 video codecs.
Pre-defined video profiles for support of multiple connected devices without additional video encoding sessions. For example, a single VP9 /1080P can stream up to 100 clients – where the only limitation is the network bandwidth

Multi-platform capable of running on multiple SoCs and operating systems:

Server:

– Linux – x86, x86_64, armv6, armhf, armv7, armv8, arm64
– Android armeabi_v7a, arm64, x86, and x86_64 – Android 5.0+ Lollipop, API level 21
– iOS 10.1 +
– macOS (arm64 and x86_64)
– Windows 7+

Client:
– Web and PWA

– Linux (x86_64)
– Windows 7+
– iOS 10.01+
– macOS (arm64 and x86_64)
– Android (x86, x86_64, armeab_v7a, arm64) Client software is implemented using emerging technologies compiled to native code –
including Web using WebAssembly – for speed. All clients have similar behavior and look yet

offer full capabilities of their platforms (iOS, macOS, Web, Windows, Android, and Linux
desktop).