Introduction

Besides capturing audio and video from a Windows desktop or app, Rainway also supports “Bring Your Own Frame Buffer” (BYOFB) mode.

In this mode, Rainway expects your host-side app to submit custom audio and video frames to the Rainway SDK. This submitted media is what gets encoded and transmitted to the client.

At the same time, the client's remote keyboard/mouse/gamepad inputs are not fed back into Windows, but fed into your app as input events.

Who is it for?

This mode is geared towards integration of Rainway into a single complex graphical application (like a game) that can provide such frame buffers and isn't interested in going through the OS layer.

Currently, BYOFB works with DirectX 11+ applications. Future integrations with Unity and Unreal Engine are on the roadmap.

Getting started

In RainwayStreamConfig, set the RainwayStreamType to Byofb.

The resulting stream is in BYOFB mode: you can now use the submitAudio and submitFrame methods on the Stream object. Until you do so, the stream is blank and silent. The desktop is not captured.

Listen to onStreamInput to handle remote input events and feed them back into your app.

Submitting frame buffers

Audio

Your app can pass PCM audio buffers in two formats: 16-bit signed integer and 32-bit float. See the API documentation for submitAudio.

Your app decides the length of the individual audio buffers. Call submitAudio again precisely at the end of the last buffer: Rainway will discard any previously playing audio buffer and play the new buffer immediately.

The maximum supported sample rate is 48,000 Hz.

Video

Your app passes D3D11 texture buffers into submitFrame. This sets the "current video frame", which Rainway polls at the frame rate of the stream (i.e. there is no buffering). To ensure smooth video, this should be done once every 1/60 seconds or faster.

Example app

An example C++ app using Rainway's BYOFB mode and MediaFoundation is available here.