Besides capturing audio and video from a Windows desktop or app, Rainway also supports “Bring Your Own Frame Buffer” (BYOFB) mode.
In this mode, Rainway expects your host-side app to submit custom audio and video frames to the Rainway SDK. This submitted media is what gets encoded and transmitted to the client.
At the same time, the client's remote keyboard/mouse/gamepad inputs are not fed back into Windows, but fed into your app as input events.
This mode is geared towards integration of Rainway into a single complex graphical application (like a game) that can provide such frame buffers and isn't interested in going through the OS layer.
Currently, BYOFB works with DirectX 11+ applications. Future integrations with Unity and Unreal Engine are on the roadmap.
In RainwayStreamConfig, set the RainwayStreamType to
The resulting stream is in BYOFB mode: you can now use the
submitFrame methods on the Stream object. Until you do so, the stream is blank and silent. The desktop is not captured.
onStreamInput to handle remote input events and feed them back into your app.
Your app can pass PCM audio buffers in two formats: 16-bit signed integer and 32-bit float. See the API documentation for
Your app decides the length of the individual audio buffers. Call
submitAudio again precisely at the end of the last buffer: Rainway will discard any previously playing audio buffer and play the new buffer immediately.
The maximum supported sample rate is 48,000 Hz.
Your app passes D3D11 texture buffers into
submitFrame. This sets the "current video frame", which Rainway polls at the frame rate of the stream (i.e. there is no buffering). To ensure smooth video, this should be done once every 1/60 seconds or faster.
An example C++ app using Rainway's BYOFB mode and MediaFoundation is available here.
Updated 3 months ago