Realtime Audio

The Flora Engine Plus plugin also provides a realtime audio capture system. For the simplest setup follow these steps:

1. In a blueprint, use the Get WhisperSubsystem node and drag off of it to use the Initialize Whisper node.
2. From the Whisper Subsystem, use the Bind Event to On Realtime Whisper Processed and create a custom event for it.
3. After binding the event, use the node Whisper Realtime Capture and set Realtime Whisper to true.
4. From the bound event, process the transcribed audio data as required.

This system will use Voice Activity Detection by default to detect when there is a speaker. Other processing layers including noise suppression and endpoint detection can be enabled. View the settings documentation for more information.

An interrupt system is also available. By setting the Interruptable variable to true on the WhisperSubsystem, the realtime capture will then switch to detecting if the user speaking matches the InterruptVoiceName (can also be set on the Subsystem) and the OnInterruptDetected event will fire if detected. This can be useful to detect if the user is speaking while another character is speaking in order to interrupt their dialogue.