Some research on latency, jitter and buffer size

Hello,

tl,dr:
Gig Performer offers sufficient low latency and negligible jitter at a buffer size of 128 samples.
Full story here:
http://www.powerlord.de/site/angel/LatencyTestReport2021.pdf

I have been messing around with this topic for some time, and now I finally did it: Measuring the MIDI-to-audio real-world latency and timing jitter depending on the buffer size, comparing Gig Performer and Brainspawn Forte, my old host used for over 15 years.

Latency and minimizing it is one of the most discussed topics for digital audio. More than latency, the random latency jitter makes live playing feel untight and sluggish. So time for some facts on it.

Here’s a sketch of my experiment: Pressing a key on the master keyboard, using the MIDI message on the cable to the MIDI interface as a trigger for a digital storage oscilloscope (DSO), and let the oscilloscope measure the time from the last bit of the MIDI note message until a first sound comes out of the audio interface. By using the statistics functions of the DSO, the latency jitter is calculated as standard deviation of the latency for 16 repeated note events.

Tested VST plug-ins were NI Kontakt 6.4 and Spectrasonics Omnisphere 2.6. Hosts were Brianspaen Forte 4.5 and Gig Performer 3.8.1. Stand-alone was compared to the plug-ins in the hosts.

Here are the results for the latency of NI Kontakt 6.4 at 44.1 kHz (Omnisphere is quite similar):

As a reference, Kontakt was used in stand-alone mode (blue trace). Almost the same latency has been measured for Kontakt as a VST plug-in inside of Gig Performer (green trace). A significantly higher latency at same buffer sizes is observed for Kontakt as VST plugin inside of Forte (red trace).

For comparison with theory, linear extrapolation functions as well as the theoretical latency for different numbers of buffers and different buffer sizes (black dashed traces) at a sample rate of 44.1 kHz are shown. From this it can be concluded that Gig Performer uses only the two buffers required for ASIO drivers, whereas Forte uses a third buffer for internal processing, increasing total latency. From the linear extrapolation and the intersection with the vertical axis, a very similar overhead ranging from 1.3 ms to 1.8 ms is found on top of the latency generated by two or three buffers. This overhead might be for some safety buffers inside of the hosts, or for MIDI processing of the USB MIDI interface.

Interestingly, the measured audio-out to audio-in roundtrip latency of the audio interface is almost the same as the MIDI-to-audio latency of Kontakt stand-alone or within Gig Performer. The internal roundtrip latency measurement function inside Gig Performer reported exactly the same numbers as the RTL Utility by Oblique Audio, a dedicated program to measure round trip latency of audio interfaces. Consequently, the simple roundtrip latency measurement is not only relevant for effect chains for vocals or guitars, but also for keyboard or drum players triggering sounds and samples via MIDI.

The most interesting results are seen in the measured latency jitter as shown in the next figure for NI Kontakt. In stand-alone mode, NI Kontakt 6.4 has a jitter of only 67 µs (standard deviation) from note to note. This is beyond significance for any musician, even for the tightest machine-like drummer. For Kontakt as a VST plug-in in Gig Performer, the typical jitter is a little bit larger and around 250 µs. If we remember the speed of sound, then this jitter is equivalent to moving your head 7 cm towards or away from the sound source. Obviously, such a low jitter is of no significance for any musician, too. Remarkably, the jitter for these two cases is independent of the buffer size. Both, Kontakt stand-alone and Gig Performer seem to use a philosophy of an equal delay for each note while processing the two audio buffers. Although this results in a slightly larger latency on average, this is less perceived as a bad timing than a jitter would be.

Very different results can be seen for Forte. The jitter is much larger as for Kontakt stand-alone or within Gig Performer and is rising with buffer size. This clearly indicates that Forte uses the philosophy of generating audio at the soonest possibility after an incoming note-event within one buffer time frame, reducing average latency, but introducing jitter. However, even average latency is larger for Forte as it uses at least three buffers instead of two like Kontakt alone or Gig performer.

Starting from a jitter of 1 ms or larger, musicians start to experience a somewhat untight playing of a groove. Keep in mind that these numbers for the jitter are a standard deviation or rms (root mean square) value, so by chance there are also some notes that are out of timing by three times the rms jitter, at a completely random distribution over repeated notes. This timing jitter has nothing to do with a dedicated note delay for a certain human groove, where within the groove pattern always the same note is shifted by the same amount in time, and this shift can be much larger than 1 ms for many great sounding human grooves. In contrast, the latency jitter discussed here is random in time and random on the notes of a pattern, so even a small jitter of only a few ms sounds unprofessional. I don’t want to exaggerate here, but you can feel it.

Summary and conclusion: What is the impact of these results for live musicians?

I have been using Brainspawn Forte as plug-in host for live performance for over 15 years. During this time, I have optimized all my notebooks, audio interfaces (Echo Indigo DJx, then RME Babyface Pro) and Windows systems (XP, Win7, Win10) such, that I could play Forte with 128 samples buffer size at first, and in later years with 96 samples or lately even with only 64 samples buffer at 44.1 kHz sample rate. The later achieved a quite good feeling for the groove and timing while playing keyboards. Forte was discontinued by Brainspawn in 2017. When switching to a new laptop, also a change of the plug-in host was on the schedule. When testing Gig Performer, I noticed subjectively, that Gig Performer offered the same or even better timing perception with 128 samples buffer, compared to Forte at 96 or 64 samples. The measurements presented here are an objective proof of this perception, and a final reason to make the switch from Forte to Gig Performer.

To sum up, Gig performer 3.8 adds no extra MIDI-to-audio latency for synth and sampler plug-ins like Kontakt or Omnisphere. In addition, the latency jitter is very small and beyond the perception of humans, and independent on the buffer size. Lowest possible jitter is measured for Kontakt stand-alone, but only robots could hear an improvement. Gig Performer together with an proper audio interface and drivers offers a very immediate and tight playing experience already at a buffer size of 128 samples at 44.1 kHz sample rate.

Now that this has been clarified, I can start over to making music again …

PS:
I you really have same time and dare to read the full story, please have a look here:
http://www.powerlord.de/site/angel/LatencyTestReport2021.pdf

PPS:
Comments, questions and corrections are welcome. Maybe I like to post it to KVR Audio forum, Sound on Sound forum, RME Adio forum, too, to get some additional feedback.

12 Likes