Audio and midi metronome not in sync

I use the Gigperformer metronome on all songs and send the signal to the drummer. On the one hand, to always have the same tempo (we often play with substitute drummers) and on the other hand to be in sync with arpeggios (I use the BlueARP Arpeggiator and also the NI Session Guitarrist) and to be in sync with backing tracks in Ableton Live (using Ableton Link).
I have now noticed, that the Gigperformer Midi Metronome is not perfectly in sync with Ableton Live (the audio metronom does) and the audio metronome on the other hand is not in sync with the BlueARP Arpeggiator.
That’s why I now use the MIDI metronome in Gigperformer (to be in sync with arpeggios) and put a delay of 40 (!) ms to the main channel of Ableton Live to be in sync with Ableton at the same time.
I can live with this workaround but I still wonder if this is a normal behavior (?)
My audio interface is a Focusrite Scarlett 18i8, buffer size 128 and the measured round trip latency is 14 ms.

It’s not clear to me what plugins are in GP and what plugins are in Ableton. Or is it only backing tracks that are in Ableton? Can you please provide some more explanation of your setup.

I use only audio backing tracks in Ableton and the metronome from Gigperformer. You can easily reproduce this problem without Ableton: Use the Midi and Audio Metronome in Gigperformer simultaneously and you can hear that they are not in sync. I use the midi metronome as described here: How to create a custom metronome with visual feedback - Gig Performer®