I always set the sound level with the IEMs firmly in place in my ears and never change it. I have IEMs molded to my ears, and taking one out, just a little bit, from one ear, is enough for me to hear the crowd’s responses.
And when I need precision listening and I no longer need to listen to the audience, I put it back. But it’s all fiddling for when we don’t have an ambient mic.