Haven't tried it myself, but this looked like a possibility, for me anyway. May drop the pistol just far enough under the pack belt to be out of the way, but not halfway down your thigh like the typical "tactical" leg holsters. They have chest holsters as well, but I generally have binoculars and a range finder there.

https://gunfightersinc.com/shop/micro-aggression-thigh-panel/