Don't carry it on your backpack strap. The minute you take your pack off, you are unarmed. There is a reason why chest rigs like the Hill People Gear are so popular. Because the work perfectly for this.
I agree. Can also carry a few essential items that never leave you with the pack off.