I think you’ll be fine not carrying a pistol for backup. 😉
I wouldn’t throw my rifle down and Wild Bill Hickok a charging bear.
I never carry a handgun if I'm carrying a rifle. OTOH when we go back to camp after shooting game and get all the stuff to clean etc... I swap the rifle for a handgun...
I just remembered to check this thread and got as far as your post, Jeff, but apparently once again we are similar. 😁. When I hunt with a rifle I’ll carry my Ruger 22/45 for grouse and general duty. When fishing, collecting wood or other chores that preclude the carrying of a rifle I carry a Glock 20 with either 200gr HC or 220 hardcast. As accurate as my 22/45 is and considering how well I shoot it, I don’t feel under gunned when I carry it.