You may be interested in building rifles but people who can afford to buy them usually want it built by somebody with a reputation. It takes time to build that. You will end up putting on recoil pads and mounting scopes.
You will have to deal with people. Some are a pleasure, others may be a challenge.
See if you can visit a local gunsmith after hunting season is over and see how he keeps busy.

I like to do my hunting BEFORE I pull the trigger!
There is only one kind of dead, but there are many different kinds of wounded.