I generally can't tell the difference between deer/elk/caribou/moose/any other big game animal, assuming they were cared for the same way. I have weak taste buds I guess however a few bighorn that I have tried has had a very slight mutton taste to it. It hasn't been bad or offsetting at all, but is something I have noticed. It is nothing like rack of lamb IMO, as it is still a lean game meat.
I can say though that dall sheep (read: not bighorn) has been the most tender and non gamey game meat I have ever tried.
Your taste buds are not too weak.
The big horns and 1 desert big horn I have eaten do a very slight mutton taste. I like lamb chops so it adds a little variety.
I think caribou has a slightly different flavor that say elk but not better or worse, just a bit different.