The big difference I've noticed is mature mule deer bucks SOMETIMES get raunchy during the rut, and whitetails normally don't, but even then there are exceptions. My wife killed a big 3x3 mule deer on the Saturday after Thanksgiving a few years ago, so rutted-out there were only a few tiny specks of fat left on his meat. We though he'd be gamey and maybe tough as well, but after aging a week the meat was mild and tender.
If a rutty muley is gamey, the taste tends to concentrate in the connective tissue. Often the bigger cuts taste fine, but the stew and burger aren't as good, and get gamier as they spend more time in the freezer.
Also, over-cooking tends to make any wild game taste gamier. The surest way to make the meat of many big game animals and dark-meated birds taste like liver is cook it too much, but a lot of people do exactly that. Some kinds of game are more tolerant of such abuse, which is why some people think those animals always taste better, but the cooking technique's at fault, not the meat.
Have killed plenty of mature mule deer bucks before the rut and cannot remember one that wasn't very good, and indistinguishable from good whitetail. But here in Montana the regular rifle season goes through the rut, and most people don't kill larger bucks until they're rut-stupid, whereupon they complain about how mule deer taste gamey.
Have also heard about how mule deer from sagebrush country don't taste good, but have killed plenty and before the rut never found any significant difference in taste. As a result, tend to believe something else is going on, whether the bucks are killed in the rut or over-cooked--which as mentioned above tends to exaggerate some flavors.
These opinions also aren't just those of me and my wife, but come from the many people whom we've served both kinds of deer (and other kinds of big game). More than a few have even commented on how good "young" deer taste during the meal, and are astonished when I show them a photo of the big muley buck they just ate.
