Mine was a Savage 99 30/30. Still have it, still use it occasionally, still kills deer, and it turned 99 yrs old this year. It was also my Father's first deer rifle - he passed it to me when I turned 16 (minimum big game age when I was growing up).
I also still think that the old medium calibers like 30/30 and 32 Win Special are good beginners/youth choices. Moderate recoil, good performance on game - but availability of rifles is getting pretty limited if you don't like the old stuff.
These days you'll have a lot more choices in the 6 to 7mm range: .243, .260, one of the 6.5s, and 7mm/08. You can also take a look at the old 250 Savage and 257 Roberts if you come across one. Any of them will kill deer if you teach him to hunt and shoot, recoil and muzzle blast won't be excessive, and there are lots of choices in pretty much any price range. Buy an extra stock to cut down until he finishes growing.
I don't think long action cartridges are the best starting points, let him make that choice later on. And I definitely don't think anything smaller than 6mm/.243 is good for a beginner - precise shot placement gets critical, leave him some room for error.
Now in my mid 60s, my "deer rifles" are almost all short action 30 calibers - 30/30, 300 Savage and .308, with a 250 Savage hanging around on the fringe. Some are lever actions, some are bolt guns, some are scoped, and some have receiver aperture sights. Believe I'm pretty well covered for any weather I care to hunt in and any range I'd care to shoot at.
