I don't find using a 300 mag or whatever for deer silly in and of itself. What I may find silly is someone's reason for doing so.
I understand what you’re saying but...how about this.
I used my 300 mags deer hunting TO get more familiar with the rifle itself & its recoil while hunting AS I intended to use it for Elk Hunting. Someone else might use one in preparation for hunting something like Moose.
I know you don’t need a 300 to kill deer but I wanted to be accustomed to the rifle for hunting something else.