I'd still go with a .22mag if it were me (actually a 25-20 or downloaded .357 if it were me). .22mag is leagal for deer in Maine and SC, if I remember correctly.
If you are determined to go this route, look at an actual deer skull, the temple is obviously the softest spot.
I do not know about private land, but the minimun caliber for deer in S.C. is still .243/6mm on public land. DO NOT try using a rimfire for deer on public land. I have known several of the game wardens down there. They WILL find you, and forgiveness, especially for Yankees or Damn Yankees, is not in their lexicon.