This might be a radical approach, but why not contact the neighboring landowner and talk. While you are talking, maybe try to come up with a joint strategy that provides opportunity on both sides of the fence. Deer are going to cross back and forth regardless. Neither property is big enough to fully hold the local deer population, especially rutting bucks.
Also a consideration: do all involved parties actually know where the property lines are? Have seen a lot of times where landowners and others assume things like fences or roads are always on property lines. Have seen faulty memories of where lines start, end, and run. Have seen people who assume they own a property they in reality do not own (even to the extent of selling what they think they own). One guy even built a big fancy barn on another man's property. Have seen moved survey pins that have shown lines not as actually surveyed. And even using GPS vs USGS quads, lines can be significantly off between systems of coordinates or between GPS coordinates and USGS maps. Never a bad idea to have an actual survey and marked property lines before getting into a squabble.
Aside from that, maybe there is some non-sinister reason for how the stand faces, like it is just seasonal storage and not intentionally installed overlooking your property. If the neighboring property is good and of some size what is his incentive for hunting just over your fence?
Besides, if you build animosity, how likely is he to let you cross to his side for that wounded big buck that made it over to his side of the fence?
