This is one of those lose lose situations you get when you don't work together with your neighbors in a rural area. You let your neighbors know your planned use, ask them what you can do to make it work for them (limited flights and hours and avoiding homes on the path, the Fred a ride in a medical emergency) feed them at a BBQ and see if that works. At worst your neighbors think you are an ok guy and don't look to cause problems for you.
If that doesn't work screw them and build a long strait road to your new big barn that happens to fit a plane.
I like how you think.
Its not unusual to see a private airfield, of sorts, out in the WY boonies. Crop dusters, outfitters, etc. I'll bet its not unusual for Idaho either. Small rural communities tend to be clannish. Best though to shake some hands and try to negotiate first.
