Why not just go talk to the neighbor and explain the dilemma he is creating for you?