Another thing to consider is resalability. Buying a house with a known flaw can be risky because when you want to move out, you'll be in the same spot as the couple who own the house now. And yes, a neighbor who is known for imposing on others in the neighborhood is a known flaw. I don't see anything rude about deciding not to buy a house, or anything else for that matter, for any reason in the world. My money, my decision. Nothing rude about that.