I don't typically acknowledge panhandlers, either, for all of the reasons mentioned above. In my experience, a significant number will take any interaction as a reason to continue to ask, try to make me feel guilty, get threatening, etc.
I work in the downtown area of my city and have for many years now. And for quite a long time, a stranger approaching me likely fell into one of three categories. They were most likely going to: 1) ask me for money, 2) push their particular brand of religion, or 3) try to pick me up. So I pretty much started ignoring approaches from strangers, except for maybe a headshake followed by moving away unless they move away first. I don't think it's retaliatory rudeness, I think it's self-protection.
Frankly, I think that someone approaching a woman on the street and expecting a "nice" response is at best a bit naive, and at worst has intentions I want nothing to do with, and I'm not going to take the chance if I can help it.
Very rarely I'd get someone asking for directions, and if they started with, "Excuse me, can you tell me where X is," and I didn't get a creepy vibe, I'd try to point them in the right direction. Otherwise, I'd just say, "Sorry, no," without slowing down.