I have a few funny examples I can add to this...
One of the first friends I met when we moved here (to Hong Kong) is caucasian/American, but at the time had already lived here for about 5 years, and had previously served a mission for our church here and learned to speak the language fluently. She is quite a bit larger than your average asian woman. One time she got into an elevator with some elderly Chinese women and they immediately started talking rudely about her, something to the effect of not wanting to be in the same elevator as the fat gweilo. (Gweilo being a term used for foreigners, sometimes not a nice thing). My friend turned to them and in perfect Cantonese basically told them if they didn't want to be in there with her, then they could get out. (yes, it was rude, but seriously, how would you respond???) The ladies were quite shocked and embarassed and didn't say another word until it was their turn to get off.
The other is with my dh. He is HK Chinese, born here, and lived here until he was about 13 when his family immigrated to the US. (He speaks Cantonese, Mandarin, and English all fluently.) I learned quite quickly here that most people do speak at least a little English, so you really do have to be careful what you say. I'm sorry to say that my dh still hasn't learned that lesson.

Anyway, we were waiting at a bus stop, and there was a couple there looking at the route map trying to decide if that was the right bus for them. They were obviously from mainland China (can tell by their accent and dress) and dh said something naughty in English about mainland Chinese (can't remember for the life of me what it was, though). I quickly hushed him and reminded him that they probably spoke English and he resisted saying that he could tell by their accent that they were from a poor province, probably didn't have much schooling, that sort of thing. Boy was he surprised a minute later when they came up to him and asked him for directions... in English! My poor dh was so embarassed, but I have to say that I'm glad it happened because he does need to learn to control himself, and think before he speaks.
And in regards to if you are living in a foreign country that you should learn the language... I completely agree. However, that makes me a bit of a hypocrite because we've been here 3.5 years, and I know a few phrases and things, but am nowhere near being able to carry on any sort of conversation. I guess it's because I haven't felt like I really needed to learn it. Like I said earlier, pretty much most of the population knows at least a little English, and so I can get by in the stores or in the taxi or whatever. Dh handles pretty much anything else, and I'm surrounded by English speaking people (church, other activities). I have tried a few things to learn more, but each time it just hasn't panned out. (Not enough money for private lessons, no one to watch my daughter etc etc.) So I don't think you (general you, no one in particular) should be too harsh on people who don't speak the language because you don't know their specific circumstances. Am I sick of dh's relatives asking him if I can speak it yet? YES! But that doesn't change the fact that I don't have money/babysitter/tons of time to dedicate to it at this point in my life when I can get by without it.