This is a very rude thing to say to anyone. It's like asking someone, "oh, was that recent baby of yours an accident? Did the condom break?" Also it's kind of stupid: who cares what gender you want? You can't exactly choose. It's like asking someone what age they'd like to go grey at, or what telephone number they'd prefer. We all might have preferences, but we don't bother asking because it's stupid since we have absolutely no control over it. I don't know why the gender of a baby is an exception here, but it sure is silly.
The comments might come from ignorance, but they're still rude. If it's said around the children in question, definitely do
not just laugh it off or bean dip. The children need to know that they are loved and not inadequate. A stern response of, "we are happy with and love our children just as they are, thank you," or anything similar, will do. I wouldn't worry about being harsh if your kids are around: the potential collateral damage just isn't worth the risk. CLD's grandfather's response is pretty perfect in this regard.
I've heard people complain of this both ways: families with all boys get asked if they'll "ever get that girl we know you've been trying for", and vice versa. I therefore don't see it as a "this gender isn't as good as that gender," but as a "you need equal numbers" (heavens know why). It's not quite the same as sexism (since it's not stating that only specific gender is intrinsically more desirable or whatever than the other), but close.
Mathematically, for those who are nerdishly inclined, this is actually called a "1-step walk". You can model it, and you get some pretty startling results: if you were to keep having children until you got equal numbers of each gender (because that's a perfectly good reason to have more kids

), although you might luck out early on, on average a person will need to have infinity children for it to even out. So, you know, don't try it. Unless your first is one gender and your second is the other, chances you're not ever gonna get there.