The problem with retaliation or "calling out" with people like this, is outsiders never see what HE did to cause YOUR "rude" behavior, so it's a pretty slippery slope to try to call him out because you'll end up looking like the fool or the witch or the emotional, unreasonable one, etc. Apparently there are plenty of people (like DH) who don't really recognize the full scope of his abusive tendencies, so it's probably best just to avoid him as best as possible and avoid being alone with him. This will take some coordination with DH on longer trips when it's a smaller group.
I could say that you can either call him out on his behavior when he's alone with you or just pretend like you are clueless and don't take the bait. Either say, "Why are you speaking to me in such a condescending tone? I did enjoy this vacation, thank you for asking." Or simply say, "Yes, I enjoyed myself." "You seem upset. Did you enjoy yourself?" Quickly exit the kitchen.
I would not go on holidays with this person again. Hubby, if you want to spend a week with your brother, have fun. I'm not going. Outside of family situations, I would refuse to spend any time with this person. It's a little disheartening that he requires your presence when this person is clearly a bully. I would not want to deny my husband time with is brother, but that doesn't mean I have to spend quality time with him as well. Family gatherings and reunions, etc., where there are lots of other people would be doable. I don't know if there's a way to coordinate where part of the vacation is you and DH while the rest includes brother or two holidays, or something. You don't want to be stuck at home while hubby goes on vacation, so something needs to be arranged.