Ah, yes, the "You are too sensitive," card. Played by people who know they have overstepped the boundaries of polite behavior, but who will never, ever admit that they might have done something wrong. Because that would make them feel bad. While they have no problem making other people feel bad, they don't much like feeling bad themselves.
Try, "And you are not sensitive enough," in reply, in a calm, gentle voice, with a pleasant smile. Then just walk away.
If your sister tries to bring up how much other people just lllllooovvvveeee this aspect of BIL's personality, you are free to point out that you, personally, don't. And now that both your sister and BIL know this, it would of course be rude for him to act this way towards you in the future. After all, they are expecting you to make allowances for BIL's personality. Surely they will understand that he must do the same for you?