What does he bring up about "man" things? If you give us some specific examples of things he has said, we can help you plan your future responses.
He asked me last time I visited if I was trying to be Dad, and that he and Dad had noticed that my sister and Grace and I have both tried to emulate our father (Dad denied this - he says he tends to tune Jack out when Jack starts lecturing). My brother definitely doesn't mind the intellectual pursuits - his wife is a physicist. But if I talk about how much iron I'm pressing, or good grouping, or talk about military history (especially the parts like genocides and torture), he starts up on the "Is this the real you? You were such a girly-girl when you were little", except I was a weight lifter and knew more about torture than your average college student as a little girl.
From what I've gathered, it's fine for me to be smart and geeky. It's fine for me to be into science fiction and some video games. But if I'm into something our Dad is passionate about, then I'm not being true to myself. He has also been rather vocal that I should teach elementary school instead of high school, even though I do not want to (I admire those who have a gift with small children, but I don't have it). I don't know if that's part of his religion (women should be meek and submissive), or if he genuinely thinks I like the things Dad's into only because Dad is into them, not because Dad introduced me and I think they're fun. My brother has always had a bit of eyebrow-raising that our sister is good at car repair and enjoys it.
I think the abused child bit was because he was coming over on weekends and not spending a huge amount of time around me when I was little. He saw me as gentle, timid, and happily off in a world of my own (often because the real world wasn't much fun to face). Fluttershy, for the My Little Pony fans. I think he still has this image of a dreamy little girl in a field of flowers, singing to herself, without wanting to face the ugly reality that I was that way because I never knew what would set my sister off before she began getting the help she needed. He thought he was seeing happiness, when he was generally seeing terror.