I definitely would put my foot down about my own birthday party.
And I would probably do the same for New Years, simply because if you find Steve offensive to be around, your friends might too. And they might not have fun at your party. Why have a bunch of guests over if some of them are going to get offended by Steve and have a bad evening?
My DH had an acquaintance that he met and he invited him to our 4th of July party one year. He seemed really cool and I liked him. He was talkative and friendly. The next year, we invited him again. But this time, I found him to be really dismissive of my feelings, requests and rules in my own house. He blatantly disregarded things I told him and was being dangerous (for example, lighting fireworks in our garage right next to my classic car which has a bad valve and emits gasoline fumes).
DH and I were planning to have another 4th of July party the next year and I told DH several months in advance that I was not ok with that guest blatantly endangering us all, our cars and our house, especially after I specifically told him to stop and he disregarded me. So we planned not to invite him.
Then, the guy sent my DH a text asking "Are you having a 4th of July party this year?" DH asked me what he should reply. I told him to reply "No." And I said "I was looking forward to the party. I was looking forward to having the neighbors over, and baking pies and starting the BBQ. But if this guy is invited, I will be dreading the party. And there is no point in going through all the work and hassle of cleaning the house, shopping for groceries, cooking and baking to have a party that you're dreading. So I will cancel. And then it will be the truth that there is no party." So DH told him we weren't having anything.
I think it's nice if one spouse endures a guest who isn't too bad, to please their spouse. But if one spouse is beyond merely bored and is actually uncomfortable in their own home and feels demeaned, by guests or relatives, then they should be able to nix that invitation and say they don't want to feel lousy in their own home. Home should be a safe, comfortable non-toxic place. And even if your DH doesn't understand that, or thinks you're snobby, prissy, whatever word he wants to insert, he should still respect your feelings. Like you would most likely respect his if the tables were turned.
ETA: changed to putting my foot down, because that's more effective than just putting your food down