I don't know why there's so much insistence that if an answer might be "no", it's rude to ask. I don't think asking puts someone on the spot. If you can't answer right away, you can say, "Oh, well - let me get a head count and I'll call you back!"
If you have to say no, say no. The asker knew that was a possibility when they asked.
If only it were that simple all the time.
In my experience, saying no to a request like this would lead to the questioner asking why. Then you have to explain that you really don't like the person, which can lead to a discussion you'd rather not have.
Or you have to explain that there's limited space, or you've already bought just enough food, or whatever, and the questioner might push a bit--"Oh, we can sit anywhere, don't need to sit at the table," "Susie doesn't eat much," that sort of thing.
I don't see "I can't come to your event because my cousin's in town," as dancing around the issue. It's a statement of fact. It does not put the host on the spot the way asking if said Cousin can attend the event. It's far more pleasant for the person hosting, than getting a direct question that they may not want to answer as directly.