I don't think the OP is being rude at all; I think she's being very sensible, actually. 2 year olds can be very unpredictable, and I totally agree with those saying that there's a good chance he'd refuse to be kept quietly entertained by Dad and insist (loudly) on being where his mother and the action are. Maybe that wouldn't be the case, but it's not the way I'd bet if I were a betting woman. Setting up for a major party can be stressful enough, and for me, a potentially-tantruming toddler would be just too much.
Given that he's allowed, at home, to take food off tables as he wants, I see huge potential for disaster. A two year old normally allowed to grab food as he wants around all kinds of party food? ::shudder:: For that reason alone, I think the OP should stick to her guns on not wanting the child in the house before the party. And even if his father is usually really good about watching him, all it would take is a few seconds of inattention on the father's part or a quick run and grab by the toddler...and they always seem to be faster than you think they're going to be.
The grandparents' habit of being late is just, in my opinion, another reason to say "no" to having the little one there during setup.
Now, that doesn't mean I think the friend is an awful person, or entitled, or any of that. I don't, not just based on the evidence provided by the OP. But I do think that the friend is just very much wanting things to work out, and she's maybe not really thinking through the practicalities and potential problems.