The first time I went to a big family event after I turned 21, I poured a glass of wine and went to sit in the "adult room" and catch up with my guy cousins. My uncle came over to me and tried to take my glass, saying I wasn't allowed to drink because I would make the "girls" want some and I should stop bothering the adults and go play in the "kid room." I stole my glass back and said he wasn't my parent and this wasn't his home, so he couldn't tell me what to do. He replied that it was my job to monitor the girls so they didn't get into trouble.
I took a big chug of wine and told him I was already too drunk to supervise but he could totally go keep his own children from destroying everything in my grandmother's house.
OK, so this is ridiculous. It's a great example of how telling people what they can't eat/drink it can be rude and kind of weird. That said, I can think of situations where it wouldn't be rude. I don't think a blanket "rude" or "polite" does it. It's too situation-dependent.
Example: I was at a workplace end-of-year party where not only each of the workers, but also their families were invited if they RSVPed. So many people brought their children, of whom most were quite little. There was a children's play area set up just for them, and a table in the corner with special party bags full of lollies and small toys. It was clearly labelled with "children's gifts to take home" or something. One adult decided he liked the look of the sweets and took a bag for himself. He was promptly told politely but firmly by the hostess (our boss) that those were for the children, and the number was exact, (i.e. if he took one, a child would miss out) so could he please put it back. I think this wasn't at all out of line.