I think it depends, but wouldn't bother me. It's not like a formal dinner party, where the guest said oh, i'm having drinks and dinner with friend a, but will come by you (friend b) in time for dessert. To me, its more casual, where someone dropping in later on would be ok. I've been to SB parties where people come and go. And now that the game begins later and later, sometimes its just not possible to stay the entire time. I know i'm going to friends, but will most likely leave at halftime, and my friends are fine with that. I get up early for work, so I'd like to be home at a fairly reasonable hour, so i can get to bed, etc. in time.
Also, a week and a day or two doesn't seem to be all that much notice; esp for a big event like the Superbowl. So I don't blame the invitee for accepting another invite, when, even though it was talked about, no firms plans were made.