In my experience (most Super Bowls of my life, plus limited baseball, playoff football and basketball, and some World Cup), there's a definite "The more the merrier" atmosphere around sports parties. There's always space for people to glue themselves to the TV, plus space to mingle and chat.
The gender divide is more or less obvious, but hardly set in stone. If someone invited my husband to a football-watching party alone, he'd decline -- he needs me there to explain what's happening! (And both of us would be there more for the social aspect than the game-watching, which we nearly could not care less about.)
I don't think there's anything wrong with expecting to socialize (even with the host) at a sports-oriented event; I also don't think there's anything wrong with the host wanting to watch the game.
As long as both sides respect each other and there's plenty of food to share, I'm happy with whatever happens.