If I were told "come hungry" at any time of day, I would expect a meal or at least heavy snacks.
OTOH, maybe wherever the neighbors are originally from, "come hungry" is the equivalent of saying "See you there" or "we'll leave the light on".
Hm, they're from Texas and now they're in New Jersey. I don't know if "come hungry" means something different in Texas?
Well, that's interesting. I'm in Texas, and our rendition of hospitality runs along the lines of pushing food at you until you can't move. Maybe that's why they had to leave.
I'm in Texas now, and I have found it to be much like Minnesota. We pretty much stuff you until you pop! But in either place "Come hungry" for a Super-bowl party would mean at least heavy snacks. In my family that means I have made a pot of chili, pan of lasagna, or a pot of home-made spaghetti sauce with meatballs. It does not mean an assortment of sweets.
Different areas do things different ways, but that's one I haven't encountered. I'm NOT saying its rude, just that I've never heard of doing it that way.