There are places that are sufficiently defined-as-public that they won't host private parties: the first place I contacted about my sister's memorial service, I got back a nice note saying that unfortunately, that sort of gathering is considered a private party, so they couldn't accommodate us. But most places where you can arrange to rent a specific space for a specific time don't count as "public" in that sense. They might be "public accommodations" in some legal sense--you might not be able to rent them for an event that was limited to members of the Orange religion or ethnic Plaids--but that doesn't mean you can't have a members-only event as long as it fits those parameters.
Also, if there really was any sort of "you have to let random wandering people in" rule about that location, they should have told your coordinator when she arranged the rental. If there isn't, the people renting the site ought to be told that this happened, so they can warn future renters and/or help find ways of preventing it. Even something as simple as labeling it "Yoga Society Business Meeting" instead of "Yoga Society Potluck," which is hard to change mid-event.
It takes a huge amount of chutzpah to argue, in effect, "I know about this event, therefore I have more right to the food you cooked than you do." And that's what this person was saying: not just taking a plate while things were out for everyone, but that his "I want it" should override yours and your husband's.