The problem here seems to be that you are mixing the rules of social etiquette with work etiquette.
The conflict between this arises when you schedule something that is for work during off work hours. You are pulling employees into the private sphere of socializing, but still applying the expectations of the workplace.
This sounds like it's definitely an issue.
The description of the party sounds purely social - playing games, gag gifts, real gifts, ugly sweater theme. But the expectations sound work related - children are not invited, employees are expected to come, leaving their spouse behind to tend to family matters if needed, and the party comes before family commitments. And when two employees RSVP'd no, the party was rescheduled, emphasizing that attendance was important.
And one thing that happens when you schedule a work party outside of office hours, with the expectation that your employes should be there, in the busiest social season of the year, is that you get employees who would really rather not be spending their evening with you, but feel that they have to, or risk their jobs.
The babysitting, I think, is a valid response - I'm guessing they thought they could get a babysitter, but their regular wasn't available, or fell sick. The second - technically bad etiquette, but at the end of the term I'd be pretty sympathetic to sudden scheduling conflicts involving picking up a kid at university.
There's also the issue that if someone doesn't really want to come, they're under a fair amount of pressure to be there, and coming up with a last minute excuse could be a way around that. If the two couples who cancelled at the last minute were the ones who originally RSVP'd no, I'd bet on that.
In the future, I think it would be better to either hold a small party in office hours, providing food and drink, or take the employees only out to lunch, or to schedule a party but not worry about rescheduling if people RSVP no, or if they have conflicts. Make it a truly optional event, and don't pressure people to be there (and when your boss is rescheduling around your attempts to RSVP no, that is pressure).