Just because I "have nothing in my life" does not mean I'm babysitting YOUR children - for FREE - on New Year's Eve. I have plans. No, I'm not telling you what they are. I'm not changing them because you have reservations to go out to dinner. Your problem. I refuse to make it mine and I refuse to fix it for you.
In your shoes, if I didn't have plans, I might quote him a high enough price that it would either deter him, or make it worthwhile: "that will be $400 for babysitting for the evening, payable in cash at the beginning of the night, plus an extra $100/hour if you aren't back by 1 a.m." But only if I genuinely didn't have plans (as I don't this year): I wouldn't cancel anything interesting or social, whether or not it was specifically holiday flavored. "Might" because it depends on the children: the offspring of someone like that might be the sort of handful that I wouldn't want to take on even for $100/hour.