It's not rude to inform someone, especially someone who drops by unannounced, that you have plans and he isn't included in them. It will probably feel unkind, but that's what he gets for dropping by unannounced. He has to accept the fact that if he doesn't call you ahead of time, sometimes you will not be able to invite him in.
There were a few missed opportunities here. First, when Mr. Oblivious arrived. Your DH could have told him it wasn't a good time for a visit, or let him in, but with a deadline--"We're having a special Valentine's dinner tonight, but you can stay for an hour/until DarkPrincess gets home."
When the OP arrived home. How long does it take to drive him home? Could your daughter have missed the cuddling for 20 minutes or so to get Mr. Oblivious safely home? I realize she was sick and wanted her mom, but she would have been safe at home with her dad--and all of you would have been spared the company of an outsider for the entire evening. Of course, if he lives an hour away, driving him home wasn't an option. But calling him a cab was.
And if he's pouring such stiff drinks that one drink makes him unable to drive for what appears to have been several hours, then, yes, I'd lock up the liquor. Or at least put it where he'll have to ask for it in the future.
But really, I think something should have been said about the plans right when he arrived, so that he was aware that it wasn't just a regular weeknight for your family. You and your DH should discuss this and work out what you will do, so that the next time Mr. Oblivious arrives and you have plans you are able to send him away and not have to adapt your plans to his presence.