Perhaps you could say something like this:
"We really like Jim, and he is welcome to our home. But we don't like how you have been treating us lately. We don't mind having him sleep over when there is a real, unforeseen emergency, or when we specifically invite him.
But there are a couple of problems. One is that you have begun to treat us like day-care providers, expecting us to pick him up at school or sports and drop him off, and expecting us to be able to take him on a moment's notice. Most parents pay for such things. Nobody likes being taken advantage of.
You do not keep your promises. You say you will pick him up in the morning, and show up at 1 pm.
You have forced us to change our plans, so that you didn't have to change your plans to lunch with friends. We thought we were helping with an emergency. Having lunch before you left for the wedding was not one.
Again, we don't want to stop seeing Jim. None of this is his fault, and we don't blame him at all. We want to keep this friendship going. "
Then make sure that you tell Jim that he is still and definitely welcome, and you want to keep him in your life. Just in case his dad gives him another story.