One way around this is to tell the person, "Let me check my schedule./Let me check with my spouse." Then when you get back to them and say that it won't be possible, you'll have an excuse in mind.
You can also think of some stock answers to have ready:
I have very little free time as it is, and I promised myself not to take on anything else.
Did you say Wednesday? Sorry, I have a class on Wednesday nights.
Sorry, I carpool to work and won't be able to get there.
We're planning a quick out of town trip for the weekend. Maybe next time.