I don't think you're unreasonable at all. I'd be none too pleased to have my relaxing vacation getaway in Nice Town with Friend changed to "hang about at Friend's house for three days with pill of a husband so he doesn't have to put cream on dog's face."
You must know my sister. She also has a pill of a husband, who finds ways to sulk and guilt her into not leaving him alone for any length of time. And she always caves to him. Everyone around her is tired of hearing the phrase, "Husband wants...." when referring to family outings.
Your story sounds tiresomely familiar. Sis and BIL lived out of country for a while, and when they came back to visit I was SO looking forward to seeing my sister. We planned, in advance, a day together to explore Nice Town, just us. I was to meet her in Nice Town as this fit in with their itinerary (they were going there anyway). Two days before, it was announced that BIL was joining us. Not only was BIL joining us, but the plan was now, "meet in Nice Town, but immediately get in the car and drive somewhere else 2 hours away, that BIL wants to go to." I protested that I was already travelling 2 hours to Nice Town, so it would be a pointless day trip for me to get into a car and travel somewhere further; the entire day would be taken up with travelling. BIL said to me, "Oh, well you don't have to come if you don't want to." (Classic PA move or what?)
You were looking forward to a weekend at Nice Town Resort and I think you should go ahead with it and not indulge any lame excuses.