I have two questions (tried to post before but then my computer ate my post...ugh!): 1) what does your DH think of this; what are his feelings? 2) you mentioned that they wanted to have 'all' of the grandchildren with them for the week. How many is 'all', and how old are they?
My personal experience is clouding this a bit, but I don't think I would have sent my 5 year old off to the GP's house (or vacation house) unless it was going to be just the kid and the GPs. In my case, my ILs knew the other grandchildren better, and would cater to them more. Unconsciously. So I was always afraid of my kiddo(s) being 'handled' as the other kids were, instead of for themselves, if that makes sense.
And honestly, how much quality time can the grandparents spend with the grandkids if they're all there at the same time, for a week? During a whole week, they'd fall into the role of caretaker, so it wouldn't be all fun and games. There would still be dishes, cooking, laundry, messes to clean up, etc.
And how do all of the kids get along? How would they plan to manage it when the inevitable "I don't feel like doing that" comes from one child, where everyone else is ready to do something? Can grandma and grandpa keep up with that many kids for an entire week? Face it...grandma and grandpa haven't actively parented several younger children for any length of time in a LOT of years. (And I say 'parented' because that's essentially what they're proposing to do for the week.)
In reading about this grand plan of theirs, it just seems more like they want the bragging rights of being able to say that they took all of their grandkids to a cabin for a week. "Yes, Marge, it was wonderful! We had ALL of the grandchildren there!" "Oh, Jane, you are just such a lucky woman to have all of your grandchildren around you like that! I envy you so much!"
And actually, since you're having a baby, that would mean that ALL of the grandchildren won't be there anyway, right?