It's a talk show. They ask personal questions on talk shows. It sort of expected. Is it rude? I don't know. I think there are different "guidelines" that we follow with celebrity talk shows and the like. Of course, there are definitely things that are obviously crossing the line, but I don't know if I think this is one of them.
Besides, I actually don't think it would be an unreasonable assumption to think that the Clinton's would invite Mr. Obama. He's the President,and Clinton was a former President, and the reality is they are both members of the same party. It isn't uncommon for members of the same political party who both hold and have held very very public offices to be friends. It's a natural assumption that the former President would invite the current one to a family event like that even if it's not true that they did invite the Obama's. If Mr. Obama responded as PeasNCues said, I think that's a really classy answer.
On a personal level though, I don't necessarily know how I feel about the current President going on shows like that... while in office anyway. I think their time is actually better spent, well, at work.
But, if this is part of his upcoming re election campaign in two years then I guess it makes sense that he would want to stay connected to his voters that way. But still, I am torn.