I agree with SamiHami that your best bet at this time would be to say something along the lines of "we'd love to visit; we'll see how things turn out." Don't need to commit now to an actual visit. Also, giving your (very understandable) excuses might just make them try to work around or fix your excuses (or dismiss them).
I agree with other posters that a lot of it has to do with expectations. I've been the one to live far away from the majority of my family; the greatest distance recently was US east coast (family) and me on the west coast. In the four years I lived out there, none of my family visited me. But, there was the expectation that I was the one that should/will visit them, at least twice a year. That bred resentment on my part, because I was the only one making the effort and putting in the expense. Their excuses were/are not based on money, it was more based on comfort, vacation time, convenience, etc. But my own comfort, vacation time and convenience were/are pretty much dismissed.
Even now I'm a two-flight, or 13 to 28 hour drive away (depending on the family member)--again, I travel to visit family twice a year, various family members have visited me once in 6 years. If its off-balance, in my opinion and situation only, it can breed resentment.