I think people kinda had this problem with ME when my kids were smaller. I have a pool, as do at least 60% of the rest of the people in this town- southwest region of the country. So it's not uncommon to have one, but in my case I chose not to fence mine. I'm sure this was the reason we got invited to other people's houses and to the park A LOT. People are hyper vigilant about pool fences and I guess probably just didn't trust me to watch their kids around water. It was cool. I mean, I was initially annoyed, but then it worked out that I wasn't the house that the kids always flocked to. Who wants to host the entire neighborhood of kids all the time anyway?
I only had one mom tell me that if the kids were going to be swimming she wanted to be there to watch her child. She was nice and not judgmental about it, and I was like- sure! Come on over! .
Anyway, In answer to your question, if there's that much going on in the household as you say, the mom might be mildly annoyed like I was about hosting and who knows, with as much as you say she has going on, she might be relieved to not have kids over at the house. I'd be polite, tactful and generally truthful.