Almost as soon as my husband and I got married, we started hearing "so, when will you have kids?" or, from other parents "won't be long until this is your life!" These comments were annoying and intrusive, but not hurtful, so bean dipping or brushing off the comments were easy enough. After our daughter was born, the "when are you going to give her a sibling?" questions started. Again, very annoying, but not yet hurtful. Then, secondary infertility hit. Those stupid questions and jokes were suddenly unbearable. I could, and did, happily participate in baby showers and baptisms for friends; but I did not take kindly to jokes or lectures about our family size.
I know exactly the kind of conversations OP is talking about. These women are not discussing pregnancy in general, or specific people who they know beyond a doubt are planning to have children. They are poking someone in the ribs, saying "ha! What's taking you guys so long? Are our kids scaring you off? Ha ha ha!" Or, they're scrutinizing every woman's midsection, and any bloating is the subject of "ohhh, is there something Karen's not telling us?"
To sum up, general pregnancy and baby talk, fine. Teasing and speculation, not fine. There is a world of difference between the two, and it is rather offensive to me that some people believe that I am a SS for differentiating between them.
OP, the only way I think you erred was in identifying that there was someone in the group struggling. You could have suggested that given the size of the crowd, and the prevalence of fertility issues in general, that it was likely someone in the room might be bothered. But to me, that is a very small error on your part.