I'd check again with the groom's mother and mention you hadn't received an invite.
I was in a similar situation and it turned out that the B&G didn't have my correct address.
The other person who told us we were invited actually gave the HC our address when they requested it.
I had an odd situation a few years ago, that I've mentioned here before. In brief, Mike invited me to his wedding, and also asked me for Carrie's mailing address as he was thinking of inviting her, but hadn't yet decided. He ended up not
inviting her, due to space concerns. I was very glad I'd never said anything to Carrie about it. Mike was very clear he hadn't completely decided, though. But, I could see someone skipping over that nuance, and just assuring you that you had been invited because the HC asked them for your address.
However, as it's your DH's nephew, it does seem like you're probably invited, and the invitation has just gone missing. I would wait for them to call you regarding your RSVP, or the next time DH's sister brings it up, tell her you never got an invitation, and you don't want to step on any toes.