We had the same issue with my ILs when we had our first child. They babysat for the other grandchild three days a week, and the days changed each week. By the end of the first year, I can't tell you how sick I was of hearing that they could come and visit DS, but always with the caveat of if they didn't have to watch the other grandchild. Hearing it like that was like a slap in the face every single time that our kid was second best in their eyes, and first consideration always belonged to the other grandchild.
Pretty much everything revolved around the first grandchild, and just because they babysat her, it seemed like she (they) got first dibs on everything and DS (and us) were the afterthoughts. And yes, to me, SIL always acted very entitled about her parents' time. She had no backup plan for anyone else to watch her child. One time we asked her a month in advance if she could use her backup for one day, so ILS could watch DS when we knew our daycare would be closed. DH (and I) viewed it like....ILs may babysit for their grandchild, and she may pay them, but there are TWO grandchildren. Do we not have the right to ask for the same type of arrangement once in a while (NOT on a regular basis)? If it helps, we were going to pay them too so they didn't lose any money. She agreed, and DH thought it was settled, and we saw them again a week beforehand and I told him to make sure. He was pretty surprised when she went absolutely ballistic when he brought it up again and if everything was settled for that day! Of course, she hadn't done a darn thing about securing a backup for that one day despite a polite and reasonable request with a month's notice. She made it seem like she would, but then she just never did, which is why she went nuts when DH brought it up again. How DARE anyone challenge what she viewed as her exclusive right to her parents' time!
See, the lines were blurred b/c of the grandparent thing. DH was just as much of their child as his sister, and yes, we felt we deserved some of the same considerations. Not equal time, but something fair. And we didn't feel that giving us 1 day with a month's notice was unfair (even though SIL did, obviously). And SIL took advantage in other ways, like if we would visit ILS for the weekend, SIL would drop off her child and then go run errands. Hello? Our kid gets to see his GPs maybe once a month. Why did she have to infringe on that little bit of time that my kid had with his GPs? And no, she wasn't paying ILs for those times, so they weren't 'working'. I never bought the 'we want to visit with you,' because they'd do the 'dump and run' with the kid, or they'd sit there and not talk. That's not visiting, that's....almost like marking your territory.
If it bothers your DH as much as it appears to (and like I said, I TOTALLY get that), then I'd advise your DH to bring it up to them NOW. They have 6 years of entitlement and 6 years of having their world revolve around one grandchild that they have to undo. And I think perhaps it would be a good thing to get some of that out of the way earlier rather than later. Give them a heads up that you're already starting to feel favoritism, and the baby isn't even here yet. Ask them how they're going to handle it when the baby does get here. Make them aware that you don't want to feel like you have to have an appointment with them or jump through hoops for them to see and spend some time with their OTHER grandchild. And also stress that you want some time with just them. SIL and her son get that, so you want to make sure that your child has alone/exclusive time with his grandparents too, so they can get to know him/her and he/she can get to know them. That is NOT unreasonable. And no, the cousins don't need to spend time together. They're 6 years apart, and yours is a baby. Grandparent time is much more precious right now.
And if SIL keeps showing up when you're there, or if MIL/FIL won't stop trying to include her in your plans/invite her or GS over when you're there, then employ the tactic I used once (worked like a charm): "Oh, if you'd rather visit with SIL now, then I guess we'll just go. We'll make plans another time." And then leave. There will probably be the protestations that SIL wants to see you too, and then you can just say that you were looking forward to some time with ILS, and let's be honest: SIL doesn't seem to enjoy your company very much anyway. (I was blunt like that with MIL once and she sputtered, but it worked.) SIL can call you and set up a visit if she wants to see you that badly. But for now, you'll just reschedule your visit for when they don't have any other committments with SIL. Yes, your contact may decrease for a bit, and maybe it will hurt. But if his parents care about seeing you, then they can make it work. And you won't have to put up with SIL or the nephew, so that's a plus.
Sorry for the book; this just brought back a lot of unpleasant memories.