My first inclination was to say "C'mon, it's your best friend and she's helping you out. Surely you can accommodate her request?"
But then I thought about it some more and changed my mind. Setting up for a party is stressful and busy. And there's no guarantee that the toddler would be kept under adequate control. So I think you were fine in telling your friend she couldn't bring her child (and husband) to "setting up time".
And given the update - the grandparents were an hour late in arriving to babysit - your decision was even more justified.
Another thing too, as a host I would have felt obliged to offer the grandparents a drink when they arrived. And yet another thing, other guests who had to find babysitters for their own children might have felt rather disgruntled to have seen your BF's toddler running around at the party.