If you are concerned about the treatment a young lady might receive at the hands of individuals you've hired, perhaps it would be appropriate to strike up a conversation with her? Not necessarily about her possibly being harassed, as it might be something embarrassing she didn't want passed around or may be something your neighbor fabricated that could end up circling around and causing trouble. Perhaps something that implies you just want to be neighborly and keep the disturbances down, such as;
"Hey so and so, I am having some roofers over during x and y times when I may not be home. If they start bothering you please let me know so I can talk to them about it. I hope the noise wont' be too much trouble."
That way you take off any implications you've heard something unsavory happening, are not confronting the roofers about something they may very well not have been involved with, and can maintain the privacy of the young lady if something DID happen that she does not wish to pursue.