I think the right response differs depending on whether people are saying something because 1) they're ignorant of what your job/company does, 2) they hear your industry/company name and assume you must do [unpleasant job] when your job actually has nothing to do with that, or 3) they really and truly believe what you do is immoral or unethical.
In the first case, I think you can include a statement about what your job/company actually does and that might head off a lot of the speculation. "I work as a product spokeswoman at conventions - some people call it "booth babes" but the job is really more about knowing our product and being able to answer questions than it is about looking pretty."
In the second case, you can either defend your company/industry or you can clarify what your job is right off the bat. "I'm an IT specialist for BigInsuranceCompany - I don't deal with claims, but I do keep their computers going."
In the third case, I think you just have to accept that what you do is controversial and some people are going to take issue with it. If nothing else, it gives you a quick litmus test for which people aren't worth putting energy toward being friends with

(I'll admit, I can kind of see this from both sides. My best friend from college now does graphic and web design for a national group with a lot of political clout, which I'll say is PETA for the sake of example even though it isn't. She believes very strongly in her cause, as does everyone else she works with, but obviously there are a lot of people out there who think the whole organization is wrong and should not exist. I happen to think her organization is a good thing and does a good job raising awareness about their issue, so it doesn't bother me when she gets a bit political in her Tweets/Facebook status updates. On the other side of the coin, I know a lady at church who has a fairly distasteful job for a large oil company. I like her fine as a person, but I just can't get past the fact that she's built a career out of doing something I consider unethical and sleazy.)