This guys sounds really juvenile. Like you said, you're in college, you're all (probably) legal adults now, people should be beyond this ridiculous behavior. One great thing about college, at least IME, is that there's a whole bureaucracy in place that I at least didn't have in my high school--if you've talked to your direct instructor and they won't do anything, talk to the next person in line above them. For example, if this is an art class, there is probably an Art Department which has a Head or Dean. If you have an academic advisor you could also talk to them and ask them what the proper channels are.
I went to a very large university, so this may be a different situation from you, but rumors about who was dating who were of exactly zero interest to anyone. Again, you are adults now, and who you're dating (or not) is no one's business but yours. I would be tempted to laugh in Joe the Jerk's face at his pathetic attempt to stir up drama with this. Don't act like it's a big deal and it will die out.
It is definitely fine to ignore someone who is bothering you. Don't even look over at him or acknowledge him in any way. If possible, sit on the opposite side of the room from him. Tell the teacher that if Joe bothers you in class again you're going to move, so she knows. And then do it, even in the middle of class. I would expect the teacher to then not allow Joe to move to follow you; if she doesn't stop him, that's one more thing you can talk to the Dean about. Frankly I would expect her to have kicked a mouthy brat like Joe out of her class a long time ago--this is not like high school where there's some obligation to somehow teach all the students who show up. The students here in college are paying for their education and Joe is ruining that for at least one of them.