First, I'm really sorry that you had to put up with this. I think you and your boyfriend did the best you really could. I mean, you came in to work out and this guy was horribly over the line. He couldn't even see it anymore.
Like others have said, this isn't flirting. Flirting is the guy next to me on the treadmill who pretended to not know how to use the machine and asked nicely for my help. I knew he was full of it, he knew I knew, but we had a pleasant conversation.

I have learned that any reaction will act as fuel for scumbags like this. I most likely would have stopped what I was doing and said keep it up, I'll have you tossed out. If he kept it up, I would have had a staff member come talk to him.
Two previous posters had some great thoughts:
1. Rebuking him *very loudly* would have drawn attention and possibly really embarrassed him--especially with other "hot" women around. I can guarantee that most men there would not have taken kindly to the fact a woman was being harassed. And the females would have been on alert to *not* work out anywhere near this dude.
2. And I agree with another poster who said the staff most likely knows about this guy, but maybe needs more complaints or to catch him in the act in order to throw him out. Or, he might be a new guy who needs to be on their radar. Either way, when you go in, ask to speak to management, explain the situation and say you want them to be aware that females may be reluctant to come in if this guy is a regular.
Given the number of females who are serious athletes, or just serious about getting into/staying in shape, a gym manager would be an idiot to allow someone like this to stay around. And another reason to put the gym on alert? Not to be an alarmist or sexist, but I'm super glad your boyfriend was there. This jerk may have been aggressive enough to follow you to your car. He might follow another "hot" woman to the parking lot if he didn't feel gratified in a reaction within the gym walls.
So, yeah. Still tell management.