I once had a supervisor who was a Supreme Whiner. It was a small, privately owned company. The powers-that-be (partners) were out of the office a lot and she spent several hours every.single.day standing at the reception desk whining and gossiping. She didn’t have a lot to do, obviously.
Whiners can be very manipulative. For example, if you agree with what they’re saying or make a comment, next thing you know you find out Whiner has told someone that you agree or that you said (whatever), and make it look as if you brought the subject up.
I think your best bet is to continue with your headphones and come up with a few stock phrases:
"It's never bothered me. If it's a problem for you, speak to your manager"
"If you're not happy here, perhaps you should look for a job elsewhere"
"I'm busy. Is there anything you needed to say to me about [actual stuff you need to work with him on]?"
Depending on what he whines about, an expression of surprise that he should think it's worthy of complaint may help.
"Wow - I can't believe you're complaining that the free cookies in the break room aren't chocolate. I've never come across another office that gives their employees free cookies at all. I think we're really lucky!"
(I work with a whiner. I've learned that he is not going to change. I find a slightly bored "oh" or "really" works quite well to cut down on whining-to-me.
I also have an e-mail folder labelled "c**p" . It makes me feel a little better to just shunt his whine-y e-mails there.
I'm hoping my whiner may be gone soon. He recently gave us an ultimatum basically telling us if we don't give him what he wants he will resign. I don't think he was expecting that our response would be, basically, "OK". (the *actual* response was "YES!! Don't let the door hit you on the way out", but we didn't say that out loud..)