I think being direct and up-front about it is the way to go. My dad's hearing isn't as good as it used to be and rather than say something about it, he just sits there and has no idea what anyone is saying, doesn't really respond, and then later gets mad because he has no idea what anyone is referring to. That's really frustrating.
You could also try something like, "Say that again to my good ear," which is more casual and kind of puts a positive spin on it.