I've been meaning to update this.
After reading all the responses, I realized two things: 1) I really prefer that people identify themselves but 2) it's also not worth a fuss.
I did go out with the guy again and it went poorly for reasons totally unrelated to phone calls. I would ask him something about himself, he would answer, and then he would look expectantly at me. So, it was less of a conversation and more like an interview. Also, I think he was trying to get me to invite him to stay at my place that evening because it was hot and his power was out.