I think asking him out "just as friends" would be misleading, since you're really wanting to see if there's a romantic interest.
Believe me, I understand the desire to make it casual.. in my dating days, there's no WAY I would have explicitly expressed an interest in a man unless he had done so first. Pathetic, yes, but my fears were that he would be thinking, "Are you KIDDING? Of course I'm not interested in YOU..". And then go laugh about it with his friends or something.
Looking back, though, I missed quite a few opportunities with truly nice guys who WERE trying to send signals but maybe too shy/unsure to do anything about it themselves, because I assumed they were just being nice, and didn't want to take that risk.
But, dating is a risk, you have to put yourself out there. Yes, sometimes you'll feel silly, and sometimes get hurt, but that's an unfortunate part of the game. Just ask him if he would like to get coffee some time. If he's shy, he's most likely not going to want to join a group of your friends that he doesn't know well, and besides, is that really going to give you an opportunity to get to know him one-on-one?