Interesting that you say that because I was about to say the exact opposite. I don't think the allergy need be mentioned at all. I mean, sure, you can mention it if you feel the need. But in my opinion, people's preferences absolutely do need to be taken into account when a group is making a decision on what restaurant to go to.
It would take some amount of coercion to get me to go to Chili's. And if I were to say, "sorry, I don't like Chili's, can we go to BRC instead?" for example, and someone were to say, "No, I insist, we're going to Chili's regardless of the fact that you don't like it" well, honestly I don't know how I would react because I simply can't imagine someone saying that.
It's one thing if someone is hosting and invites everyone. "Please join us at Chili's for a celebration dinner that we're hosting." Then fine, Chili's it is and I'll make do. But if it's a group dinner as in, "Let's all go out to dinner to celebrate. How about Chili's?", then it's neither rude nor even out of the ordinary for someone else to say no to Chili's, for whatever reason, spoken or not, and suggest an alternative.