It's not rude not to include a response card. And although I know that technically it is rude (or perhaps "insulting" or "contrary to etiquette" would be a better description) to include one, I don't think that including the is rude, either. I always appreciate it.
But if you are inviting a large number of people, my advice is either to include a response card or to put your preferred method:
RSVP to email@example.com
or (123) 456-7890
You will save yourself a LOT of trouble. If the card is right there, stamped and ready to go, your guests are much, much more likely to reply promptly. Otherwise, even those who know already whether they can come are likely to put the invitation in a pile of mail they are going to deal with later, and we know what happens all too often then .... An email address is almost as good for that.
Lynn2000 is right. Unfortunately, some people see absence of any reference to responses as the hosts' telling them that they don't need a response, and they should just come or not at the last minute -- the way you would handle an invitation to a public or charity event, or, as she says, an open house or a "bulletin-invitation" wedding at some churches.
I know that the response cards are an extra expense, especially with the current high rates of postage, because I do agree that if you use them, you should stamp them. But they are really fun to get back! People often write such sweet and funny things on them, especially if you don't crowd the card with a lot of check boxes and such.