I do think that a lot of the people who dismiss allergies as 'something to get over' or regard it as a flimsy excuse, or something negotiable, haven't had serious allergies themselves.
I also have allergies, and medication helps mute the symptoms, as does regular housecleaning. They do not remove the symptoms - I still feel dragged out and tired even when the obvious symptoms such as sneezing, runny nose, hives, etc. are supressed. It also depends where I am living. If my regular hay fever is minimal, I can tolerate cats better, if it is acting up, it's pure misery. A fluffy cat in a small apartment with poor housekeeping will make me miserable enought that I can't last out an evening party without an attack, even with medication.
Living with a cat would be actively unpleasant for me. I would have to balance the pleasure of someone's company versus spending triple my time housecleaning (in a face mask), having to hide in the cat free parts of the house (office and bedroom), sleeping eleven hours a night, and feeling cruddy. You can see why this isn't appealing. Even a couple of months of adapting to a new pet while feeling like garbage is not something I want to deal with.
On a personal note, words like 'furbaby', or the insistence that someone's pets are just like children to them are a real turn-off for me, even without an actual allergy involved. I don't mind other people doing it away from me, but I don't think I could be in a relationship with someone who did.