If they were exclusively handicap only they would be marked and policed like the spots are.
They are handicap accessible, just like many doors, hotel rooms, ramps, fairground rides, tables, bus seats, movie aisles, etc- which means that while they are designed with mobility issues in mind, they are not usually meant for the exclusive use of those they are accessible to. It's polite (and sometimes the posted rules) to allow those who "need" them to go ahead of you, or to offer their use immediately if someone who needs it approaches (ie bus seats)- but they are meant to be used by anyone.