I've always hated giving rides regularly to people. When I worked retail, a lot of my coworkers didn't have cars and thought as long as they offered to pay or actually paid for gas, that I was almost obligated to give them rides.
If it was miserable weather or something, or if it was MY idea to be nice and offer a ride, it was fine. But one guy in particular was such a horrible offender, asking me all the time. I used to be less assertive and would usually give him one but most of the time I'd resent it. It may be petty but after a long shift on my feet, that extra ten minutes it took me to get home was time being taken from me.
Unless someone is RIGHT on the route, where I can drop them off without deviating from my usual way home, I don't like being asked. I may offer, but I really resent being asked. Because then I feel mean for saying no.
Luckily it's almost never an issue anymore, and now I think it wouldn't bother me actually. But back in the days where I knew lots of carless people, it was just another one of those annoying drip drip Chinese water torture things that drove me nuts. It's not the one drop that will bother you, it's the cumulative effect of the whole thing.