But it isn’t a back-handed compliment. You are taking a simple well-intentioned statement far too literally. Do you really think that what people are really trying to say is, “All your hard work and perseverance was for naught. You got what you did because you have better circumstances than most.” Really?
Wait, how can you assume that it wasn't meant a back-handed compliment? None of us know what people are thinking, so we can't say for sure how it was meant. But I can tell you, sometimes using "lucky" like that definitely is. It's a classic passive-aggressive maneuver. Siblings, parents with children etc often use that phrase to diminish and belittle family members in a classic PA way. It's hard to defend precisely because it can easily be dismissed as being a compliment. It took some serious soul-searching to realize that my mother's constant referring to any success of mine as "wow, you're sure lucky" was part of why I never felt recognized for my hard work.
It's not that people literally think it's luck. It is, however, a shortcut to dismiss or not have to recognize someone's hard work. Even if it isn't an intentional slight, that's how it comes across to many people. It's right up there with answering someone saying "I got that job I wanted!" with "wow, who did you play scrabble with?" It's meant as a joking way to congratulate someone about pulling off a big feat, but it comes across as dismissive and belittling.
It can be meant innocently enough, but a lot of slurs are that way. It's polite to realize how what you say comes across to other people, not defend it as "that's not how it was meant." Nobody else knows what you meant or thought, they can only judge by what you say.