Born and raised in America (Utah and Colorado, to be precise). My family is at least four generations from immigrant status. I shouldn't have much of an accent, if any at all. One of my first jobs in a call center, I had a person accuse me of lying to them about where I was located, because "You sound like one of those *expletive* Indians!", referring to the call center we had in India. There is no way I had one of the awesome British-flavored accents of our India reps.
Nowadays, when some people learn I live in Minnesota, they're very disappointed to learn that nobody I know talks like the people in "Fargo".
People will believe and say what they want to believe, and that may not have much, if anything, to do with reality.