Hmmmm. I can always tell american tourists. I think, as another poster said, the casual clothes are just more casual.
That's not to say that people in the UK don't wear jeans and trainers! But, the jeans will be 'smarter' (fitted, probably a darker colour), and the trainers will be more of a 'casual shoe' . Most women I know wear proper shoes (mostly ballet flats or similar) or boots with their jeans.
A looser cut of jeans is generally for things that you need to be a bit scruffy for. Jeans and trainers (as in running shoes) are basically for teenagers or *very* casual situations. Actually, most teenagers/young people I know wear more of a converse-type shoe.
So yes, If I see anybody over 25 in joggers, jeans-with-trainers or similarly casual attire, and they aren't at the gym or doing their gardening, then I'm pretty confident that I could peg them as american (or as scallies/neds, but that's a completely different kettle of fish