For a casual wedding, I'm very likely to just carry my regular purse. Mine isn't particularly large, and is made of nice looking leather, so it saves me having to transfer stuff into another bag (and possibly forget something important). Plus, my car keys are kind of bulky, so if I'm driving anywhere, it's nice to have my regular purse since I know they fit comfortably in it.
If your normal purse isn't something you'd be comfortable with, though, I'd go for a wallet-with-a-strap or a wristlet. Those have the advantage of potentially being useful for everyday purposes (either to toss into a larger bag, or take with you for an evening out when you don't want a ton of stuff). But, most of the clutches or evening bags I've seen usually have at least a short strap that can be tucked into the bag or pulled out, depending on what you want. It's often a wrist-length gold chain, but sometimes they have longer ones.
I'd agree that thrift stores might have a good selection, since evening bags tend not to get heavy use. I'd imagine they'd be likely to have at least a handful of nice ones that were in pretty good condition.