I think this depends - were people milling about, were other people occasionally moving seats? And/or was she talking with the same person each time you returned? And was this seat just casual seating, or was this your seat at a dinner table? And your phone - did you leave it on the seat or just at the table near the seat?
At a dinner party, I feel like seats become people's seats, but at a milling about party, once you get up, the seat is fair game for anyone (unless its one of those backyard parties where people literally bring their own chairs... and even at those, IME, often seats are shared among folks).