I don't think it's likely that all the employees were on break at the same time. I never have a problem with shopping around employees who are stocking shelves, but it would bother me to see empty shelves and trash in the aisles and no one actively doing any work.
It wouldn't bother be enough to never go back, and I think I'd probably wait and see if things improve before contacting corporate.