To me (and a lot of other people who like science fiction, fantasy, and Renn Faires) the term "Mundane" is a short hand way of referring to people who do not read science fiction or fantasy literature, do not watch science fiction or fantasy movies or television shows, or (in some but not all cases) know that there is a difference between Star Trek, Star Wars, Harry Potter, Valdemar, Pern, Xanth, and other really popular series in the sort-of-related genres (both are fiction set in locations that did not happen in our documented past, present, or under our present understanding of physics). Vampires, werewolves, faeries, witches, wizards, dinosaurs brought back by DNA reconstruction, and other creatures not able to be seen in everyday life are included, more or less. At least for now - DNA reconstruction may make advanced in the near future so that field is no longer "fiction" but fact (with the references in scientific journals to document it).
I don't see it as pejorative - unless someone is poking fun at the mundanes. I see it as a valid description - although one that could easily be twisted by those with that sort of bent in their brains....