OP, I live in Central NJ, an hour's train ride from Manhattan. I tell my friends to save time and stay in NYC. It may be more expensive, but a train ride, while sounding easy, can be a hassle (esp during rush hour).
So much to do in NYC. I love the Met, the Guggenheim, Central Park, and the big Macy's. I really like Greenwich Village, too. It has a nice vibe. If you like history, I suggest visiting Greenwood Cemetery in Brooklyn. It's more than a cemetery - it's full of famous people (dec'd, obviously!) and interesting architecture.
And, of course, Ellis Island and the Statue of Liberty (although, with gov't shutdowns those aren't currently available).
Speaking of gov't shutdowns, I suggest checking places out to make sure they're open.