Philadelphia is great for a few days. Rhode Island is lovely too.
Seaside holidays? NY'ers hit the beaches in 'the season'. Some even rent houses in the Hamptons (area towards east end of Long Island) for the whole period. It is quite possible to do a day trip to lovely beaches from Manhattan, using public transport. The Long Island shore has some fantastic towns/beaches.
NJ'ers have the Jersey Shore, again some fantastic beaches, though several only allow access for town residents (one called (I think) Lavalett comes to mind) so making public beaches (Point Pleasant comes to mind) a bit of a zoo. There are exceptions where you can get day passes (example: Sea Gert) for under $10. The most popular (thought of as the best) is Cape May. But if you are heading there over a summer weekend, you would probably need to book accomodation weeks in advance...