I think it's perfectly reasonable to say "You have to play outside now." I think it's slightly less reasonable to say "You have to play at the park with these four other kids right now, without an adult watching over you." Because even if he's the oldest, it sounds like he's an introvert and it's entirely possible that even if he wants to be left alone, the other kids won't let him be. Since you weren't there at the park to intervene, who knows what they were doing? (Not a judgement against you, btw - it's just that kids can be both creative and cruel at that age.)
If this comes up again, I think you're reasonable in saying "It's time to be outside right now. If you'd like to take a folding chair out on the porch [or the driveway, or the backyard, or whatever you've got] you're welcome to do so, but you need to either be outside in my yard or outside at the park."