If you were in one of the back seats and not noticeable, and it was a long ride, then I think he was being overly picky about it. But I do think that if it is at all possible for to stay awake, then you should try to, even if a nap would be convenient for you. But if it's not possible, it's not possible.
If another outing like this is being arranged and he offers to drive again, I think it would be a good idea to let him know in advance that you get wiped out by the long hikes, and you may not be able to stay awake on the ride back. Then he can decide to make it a shorter hike, or let you know that you'll need to find your own ride. For some drivers, just seeing a sleepy person can make them sleepier, so if it is a safety issue for him, I don't he would be rude to tell you he couldn't drive you, if that was in fact what his problem was.