I was all for the generic bag concept and handing out bags as treats until I started thinking about how my sis and BIL does their homemade treats. They package them specific for each receiver. More peanut brittle for DD, lots of rum balls for me, tons of turtles for DS, and very little chocolate for DH. And they wrap,them beautifully or creatively and they are under the tree and are just as anticipated a gift every year as anything else and are opened with all the other gifts. I'd hate to start getting a generic bag as a party favor because someone else doesn't like gifts. and I know my sis would feel like she was no longer giving personalized gifts to us.
So, OP, if you give the same to everyone, then I think having a an extra for him is fine. But if you hand them out when other gifts are distributed, then I probably wouldn't give him one then, but instead hand it to him when he leaves, and make it look as non-gifty as possible.