For forty people at 1/4 lb meat per person, 10 lb is just barely enough. I'd probably do 13-15 lbs and be ok with leftovers. It's always better to have too much food than not enough.
And I'd probably add some more snacky things. Another chip & dip such as potato chips & Onion dip. Maybe raw veggies & ranch dip. I like the idea of cheese & crackers which are very easy to do. Just buy a few large chunks of cheese and set them out with a knife for each kind and some crackers. Very popular and very easy. Another easy filling appetizer is to take a block of cream cheese and top it with a sweet/hot pepper jelly. Serve with crackers.
And I'm wondering about serving the food at 4. I'd let people hang out and swim and socialize for a while before serving the food. I'd probably put out the snacky things at 4, and let people know that the main dinner will be served at 6. Otherwise people will eat, then won't be able to swim, and the purpose of having it by the pool will be lost.
I also don't see the point of doing both a cake and cupcakes. I'm guessing you were thinking cupcakes for the kids? I wouldn't bother. Instead I'd do the cake, the rice krispie bars, and maybe one more dessert such as brownies or cookies. Something easy to eat by the pool.
When you're feeding 40 people, you need to have enough of the main item PLUS enough variety so that people have enough to eat to supplement the main item. If you have a bunch of munchies and ample desserts, one sloppy joe sandwich should be enough for most people. Most, not all, though, so make sure to have enough for those who do want seconds.