Donnamos2, let me tell you my (long) story. Granted, this was ~30 years ago, but it seems that the human condition never changes.

My XH and I had two children (2 and 5 years old) when we split. I had primary custody, and he had visitation every other weekend, and also any time between that worked for our schedules. Now, XH had to pay child support, but he and I worked it out to a sum that he could afford and still enabled me to adequately provide for our kids’ needs. He, our attorneys, and the judge agreed that the amount was fair to all concerned.
I wanted to do fun things with the kids, but money was tight. So I would scrimp and save during a 4 to 6 week period in order to have the funds to do those things. My scrimping always was in the terms of denying myself something that was just “fun”, but not necessary. For instance, I might have been running out of my favorite shade of lipstick, but instead of buying a new one before <saving for fun thing to do with kids> I simply made do with what I had. Or, my girlfriends would invite me for drinks on a Friday night when XH had the kids for the weekend, but I stayed home because buying drinks at a club was way too expensive. In other words, I denied myself fun (unnecessary) stuff in order to do fun activities with my children. For me, it was a no-brainer.
It came to my attention via “friends” that XH was unable to do fun stuff on the weekends he had our children because I was “taking so much of his paycheck in child support” that I left with him with no money to show the kids a good time. Remember, he and I agreed to a specific dollar amount for child support. What those supposed friends were not considering was XH would hit the bars at least 3 times a week after work (small town and word got around) and blow all his expendable cash.
I told those people that how XH managed his budget was none of my concern and certainly none of theirs. Some people that I thought were friends dropped off the radar, but I realized they were not really friends in the first place.
So, to finally answer your question:
No, your STBXH and his extremely immature friend are not right. Your reply to STBXH’s friend was perfect. Keep polishing that shiny spine!