I think that this is iffy. If you are planning on doing your normal thing, but just not telling him that everyone else will be there I think that it's OK. IF he likes surprises and won't mind being the center of attention in way that is different than a normal birthday party. I think that the important thing is to honor his wishes.
DH loves to give presents. I do not care about getting a gift from him because if there is something I want, I will buy it myself!
Every birthday or holiday he always asks what I want. I am very simple. I want to go out for a nice, expensive dinner, and have him write me a nice love letter. I say this every single time. He is not good at expressing himself on paper, so this is difficult to get him to do. For my birthday this year I gave him a list of restaurants and told him to surprise me. They were all quite pricey so I really did not want him to get me a gift. Well, surprise, he bought me a gift card that had quite a bit of money on it. I understand that it came from a really sweet genuine place, but because he spent so much on the card, we didn't end up going to dinner. (I do the finances so I was uncomfortable spending more money.) I didn't tell him this, but I was really disappointed.
I agree with what the other posters said about making sure that his
wishes are honored in this situation. If he has told you not to spend more money on this, don't do it. It will cause resentment.