When my dad retired from the Army as an O6, my mom and I wore skirts and jackets (not suits, per se, but navy blue skirts w/ jackets of a print material). My brother wore khakis, shirt, tie, and sport coat. I think -- bro may have put on a suit. Now, it was our own father, but I believe that was pretty much the dress of the day. For his dinner that night it was cocktail attire/dress mess.
But, like the OPs wrote, go with what the invitation listed. If it's service dress blues, that's not as formal, and you probably could go with a dressy pantsuit as a civilian.
ETA: Uniform of the day that day was BDU's (camouflage uniform) so my dad and all others there were in that uniform during the ceremony, but in dress mess/tuxedo equivalent for the dinner that night. BDU's definitely don't look formal!