I think someone needs to grab the bull by horns and talk to the boss.
OP keeps referring to this as a "management team" that meets twice a week. I assume these are people with their own to-do lists and responsibilities. How much time is lost in waiting for boss to finish talking, then following up on things he missed, reminder emails, etc?
On top of that, OP said in the update that the higher ups were in a meeting once and seemed irritated when Boss got on his phone. This tells me Boss is kind of clueless about when and where he takes calls or texts.
So one solution might be for someone to volunteer to talk to him, and find a direct but polite way to ask if meetings can't be electronic free. It's important that whoever does this have the full support of the other managers, because this would be mentioned in the convo. Nobody wants to fly solo on this sort of thing.