If you were the only supervisor, my advise would be to start a paper trail of documentation of errors made.
I agree with the post to text him that an e-mail was sent...it satifies your bosses statement, but still creates a papertrail. On each e-mail give a "respond by date."
Each time he commits an error, document it in writing with a request to him, in writing, to explain himself.
When he doesn't respond to the request, document that failure also, then request him to respond why he didn't respond.
Death by a million cuts.
This is important in job situations, as an employer is required to have evidence to show unemployment or the union that there was a paper trail.
That being said, I'm quite concerned that your supervisors are properly concerned. I noted the college aspect, and am also wondering if they are students. It doesn't make sense that they are not concerned.
You should Copy your supervisors on each e-mail.
Once there are enough documented failures, then you can force a Plan of Action that he must sign, that lists the areas in which he is deficient, and that he has X amount of time to correct the errors or he faces dismissal.
Document each new failure, with the statement that he is not following the Plan of Action.
Be careful, without your supervisors full support, you are potentially putting yourself in jeopary, especially if he has the ability to cry discrimination.
If you have the ability to use only the other workers, and assign work to only them, that is good for you. I fear it's what he wants...to get paid without being assigned work. You need to look out for yourself.