In general, the priest during Mass should only touch the large Communion host. The little round ones are set in the plate ahead of time by the EMs and don't get touched.
Of course, then he does touch them while distributing Communion, but then you can go to one of the EMs instead.
When I served as an EM, I always handled the cup if I were sick or else found someone to replace me, since the cup minister didn't have to drink from the cup until the very end and didn't have to handle the hosts.