I'm in a similar position at the moment. I've been working part time in an office for almost a year now, while trying to get into museum work by job-seeking and bulking up my CV with volunteering/training days.
I know my boss would like me to go full time and stay with the company, but, obviously, I don't want to lose what precious time I have each week to pursue the career I really want. I have explained this to him, and he's fine with it. It's a small company, so me being there only 3 days a week isn't so much different than 5.
With my coworkers, I work hard, but I tend to tell them things about the volunteer work I'm doing, the training days I've been on, the jobs I'm applying for. I make sure all of this happens on my non-working days, so they can't get upset about that, but it seems to help them remember that this job, while being brilliant for the moment, isn't my endgame.
Is it possible you could tell your coworkers directly that while you're doing this for now, what you really want to do is <insert profession here>? How did you explain it to your supervisors?