I don't have a PhD or a degree from Oxbridge, but I am very good at learning languages (I am self-taught and nearly fluent in Mandarin Chinese and have a reasonable level of reading and writing ability, and live in Taiwan).
And
I think Japanese is impossibly hard. The grammar is insane. Truly insane. Like, designed by an insane person. It's hard to pronounce the longer words ("please" is six syllables! On-a-gai-shi-ma-su. CRIVINS! is up with that?). So do not beat yourself up.
I think "I am studying conversational Japanese in another setting. Have you tried the wasabi mayonnaise?" - ie a quick, vague answer and immediate beandip, pre-rehearsed, is the way to go. People will get the hint. Say the first part with a tone of finality that says "I do not wish to discuss this".
By the way, I teach foreign languages, too (OK, I work in corporate training, but it involves a lot of language teaching). I can say, honestly, that those "formally taught" language classes that don't start with 'how to say things' are useless for the casual learner, businessperson or expat - they're really only good for long-term dedicated students and aspiring academics (ie the ones who go on to study Japanese literature or political history). I took some of those "formal" classes for Chinese and quit after two semesters. It wasn't that I was doing badly or was frustrated; I simply felt that I wasn't learning anything useful or practical, and the "useful" stuff was all too formal for everyday discussion: I wanted to learn the equivalent of "How've you been?", not the equivalent of "How do you do, my good sir?" (when I was lower level) or "What's up with that?", not "Please clarify your logic" (at a higher level).
So...hey. As a professional in this field I can tell you that I am right with you. That class format simply does not work for you, and I am honestly not surprised that it doesn't. It doesn't work for a lot of people. One of the big downfalls of foreign language education, especially in Asia and/or among stodgier teachers is this idea that "formal = better", and this assumption that we all want to learn how to carefully craft literary sentences rather than, I dunno, buy a tomato! This is something that I sincerely hope to change from the inside!
Of course, if the others all have or are getting PhDs, they are probably more enamored with the academic side of it and don't see it as a disadvantage. That's fine, and they are free to enjoy it. (That said, I know a few people like this in Taiwan and honestly, they may be able to read classical literature but put us in a conversation with locals and I can guarantee that I'll do better. They tend to clam up when asked to actually speak in a friendly, conversational way).
My advice, as someone who does this for a living, is to ditch the class for as long as you need or want to, including permanently. That's OK. Instead, keep on studying at home and maybe consider (NOT trying to pressure you - just thinking you might consider) a language exchange partner. If you find one with which you have good chemistry, then use that time to just hang out and socialize without the pressure of language learning at first. Then, start asking simple questions in English (like "How do I say 'I'd like a serving of eel nigiri?' or 'How do you say 'toilet paper'?) and keeping notes that you can look at later at home. Get a jumbo pack of post-its and write the names of household items in Japanese - both in English with pronunciation and in Hiragana which you can review later. Look at the post-it every time you use that item. When out and about, train your eyes on various objects and repeat anything you can remember in Japanese - and don't fret if you can't remember. It can take many repetitions before it sticks (carrying around a notebook full of, say, food words at the supermarket to 'cheat' at first can help). It took me five or six rounds of a supermarket looking at things and going "tomato - fanqie / soy sauce - youjiang / soda crackers - sudebing" before I really started to get it.
And don't worry that people will think you're weird. 1.) you're a foreigner in Japan so you are automatically weird to locals
(just joking...kinda) and 2.) most Japanese are trying to learn English so if they see you doing this, they'll totally get it.
Anyway. You don't have to do this but it's something to think about.
