We might be getting farther afield from the situation in the OP; but, I can agree that a couple ought to have some fun things they both do together, if not everything. Imagine if the OP had been that the boyfriend was having fun doing this sport with his ex, but there was no comparable activity that the boyfriend and his current girlfriend liked doing together. That could spell trouble for the relationship--he's always having fun with other people, not his girlfriend. So I think it would be sensible to suggest that the girlfriend try to find fun things they can do together as a couple--maybe trying something completely new to both of them, or making a fair compromise of their interests somehow. Not so that they can spend 100% of their time doing things together, but so they can have some fun things they do together, which might offset the girlfriend's discomfort with him doing fun things with his ex.
Of course, in the OP IIRC, the boyfriend also felt somewhat uncomfortable about sparring with his ex, and didn't he say he didn't want to do it again? So that's not quite the same thing, I guess.