As I understand your post, you have an actual offer for both positions?
No offer on the table, but they both strongly advised me to apply for their positions. I don't want to apply for Group A but I don't know how to turn them down.
Apply for them both. Don't turn anyone down until you have an actual, enforceable offer.
I nth this.
And if/when you turn down Group A, you can say that while you enjoyed working with Group A, Group B offered specific work in X field/language/program, which is a better career choice for you.
Even though both strongly suggested that you apply with them, and even though both say that there is an open job within their dept, you *never* know what will happen. Job/funding falls through, group is forced to hire someone else, and so on.