Friends of mine did this. They had a small destination wedding with just their immediate families present, so I think a total of 9 or 10 people. They had a very elegant dinner after the wedding, but no dancing.
Two months after their wedding, they hosted a reception for all their family and friends, with dinner and dancing. And a very good time was had by all.
The reason it didn't feel like an A list/B list situation was that their wedding guest list was so limited--parents, siblings, and children of siblings (there weren't any grandparents to be invited). It was very clear that it was an "immediate family only" wedding. I think if they had invited some friends and not others, there might have been some hurt feelings, as various friends realized they weren't as close to the happy couple as they thought they were, and didn't rate an invitation to the wedding.
So I think you need to be careful who gets invited to the actual wedding, because that's where people are going to get upset. Define your immediate family--parents? siblings? siblings' children? grandparents? aunts and uncles? Draw clear lines.
Then think carefully about friends. Are there clear categories of friends for both you and your DF? Do you each have one or two very best friends whom everyone could see just had to be invited? Or do you have masses of friends, and picking just a few has the potential for hurt feelings among the rest? I'd consider inviting one friend, plus spouse/SO for each of you. Those friends could then be the MOH and Best Man.
Obviously, you can invite anyone you want. But since you are expressing a concern about hurt feelings, my guess is that you are aware that some potential wedding guests will be upset if they aren't invited. Or if they aren't invited and other people are.