OP, I'm going to stand up a little bit for your DH, not because i think his behavior is acceptable, but because I think he's in a hard position.
DH and I are facing a somewhat similar situation with my brother so I'm putting myself somewhat in your DH's shoes. This is his brother, whom he loves, even if he doesn't like him. They have a lot of history together. Both parents are dead, and you don't mention any other siblings, so DH's brother may be the only person who shares DH's memories.
You probably have a clearer view of your BIL than your DH does. My DH, for some time now, has thought that my brother was mentally ill; it is only fairly recently, with some really egregious actions on my brother's part, that I can acknowledge that, at the least, he is mentally unstable. It has taken me much longer to get to that point--my vision is clouded by those years that my brother and I shared.
So, yes, your DH may be hoping that the situation isn't quite as bad as it seems, and that if you and he back off a little, that maybe the situation with your BIL will resolve and he won't need to cut his brother out of his life.
Your DH needs to learn to stand up for you. It sounds as if this process is starting. You've gotten some great suggestions on ways to proceed, but be aware that as hard as it is for you, it may be even harder for your DH. It has been so hard for me to realize that I do not want my brother in my life.
Hugs to you and your DH.