Interesting thread. I would have been put off by the guest's behavior, too. I don't know about the whole, "he's more DH's guest than mine" thing, but he's definitely more DH's friend than the OP's, as in, the OP doesn't know Gary nearly as well or feel as comfortable with him. If someone I didn't know well acted in such a familiar way with me as Gary did, it would make me uncomfortable. Then for him to take one of my possessions and throw it into the guest room for his own use later, without asking, even though I was standing right there and we were having a conversation about those possessions--this would just be too familiar for me, and I would be put off by it. I get the OP's issue with Gary's timing, as well, because he didn't even come in, settle down, and reacquaint himself with the OP--he just immediately assumed it was okay to comment on her reading material, at length, with humor/tone that she may or may not have appreciated.
I'm not sure if I would call Gary rude, but I would probably feel like our personalities were not compatible, and I wouldn't want to socialize with him much anymore.