Where I live, a shared fence is a shared expense by both neighbors when it comes to repairs/replacements.
so, that would be a longer conversation. I think you both did ok in a new situation- he is new, you were sick, but you did ok.
Set up a time and have a meeting, and figure out the dates and other info- when it will happen for workmen around, noise, etc...
Hope you feel better soon!