Just keep in mind that yesterdays pox may NOT be ALL the pox... When our three had it, in two waves, pox showed for for several days. Not a fun time.
IMO, you need to stay home. It's really not worth the risk to others, be they babies, possibly pregnant adults, or the elderly. Chicken pox can lead to shingles, which can last for months and be hideously painful. My DH and Mother both get it, and for my mother, it lasts for months. DH only has an episode for days, but it's not a fun time for anybody. It's pretty unhappy timing for you and your family, but there IS no good time for it.
If the itching gets really intolerable, see if the Dr can prescribe some Atarax. Calamine lotion and oatmeal baths did zip for my kids, only the Atarax helped with the awful itching.