BG: My dad was a preacher and my mom the pianist/organist (of course, I believe there is some unwritten law somewhere) of a very conservative fundamentalist Baptist church. What passed for decent clothing - especially for women - was
very strict. FF to now: I haven't attended church for several years. I love my lazy Sundays!
Last year I was visiting my mom to help her prepare for a move. Her church was having a farewell party for her (she'd attended for over 20 yrs). I mentioned I only had blue jeans with me, and she replied without skipping a beat: "Then it's a shame you'll miss my party."

I wasn't angry; in fact, I had to hold back the urge to laugh. Wearing jeans in mom's church - even on a Saturday afternoon when there's nothing going on - is considered a sin.
All this to say that I'm sorry you're getting the 3rd degree for your clothing choice, OP. This older lady is a busy-body and should concentrate on other things besides your wardrobe.