Say "pie" to the average American, and s/he will think of a sweet dessert. Non-dessert (savory) pies are not unheard of here, but unless they are prefaced with "pot" or "shepherd's," we will assume that a pie is dessert. (And shepherd's pie isn't really a pie by our standards, because it doesn't have a pie crust. BTW, when I was a tyke, the school cafeteria called shepherd's pie "Chinese pie." Go figure. I've never seen mashed potatoes in a real Chinese dish.)
The other exception is pizza, but IME "pie" is only used in the ordering process: "I'd like a large pie with mushrooms and anchovies."
It was only when my job as a copy editor had me dealing with Canadian and British content that I learned "savoury" meant "not sweet." For decades before that, I'd assumed it was just a synonym for "tasty."