I can understand your reasoning, but, please, stop doing this. If for no other reason that it's counter-productive.
As others have remarked: people will remember the criticism and not the compliment, they will believe nothing is ever good enough for you and at some point they'll stop trying to please you.
Recently the mistress of the back-handed compliments, and the queen of *either my way or my way*, found it necessary to tell me I'm looking good, nowadays, with extra remarks of how bad I used to look before. Does that make me happy about looking good? No, it makes me angry because she criticises me, and I don't care if she's right or wrong to think like that, but I do care she feels the need to tell me. (And even if she was right, what's the use of telling me now?)