I see a lot of people do it (even here on the Grammar Quirks thread), and I realize that it is "becoming more accepted", but I hate seeing a sentence that begins with "but" or "and". It briefly annoys me in informal writing but drives me up the wall in formal writing. The sentence can always easily be restructured to not begin that way or it can be added to the sentence before it with proper punctuation.
I know, I know, lots of people do it, but it still bothers me. You can flame me now. 
Well, you're certainly entitled to be bothered by it - this thread is about "quirks," after all - but it's not incorrect.
http://www.chicagomanualofstyle.org/16/ch05/ch05_sec206.html
I can't access the link because I am not a subscriber. It may not be incorrect according to the Chicago style, but it is incorrect according to others. APA says it is inappropriate because they are joining words; if they are used at the beginning of a sentence, they aren't joining anything.
Sorry, here's the text (odd, I didn't realize I was a subscriber!):
"5.206Beginning a sentence with a conjunction
There is a widespread belief—one with no historical or grammatical foundation—that it is an error to begin a sentence with a conjunction such as and, but, or so. In fact, a substantial percentage (often as many as 10 percent) of the sentences in first-rate writing begin with conjunctions. It has been so for centuries, and even the most conservative grammarians have followed this practice. Charles Allen Lloyd’s 1938 words fairly sum up the situation as it stands even today:
Next to the groundless notion that it is incorrect to end an English sentence with a preposition, perhaps the most wide-spread of the many false beliefs about the use of our language is the equally groundless notion that it is incorrect to begin one with “but” or “and.” As in the case of the superstition about the prepositional ending, no textbook supports it, but apparently about half of our teachers of English go out of their way to handicap their pupils by inculcating it. One cannot help wondering whether those who teach such a monstrous doctrine ever read any English themselves.7
Still, but as an adversative conjunction can occasionally be unclear at the beginning of a sentence. Evaluate the contrasting force of the but in question, and see whether the needed word is really and; if and can be substituted, then but is almost certainly the wrong word. Consider this example: He went to school this morning. But he left his lunch box on the kitchen table. Between those sentences is an elliptical idea, since the two actions are in no way contradictory. What is implied is something like this: He went to school, intending to have lunch there, but he left his lunch behind. Because and would have made sense in the passage as originally stated, but is not the right word—the idea for the contrastive but should be explicit. To sum up, then, but is a perfectly proper word to open a sentence, but only if the idea it introduces truly contrasts with what precedes. For that matter, but is often an effective word for introducing a paragraph that develops an idea contrary to the one preceding it."
