You should never feel guilty for saying "no". The pencils are special treats. If they appear too regularly they will not be appreciated.
DS had a teacher's aide in elementary school who gave candy for every small "holiday" that was listed on the calendar. St. Patrick's Day, Lincoln's birthday, Washington's birthday, Columbus Day, National Smelly Feel Day, etc.
It was overwhelming and the kids didn't appreciate the treats.