Depending on how much room there is at the table, you could also just shove the food further along--not in front of any specific person, but maybe in front of an empty chair or behind the centerpiece so you can't see it as easily, whatever distance is needed so you won't be tempted by it. Kind of pretend that's what Grandma was trying to do, set the food "aside" so the server would know to take it away, but she couldn't reach far enough and it ended up at your spot. So you're just passing it on to "aside" to be gotten rid of. You don't even have to say anything, just push it along and continue your conversation, and if anyone asks just say, "No thanks, I don't want it," elaborating as much as you want if they continue to question (including the part about you asking Grandma not to do this).