Because it's gracious to let those not as able as you go first. It shows empathy and humanity to acknowledge that they have it harder than you at this point and they need some support and assistance. It's supporting someone who needs it, not putting them above people on this imaginary human hierarchy of disabled vs. single people. If it's more important that you get the "best" stuff than show compassion for your fellow man who aren't as able as you, well - that really says a lot.
It's a potluck. Unless it's the last potluck before the Apocalypse, I promise you it won't be the last time you go to one. It's called a pot luck because you never know what you're going to get. If there's never enough food, that's one thing, but the idea that all the "best" things are taken by the disabled, elderly and children when they get to go in line ahead of you is eye-roll worthy.