You are not obligated to "give him a chance" if you don't want to. No one is ever
obligated to "give someone a chance" romantically. I hate the attitude that someone owes
anyone a date--that if a woman doesn't want to go out with a guy (because she just isn't interested in him), she's being mean or snobby or female-dog-ish. Or that she should be so pathetically grateful that a MAN, any man, asked her out, that she must of course say yes.
However, as PPs have said, sometimes you don't feel anything particular for a guy but you might get interested if you get to know him a little better--the "unsure that could become a yes." That could be thought of as "giving me
a chance" (a chance to warm up to someone new), or even "giving us
a chance" (to see if we have a spark).
In that case, it's fine to go out a few times to see if anything develops.
When you reach the point where you are sure you aren't interested--then no more dates. If you reach that point the second you lay eyes on a guy, oh well.