It might be because we don't know what (if anything) happens after. I'm speaking as one who doesn't believe in an afterlife, and the thought of oblivion can be disturbing - but hopefully not as disturbing as people who think they're going to be tortured for eternity.

I think it's the unknown that we fear, mostly. And because we know we can't come back from death.