As a Christian and as a rational person who can observe reality, I hold that all things must have a cause. If you have nothing to begin with, you get nothing. So if there's something, there must a cause of what is. Most atheists would, I think, agree.
But here is where atheists often go off track. As Bertrand Russell puts it, "If everything must have a cause, then God must have a cause. If there can be anything without a cause, it may just as well be the world as God."
Russell thinks he's trapped the theist. "Aha! God violates your premise that what exists must have a cause! If you're going to say that, why not just say the world is uncaused? Or that 'it's turtles all the way down!'"
I have sympathy for Russell. I really do. He's like the Detroit Lions--all the right pieces, just not putting it together in the right way. Because here's the point: of course God has a cause...himself! (The fancy term for this is 'aseity'--use it to impress your friends and family).
Now Russell or Richard Dawkins (the single A version of the major leaguer Russell) might retort, "Well, that's convenient. You're telling me that God is the cause of himself? I don't know anything else in existence that operates this way!"
To which the reply is: "Exactly. Who is like the Lord? There is no one." In other words, the theistic claim about God being the cause of himself is intrinsic to what theists mean when we talk about "God." But this is not irrational, but precisely what you would expect of God, if you understand how philosophers since Plato and Aristotle have talked about God. Because if you've got a 'god' who is not self-sufficient and not self-causing, well then, you've got a creature, not the Creator. Russell can deny that 'god' all day long, and it doesn't harm belief in the true God one bit.