18 For the wrath of God is revealed from heaven against all ungodliness and unrighteousness of men, who suppress the truth in unrighteousness,
19 because that which is known about God is evident within them; for God made it evident to them.
20 For since the creation of the world His invisible attributes, His eternal power and divine nature, have been clearly seen, being understood through what has been made, so that they are without excuse.
21 For even though they knew God, they did not honor Him as God, or give thanks; but they became futile in their speculations, and their foolish heart was darkened.
22 Professing to be wise, they became fools,
23 and exchanged the glory of the incorruptible God for an image in the form of corruptible man and of birds and four-footed animals and crawling creatures.
24 Therefore God gave them over in the lusts of their hearts to impurity, that their bodies might be dishonored among them.
25 For they exchanged the truth of God for a lie, and worshiped and served the creature rather than the Creator, who is blessed forever. Amen.
26 For this reason God gave them over to degrading passions; for their women exchanged the natural function for that which is unnatural,
27 and in the same way also the men abandoned the natural function of the woman and burned in their desire toward one another, men with men committing indecent acts and receiving in their own persons the due penalty of their error.
28 And just as they did not see fit to acknowledge God any longer, God gave them over to a depraved mind, to do those things which are not proper,
29 being filled with all unrighteousness, wickedness, greed, evil; full of envy, murder, strife, deceit, malice; they are gossips,
30 slanderers, haters of God, insolent, arrogant, boastful, inventors of evil, disobedient to parents,
31 without understanding, untrustworthy, unloving, unmerciful;
32 and, although they know the ordinance of God, that those who practice such things are worthy of death, they not only do the same, but also give hearty approval to those who practice them.
- Romans, chapter 1 (New American Standard Bible), my emphasis
I interpret this passage to mean that we all know there's a God. He communicates his attributes, power, and nature to us directly and through His creation. "Atheists" know God exists, but they choose not to honor and thank Him.