I've heard it said that "Jewish" is both a race and a religion, but to be honest that really only applies to Israel. Hence why, ethnically, people of Palestine are essentially the same race as people from Israel, but their religion differs so much that they call themselves a different race.
In other countries though, to be Jewish really means you follow Judaism. Nobody for example could possibly be a Jewish Christian (as someone a few posts above suggested) because it is a fundamental belief of Jews that the saviour has not yet arrived, whereas it is a fundamental belief of Christianity that Jesus was the saviour and he very much already arrived. You really cannot contradict two major religions like that with such an oxymoron.
I would suggest that the OP does a little more research before calling himself a Jewish Atheist. If he means he has some Israeli lineage, then okay, he has Israeli lineage, and comes from a Jewish family, but by choosing atheism he has renounced his religion and is therefore only Israeli by heritage, not Jewish.