Oklo Fossil Reactors
In 1972 in Oklo, Gabon, while mining for uranium a very strange thing was discovered: the fossilized remains of a naturally occuring nuclear reactor. Later several other similar reactors were discovered, each dating back about two billion years. This is one of the few sites I've been able to find with explanations of the phenomenon and photographs of the site.
dead link: http://www.curtin.edu.au/curtin/centre/waisrc/OKLO/
2020 Update: That site is dead, but there are lots of sites about the Oklo Natural Fission Reactors now. And there's a start-up company named Oklo trying to make small fission reactors.