At Google’s Blog, they’ve declare this week the 404 Week At Webmaster Central. It’s meant to discuss how “page not found”-errors are still throwing a 200 status code (which means “OK”), instead of a 404 statuscode (“Page Not Found”).
This issue is called a “Soft 404”, because the user gets a clear message stating the page no longer exists, but crawlers assume (because of the 200 statuscode) that the page does exist, and they keep indexing it.
Google has published a good post about soft 404’s, together with some examplary images at their blog: Farewell to Soft 404’s.
They also go on to explain that a 410 statuscode (“gone”), is treated exactly as a 404 page. Content shown in a 404 error page isn’t indexed, but its links will be followed (such as helpful pages to link to, from a 404 page).
In their latest post they also discuss the importance of 301 redirects, if your content was moved but still exists.
And for those of us who use custom error pages, in combination with PHP, I’ve recently shared how to use proper headers for redirects, and 404 pages.