Re: Cloning from sites with 404 overridden

If someone feels ambitious, you can detect this condition automatically by searching for a file that you know won't be there and seeing if you get a 404 response to that.

To avoid punishing good servers, it would be nice to defer the test until reciving the first corrupted object.

I'm not sure what the best "object that's not supposed to be there" is. It could just be a random hash, or would a malformed object file name be better? Any fixed name has a finite chance of being created by someone somewhere, but generating 160-bit random numbers is a PITA on non-freenix platforms.

(As an aside, I suspect this is all caused by Microsoft's "friendly HTML error messages" invention.)