From: Nguyen Thai Ngoc Duy Date: Mon, 06 Feb 2012 08:54:37 GMT Subject: Re: [PATCH 6/6] Automatically switch to crc32 checksum for index when it's too large Message-ID: In-Reply-To: 2012/2/6 Dave Zarzycki : > Which crc32 polynomial is being used? crc32c (a.k.a. Castagnoli)? It would be great if this were the same polynomial that Intel implements in hardware via SSE4.2. It's zlib's crc32. > On Feb 5, 2012, at 9:48 PM, Nguyễn Thái Ngọc Duy wrote: > >> An experiment with -O3 is done on Intel D510@1.66GHz. At around 250k >> entries, index reading time exceeds 0.5s. Switching to crc32 brings it >> back lower than 0.2s. >> >> On 4M files index, reading time with SHA-1 takes ~8.4, crc32 2.8s. -- Duy