git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Performance problem, long run of identical hashes

From
Jon Smirl <jonsmirl@gmail.com>
Date
Dec 10, 2007, 15:07 UTC
Message-ID
<9e4733910712100707i66e185bofe22805b8e0ba4d8@mail.gmail.com>

Running oprofile during my gcc repack shows this loop as the hottest place in the code by far. I added some debug printfs which show that I have a 100,000+ run of identical hash entries. Processing the 100,000 entries also causes RAM consumption to explode.

create_delta()
	for (entry = index->hash[i]; entry < index->hash[i+1]; entry++) {
		const unsigned char *ref = entry->ptr;
		const unsigned char *src = data;
		unsigned int ref_size = ref_top - ref;
		if (entry->val != val)
			continue;
		if (ref_size > top - src)
			ref_size = top - src;
		if (ref_size <= msize)
			break;
		while (ref_size-- && *src++ == *ref)
			ref++;
		if (msize < ref - entry->ptr) {
			/* this is our best match so far */
			msize = ref - entry->ptr;
			moff = entry->ptr - ref_data;
			if (msize >= 4096) /* good enough */
				break;
		}
	}
-- 
Jon Smirl
jonsmirl@gmail.com
Next: Nicolas Pitre
Message 1 of 7 in “Performance problem, long run of identical hashes”
  1. Jon SmirlDec 10, 2007
  2. Nicolas PitreDec 10, 2007
  3. David KastrupDec 10, 2007
  4. Jon SmirlDec 10, 2007
  5. Nicolas PitreDec 10, 2007
  6. David KastrupDec 10, 2007
  7. Nicolas PitreDec 10, 2007

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.