git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: [PATCH v3 5/5] sha1_name: Minimize OID comparisons during disambiguation

From
Jeff King <peff@peff.net>
Date
Oct 5, 2017, 09:44 UTC
Message-ID
<20171005094418.irm6omly67bgyvo7@sigill.intra.peff.net>
In-Reply-To
<20171002145651.204984-6-dstolee@microsoft.com>
On Mon, Oct 02, 2017 at 10:56:51AM -0400, Derrick Stolee wrote:
Show 12 quoted lines
> +static void find_abbrev_len_for_pack(struct packed_git *p,
> +				     struct min_abbrev_data *mad)
> +{
> +	int match = 0;
> +	uint32_t num, last, first = 0;
> +	struct object_id oid;
> +
> +	open_pack_index(p);
> +	num = p->num_objects;
> +	last = num;
> +	while (first < last) {
> +		uint32_t mid = (first + last) / 2;

This can overflow if we are looking past 2^31. Probably not very common (things seem to get pretty painful at hundreds of millions of objects). But it can be written as:

  uint32_t mid = lo + (hi - lo) / 2;

Sadly, this overflow case seems to be present in many of our binary searches.

> +		const unsigned char *current;
> +		int cmp;
> +
> +		current = nth_packed_object_sha1(p, mid);

nth_packed_object_sha1() has to do some minor computation to come up with the correct pointer. The normal packfile lookup precomputes the initial offset and stride outside the loop. I have no idea if that micro-optimization is actually noticeable, but I thought I'd mention it as a possible avenue for investigation since you're obviously interested in squeezing out performance. ;)

-Peff
Previous: Derrick StoleeNext: Derrick Stolee
Message 42 of 47 in “Improve abbreviation disambiguation”
  1. 0/5 Improve abbreviation disambiguationDerrick Stolee, Sep 25, 2017
  2. 2/5 p0008-abbrev.sh: Test find_unique_abbrev() perfDerrick Stolee, Sep 25, 2017
  3. Junio C HamanoSep 26, 2017
  4. Jeff KingOct 5, 2017
  5. Jeff KingOct 5, 2017
  6. 5/5 sha1_name: Minimize OID comparisons during disambiguationDerrick Stolee, Sep 25, 2017
  7. 1/5 test-list-objects: List a subset of object idsDerrick Stolee, Sep 25, 2017
  8. Junio C HamanoSep 26, 2017
  9. Jeff KingOct 5, 2017
  10. Junio C HamanoOct 5, 2017
  11. Jeff KingOct 5, 2017
  12. Junio C HamanoOct 5, 2017
  13. Derrick StoleeOct 5, 2017
  14. Jeff KingOct 6, 2017
  15. Derrick StoleeOct 7, 2017
  16. Jeff KingOct 7, 2017
  17. Junio C HamanoOct 8, 2017
  18. 4/5 sha1_name: Parse less while finding common prefixDerrick Stolee, Sep 25, 2017
  19. Stefan BellerSep 25, 2017
  20. Derrick StoleeOct 2, 2017
  21. 3/5 sha1_name: Unroll len loop in find_unique_abbrev_rDerrick Stolee, Sep 25, 2017
  22. 1/5 test-list-objects: List a subset of object idsDerrick Stolee, Oct 2, 2017
  23. Junio C HamanoOct 3, 2017
  24. 2/5 p0008-abbrev.sh: Test find_unique_abbrev() perfDerrick Stolee, Oct 2, 2017
  25. 0/5 Improve abbreviation disambituationDerrick Stolee, Oct 2, 2017
  26. Jeff KingOct 5, 2017
  27. 4/5 sha1_name: Parse less while finding common prefixDerrick Stolee, Oct 2, 2017
  28. Junio C HamanoOct 4, 2017
  29. 3/5 sha1_name: Unroll len loop in find_unique_abbrev_rDerrick Stolee, Oct 2, 2017
  30. Junio C HamanoOct 3, 2017
  31. Derrick StoleeOct 3, 2017
  32. Junio C HamanoOct 4, 2017
  33. Derrick StoleeOct 4, 2017
  34. Junio C HamanoOct 4, 2017
  35. Derrick StoleeOct 4, 2017
  36. Junio C HamanoOct 5, 2017
  37. Jeff KingOct 5, 2017
  38. Junio C HamanoOct 5, 2017
  39. 5/5 sha1_name: Minimize OID comparisons during disambiguationDerrick Stolee, Oct 2, 2017
  40. Stefan BellerOct 3, 2017
  41. Derrick StoleeOct 3, 2017
  42. Jeff KingOct 5, 2017
  43. cleanup: fix possible overflow errors in binary searchDerrick Stolee, Oct 6, 2017
  44. Jeff KingOct 6, 2017
  45. Derrick StoleeOct 6, 2017
  46. cleanup: fix possible overflow errors in binary searchDerrick Stolee, Oct 8, 2017
  47. Jeff KingOct 9, 2017

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.