git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: [PATCH v2 06/11] get_short_oid: sort ambiguous objects by type, then SHA-1

From
Derrick Stolee <stolee@gmail.com>
Date
May 1, 2018, 14:15 UTC
Message-ID
<00b64485-44ef-709c-2f9d-56e790ca3a5e@gmail.com>
In-Reply-To
<874ljrzctc.fsf@evledraar.gmail.com>
On 5/1/2018 10:10 AM, Ævar Arnfjörð Bjarmason wrote:
Show 24 quoted lines
> Actually I'm having second thoughts about that and thinking I might keep
> my original approach (with a better explanation).
>
> A few more lines of code seems worthwhile in order to not break the
> assumptions a documented API is making, no matter how briefly, so I set
> about documenting this case and supporting it, since
> e.g. oid_array_lookup() will completely fail with the hack of setting
> the .sorted member, and came up with this:
>
> diff --git a/Documentation/technical/api-oid-array.txt b/Documentation/technical/api-oid-array.txt
> index b0c11f868d..ff87260220 100644
> --- a/Documentation/technical/api-oid-array.txt
> +++ b/Documentation/technical/api-oid-array.txt
> @@ -16,6 +16,20 @@ Data Structures
>   	the actual data. The `nr` member contains the number of items in
>   	the set.  The `alloc` and `sorted` members are used internally,
>   	and should not be needed by API callers.
> ++
> +Both the `oid_array_lookup` and `oid_array_for_each_unique` functions
> +rely on the array being sorted. For the former it's an absolute
> +requirenment that the internal `oid_array_sort` function has been
> +called on it, bu for the latter it's enough that the elements are
> +ordered in such a way as to guarantee that identical object IDs are
> +adjacent in the array.
s/bu/but/
Show 33 quoted lines
> ++
> +This is useful e.g. to print output where commits, tags etc. are
> +grouped together (barring a hash collision they won't have the same
> +object ID), in such cases the `custom_sorted` member can be set to `1`
> +before calling `oid_array_for_each_unique`, and it'll skip its own
> +sorting. Once it's been set calling e.g. `oid_array_lookup` without it
> +being cleared again will cause an internal panic, so use it carefully.
>
>   Functions
>   ---------
> diff --git a/sha1-array.c b/sha1-array.c
> index 466a926aa3..cbae07ff78 100644
> --- a/sha1-array.c
> +++ b/sha1-array.c
> @@ -18,6 +18,7 @@ static void oid_array_sort(struct oid_array *array)
>   {
>   	QSORT(array->oid, array->nr, void_hashcmp);
>   	array->sorted = 1;
> +	array->custom_sorted = 0;
>   }
>
>   static const unsigned char *sha1_access(size_t index, void *table)
> @@ -28,6 +29,13 @@ static const unsigned char *sha1_access(size_t index, void *table)
>
>   int oid_array_lookup(struct oid_array *array, const struct object_id *oid)
>   {
> +	if (array->custom_sorted)
> +		/*
> +		 * We could also just clear custom_sorted here, but if
> +		 * the caller is custom sorting and then calling this
> +		 * that's likely something they'd like to know about.
> +		 */
> +		BUG("PANIC: Cannot lookup OIDs in arrays with a custom sort!");
Probably don't need the "PANIC: " here.
Show 56 quoted lines
>   	if (!array->sorted)
>   		oid_array_sort(array);
>   	return sha1_pos(oid->hash, array->oid, array->nr, sha1_access);
> @@ -39,6 +47,7 @@ void oid_array_clear(struct oid_array *array)
>   	array->nr = 0;
>   	array->alloc = 0;
>   	array->sorted = 0;
> +	array->custom_sorted = 0;
>   }
>
>   int oid_array_for_each_unique(struct oid_array *array,
> @@ -47,7 +56,7 @@ int oid_array_for_each_unique(struct oid_array *array,
>   {
>   	int i;
>
> -	if (!array->sorted)
> +	if (!array->sorted && !array->custom_sorted)
>   		oid_array_sort(array);
>
>   	for (i = 0; i < array->nr; i++) {
> diff --git a/sha1-array.h b/sha1-array.h
> index 1e1d24b009..bfa77ba1e4 100644
> --- a/sha1-array.h
> +++ b/sha1-array.h
> @@ -6,6 +6,7 @@ struct oid_array {
>   	int nr;
>   	int alloc;
>   	int sorted;
> +	int custom_sorted;
>   };
>
>   #define OID_ARRAY_INIT { NULL, 0, 0, 0 }
> diff --git a/sha1-name.c b/sha1-name.c
> index b81e07adbb..d190800db0 100644
> --- a/sha1-name.c
> +++ b/sha1-name.c
> @@ -490,9 +490,11 @@ int for_each_abbrev(const char *prefix, each_abbrev_fn fn, void *cb_data)
>   	find_short_packed_object(&ds);
>
>   	QSORT(collect.oid, collect.nr, sort_ambiguous);
> -	collect.sorted = 1;
>
> +	collect.custom_sorted = 1;
>   	ret = oid_array_for_each_unique(&collect, fn, cb_data);
> +	collect.custom_sorted = 0;
> +
>   	oid_array_clear(&collect);
>   	return ret;
>   }
>
> So maybe I should just stop worrying and YOLO it, it just seems wrong to
> leave such a fragile setup in place where we set .sorted=1 and some
> future refactoring reasonably tries to call oid_array_lookup() on it and
> silently fails.
>
> What do you think?

I think this extra custom_sort check is worth keeping the API stable to future changes.

Thanks, -Stolee

Previous: Ævar Arnfjörð BjarmasonNext: Ævar Arnfjörð Bjarmason
Message 38 of 99 in “get_short_oid UI improvements”
  1. 0/9 get_short_oid UI improvementsÆvar Arnfjörð Bjarmason, Apr 30, 2018
  2. 1/9 sha1-name.c: remove stray newlineÆvar Arnfjörð Bjarmason, Apr 30, 2018
  3. 2/9 sha1-array.h: align function argumentsÆvar Arnfjörð Bjarmason, Apr 30, 2018
  4. 3/9 sha1-name.c: move around the collect_ambiguous() functionÆvar Arnfjörð Bjarmason, Apr 30, 2018
  5. 4/9 get_short_oid: sort ambiguous objects by type, then SHA-1Ævar Arnfjörð Bjarmason, Apr 30, 2018
  6. Derrick StoleeMay 1, 2018
  7. Ævar Arnfjörð BjarmasonMay 1, 2018
  8. Derrick StoleeMay 1, 2018
  9. Ævar Arnfjörð BjarmasonMay 1, 2018
  10. Derrick StoleeMay 1, 2018
  11. 6/9 get_short_oid: learn to disambiguate by ^{blob}Ævar Arnfjörð Bjarmason, Apr 30, 2018
  12. 7/9 get_short_oid / peel_onion: ^{tree} should mean tree, not treeishÆvar Arnfjörð Bjarmason, Apr 30, 2018
  13. brian m. carlsonMay 1, 2018
  14. 5/9 get_short_oid: learn to disambiguate by ^{tag}Ævar Arnfjörð Bjarmason, Apr 30, 2018
  15. 9/9 config doc: document core.disambiguateÆvar Arnfjörð Bjarmason, Apr 30, 2018
  16. 8/9 get_short_oid / peel_onion: ^{tree} should mean commit, not commitishÆvar Arnfjörð Bjarmason, Apr 30, 2018
  17. Eric SunshineApr 30, 2018
  18. Stefan BellerApr 30, 2018
  19. brian m. carlsonMay 1, 2018
  20. Derrick StoleeMay 1, 2018
  21. 00/12 get_short_oid UI improvementsÆvar Arnfjörð Bjarmason, May 1, 2018
  22. 01/12 sha1-name.c: remove stray newlineÆvar Arnfjörð Bjarmason, May 1, 2018
  23. 02/12 sha1-array.h: align function argumentsÆvar Arnfjörð Bjarmason, May 1, 2018
  24. 03/12 git-p4: change "commitish" typo to "committish"Ævar Arnfjörð Bjarmason, May 1, 2018
  25. 04/12 cache.h: add comment explaining the order in object_typeÆvar Arnfjörð Bjarmason, May 1, 2018
  26. 05/12 sha1-name.c: move around the collect_ambiguous() functionÆvar Arnfjörð Bjarmason, May 1, 2018
  27. 07/12 get_short_oid: learn to disambiguate by ^{tag}Ævar Arnfjörð Bjarmason, May 1, 2018
  28. 06/12 get_short_oid: sort ambiguous objects by type, then SHA-1Ævar Arnfjörð Bjarmason, May 1, 2018
  29. 09/12 get_short_oid / peel_onion: ^{tree} should be tree, not treeishÆvar Arnfjörð Bjarmason, May 1, 2018
  30. 10/12 get_short_oid / peel_onion: ^{commit} should be commit, not committishÆvar Arnfjörð Bjarmason, May 1, 2018
  31. 12/12 get_short_oid: document & warn if we ignore the type selectorÆvar Arnfjörð Bjarmason, May 1, 2018
  32. 11/12 config doc: document core.disambiguateÆvar Arnfjörð Bjarmason, May 1, 2018
  33. 08/12 get_short_oid: learn to disambiguate by ^{blob}Ævar Arnfjörð Bjarmason, May 1, 2018
  34. 06/11 get_short_oid: sort ambiguous objects by type, then SHA-1Derrick Stolee, May 1, 2018
  35. Ævar Arnfjörð BjarmasonMay 1, 2018
  36. Derrick StoleeMay 1, 2018
  37. Ævar Arnfjörð BjarmasonMay 1, 2018
  38. Derrick StoleeMay 1, 2018
  39. 00/12 get_short_oid UI improvementsÆvar Arnfjörð Bjarmason, May 1, 2018
  40. 01/12 sha1-name.c: remove stray newlineÆvar Arnfjörð Bjarmason, May 1, 2018
  41. 02/12 sha1-array.h: align function argumentsÆvar Arnfjörð Bjarmason, May 1, 2018
  42. 04/12 cache.h: add comment explaining the order in object_typeÆvar Arnfjörð Bjarmason, May 1, 2018
  43. Junio C HamanoMay 3, 2018
  44. Duy NguyenMay 8, 2018
  45. pack-format.txt: more details on pack file formatNguyễn Thái Ngọc Duy, May 8, 2018
  46. Stefan BellerMay 8, 2018
  47. Duy NguyenMay 8, 2018
  48. Stefan BellerMay 8, 2018
  49. Ævar Arnfjörð BjarmasonMay 8, 2018
  50. Duy NguyenMay 8, 2018
  51. pack-format.txt: more details on pack file formatNguyễn Thái Ngọc Duy, May 10, 2018
  52. Stefan BellerMay 10, 2018
  53. Duy NguyenMay 11, 2018
  54. Junio C HamanoMay 11, 2018
  55. pack-format.txt: more details on pack file formatNguyễn Thái Ngọc Duy, May 11, 2018
  56. 03/12 git-p4: change "commitish" typo to "committish"Ævar Arnfjörð Bjarmason, May 1, 2018
  57. 05/12 sha1-name.c: move around the collect_ambiguous() functionÆvar Arnfjörð Bjarmason, May 1, 2018
  58. 08/12 get_short_oid: learn to disambiguate by ^{blob}Ævar Arnfjörð Bjarmason, May 1, 2018
  59. 09/12 get_short_oid / peel_onion: ^{tree} should be tree, not treeishÆvar Arnfjörð Bjarmason, May 1, 2018
  60. Junio C HamanoMay 3, 2018
  61. Ævar Arnfjörð BjarmasonMay 3, 2018
  62. Junio C HamanoMay 4, 2018
  63. Ævar Arnfjörð BjarmasonMay 4, 2018
  64. Junio C HamanoMay 7, 2018
  65. Jeff KingMay 8, 2018
  66. Ævar Arnfjörð BjarmasonMay 8, 2018
  67. Jeff KingMay 9, 2018
  68. Ævar Arnfjörð BjarmasonMay 9, 2018
  69. Junio C HamanoMay 10, 2018
  70. Jeff KingMay 10, 2018
  71. 0/6 get_short_oid UI improvementsÆvar Arnfjörð Bjarmason, May 10, 2018
  72. 1/6 sha1-name.c: remove stray newlineÆvar Arnfjörð Bjarmason, May 10, 2018
  73. 2/6 sha1-array.h: align function argumentsÆvar Arnfjörð Bjarmason, May 10, 2018
  74. Jeff KingMay 10, 2018
  75. Junio C HamanoMay 11, 2018
  76. Junio C HamanoMay 11, 2018
  77. 3/6 git-p4: change "commitish" typo to "committish"Ævar Arnfjörð Bjarmason, May 10, 2018
  78. Luke DiamandMay 10, 2018
  79. 4/6 sha1-name.c: move around the collect_ambiguous() functionÆvar Arnfjörð Bjarmason, May 10, 2018
  80. 5/6 get_short_oid: sort ambiguous objects by type, then SHA-1Ævar Arnfjörð Bjarmason, May 10, 2018
  81. Jeff KingMay 10, 2018
  82. Junio C HamanoMay 11, 2018
  83. 6/6 get_short_oid: document & warn if we ignore the type selectorÆvar Arnfjörð Bjarmason, May 10, 2018
  84. Martin ÅgrenMay 10, 2018
  85. Jeff KingMay 10, 2018
  86. Jeff KingMay 10, 2018
  87. Jeff KingMay 10, 2018
  88. Jeff KingMay 10, 2018
  89. 10/12 get_short_oid / peel_onion: ^{commit} should be commit, not committishÆvar Arnfjörð Bjarmason, May 1, 2018
  90. 11/12 config doc: document core.disambiguateÆvar Arnfjörð Bjarmason, May 1, 2018
  91. Jeff KingMay 8, 2018
  92. 12/12 get_short_oid: document & warn if we ignore the type selectorÆvar Arnfjörð Bjarmason, May 1, 2018
  93. 06/12 get_short_oid: sort ambiguous objects by type, then SHA-1Ævar Arnfjörð Bjarmason, May 1, 2018
  94. Junio C HamanoMay 3, 2018
  95. Jeff KingMay 8, 2018
  96. 07/12 get_short_oid: learn to disambiguate by ^{tag}Ævar Arnfjörð Bjarmason, May 1, 2018
  97. Derrick StoleeMay 2, 2018
  98. Derrick StoleeMay 2, 2018
  99. Jacob KellerMay 3, 2018

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.