git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: [PATCH 4/9] get_short_oid: sort ambiguous objects by type, then SHA-1

From
Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Date
May 1, 2018, 12:36 UTC
Message-ID
<878t93zh60.fsf@evledraar.gmail.com>
In-Reply-To
<348f193f-47bb-112f-6ef5-571749bb01e2@gmail.com>
On Tue, May 01 2018, Derrick Stolee wrote:
Show 59 quoted lines
> On 5/1/2018 7:27 AM, Ævar Arnfjörð Bjarmason wrote:
>> On Tue, May 01 2018, Derrick Stolee wrote:
>>
>>> On 4/30/2018 6:07 PM, Ævar Arnfjörð Bjarmason wrote:
>>>> Since we show the commit data in the output that's nicely aligned once
>>>> we sort by object type. The decision to show tags before commits is
>>>> pretty arbitrary, but it's much less likely that we'll display a tag,
>>>> so if there is one it makes sense to show it first.
>>> Here's a non-arbitrary reason: the object types are ordered
>>> topologically (ignoring self-references):
>>>
>>> tag -> commit, tree, blob
>>> commit -> tree
>>> tree -> blob
>> Thanks. I'll add a patch with that comment to v2.
>>
>>>> @@ -421,7 +451,12 @@ static int get_short_oid(const char *name, int len, struct object_id *oid,
>>>>    			ds.fn = NULL;
>>>>      		advise(_("The candidates are:"));
>>>> -		for_each_abbrev(ds.hex_pfx, show_ambiguous_object, &ds);
>>>> +		for_each_abbrev(ds.hex_pfx, collect_ambiguous, &collect);
>>>> +		QSORT(collect.oid, collect.nr, sort_ambiguous);
>>> I was wondering how the old code sorted by SHA even when the ambiguous
>>> objects were loaded from different sources (multiple pack-files, loose
>>> objects). Turns out that for_each_abbrev() does its own sort after
>>> collecting the SHAs and then calls the given function pointer only
>>> once per distinct object. This avoids multiple instances of the same
>>> object, which may appear multiple times across pack-files.
>>>
>>> I only ask because now we are doing two sorts. I wonder if it would be
>>> more elegant to provide your sorting algorithm to for_each_abbrev()
>>> and let it call show_ambiguous_object as before.
>>>
>>> Another question is if we should use this sort generally for all calls
>>> to for_each_abbrev(). The only other case I see is in
>>> builtin/revparse.c.
>> When preparing v2 I realized how confusing this was, so I'd added this
>> to the commit message of my WIP re-roll which should explain this:
>>
>>      A note on the implementation: I started out with something much
>>      simpler which just replaced oid_array_sort() in sha1-array.c with a
>>      custom sort function before calling oid_array_for_each_unique(). But
>>      then dumbly noticed that it doesn't work because the output function
>>      was tangled up with the code added in fad6b9e590 ("for_each_abbrev:
>>      drop duplicate objects", 2016-09-26) to ensure we don't display
>>      duplicate objects.
>>           That's why we're doing two passes here, first we need to
>> sort the list
>>      and de-duplicate the objects, then sort them in our custom order, and
>>      finally output them without re-sorting them. I suppose we could also
>>      make oid_array_for_each_unique() maintain a hashmap of emitted
>>      objects, but that would increase its memory profile and wouldn't be
>>      worth the complexity for this one-off use-case,
>>      oid_array_for_each_unique() is used in many other places.
>
> How would sorting in our custom order before de-duplicating fail the
> de-duplication? We will still pair identical OIDs as consecutive
> elements and oid_array_for_each_unique only cares about consecutive
> elements having distinct OIDs, not lex-ordered OIDs.

Because there's no de-duplication without the array first being sorted in oidcmp() order, which oid_array_for_each_unique() checks for and re-sorts if !array->sorted. I.e. its de-duplication is just a state machine where it won't call the callback if the currently processed element has the same SHA1 as the last one.

> Perhaps the noise is because we rely on oid_array_sort() to mark the
> array as sorted inside oid_array_for_each_unique(), but that could be
> remedied by calling our QSORT() inside for_each_abbrev() and marking
> the array as sorted before calling oid_array_for_each_unique().

As noted above this won't work, because the function inherently relies on the array being sorted to be able to de-duplicate. Doing this will yield duplicate entries.

Previous: Derrick StoleeNext: Derrick Stolee
Message 9 of 99 in “get_short_oid UI improvements”
  1. 0/9 get_short_oid UI improvementsÆvar Arnfjörð Bjarmason, Apr 30, 2018
  2. 1/9 sha1-name.c: remove stray newlineÆvar Arnfjörð Bjarmason, Apr 30, 2018
  3. 2/9 sha1-array.h: align function argumentsÆvar Arnfjörð Bjarmason, Apr 30, 2018
  4. 3/9 sha1-name.c: move around the collect_ambiguous() functionÆvar Arnfjörð Bjarmason, Apr 30, 2018
  5. 4/9 get_short_oid: sort ambiguous objects by type, then SHA-1Ævar Arnfjörð Bjarmason, Apr 30, 2018
  6. Derrick StoleeMay 1, 2018
  7. Ævar Arnfjörð BjarmasonMay 1, 2018
  8. Derrick StoleeMay 1, 2018
  9. Ævar Arnfjörð BjarmasonMay 1, 2018
  10. Derrick StoleeMay 1, 2018
  11. 6/9 get_short_oid: learn to disambiguate by ^{blob}Ævar Arnfjörð Bjarmason, Apr 30, 2018
  12. 7/9 get_short_oid / peel_onion: ^{tree} should mean tree, not treeishÆvar Arnfjörð Bjarmason, Apr 30, 2018
  13. brian m. carlsonMay 1, 2018
  14. 5/9 get_short_oid: learn to disambiguate by ^{tag}Ævar Arnfjörð Bjarmason, Apr 30, 2018
  15. 9/9 config doc: document core.disambiguateÆvar Arnfjörð Bjarmason, Apr 30, 2018
  16. 8/9 get_short_oid / peel_onion: ^{tree} should mean commit, not commitishÆvar Arnfjörð Bjarmason, Apr 30, 2018
  17. Eric SunshineApr 30, 2018
  18. Stefan BellerApr 30, 2018
  19. brian m. carlsonMay 1, 2018
  20. Derrick StoleeMay 1, 2018
  21. 00/12 get_short_oid UI improvementsÆvar Arnfjörð Bjarmason, May 1, 2018
  22. 01/12 sha1-name.c: remove stray newlineÆvar Arnfjörð Bjarmason, May 1, 2018
  23. 02/12 sha1-array.h: align function argumentsÆvar Arnfjörð Bjarmason, May 1, 2018
  24. 03/12 git-p4: change "commitish" typo to "committish"Ævar Arnfjörð Bjarmason, May 1, 2018
  25. 04/12 cache.h: add comment explaining the order in object_typeÆvar Arnfjörð Bjarmason, May 1, 2018
  26. 05/12 sha1-name.c: move around the collect_ambiguous() functionÆvar Arnfjörð Bjarmason, May 1, 2018
  27. 07/12 get_short_oid: learn to disambiguate by ^{tag}Ævar Arnfjörð Bjarmason, May 1, 2018
  28. 06/12 get_short_oid: sort ambiguous objects by type, then SHA-1Ævar Arnfjörð Bjarmason, May 1, 2018
  29. 09/12 get_short_oid / peel_onion: ^{tree} should be tree, not treeishÆvar Arnfjörð Bjarmason, May 1, 2018
  30. 10/12 get_short_oid / peel_onion: ^{commit} should be commit, not committishÆvar Arnfjörð Bjarmason, May 1, 2018
  31. 12/12 get_short_oid: document & warn if we ignore the type selectorÆvar Arnfjörð Bjarmason, May 1, 2018
  32. 11/12 config doc: document core.disambiguateÆvar Arnfjörð Bjarmason, May 1, 2018
  33. 08/12 get_short_oid: learn to disambiguate by ^{blob}Ævar Arnfjörð Bjarmason, May 1, 2018
  34. 06/11 get_short_oid: sort ambiguous objects by type, then SHA-1Derrick Stolee, May 1, 2018
  35. Ævar Arnfjörð BjarmasonMay 1, 2018
  36. Derrick StoleeMay 1, 2018
  37. Ævar Arnfjörð BjarmasonMay 1, 2018
  38. Derrick StoleeMay 1, 2018
  39. 00/12 get_short_oid UI improvementsÆvar Arnfjörð Bjarmason, May 1, 2018
  40. 01/12 sha1-name.c: remove stray newlineÆvar Arnfjörð Bjarmason, May 1, 2018
  41. 02/12 sha1-array.h: align function argumentsÆvar Arnfjörð Bjarmason, May 1, 2018
  42. 04/12 cache.h: add comment explaining the order in object_typeÆvar Arnfjörð Bjarmason, May 1, 2018
  43. Junio C HamanoMay 3, 2018
  44. Duy NguyenMay 8, 2018
  45. pack-format.txt: more details on pack file formatNguyễn Thái Ngọc Duy, May 8, 2018
  46. Stefan BellerMay 8, 2018
  47. Duy NguyenMay 8, 2018
  48. Stefan BellerMay 8, 2018
  49. Ævar Arnfjörð BjarmasonMay 8, 2018
  50. Duy NguyenMay 8, 2018
  51. pack-format.txt: more details on pack file formatNguyễn Thái Ngọc Duy, May 10, 2018
  52. Stefan BellerMay 10, 2018
  53. Duy NguyenMay 11, 2018
  54. Junio C HamanoMay 11, 2018
  55. pack-format.txt: more details on pack file formatNguyễn Thái Ngọc Duy, May 11, 2018
  56. 03/12 git-p4: change "commitish" typo to "committish"Ævar Arnfjörð Bjarmason, May 1, 2018
  57. 05/12 sha1-name.c: move around the collect_ambiguous() functionÆvar Arnfjörð Bjarmason, May 1, 2018
  58. 08/12 get_short_oid: learn to disambiguate by ^{blob}Ævar Arnfjörð Bjarmason, May 1, 2018
  59. 09/12 get_short_oid / peel_onion: ^{tree} should be tree, not treeishÆvar Arnfjörð Bjarmason, May 1, 2018
  60. Junio C HamanoMay 3, 2018
  61. Ævar Arnfjörð BjarmasonMay 3, 2018
  62. Junio C HamanoMay 4, 2018
  63. Ævar Arnfjörð BjarmasonMay 4, 2018
  64. Junio C HamanoMay 7, 2018
  65. Jeff KingMay 8, 2018
  66. Ævar Arnfjörð BjarmasonMay 8, 2018
  67. Jeff KingMay 9, 2018
  68. Ævar Arnfjörð BjarmasonMay 9, 2018
  69. Junio C HamanoMay 10, 2018
  70. Jeff KingMay 10, 2018
  71. 0/6 get_short_oid UI improvementsÆvar Arnfjörð Bjarmason, May 10, 2018
  72. 1/6 sha1-name.c: remove stray newlineÆvar Arnfjörð Bjarmason, May 10, 2018
  73. 2/6 sha1-array.h: align function argumentsÆvar Arnfjörð Bjarmason, May 10, 2018
  74. Jeff KingMay 10, 2018
  75. Junio C HamanoMay 11, 2018
  76. Junio C HamanoMay 11, 2018
  77. 3/6 git-p4: change "commitish" typo to "committish"Ævar Arnfjörð Bjarmason, May 10, 2018
  78. Luke DiamandMay 10, 2018
  79. 4/6 sha1-name.c: move around the collect_ambiguous() functionÆvar Arnfjörð Bjarmason, May 10, 2018
  80. 5/6 get_short_oid: sort ambiguous objects by type, then SHA-1Ævar Arnfjörð Bjarmason, May 10, 2018
  81. Jeff KingMay 10, 2018
  82. Junio C HamanoMay 11, 2018
  83. 6/6 get_short_oid: document & warn if we ignore the type selectorÆvar Arnfjörð Bjarmason, May 10, 2018
  84. Martin ÅgrenMay 10, 2018
  85. Jeff KingMay 10, 2018
  86. Jeff KingMay 10, 2018
  87. Jeff KingMay 10, 2018
  88. Jeff KingMay 10, 2018
  89. 10/12 get_short_oid / peel_onion: ^{commit} should be commit, not committishÆvar Arnfjörð Bjarmason, May 1, 2018
  90. 11/12 config doc: document core.disambiguateÆvar Arnfjörð Bjarmason, May 1, 2018
  91. Jeff KingMay 8, 2018
  92. 12/12 get_short_oid: document & warn if we ignore the type selectorÆvar Arnfjörð Bjarmason, May 1, 2018
  93. 06/12 get_short_oid: sort ambiguous objects by type, then SHA-1Ævar Arnfjörð Bjarmason, May 1, 2018
  94. Junio C HamanoMay 3, 2018
  95. Jeff KingMay 8, 2018
  96. 07/12 get_short_oid: learn to disambiguate by ^{tag}Ævar Arnfjörð Bjarmason, May 1, 2018
  97. Derrick StoleeMay 2, 2018
  98. Derrick StoleeMay 2, 2018
  99. Jacob KellerMay 3, 2018

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.