git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: [PATCH v3 3/3] rev-list: add commit object support in `--missing` option

From
Karthik Nayak <karthik.188@gmail.com>
Date
Oct 24, 2023, 11:34 UTC
Message-ID
<CAOLa=ZTUiYi1YqSuirmB9e_7B8ihC6rsFAR3VTDCL4SoLrEX_Q@mail.gmail.com>
In-Reply-To
<xmqq1qdptffk.fsf@gitster.g>
On Fri, Oct 20, 2023 at 6:41 PM Junio C Hamano <gitster@pobox.com> wrote:
Show 59 quoted lines
>
> Junio C Hamano <gitster@pobox.com> writes:
>
> > Rather, I was wondering if we need to use object flags to mark these
> > objects, or can do what we want to do without using any object flags
> > at all.  For the purpose of reporting "missing" objects, wouldn't it
> > be sufficient to walk the object graph and report our findings as we
> > go?  To avoid reporting the same object twice, as we reasonably can
> > expect that the missing objects are minority (compared to the total
> > number of objects), perhaps the codepath that makes such a report
> > can use a hashmap of object_ids or something, for example.
>
> Digging from the bottom,
>
>  * builtin/rev-list.c:show_commit() gets "struct rev_list_info *"
>    that has "struct rev_info *" [*].
>
>  * list-objects.c:do_traverse() calls revision.c:get_revision() to
>    obtain commits, some of which may be missing ones, and things
>    behind get_revision() are responsible for marking the commit as
>    missing.  It has "struct traversal_context *", among whose
>    members is the "revs" member that is the "struct rev_info *".
>
>  * revision.c:get_revision() and machinery behind it ultimately
>    discovers a missing commit in the revision.c:process_parents()
>    that loops over the parents commit_list.  It of course has access
>    to "struct rev_info *".
>
> So, presumably, if we add a new member to "struct rev_info" that
> optionally [*] points at an oidset that records the object names of
> missing objects we discovered so far (i.e., the set of missing
> objects), the location we set the MISSING bit of a commit can
> instead add the object name of the commit to the set.  And we can
> export a function that takes "struct rev_info *" and "struct object
> *" (or "struct object_id *") to check for membership in the "set of
> missing objects", which would be used where we checked the MISSING
> bit of a commit.
>
> I do not know the performance implications of going this route, but
> if we do not find a suitable vacant bit, we do not have to use any
> object flags bit to do this, if we go this route, I would think.  I
> may be missing some details that breaks the above outline, though.
>
>
> [Footnotes]
>
>  * A potential #leftoverbits tangent.
>
>    Why is "rev_list_info" structure declared in <bisect.h>?  I
>    suspect that this is a fallout from recent header file shuffling,
>    but given who uses it (among which is rev-list:show_commit() that
>    has very little to do with bisection and uses the information in
>    rev_list_info when doing its normal non-bisect things), it does
>    not make much sense.
>
>  * When .do_not_die_on_missing_objects is false, it can and should
>    be left NULL, but presumably we use the "do not die" bit even
>    when we are not necessarily collecting the missing objects?  So
>    the new member cannot replace the "do not die" bit completely.

Thanks for the suggestion, this does seem like a good way to go ahead without using flags. The only performance issue being if there are too many commits which are missing, then oidset would be large.

But I think that's okay though.
Show 6 quoted lines
> Thanks for researching.  It sounds like it may be a better bit to
> steal than the one used by the commit-graph, as long as there is no
> reason to expect that blame may want to work in a corrupt repository
> with missing objects, but when it happens, we may regret the
> decision we are making here.
>

I don't see blame working with missing commits though, because it relies on parsing commits to get information to show to the user. So I think it's a safe bit to steal. Also, when the time comes we could always release the bit and move to the solution you mentioned above.

Anyways on the whole I think keeping it future compatible makes a lot more sense. I'll send a patch series to implement an oidset instead of flags soon.

- Karthik
Previous: Junio C HamanoNext: Karthik Nayak
Message 45 of 61 in “rev-list: add support for commits in `--missing`”
  1. 0/3 rev-list: add support for commits in `--missing`Karthik Nayak, Oct 9, 2023
  2. 1/3 revision: rename bit to `do_not_die_on_missing_objects`Karthik Nayak, Oct 9, 2023
  3. 3/3 rev-list: add commit object support in `--missing` optionKarthik Nayak, Oct 9, 2023
  4. 2/3 rev-list: move `show_commit()` to the bottomKarthik Nayak, Oct 9, 2023
  5. Junio C HamanoOct 9, 2023
  6. Patrick SteinhardtOct 10, 2023
  7. Junio C HamanoOct 10, 2023
  8. Karthik NayakOct 11, 2023
  9. Junio C HamanoOct 11, 2023
  10. Karthik NayakOct 12, 2023
  11. Patrick SteinhardtOct 12, 2023
  12. Karthik NayakOct 12, 2023
  13. Junio C HamanoOct 12, 2023
  14. Patrick SteinhardtOct 13, 2023
  15. Patrick SteinhardtOct 13, 2023
  16. commit: detect commits that exist in commit-graph but not in the ODBPatrick Steinhardt, Oct 13, 2023
  17. Junio C HamanoOct 13, 2023
  18. Patrick SteinhardtOct 17, 2023
  19. Junio C HamanoOct 17, 2023
  20. Patrick SteinhardtOct 19, 2023
  21. Patrick SteinhardtOct 19, 2023
  22. Junio C HamanoOct 19, 2023
  23. Jeff KingOct 20, 2023
  24. Junio C HamanoOct 20, 2023
  25. Patrick SteinhardtOct 23, 2023
  26. Junio C HamanoOct 13, 2023
  27. Junio C HamanoOct 12, 2023
  28. 0/3 rev-list: add support for commits in `--missing`Karthik Nayak, Oct 16, 2023
  29. 1/3 revision: rename bit to `do_not_die_on_missing_objects`Karthik Nayak, Oct 16, 2023
  30. 2/3 rev-list: move `show_commit()` to the bottomKarthik Nayak, Oct 16, 2023
  31. 3/3 rev-list: add commit object support in `--missing` optionKarthik Nayak, Oct 16, 2023
  32. Junio C HamanoOct 16, 2023
  33. Karthik NayakOct 16, 2023
  34. Junio C HamanoOct 16, 2023
  35. 0/3 rev-list: add support for commits in `--missing`Karthik Nayak, Oct 19, 2023
  36. 1/3 revision: rename bit to `do_not_die_on_missing_objects`Karthik Nayak, Oct 19, 2023
  37. 2/3 rev-list: move `show_commit()` to the bottomKarthik Nayak, Oct 19, 2023
  38. 3/3 rev-list: add commit object support in `--missing` optionKarthik Nayak, Oct 19, 2023
  39. Junio C HamanoOct 19, 2023
  40. Junio C HamanoOct 19, 2023
  41. Karthik NayakOct 20, 2023
  42. Karthik NayakOct 20, 2023
  43. Junio C HamanoOct 20, 2023
  44. Junio C HamanoOct 20, 2023
  45. Karthik NayakOct 24, 2023
  46. 0/3 rev-list: add support for commits in `--missing`Karthik Nayak, Oct 24, 2023
  47. 1/3 revision: rename bit to `do_not_die_on_missing_objects`Karthik Nayak, Oct 24, 2023
  48. 2/3 rev-list: move `show_commit()` to the bottomKarthik Nayak, Oct 24, 2023
  49. 3/3 rev-list: add commit object support in `--missing` optionKarthik Nayak, Oct 24, 2023
  50. Junio C HamanoOct 24, 2023
  51. Junio C HamanoOct 25, 2023
  52. Karthik NayakOct 25, 2023
  53. Patrick SteinhardtOct 25, 2023
  54. Junio C HamanoOct 26, 2023
  55. 0/3 rev-list: add support for commits in `--missing`Karthik Nayak, Oct 26, 2023
  56. 1/3 revision: rename bit to `do_not_die_on_missing_objects`Karthik Nayak, Oct 26, 2023
  57. 2/3 rev-list: move `show_commit()` to the bottomKarthik Nayak, Oct 26, 2023
  58. 3/3 rev-list: add commit object support in `--missing` optionKarthik Nayak, Oct 26, 2023
  59. Patrick SteinhardtOct 27, 2023
  60. Karthik NayakOct 27, 2023
  61. 3/3 rev-list: add commit object support in `--missing` optionKarthik Nayak, Oct 27, 2023

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.