git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: [RFC PATCH] During a shallow fetch, prevent sending over unneeded objects

From
Junio C Hamano <gitster@pobox.com>
Date
Aug 8, 2013, 01:01 UTC
Message-ID
<7v61vhc7wn.fsf@alter.siamese.dyndns.org>
In-Reply-To
<20130807102716.GA10217@login.drsnuggles.stderr.nl>
Matthijs Kooijman <matthijs@stdin.nl> writes:
Show 6 quoted lines
>> > In your discussion (including the comment), you talk about "shallow
>> > root" (I think that is the same as what we call "shallow boundary"),
>> I think so, yes. I mean to refer to the commits referenced in
>> .git/shallow, that have their parents "hidden".
> Could you confirm that I got the terms right here (or is the shallow
> boundary the first hidden commit?)

As long as you are consistent it is fine. I _think_ boundary refers to what is recorded in the .git/shallow file, so they are commits that are missing from our repository, and their immediate children are available.

Show 26 quoted lines
> My proposal was to only apply the fix for all have revisions when the
> previous history traversal came across some shallow boundary commits. If
> this happens, then that shallow boundary commit will be a "new" one and
> it will have prevented the history traversal from finding the full list
> of relevant "have" commits. In this case, we should just use all "have"
> commits instead.
>
> Now, looking at the code, I see a few options for detecting this case:
>
>  1 Modify mark_edges_uninteresting to return a boolean (or have an
>    output argument) if any of the commits in the list of commits to find
>    (not the edges) is a shallow boundary.
>  2 Modify mark_edges_uninteresting to have a "show_shallow" argument
>    that gets called for every shallow boundary. The show_shallow
>    function passed would then simply keep a boolean if it is passed at
>    least once.
>  3 Add another loop over the commits _after_ the call to
>    mark_edges_uninteresting, that simply looks for any shallow boundary
>    commit.
>
> The last option seems sensible to me, since it prevents modifying the
> somewhat generic mark_edges_uninteresting function for this specific
> usecase. On the other hand, it does mean that the list of commits is
> looped twice, not sure what that means for performance.
>
> Before I go and implement one of these, which option seems best to you?

My gut feeling without looking at any patch is that the simplest (i.e. 3.) would be the best among these three.

But I suspect, with any of these approaches, you would need to be very careful futzing with the edge ones. It may have an interesting interactions with --thin transfer.

Previous: Matthijs KooijmanNext: Duy Nguyen
Message 5 of 30 in “During a shallow fetch, prevent sending over unneeded objects”
  1. During a shallow fetch, prevent sending over unneeded objectsMatthijs Kooijman, Jul 11, 2013
  2. Junio C HamanoJul 11, 2013
  3. Matthijs KooijmanJul 12, 2013
  4. Matthijs KooijmanAug 7, 2013
  5. Junio C HamanoAug 8, 2013
  6. Duy NguyenAug 8, 2013
  7. Junio C HamanoAug 8, 2013
  8. Duy NguyenAug 8, 2013
  9. Junio C HamanoAug 8, 2013
  10. Duy NguyenAug 8, 2013
  11. Junio C HamanoAug 8, 2013
  12. Duy NguyenAug 9, 2013
  13. Matthijs KooijmanAug 12, 2013
  14. Duy NguyenAug 16, 2013
  15. 1/6 Move setup_alternate_shallow and write_shallow_commits to shallow.cNguyễn Thái Ngọc Duy, Aug 16, 2013
  16. 2/6 shallow: only add shallow graft points to new shallow fileNguyễn Thái Ngọc Duy, Aug 16, 2013
  17. Eric SunshineAug 16, 2013
  18. 3/6 shallow: add setup_temporary_shallow()Nguyễn Thái Ngọc Duy, Aug 16, 2013
  19. Eric SunshineAug 16, 2013
  20. 4/6 upload-pack: delegate rev walking in shallow fetch to pack-objectsNguyễn Thái Ngọc Duy, Aug 16, 2013
  21. Matthijs KooijmanAug 28, 2013
  22. Duy NguyenAug 29, 2013
  23. 5/6 list-objects: reduce one argument in mark_edges_uninterestingNguyễn Thái Ngọc Duy, Aug 16, 2013
  24. 6/6 list-objects: mark more commits as edges in mark_edges_uninterestingNguyễn Thái Ngọc Duy, Aug 16, 2013
  25. Matthijs KooijmanAug 28, 2013
  26. Add testcase for needless objects during a shallow fetchMatthijs Kooijman, Aug 28, 2013
  27. Duy NguyenAug 29, 2013
  28. Duy NguyenAug 31, 2013
  29. Matthijs KooijmanOct 21, 2013
  30. Duy NguyenOct 26, 2013

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.