git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: [PATCH 5/8] commit-graph: not compatible with replace objects

From
Derrick Stolee <stolee@gmail.com>
Date
Jul 18, 2018, 19:52 UTC
Message-ID
<f5d2f06d-1736-57e4-edbd-aa638ae34238@gmail.com>
In-Reply-To
<20180718194657.GC7778@sigill.intra.peff.net>
On 7/18/2018 3:46 PM, Jeff King wrote:
Show 60 quoted lines
> On Wed, Jul 18, 2018 at 08:15:41AM -0700, Derrick Stolee via GitGitGadget wrote:
>
>> From: Derrick Stolee <dstolee@microsoft.com>
>>
>> Create new method commit_graph_compatible(r) to check if a given
>> repository r is compatible with the commit-graph feature. Fill the
>> method with a check to see if replace-objects exist. Test this
>> interaction succeeds, including ignoring an existing commit-graph and
>> failing to write a new commit-graph.
> I think this approach is sensible. These are optimizations, and it's not
> a big deal to just punt no cases we can't handle.
>
> I do have one complaint about the implementation, though:
>
>> +static int commit_graph_compatible(struct repository *r)
>> +{
>> +	prepare_replace_object(r);
>> +	if (hashmap_get_size(&r->objects->replace_map->map))
>> +		return 0;
>> +
>> +	return 1;
>> +}
> If I'm reading the code correctly, this will predicate the decision
> entirely on the presence of refs in refs/replace/. But you may have a
> situation where those refs exist, but you are not respecting them in
> this run.
>
> For instance, imagine[1] a server that hosts arbitrary repositories, but
> wants to use commit graphs to speed up server-side operations (e.g.,
> serving fetches, but also perhaps a web interface doing --contains,
> etc). If it runs all of the server-side commands with
> GIT_NO_REPLACE_OBJECTS=1, then there should be no problem. But if a user
> pushes anything to refs/replace (because they _do_ use replace refs
> locally, and want to share them with other clients), that would disable
> graphs on the server.
>
> So I think this should at least be loosened to:
>
>    if (check_replace_refs) {
> 	prepare_replace_object(r);
> 	if (...)
>    }
>
> which would make this case work. I'd even go so far as to say that for
> writing, we should just always ignore replace refs and generate the full
> graph (just like pack-objects does so when writing a pack). Then the
> resulting graph can be used selectively by disabling replace refs for
> particular commands. But for the scenario I described above, the
> distinction is mostly academic, as replacements would be disabled anyway
> during the write command anyway.
>
> -Peff
>
> [1] You can probably guess that this is how GitHub handles replace refs.
>      We ran into this long ago because replacements and grafts mess up
>      any other caches external to Git that rely on the immutability of
>      the hash.
>
>      We do it with a config option, though, which requires a trivial
>      patch. I'll post that momentarily.

Thanks for the details! I never considered that someone would have these refs around, but would ignore them most of the time.

The biggest reason I wanted to punt here was that it is easy to toggle between using replace refs and not using them. Writing and reading as long as we are ignoring those refs is a good idea, and I'll use that approach in v2.

Thanks,
-Stolee
Previous: Jeff KingNext: Derrick Stolee
Message 14 of 47 in “Clarify commit-graph and grafts/replace/shallow incompatibilities”
  1. 0/8 Clarify commit-graph and grafts/replace/shallow incompatibilitiesDerrick Stolee via GitGitGadget, Jul 18, 2018
  2. 1/8 refs.c: migrate internal ref iteration to pass thru repository argumentStefan Beller via GitGitGadget, Jul 18, 2018
  3. 2/8 refs.c: upgrade for_each_replace_ref to be a each_repo_ref_fn callbackStefan Beller via GitGitGadget, Jul 18, 2018
  4. Junio C HamanoJul 18, 2018
  5. Stefan BellerJul 18, 2018
  6. Junio C HamanoJul 18, 2018
  7. 3/8 commit-graph: update design documentDerrick Stolee via GitGitGadget, Jul 18, 2018
  8. Jakub NarebskiJul 29, 2018
  9. 4/8 test-repository: properly init repoDerrick Stolee via GitGitGadget, Jul 18, 2018
  10. Jakub NarebskiJul 29, 2018
  11. 5/8 commit-graph: not compatible with replace objectsDerrick Stolee via GitGitGadget, Jul 18, 2018
  12. Jeff KingJul 18, 2018
  13. Jeff KingJul 18, 2018
  14. Derrick StoleeJul 18, 2018
  15. Derrick StoleeJul 20, 2018
  16. Stefan BellerJul 20, 2018
  17. Derrick StoleeJul 20, 2018
  18. Jakub NarebskiJul 29, 2018
  19. 6/8 commit-graph: not compatible with graftsDerrick Stolee via GitGitGadget, Jul 18, 2018
  20. Jakub NarebskiJul 29, 2018
  21. 7/8 commit-graph: not compatible with uninitialized repoDerrick Stolee via GitGitGadget, Jul 18, 2018
  22. Jakub NarebskiJul 29, 2018
  23. 8/8 commit-graph: close_commit_graph before shallow walkDerrick Stolee via GitGitGadget, Jul 18, 2018
  24. Jakub NarebskiJul 30, 2018
  25. DO-NOT-MERGE: write and read commit-graph alwaysDerrick Stolee, Jul 18, 2018
  26. Jakub NarebskiJul 30, 2018
  27. Stefan BellerJul 30, 2018
  28. Jakub NarebskiJul 31, 2018
  29. Elijah NewrenJul 31, 2018
  30. Jakub NarebskiJul 29, 2018
  31. Eric SunshineJul 29, 2018
  32. 0/8 Clarify commit-graph and grafts/replace/shallow incompatibilitiesDerrick Stolee, Aug 20, 2018
  33. 6/8 commit-graph: not compatible with graftsDerrick Stolee, Aug 20, 2018
  34. 7/8 commit-graph: not compatible with uninitialized repoDerrick Stolee, Aug 20, 2018
  35. 2/8 refs.c: upgrade for_each_replace_ref to be a each_repo_ref_fn callbackDerrick Stolee, Aug 20, 2018
  36. 3/8 commit-graph: update design documentDerrick Stolee, Aug 20, 2018
  37. 4/8 test-repository: properly init repoDerrick Stolee, Aug 20, 2018
  38. 5/8 commit-graph: not compatible with replace objectsDerrick Stolee, Aug 20, 2018
  39. Junio C HamanoAug 21, 2018
  40. Derrick StoleeAug 21, 2018
  41. 8/8 commit-graph: close_commit_graph before shallow walkDerrick Stolee, Aug 20, 2018
  42. 1/8 refs.c: migrate internal ref iteration to pass thru repository argumentDerrick Stolee, Aug 20, 2018
  43. Stefan BellerAug 20, 2018
  44. Junio C HamanoAug 21, 2018
  45. Derrick StoleeAug 21, 2018
  46. Stefan BellerAug 20, 2018
  47. Derrick StoleeAug 20, 2018

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.