git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: [PATCH v3] rev-list: exclude promisor objects at walk time

From
Josh Steadmon <steadmon@google.com>
Date
Apr 19, 2019, 20:26 UTC
Message-ID
<20190419202652.GU60888@google.com>
In-Reply-To
<20190413053434.GA31884@sigill.intra.peff.net>
On 2019.04.13 01:34, Jeff King wrote:
Show 41 quoted lines
> On Fri, Apr 12, 2019 at 03:38:47PM -0700, Josh Steadmon wrote:
> 
> > > > Thank you for catching this. I haven't yet figured out the cause. I will
> > > > look into this more tomorrow and then send a V4 once I've fixed it.
> > > 
> > > I'm concerned that this is a sign that the approach I suggested does not
> > > actually work everywhere. I.e., could this be a case where we have some
> > > non-promisor object that points to a sub-object that is reachable from
> > > the promisor pack, but not a direct tip? Before your patch we'd consider
> > > that sub-object a promisor (because we enumerate all of the graph that
> > > we do have and mark each such object), but afterwards we would not.
> > > 
> > > And I wonder if that confuses pack-objects. Though I think it would
> > > confuse it in the _opposite_ direction. I.e., using
> > > --exclude-promisor-objects would count such an object as not-a-promisor
> > > and would be more inclined to include it in the new pack.
> > > 
> > > It is curious that this only turns up with GIT_TEST_COMMIT_GRAPH=1, too.
> > > It seems like any such problem ought to be independent of that.
> > > 
> > > Puzzling...
> > 
> > Do you think this justifies going back to the V1 approach (only checking
> > presence of objects pointed to by refs when doing a partial clone)?
> 
> Yes, I think it might. Especially coupled with your other report that
> the V1 approach is 500ms compared to several seconds for this one. Which
> I'd guess is probably because we actually parse the ref tip objects in
> rev-list, whereas your V1 just skipped that step entirely (which is
> perfectly fine for a clone, as we'd have just hashed the objects via
> index-pack anyway).
> 
> It might be interesting to know if the problem is indeed insurmountable
> with the V3 approach here, or if it's simply another bug. But diving
> into it is going to be rather tricky, and I am not volunteering to do
> it. :) So if you want to punt and go back to the more clearly correct V1
> approach, I can live with that. We can always revisit this approach
> later (it wouldn't be necessary for the clone case after your V1, but in
> theory it could be helping other cases, too).
> 
> -Peff

I have not made any progress in figuring out the repack + commit-graph failure, so I will resend V1.

Previous: Jeff KingNext: Josh Steadmon
Message 25 of 27 in “clone: do faster object check for partial clones”
  1. clone: do faster object check for partial clonesJosh Steadmon, Apr 3, 2019
  2. Jonathan TanApr 3, 2019
  3. Jeff KingApr 3, 2019
  4. Jonathan TanApr 3, 2019
  5. Josh SteadmonApr 4, 2019
  6. Jeff KingApr 4, 2019
  7. rev-list: exclude promisor objects at walk timeJosh Steadmon, Apr 4, 2019
  8. Jeff KingApr 4, 2019
  9. Josh SteadmonApr 4, 2019
  10. Jeff KingApr 5, 2019
  11. Josh SteadmonApr 5, 2019
  12. Josh SteadmonApr 8, 2019
  13. rev-list: exclude promisor objects at walk timeJosh Steadmon, Apr 8, 2019
  14. Christian CouderApr 8, 2019
  15. Josh SteadmonApr 8, 2019
  16. Junio C HamanoApr 9, 2019
  17. Jeff KingApr 9, 2019
  18. Junio C HamanoApr 9, 2019
  19. Josh SteadmonApr 9, 2019
  20. SZEDER GáborApr 9, 2019
  21. Josh SteadmonApr 9, 2019
  22. Jeff KingApr 11, 2019
  23. Josh SteadmonApr 12, 2019
  24. Jeff KingApr 13, 2019
  25. Josh SteadmonApr 19, 2019
  26. clone: do faster object check for partial clonesJosh Steadmon, Apr 19, 2019
  27. Jeff KingApr 22, 2019

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.