git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: [PATCH 03/13] list-objects: filter objects in traverse_commit_list

From
JTJonathan Tan <jonathantanmy@google.com>
Date
Sep 27, 2017, 20:49 UTC
Message-ID
<20170927134934.d40b24d08749c6d4589e8b33@google.com>
In-Reply-To
<1577b87b-d6de-1434-930a-83fde384d4ca@jeffhostetler.com>

On Wed, 27 Sep 2017 15:09:43 -0400 Jeff Hostetler <git@jeffhostetler.com> wrote:

Show 12 quoted lines
> By adding it to the set of provisionally omitted objects, we
> have the option to capture a little extra information with it
> and refer to that the next time we see the object in the traversal.
> For example, in the sparse-checkout case, the first time we see the
> object we know the pathname and know that it does not need to be
> included.  The second time we see that object, we can see if the
> new pathname is the same as the previous one with a simple strcmp
> and avoid the expensive is_excluded_from_list() computation.  Keep
> in mind that rev-list or pack-objects could be called be on something
> like HEAD~100000..HEAD or that there may be 50,000 tips.  So a file
> that doesn't change across that range will be visited many times
> with the same {pathname, sha}.
Ah, capturing the extra information makes sense. I missed that detail.
Show 7 quoted lines
> Right now I want to force the tree to be shown the first time it is
> visited (because I don't want to do tree filtering yet).  I don't mark
> it SEEN yet because we may want to revisit blobs within (say, after a
> folder rename like I described previously).
> 
> I do, however, mark the tree object as SEEN (in the _END event) when I
> can verify that I've included ALL of the children.
This optimization makes sense too.
> So it might be possible that I could change the flags and not use
> FILTER_REVISIT on tree objects, I hesitate to do that right now.

You're probably right that we need some sort of flag on tree objects, and FILTER_REVISIT can do the job. (My suggestion SHOWN plays a similar role anyway.)

> Having the FILTER_REVISIT flag on blob objects means I can avoid
> doing a hash/oidset lookup on subsequent visits.

By the hash/oidset lookup, I presume you mean the lookup on the set of provisionally omitted objects? If yes, this makes sense.

Thanks for your clarifications - I'll take another look at the code here.

Previous: Jeff HostetlerNext: Jeff Hostetler
Message 12 of 16 in “RFC object filtering for parital clone”
  1. 00/13 RFC object filtering for parital cloneJeff Hostetler, Sep 22, 2017
  2. 01/13 dir: refactor add_excludes()Jeff Hostetler, Sep 22, 2017
  3. 02/13 oidset2: create oidset subclass with object length and pathnameJeff Hostetler, Sep 22, 2017
  4. Brandon WilliamsSep 22, 2017
  5. Jonathan TanSep 26, 2017
  6. Jeff HostetlerSep 27, 2017
  7. 03/13 list-objects: filter objects in traverse_commit_listJeff Hostetler, Sep 22, 2017
  8. Jonathan TanSep 26, 2017
  9. Jeff HostetlerSep 27, 2017
  10. Jonathan TanSep 27, 2017
  11. Jeff HostetlerSep 27, 2017
  12. Jonathan TanSep 27, 2017
  13. 04/13 list-objects-filter-all: add filter to omit all blobsJeff Hostetler, Sep 22, 2017
  14. Jonathan TanSep 23, 2017
  15. Jeff HostetlerSep 26, 2017
  16. Jeff HostetlerSep 26, 2017

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.