git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: [ANNOUNCE] git-sizer: compute various size-related metrics for your Git repository

From
Jeff King <peff@peff.net>
Date
Mar 16, 2018, 21:29 UTC
Message-ID
<20180316212920.GD12333@sigill.intra.peff.net>
In-Reply-To
<87370zeqmx.fsf@evledraar.gmail.com>
On Fri, Mar 16, 2018 at 09:01:42PM +0100, Ævar Arnfjörð Bjarmason wrote:
Show 8 quoted lines
> Suggestion for a thing to add to it, I don't have the time on the Go
> tuits:
> 
> One thing that can make repositories very pathological is if the ratio
> of trees to commits is too low.
> 
> I was dealing with a repo the other day that had several thousand files
> all in the same root directory, and no subdirectories.

We've definitely run into this problem before (CocoaPods/Specs, for example). The metric that would hopefully show this off is "what is the tree object with the most entries". Or possibly "what is the average number of entries in a tree object".

That's not the _whole_ story, because the really pathological case is when you then touch that giant tree a lot. But if you assume the paths touched by commits are reasonably distributed over the tree, then having a huge number of entries in one tree will also mean that more commits will touch that tree. Sort of a vaguely quadratic problem.

Doing it at the root is obviously the worst case, but the same thing can happen if you have "foo/bar" as a huge tree, and every single commit needs to touch some variant of "foo/bar/baz".

That's why I suspect some "average per tree object" may show this type of problem, because you'd have a lot of near-identical copies of that giant tree if it's being modified a lot.

Show 7 quoted lines
> But it's not something where you can just say having more trees is
> better, because on the other end of the spectrume we can imagine a repo
> like linux.git where each file like COPYING instead exists at
> C/O/P/Y/I/N/G, that would also be pathological.
> 
> It would be very interesting to do some tests to see what the optimal
> value would be.

I suspect there's some math that could give us the solution. You want approximately equal-sized trees, so maybe log(N) levels?

-Peff
Previous: Ævar Arnfjörð BjarmasonNext: Michael Haggerty
Message 3 of 5 in “[ANNOUNCE] git-sizer: compute various size-related metrics for your Git repository”
  1. Michael HaggertyMar 16, 2018
  2. Ævar Arnfjörð BjarmasonMar 16, 2018
  3. Jeff KingMar 16, 2018
  4. Michael HaggertyMar 18, 2018
  5. Johannes SchindelinMar 21, 2018

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.