git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: Performance impact of a large number of commits

From
SASamuel Abels <newsgroups@debain.org>
Date
Oct 25, 2008, 05:18 UTC
Message-ID
<1224911915.7566.35.camel@localhost>
In-Reply-To
<alpine.DEB.1.10.0810241301430.27333@asgard.lang.hm>
On Fri, 2008-10-24 at 13:11 -0700, david@lang.hm wrote:
Show 8 quoted lines
> > git commit explicitly (i.e., walking the tree to stat files for finding
> > changes is not necessary).
> 
> I suspect that your limits would be filesystem/OS limits more than git 
> limits
> 
> at 5-10 files/commit you are going to be creating .5-1m files/day, even 
> spread across 256 directories this is going to be a _lot_ of files.

The files are organized in a way that places no more than ~1.000 files into each directory. Will Git create a directory containing a larger number of object files? I can see that this would be a problem in our use case.

> packing this may help (depending on how much the files change), but with 
> this many files the work of doing the packing would be expensive.
We can probably do that even if it takes several hours.
-Samuel
Previous: david@lang.hmNext: david@lang.hm
Message 5 of 7 in “Performance impact of a large number of commits”
  1. Samuel AbelsOct 24, 2008
  2. david@lang.hmOct 24, 2008
  3. Samuel AbelsOct 24, 2008
  4. david@lang.hmOct 24, 2008
  5. Samuel AbelsOct 25, 2008
  6. david@lang.hmOct 25, 2008
  7. Samuel AbelsOct 25, 2008

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.