git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: Performance impact of a large number of commits

From
SASamuel Abels <newsgroups@debain.org>
Date
Oct 24, 2008, 19:56 UTC
Message-ID
<1224878202.7566.25.camel@localhost>
In-Reply-To
<alpine.DEB.1.10.0810241240100.27333@asgard.lang.hm>
On Fri, 2008-10-24 at 12:43 -0700, david@lang.hm wrote:
> 100,000 commits per day??
> 
> that's 1.5 commits/second. what is updating files that quickly?

This is an automated process taking snapshots of rapidly changing files containing statistical data. Unfortunately, our needs go beyond what a versioning file system has to offer, and the data is an unstructured text file (in other words, using a relational database is not a good option).

> I suspect that you will have some issues here, but it's going to depend on 
> how many files get updated each 3/4 of a second.

That would be 5 to 10 changed files per commit, and those are passed to git commit explicitly (i.e., walking the tree to stat files for finding changes is not necessary).

-Samuel
Previous: david@lang.hmNext: david@lang.hm
Message 3 of 7 in “Performance impact of a large number of commits”
  1. Samuel AbelsOct 24, 2008
  2. david@lang.hmOct 24, 2008
  3. Samuel AbelsOct 24, 2008
  4. david@lang.hmOct 24, 2008
  5. Samuel AbelsOct 25, 2008
  6. david@lang.hmOct 25, 2008
  7. Samuel AbelsOct 25, 2008

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.