git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: fast-import and unique objects.

From
Jon Smirl <jonsmirl@gmail.com>
Date
Aug 6, 2006, 15:53 UTC
Message-ID
<9e4733910608060853ua0eabc1w9b35b8414d3c9bae@mail.gmail.com>
In-Reply-To
<9e4733910608060532w51fca2c0r8038828df0d41eeb@mail.gmail.com>
On 8/6/06, Jon Smirl <jonsmirl@gmail.com> wrote:
Show 5 quoted lines
> This model has a lot of object duplication. I generated 949,305
> revisions, but only 754,165 are unique. I'll modify my code to build a
> hash of the objects it has seen and then not send the duplicates to
> fast-import. Those 195,140 duplicated objects may be what is tripping
> index-pack up.
New run is finished with duplicate removal.

Time to run is unchanged, still 2hrs. Run time is IO bound not CPU. Pack file is 845MB instead of 934MB. git-index-pack works now, it takes 4 CPU minutes to run. Index file is 18MB.

So it looks like the first stage code is working. Next I need to modify cvs2svn to keep track of the sha-1 through it's sorting process instead of file:revision.

-- 
Jon Smirl
jonsmirl@gmail.com
Previous: Jon SmirlNext: Shawn Pearce
Message 2 of 15 in “fast-import and unique objects.”
  1. Jon SmirlAug 6, 2006
  2. Jon SmirlAug 6, 2006
  3. Shawn PearceAug 6, 2006
  4. Jon SmirlAug 7, 2006
  5. Shawn PearceAug 7, 2006
  6. Jon SmirlAug 7, 2006
  7. Jakub NarebskiAug 7, 2006
  8. Jon SmirlAug 7, 2006
  9. Shawn PearceAug 8, 2006
  10. Jon SmirlAug 8, 2006
  11. Shawn PearceAug 8, 2006
  12. Jon SmirlAug 8, 2006
  13. Martin LanghoffAug 7, 2006
  14. Ryan AndersonAug 7, 2006
  15. Shawn PearceAug 7, 2006

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.