git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: packs and trees

From
DLDavid Lang <dlang@digitalinsight.com>
Date
Jun 21, 2006, 15:32 UTC
Message-ID
<Pine.LNX.4.63.0606210830140.6305@qynat.qvtvafvgr.pbz>
In-Reply-To
<Pine.LNX.4.64.0606202046290.5498@g5.osdl.org>

there are performance penalties in some cases when you use directory hashing. I did some tests last year of lots of small files in a directory tree (X/X/X/XXX of 1k files) and I found that turning on directory hashing actually slowed down the creation of this tree from a tarball significantly (I also found that in this case the ext2 allocation strategy was significantly better then the ext3 one), so test on your system.

David Lang
  On Tue, 20 Jun 2006, Linus Torvalds wrote:
Show 37 quoted lines
> Date: Tue, 20 Jun 2006 20:54:01 -0700 (PDT)
> From: Linus Torvalds <torvalds@osdl.org>
> To: Martin Langhoff <martin.langhoff@gmail.com>
> Cc: Nicolas Pitre <nico@cam.org>, Jon Smirl <jonsmirl@gmail.com>,
>     git <git@vger.kernel.org>
> Subject: Re: packs and trees
> 
>
>
> On Wed, 21 Jun 2006, Martin Langhoff wrote:
>>
>> If you are asking about the ext3 performance problems, I think Linus
>> discussed that a while ago, why unpacked repos are slow (in addition
>> to huge), and there were some suggestions of using hashed directory
>> indexes.
>
> Yes. I think most distros still default to nonhashed directories, but for
> any large-directory case you really want to turn on hashing.
>
> I forget the exact details, it's somethng like
>
> 	tune2fs -O dir_index
>
> or something to turn it on (if I remember correctly, that will only affect
> any directories then created after that, but you can effect that by just
> doing a "git repack -a -d" which will remove all old object directories,
> and now subsequent directories will be done with indexing on).
>
> Personally, I just ended up using packs extensively, so I think I'm still
> running without indexing on all my machines ;)
>
> 		Linus
> -
> To unsubscribe from this list: send the line "unsubscribe git" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>
Previous: Linus Torvalds
Message 10 of 10 in “packs and trees”
  1. Jon SmirlJun 20, 2006
  2. Martin LanghoffJun 20, 2006
  3. Jon SmirlJun 20, 2006
  4. Keith PackardJun 20, 2006
  5. Jon SmirlJun 20, 2006
  6. Nicolas PitreJun 20, 2006
  7. Martin LanghoffJun 20, 2006
  8. Nicolas PitreJun 20, 2006
  9. Linus TorvaldsJun 21, 2006
  10. David LangJun 21, 2006

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.