Re: [ANNOUNCEMENT] /Arch/ embraces `git'

Hi Tom,

just as a datapoint, here is an experiment I carried out. I wanted to evaluate how much overhead is incurred by using several levels of directories to implement a discrimating index. I used the key format you specified:

	SHA1,SIZE

As data, I used my /usr/src/linux which uses 301M and contains 20753 files and 1389 directories. To compute the key for a directory, I considered that its contents were a mapping from names to keys.

When constructing the indexed archive, I actually stored empty files instead of blobs because I am only interested in overhead.

Using your suggested indexing method that uses [0:4] as the 1st level key and [4:8] as the 2nd level key, I obtain an indexed archive that occupies 159M, where the top level contains 18665 1st level keys, the largest first level dir contains 5 entries, and all 2nd level dirs contain exactly 1 entry.

Using Linus suggested 1 level [0:2] indexing, I obtain an indexed archive that occupies 1.8M, where the top level contains 256 1st level keys, and where the largest 1st level dir contains 110 entries.

This experiment was performed on an ext3 file system.

Cheers,

--Denys