git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: [PATCH 3/6] Stop producing index version 2

From
Nguyen Thai Ngoc Duy <pclouds@gmail.com>
Date
Feb 7, 2012, 04:50 UTC
Message-ID
<CACsJy8Ayqea75xeFKJNm6iT7GSUGDEfvZD17uEv7ihr4SS2LMg@mail.gmail.com>
In-Reply-To
<CAJo=hJvtRnmvALcn3vKpYTr3j6ada8iboPjWN3cQnwwKzRvrDA@mail.gmail.com>
On Tue, Feb 7, 2012 at 10:09 AM, Shawn Pearce <spearce@spearce.org> wrote:
Show 16 quoted lines
> 2012/2/5 Junio C Hamano <gitster@pobox.com>:
>> Nguyễn Thái Ngọc Duy  <pclouds@gmail.com> writes:
>>
>>> read-cache.c learned to produce version 2 or 3 depending on whether
>>> extended cache entries exist in 06aaaa0 (Extend index to save more flags
>>> - 2008-10-01), first released in 1.6.1. The purpose is to keep
>>> compatibility with older git. It's been more than three years since
>>> then and git has reached 1.7.9. Drop support for older git.
>>
>> Cc'ing this, as I suspect this would surely raise eyebrows of some people
>> who wanted to get rid of the version 3 format.
>
> Version 3 was a mistake because of the variable length record sizes.
> Saving 2 bytes on some records that don't use the extended flags makes
> the index file *MUCH* harder to parse. So much so that we should take
> version 3 and kill it, not encourage it as the default!

Probably too late for that, but it's good to know there are strong user base for v2.

Show 20 quoted lines
> <thinking type="wishful" probability="never-happen"
> probably-inflating-flame-from="linus">
>
> I have long wanted to scrap the current index format. I unfortunately
> don't have the time to do it myself. But I suspect there may be a lot
> of gains by making the index format match the canonical tree format
> better by keeping the tree structure within a single file stream,
> nesting entries below their parent directory, and keeping tree SHA-1
> data along with the directory entry. For one thing the index would be
> able to register an empty subdirectory, rather than ignoring them. It
> would also better line up with the filesystem's readdir() handling,
> giving us more sane logic to compare what readdir() tells us exists
> against what the index thinks should be in the same file. And the
> overall index should be smaller, because we don't have to repeat the
> same path/to/a/file/for/every/file/in/that/same/directory/tree.
> Reconstructing the path strings at read time into a flat list should
> be pretty trivial, and still keep the parallel lstat calls running off
> a flat list working well for fast status operations.
>
> </thinking>

Haven't really thought through, but I suppose we could create extended tree object format (there is info in cache entry that's not in tree entry), store index in this format, then pack together and store the pack as part of index file. Append-only access to index would be possible by appending a new pack of new trees to index) I think with tree-based index, we could kill a big chunk of code (merging trees and index together) in unpack_trees(). With further efforts to remove list-based index usage, we could even kill match_pathspec_depth(), making tree_entry_interesting() the only function to match patchspec. But dreams probably never come true.

-- 
Duy
Previous: Shawn PearceNext: Nguyen Thai Ngoc Duy
Message 6 of 20 in “read-cache: use sha1file for sha1 calculation”
  1. 1/6 read-cache: use sha1file for sha1 calculationNguyễn Thái Ngọc Duy, Feb 6, 2012
  2. 2/6 csum-file: make sha1 calculation optionalNguyễn Thái Ngọc Duy, Feb 6, 2012
  3. 3/6 Stop producing index version 2Nguyễn Thái Ngọc Duy, Feb 6, 2012
  4. Junio C HamanoFeb 6, 2012
  5. Shawn PearceFeb 7, 2012
  6. Nguyen Thai Ngoc DuyFeb 7, 2012
  7. Nguyen Thai Ngoc DuyFeb 7, 2012
  8. Junio C HamanoFeb 7, 2012
  9. Thomas RastFeb 7, 2012
  10. 4/6 Introduce index version 4 with global flagsNguyễn Thái Ngọc Duy, Feb 6, 2012
  11. 5/6 Allow to use crc32 as a lighter checksum on indexNguyễn Thái Ngọc Duy, Feb 6, 2012
  12. Shawn PearceFeb 7, 2012
  13. Dave ZarzyckiFeb 7, 2012
  14. Dave ZarzyckiFeb 7, 2012
  15. 6/6 Automatically switch to crc32 checksum for index when it's too largeNguyễn Thái Ngọc Duy, Feb 6, 2012
  16. Dave ZarzyckiFeb 6, 2012
  17. Nguyen Thai Ngoc DuyFeb 6, 2012
  18. Dave ZarzyckiFeb 6, 2012
  19. Junio C HamanoFeb 6, 2012
  20. Nguyen Thai Ngoc DuyFeb 6, 2012

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.