# [PATCH] Technical details about the index file format. 9 messages from 2008-02-11 to 2010-09-01. Participants: Robin Rosenberg, Tim Stoakes, Junio C Hamano, Sverre Rabbelier, Ramkumar Ramachandra, Jonathan Nieder, Nguyen Thai Ngoc Duy. Thread: https://gitlist.dev/t/12022 ## Robin Rosenberg, 2008-02-11 06:28 Subject: [PATCH] Technical details about the index file format. Message-ID: <1202711335-12026-1-git-send-email-robin.rosenberg@dewire.com> URL: https://gitlist.dev/e/1202711335-12026-1-git-send-email-robin.rosenberg%40dewire.com ``` Signed-off-by: Robin Rosenberg --- Documentation/technical/index-format.txt | 91 ++++++++++++++++++++++++++++++ 1 files changed, 91 insertions(+), 0 deletions(-) create mode 100644 Documentation/technical/index-format.txt I believe the main index details are accurate. Anything else to explain. The TREE section probably needs more details. -- robin diff --git a/Documentation/technical/index-format.txt b/Documentation/technical/index-format.txt new file mode 100644 index 0000000..c57b382 --- /dev/null +++ b/Documentation/technical/index-format.txt @@ -0,0 +1,91 @@ +GIT index format +================ + += The git index file has the following format + + All binary numbers are in network byte order. + + - A twelve byte header consisting of + + 4 byte signature: + The signature is { 'D', 'I', 'R', 'C' } + + 4 byte version number: + The current version is 2 + + 32-bit number of index entries. + + - An entry consists of + + 32-bit ctime seconds, the last time a file's metadata changed + this is stat(2) data + + 32-bit ctime nanoseconds (modulo 1G) + this is stat(2) data + + 32-bit mtime seconds, the last time a file's data changed + this is stat(2) data + + 32-bit mtime nanoseconds (modulo 1G) + this is stat(2) data + + 32-bit dev + this is stat(2) data + + 32-bit uid + this is stat(2) data + + 32-bit gid + this is stat(2) data + + 32-bit file size + This is the on-disk size from stat(2) + + 160-bit SHA-1 for the represented blob + + A 16-bit field split into (high to low bits) + + 1-bit assume-valid flag + + 1-bit update-needed flag + + 2-bit stage (during merge) + + 12-bit name length + + Name (variable length) - encoding is undefined + + 1-8 nul bytes as necessary to pad the entry to a multiple ot eight bytes + while keeping the name NUL-terminated. + + - Extensions + + The only know index extension today is a tree cache. It contains + pre-computes hashes for all trees that can be derived from the index + + 4 byte extension signature. If the first byte is 'A'..'Z' the + extension is optional and can be ignored. + + 32-bit size of the extension + + Extension data + + - 160-bit SHA-1 over the content of the index file before this checksum. + + +== Tree cache + + - Extension tag { 'T', 'R', 'E', 'E' } + + - 32-bit size + + - A number of entries + + NUL-terminated tree name + + Blank-terminated ASCII decimal number of entries in this tree + + Newline-terminated position of this tree in the parent tree. 0 for + the root tree + + 160-bit SHA-1 for this tree and it's children -- 1.5.4.rc4.25.g81cc ``` ## Tim Stoakes, 2008-02-11 12:00 Subject: Re: [PATCH] Technical details about the index file format. Message-ID: <20080211120054.GG27275@mail.stoakes.net> URL: https://gitlist.dev/e/20080211120054.GG27275%40mail.stoakes.net In-Reply-To: <1202711335-12026-1-git-send-email-robin.rosenberg@dewire.com> ``` Robin Rosenberg(robin.rosenberg@dewire.com)@110208-07:28: > diff --git a/Documentation/technical/index-format.txt b/Documentation/technical/index-format.txt A couple of typos: > + Name (variable length) - encoding is undefined > + > + 1-8 nul bytes as necessary to pad the entry to a multiple ot eight bytes ^ of > + The only know index extension today is a tree cache. It contains > + pre-computes hashes for all trees that can be derived from the index ^ computed full stop Tim -- Tim Stoakes ``` ## Junio C Hamano, 2008-02-11 19:27 Subject: Re: [PATCH] Technical details about the index file format. Message-ID: <7vr6fj8fa0.fsf@gitster.siamese.dyndns.org> URL: https://gitlist.dev/e/7vr6fj8fa0.fsf%40gitster.siamese.dyndns.org In-Reply-To: <1202711335-12026-1-git-send-email-robin.rosenberg@dewire.com> ``` Robin Rosenberg writes: > Signed-off-by: Robin Rosenberg > --- > Documentation/technical/index-format.txt | 91 ++++++++++++++++++++++++++++++ > 1 files changed, 91 insertions(+), 0 deletions(-) > create mode 100644 Documentation/technical/index-format.txt > > I believe the main index details are accurate. Anything else to explain. You missed the most important feature. The entries are sorted in a particular order. Also I do not think we store CE_UPDATE. The tip of 'master' will soon clarify about this. Also when a name is longer than 12-bit can express, we will have a constant there and the name length could be longer (you need to strlen() it). The tip of 'master' will soon have this fix (we used to just overrun). Other than these, I think the description is fairly accurate. ``` ## Sverre Rabbelier, 2010-08-31 00:59 Subject: Re: [PATCH] Technical details about the index file format. Message-ID: URL: https://gitlist.dev/e/loom.20100831T025714-111%40post.gmane.org In-Reply-To: <1202711335-12026-1-git-send-email-robin.rosenberg@dewire.com> ``` Heya, Robin Rosenberg dewire.com> writes: > > Signed-off-by: Robin Rosenberg dewire.com> > --- > Documentation/technical/index-format.txt | 91 ++++++++++++++++++++++++++++++ > 1 files changed, 91 insertions(+), 0 deletions(-) > create mode 100644 Documentation/technical/index-format.txt This pretty much got a LGTM from Junio back when it was sent [0], Robin, can you be persuaded to resend this? I think it'd be very good if we had some documentation on the index format. [0] http://thread.gmane.org/gmane.comp.version-control.git/73471 -- Cheers, Sverre Rabbelier ``` ## Ramkumar Ramachandra, 2010-08-31 07:08 Subject: Re: [PATCH] Technical details about the index file format. Message-ID: <20100831070851.GA7543@kytes> URL: https://gitlist.dev/e/20100831070851.GA7543%40kytes In-Reply-To: ``` Hi Sverre, Sverre Rabbelier writes: > Robin Rosenberg dewire.com> writes: > > > > Signed-off-by: Robin Rosenberg dewire.com> > > --- > > Documentation/technical/index-format.txt | 91 ++++++++++++++++++++++++++++++ > > 1 files changed, 91 insertions(+), 0 deletions(-) > > create mode 100644 Documentation/technical/index-format.txt > > This pretty much got a LGTM from Junio back when it was sent [0], Robin, can you > be persuaded to resend this? I think it'd be very good if we had some > documentation on the index format. Don't we already have this in Documentation/technical/pack-format.txt? -- Ram ``` ## Jonathan Nieder, 2010-08-31 14:23 Subject: Re: [PATCH] Technical details about the index file format. Message-ID: <20100831142359.GB2315@burratino> URL: https://gitlist.dev/e/20100831142359.GB2315%40burratino In-Reply-To: <20100831070851.GA7543@kytes> ``` Ramkumar Ramachandra wrote: > Don't we already have this in Documentation/technical/pack-format.txt? No; this is about the .git/index file rather than .git/objects/pack/*.idx. ``` ## Nguyen Thai Ngoc Duy, 2010-08-31 22:12 Subject: Re: [PATCH] Technical details about the index file format. Message-ID: URL: https://gitlist.dev/e/AANLkTiktRqX%2BfhFW3mN_b5tnbKzDJY86fLpKVth7RpY9%40mail.gmail.com In-Reply-To: <1202711335-12026-1-git-send-email-robin.rosenberg@dewire.com> ``` On Mon, Feb 11, 2008 at 5:28 PM, Robin Rosenberg wrote: > +     4 byte version number: > +       The current version is 2 The version could be 3 if extended flags are used. > +     A 16-bit field split into (high to low bits) > + > +       1-bit assume-valid flag > + > +       1-bit update-needed flag I think this bit is CE_EXTENDED, an indication that this entry has extended flags > + > +       2-bit stage (during merge) > + > +       12-bit name length A 16-bit field additional flags (high to low bits), only applicable to version 3 1-bit reserved for future 1-bit skip-worktree flag 1-bit intent-to-add flag (aka "git add -N") > +  - Extensions > + > +    The only know index extension today is a tree cache. There's also "REUC" extension from read-cache.c. I personally have never touched it, so no comments. -- Duy ``` ## Sverre Rabbelier, 2010-09-01 02:28 Subject: Re: [PATCH] Technical details about the index file format. Message-ID: URL: https://gitlist.dev/e/AANLkTimwGYM-nJTks%2BE5ez_hGkmE%3DfcutPKfd8_kfEvs%40mail.gmail.com In-Reply-To: ``` Heya, On Tue, Aug 31, 2010 at 17:12, Nguyen Thai Ngoc Duy wrote: > The version could be 3 if extended flags are used. I suspect you are the person to have most recently messed around with the index, and given your corrections above, could you perhaps pick up the patch? -- Cheers, Sverre Rabbelier ``` ## Nguyen Thai Ngoc Duy, 2010-09-01 02:52 Subject: Re: [PATCH] Technical details about the index file format. Message-ID: URL: https://gitlist.dev/e/AANLkTi%3DgfGnfwXKFxr-%2BYmDoEvy5hgdG-xUYsR9AoEK3%40mail.gmail.com In-Reply-To: ``` On Wed, Sep 1, 2010 at 12:28 PM, Sverre Rabbelier wrote: > Heya, > > On Tue, Aug 31, 2010 at 17:12, Nguyen Thai Ngoc Duy wrote: >> The version could be 3 if extended flags are used. > > I suspect you are the person to have most recently messed around with > the index, and given your corrections above, could you perhaps pick up > the patch? Whoa, I did not notice the patch was from 2008. Yes, I'll fix it up and resend. -- Duy ```