{"thread":{"id":"12022","subject":"[PATCH] Technical details about the index file format.","startedAt":"2008-02-11T06:28:55Z","lastAt":"2010-09-01T02:52:42Z","messageCount":9,"participants":["Robin Rosenberg","Tim Stoakes","Junio C Hamano","Sverre Rabbelier","Ramkumar Ramachandra","Jonathan Nieder","Nguyen Thai Ngoc Duy"],"isPatch":true,"patchVersion":1,"patchTotal":null},"messages":[{"id":"68328","messageId":"1202711335-12026-1-git-send-email-robin.rosenberg@dewire.com","threadId":"12022","inReplyTo":null,"subject":"[PATCH] Technical details about the index file format.","fromName":"Robin Rosenberg","fromEmail":"robin.rosenberg@dewire.com","sentAt":"2008-02-11T06:28:55Z","receivedAt":"2008-02-11T06:28:55Z","isPatch":true,"sender":{"key":"robin.rosenberg@dewire.com","avatar":"https://avatars.githubusercontent.com/u/46357?v=4"},"body":"Signed-off-by: Robin Rosenberg \n---\n Documentation/technical/index-format.txt | 91 ++++++++++++++++++++++++++++++\n 1 files changed, 91 insertions(+), 0 deletions(-)\n create mode 100644 Documentation/technical/index-format.txt\n\nI believe the main index details are accurate. Anything else to explain. The\nTREE section probably needs more details.\n\n-- robin\n\ndiff --git a/Documentation/technical/index-format.txt b/Documentation/technical/index-format.txt\nnew file mode 100644\nindex 0000000..c57b382\n--- /dev/null\n+++ b/Documentation/technical/index-format.txt\n@@ -0,0 +1,91 @@\n+GIT index format\n+================\n+\n+= The git index file has the following format\n+\n+ All binary numbers are in network byte order.\n+\n+ - A twelve byte header consisting of\n+\n+ 4 byte signature:\n+\tThe signature is { 'D', 'I', 'R', 'C' }\n+\n+ 4 byte version number:\n+\tThe current version is 2\n+\n+ 32-bit number of index entries.\n+\n+ - An entry consists of\n+\n+ 32-bit ctime seconds, the last time a file's metadata changed\n+\tthis is stat(2) data\n+\n+ 32-bit ctime nanoseconds (modulo 1G)\n+\tthis is stat(2) data\n+\n+ 32-bit mtime seconds, the last time a file's data changed\n+\tthis is stat(2) data\n+\n+ 32-bit mtime nanoseconds (modulo 1G)\n+\tthis is stat(2) data\n+\n+ 32-bit dev\n+\tthis is stat(2) data\n+\n+ 32-bit uid\n+\tthis is stat(2) data\n+\n+ 32-bit gid\n+\tthis is stat(2) data\n+\n+ 32-bit file size\n+\tThis is the on-disk size from stat(2)\n+\n+ 160-bit SHA-1 for the represented blob\n+\n+ A 16-bit field split into (high to low bits)\n+\n+\t1-bit assume-valid flag\n+\n+\t1-bit update-needed flag\n+\n+\t2-bit stage (during merge)\n+\n+\t12-bit name length\n+\n+ Name (variable length) - encoding is undefined\n+\n+ 1-8 nul bytes as necessary to pad the entry to a multiple ot eight bytes\n+ while keeping the name NUL-terminated.\n+\n+ - Extensions\n+\n+ The only know index extension today is a tree cache. It contains\n+ pre-computes hashes for all trees that can be derived from the index\n+\n+ 4 byte extension signature. If the first byte is 'A'..'Z' the\n+ extension is optional and can be ignored.\n+\n+ 32-bit size of the extension\n+\n+ Extension data\n+\n+ - 160-bit SHA-1 over the content of the index file before this checksum.\n+\n+\n+== Tree cache\n+\n+ - Extension tag { 'T', 'R', 'E', 'E' }\n+\n+ - 32-bit size\n+\n+ - A number of entries\n+\n+ NUL-terminated tree name\n+\n+ Blank-terminated ASCII decimal number of entries in this tree\n+\n+ Newline-terminated position of this tree in the parent tree. 0 for\n+ the root tree\n+\n+ 160-bit SHA-1 for this tree and it's children\n-- \n1.5.4.rc4.25.g81cc\n"},{"id":"68379","messageId":"20080211120054.GG27275@mail.stoakes.net","threadId":"12022","inReplyTo":"1202711335-12026-1-git-send-email-robin.rosenberg@dewire.com","subject":"Re: [PATCH] Technical details about the index file format.","fromName":"Tim Stoakes","fromEmail":"tim@stoakes.net","sentAt":"2008-02-11T12:00:55Z","receivedAt":"2008-02-11T12:00:55Z","isPatch":true,"sender":{"key":"tim@stoakes.net","avatar":null},"body":"Robin Rosenberg(robin.rosenberg@dewire.com)@110208-07:28:\n> diff --git a/Documentation/technical/index-format.txt b/Documentation/technical/index-format.txt\n\nA couple of typos:\n\n> + Name (variable length) - encoding is undefined\n> +\n> + 1-8 nul bytes as necessary to pad the entry to a multiple ot eight bytes\n ^\nof\n\n> + The only know index extension today is a tree cache. It contains\n> + pre-computes hashes for all trees that can be derived from the index\n ^\n\ncomputed\n\nfull stop\n\nTim\n\n-- \nTim Stoakes\n"},{"id":"68426","messageId":"7vr6fj8fa0.fsf@gitster.siamese.dyndns.org","threadId":"12022","inReplyTo":"1202711335-12026-1-git-send-email-robin.rosenberg@dewire.com","subject":"Re: [PATCH] Technical details about the index file format.","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2008-02-11T19:27:35Z","receivedAt":"2008-02-11T19:27:35Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Robin Rosenberg writes:\n\n> Signed-off-by: Robin Rosenberg \n> ---\n> Documentation/technical/index-format.txt | 91 ++++++++++++++++++++++++++++++\n> 1 files changed, 91 insertions(+), 0 deletions(-)\n> create mode 100644 Documentation/technical/index-format.txt\n>\n> I believe the main index details are accurate. Anything else to explain.\n\nYou missed the most important feature. The entries are sorted\nin a particular order.\n\nAlso I do not think we store CE_UPDATE. The tip of 'master'\nwill soon clarify about this.\n\nAlso when a name is longer than 12-bit can express, we will have\na constant there and the name length could be longer (you need\nto strlen() it). The tip of 'master' will soon have this fix\n(we used to just overrun).\n\nOther than these, I think the description is fairly accurate.\n"},{"id":"149388","messageId":"loom.20100831T025714-111@post.gmane.org","threadId":"12022","inReplyTo":"1202711335-12026-1-git-send-email-robin.rosenberg@dewire.com","subject":"Re: [PATCH] Technical details about the index file format.","fromName":"Sverre Rabbelier","fromEmail":"srabbelier@gmail.com","sentAt":"2010-08-31T00:59:14Z","receivedAt":"2010-08-31T00:59:14Z","isPatch":true,"sender":{"key":"srabbelier@gmail.com","avatar":"https://avatars.githubusercontent.com/u/3098?v=4"},"body":"Heya,\n\nRobin Rosenberg dewire.com> writes:\n> \n> Signed-off-by: Robin Rosenberg dewire.com>\n> ---\n> Documentation/technical/index-format.txt | 91 ++++++++++++++++++++++++++++++\n> 1 files changed, 91 insertions(+), 0 deletions(-)\n> create mode 100644 Documentation/technical/index-format.txt\n\nThis pretty much got a LGTM from Junio back when it was sent [0], Robin, can you \nbe persuaded to resend this? I think it'd be very good if we had some \ndocumentation on the index format.\n\n[0] http://thread.gmane.org/gmane.comp.version-control.git/73471\n\n--\nCheers,\n\nSverre Rabbelier\n"},{"id":"149396","messageId":"20100831070851.GA7543@kytes","threadId":"12022","inReplyTo":"loom.20100831T025714-111@post.gmane.org","subject":"Re: [PATCH] Technical details about the index file format.","fromName":"Ramkumar Ramachandra","fromEmail":"artagnon@gmail.com","sentAt":"2010-08-31T07:08:53Z","receivedAt":"2010-08-31T07:08:53Z","isPatch":true,"sender":{"key":"r@artagnon.com","avatar":"https://avatars.githubusercontent.com/u/37226?v=4"},"body":"Hi Sverre,\n\nSverre Rabbelier writes:\n> Robin Rosenberg dewire.com> writes:\n> > \n> > Signed-off-by: Robin Rosenberg dewire.com>\n> > ---\n> > Documentation/technical/index-format.txt | 91 ++++++++++++++++++++++++++++++\n> > 1 files changed, 91 insertions(+), 0 deletions(-)\n> > create mode 100644 Documentation/technical/index-format.txt\n> \n> This pretty much got a LGTM from Junio back when it was sent [0], Robin, can you \n> be persuaded to resend this? I think it'd be very good if we had some \n> documentation on the index format.\n\nDon't we already have this in Documentation/technical/pack-format.txt?\n\n-- Ram\n"},{"id":"149415","messageId":"20100831142359.GB2315@burratino","threadId":"12022","inReplyTo":"20100831070851.GA7543@kytes","subject":"Re: [PATCH] Technical details about the index file format.","fromName":"Jonathan Nieder","fromEmail":"jrnieder@gmail.com","sentAt":"2010-08-31T14:23:59Z","receivedAt":"2010-08-31T14:23:59Z","isPatch":true,"sender":{"key":"jrnieder@gmail.com","avatar":"https://avatars.githubusercontent.com/u/281595?v=4"},"body":"Ramkumar Ramachandra wrote:\n\n> Don't we already have this in Documentation/technical/pack-format.txt?\n\nNo; this is about the .git/index file rather than .git/objects/pack/*.idx.\n"},{"id":"149503","messageId":"AANLkTiktRqX+fhFW3mN_b5tnbKzDJY86fLpKVth7RpY9@mail.gmail.com","threadId":"12022","inReplyTo":"1202711335-12026-1-git-send-email-robin.rosenberg@dewire.com","subject":"Re: [PATCH] Technical details about the index file format.","fromName":"Nguyen Thai Ngoc Duy","fromEmail":"pclouds@gmail.com","sentAt":"2010-08-31T22:12:03Z","receivedAt":"2010-08-31T22:12:03Z","isPatch":true,"sender":{"key":"pclouds@gmail.com","avatar":"https://avatars.githubusercontent.com/u/720?v=4"},"body":"On Mon, Feb 11, 2008 at 5:28 PM, Robin Rosenberg\n wrote:\n> +     4 byte version number:\n> +       The current version is 2\n\nThe version could be 3 if extended flags are used.\n\n> +     A 16-bit field split into (high to low bits)\n> +\n> +       1-bit assume-valid flag\n> +\n> +       1-bit update-needed flag\n\nI think this bit is CE_EXTENDED, an indication that this entry has\nextended flags\n\n> +\n> +       2-bit stage (during merge)\n> +\n> +       12-bit name length\n\n A 16-bit field additional flags (high to low bits), only\napplicable to version 3\n\n 1-bit reserved for future\n\n 1-bit skip-worktree flag\n\n 1-bit intent-to-add flag (aka \"git add -N\")\n\n> +  - Extensions\n> +\n> +    The only know index extension today is a tree cache.\n\nThere's also \"REUC\" extension from read-cache.c. I personally have\nnever touched it, so no comments.\n-- \nDuy\n"},{"id":"149517","messageId":"AANLkTimwGYM-nJTks+E5ez_hGkmE=fcutPKfd8_kfEvs@mail.gmail.com","threadId":"12022","inReplyTo":"AANLkTiktRqX+fhFW3mN_b5tnbKzDJY86fLpKVth7RpY9@mail.gmail.com","subject":"Re: [PATCH] Technical details about the index file format.","fromName":"Sverre Rabbelier","fromEmail":"srabbelier@gmail.com","sentAt":"2010-09-01T02:28:17Z","receivedAt":"2010-09-01T02:28:17Z","isPatch":true,"sender":{"key":"srabbelier@gmail.com","avatar":"https://avatars.githubusercontent.com/u/3098?v=4"},"body":"Heya,\n\nOn Tue, Aug 31, 2010 at 17:12, Nguyen Thai Ngoc Duy wrote:\n> The version could be 3 if extended flags are used.\n\nI suspect you are the person to have most recently messed around with\nthe index, and given your corrections above, could you perhaps pick up\nthe patch?\n\n-- \nCheers,\n\nSverre Rabbelier\n"},{"id":"149520","messageId":"AANLkTi=gfGnfwXKFxr-+YmDoEvy5hgdG-xUYsR9AoEK3@mail.gmail.com","threadId":"12022","inReplyTo":"AANLkTimwGYM-nJTks+E5ez_hGkmE=fcutPKfd8_kfEvs@mail.gmail.com","subject":"Re: [PATCH] Technical details about the index file format.","fromName":"Nguyen Thai Ngoc Duy","fromEmail":"pclouds@gmail.com","sentAt":"2010-09-01T02:52:42Z","receivedAt":"2010-09-01T02:52:42Z","isPatch":true,"sender":{"key":"pclouds@gmail.com","avatar":"https://avatars.githubusercontent.com/u/720?v=4"},"body":"On Wed, Sep 1, 2010 at 12:28 PM, Sverre Rabbelier wrote:\n> Heya,\n>\n> On Tue, Aug 31, 2010 at 17:12, Nguyen Thai Ngoc Duy wrote:\n>> The version could be 3 if extended flags are used.\n>\n> I suspect you are the person to have most recently messed around with\n> the index, and given your corrections above, could you perhaps pick up\n> the patch?\n\nWhoa, I did not notice the patch was from 2008. Yes, I'll fix it up and resend.\n-- \nDuy\n"}]}