{"thread":{"id":"24933","subject":"[PATCH] doc: technical details about the index file format","startedAt":"2010-09-01T09:53:45Z","lastAt":"2011-03-02T12:53:36Z","messageCount":22,"participants":["Nguyễn Thái Ngọc Duy","Ramkumar Ramachandra","Sverre Rabbelier","Robin Rosenberg","Nguyen Thai Ngoc Duy","Alex Riesen","Joshua Juran","Junio C Hamano","Erik Faye-Lund","Drew Northup"],"isPatch":true,"patchVersion":1,"patchTotal":null},"messages":[{"id":"149544","messageId":"1283334825-18309-1-git-send-email-pclouds@gmail.com","threadId":"24933","inReplyTo":null,"subject":"[PATCH] doc: technical details about the index file format","fromName":"Nguyễn Thái Ngọc Duy","fromEmail":"pclouds@gmail.com","sentAt":"2010-09-01T09:53:45Z","receivedAt":"2010-09-01T09:53:45Z","isPatch":true,"sender":{"key":"pclouds@gmail.com","avatar":"https://avatars.githubusercontent.com/u/720?v=4"},"body":"This bases on the original work by Robin Rosenberg:\n\nhttp://thread.gmane.org/gmane.comp.version-control.git/73471\n\nSigned-off-by: Nguyễn Thái Ngọc Duy \n---\n I split index entry out so the overall format is clearer.\n\n Other changes:\n - mention of version 3\n - added ino and mode\n - added extended flags (v3)\n - entry sort order\n\n Again I don't realy know REUC extension, so only placeholder\n\n Documentation/technical/index-format.txt | 139 ++++++++++++++++++++++++++++++\n 1 files changed, 139 insertions(+), 0 deletions(-)\n create mode 100644 Documentation/technical/index-format.txt\n\ndiff --git a/Documentation/technical/index-format.txt b/Documentation/technical/index-format.txt\nnew file mode 100644\nindex 0000000..3e113ca\n--- /dev/null\n+++ b/Documentation/technical/index-format.txt\n@@ -0,0 +1,139 @@\n+GIT index format\n+================\n+\n+= The git index file has the following format\n+\n+ All binary numbers are in network byte order. Version 2 is described\n+ here unless stated otherwise.\n+\n+ - A 12-byte header consisting of\n+\n+ 4-byte signature:\n+ The signature is { 'D', 'I', 'R', 'C' }\n+\n+ 4-byte version number:\n+ The current supported versions are 2 and 3.\n+\n+ 32-bit number of index entries.\n+\n+ - A number of sorted index entries\n+\n+ - Extensions\n+\n+ Extensions are identified by signature. Optional extensions can\n+ be ignored if GIT does not understand them.\n+\n+ GIT currently supports tree cache and resolve undo extensions.\n+\n+ 4-byte extension signature. If the first byte is 'A'..'Z' the\n+ extension is optional and can be ignored.\n+\n+ 32-bit size of the extension\n+\n+ Extension data\n+\n+ - 160-bit SHA-1 over the content of the index file before this\n+ checksum.\n+\n+== Index entry\n+\n+ Index entries are sorted with memcmp() by entry name. Entries with\n+ the same name are sorted by their stage.\n+\n+ 32-bit ctime seconds, the last time a file's metadata changed\n+ this is stat(2) data\n+\n+ 32-bit ctime nanoseconds (modulo 1G)\n+ this is stat(2) data\n+\n+ 32-bit mtime seconds, the last time a file's data changed\n+ this is stat(2) data\n+\n+ 32-bit mtime nanoseconds (modulo 1G)\n+ this is stat(2) data\n+\n+ 32-bit dev\n+ this is stat(2) data\n+\n+ 32-bit ino\n+ this is stat(2) data\n+\n+ 32-bit mode, split into (high to low bits)\n+\n+ 4-bit object type\n+ valid values in binary are 1000 (blob), 1010 (symbolic link)\n+ and 1110 (gitlink)\n+\n+ 3-bit unused\n+\n+ 9-bit unix permission (only 0755 and 0644 are valid)\n+\n+ 32-bit uid\n+ this is stat(2) data\n+\n+ 32-bit gid\n+ this is stat(2) data\n+\n+ 32-bit file size\n+ This is the on-disk size from stat(2)\n+\n+ 160-bit SHA-1 for the represented object\n+\n+ A 16-bit field split into (high to low bits)\n+\n+ 1-bit assume-valid flag\n+\n+ 1-bit extended flag (must be zero in version 2)\n+\n+ 2-bit stage (during merge)\n+\n+ 12-bit name length if the length is less than 0x0FFF\n+\n+ (Version 3) A 16-bit field, only applicable if the \"extended flag\"\n+ above is 1, split into (high to low bits).\n+\n+ 1-bit reserved for future\n+\n+ 1-bit skip-worktree flag (used by sparse checkout)\n+\n+ 1-bit intent-to-add flag (used by \"git add -N\")\n+\n+ 13-bit unused, must be zero\n+\n+ Entry path name (variable length) relative to top-level directory\n+ (without leading slash). '/' is used as path separator. Special\n+ paths \".\", \"..\" and \".git\" (without quotes) are disallowed.\n+ Trailing slash is also disallowed.\n+\n+ 1-8 nul bytes as necessary to pad the entry to a multiple ot eight bytes\n+ while keeping the name NUL-terminated.\n+\n+== Extensions\n+\n+=== Tree cache\n+\n+ Tree cache extension contains pre-computes hashes for all trees that\n+ can be derived from the index\n+\n+ - Extension tag { 'T', 'R', 'E', 'E' }\n+\n+ - 32-bit size\n+\n+ - A number of entries\n+\n+ NUL-terminated tree name\n+\n+ Blank-terminated ASCII decimal number of entries in this tree\n+\n+ Newline-terminated position of this tree in the parent tree. 0 for\n+ the root tree\n+\n+ 160-bit SHA-1 for this tree and it's children\n+\n+=== Resolve undo\n+\n+ TODO\n+\n+ - Extension tag { 'R', 'E', 'U', 'C' }\n+\n+ - 32-bit size\n-- \n1.7.1.rc1.69.g24c2f7\n"},{"id":"149551","messageId":"20100901103647.GA17260@kytes","threadId":"24933","inReplyTo":"1283334825-18309-1-git-send-email-pclouds@gmail.com","subject":"Re: [PATCH] doc: technical details about the index file format","fromName":"Ramkumar Ramachandra","fromEmail":"artagnon@gmail.com","sentAt":"2010-09-01T10:36:51Z","receivedAt":"2010-09-01T10:36:51Z","isPatch":true,"sender":{"key":"r@artagnon.com","avatar":"https://avatars.githubusercontent.com/u/37226?v=4"},"body":"Hi,\n\nNguyễn Thái Ngọc Duy writes:\n> This bases on the original work by Robin Rosenberg:\n> \n> http://thread.gmane.org/gmane.comp.version-control.git/73471\n[...]\n\nIt might be more profitable to mention the Message-ID instead.\n<1202711335-12026-1-git-send-email-robin.rosenberg@dewire.com>\n\n-- Ram\n"},{"id":"149611","messageId":"1283351989-19426-1-git-send-email-pclouds@gmail.com","threadId":"24933","inReplyTo":"201009012054.20482.robin.rosenberg@dewire.com","subject":"[PATCH] doc: technical details about the index file format","fromName":"Nguyễn Thái Ngọc Duy","fromEmail":"pclouds@gmail.com","sentAt":"2010-09-01T14:39:49Z","receivedAt":"2010-09-01T14:39:49Z","isPatch":true,"sender":{"key":"pclouds@gmail.com","avatar":"https://avatars.githubusercontent.com/u/720?v=4"},"body":"This bases on the original work by Robin Rosenberg.\n\nSigned-off-by: Robin Rosenberg \nSigned-off-by: Nguyễn Thái Ngọc Duy \n---\n Fixups after Robin's review\n\n Documentation/technical/index-format.txt | 144 ++++++++++++++++++++++++++++++\n 1 files changed, 144 insertions(+), 0 deletions(-)\n create mode 100644 Documentation/technical/index-format.txt\n\ndiff --git a/Documentation/technical/index-format.txt b/Documentation/technical/index-format.txt\nnew file mode 100644\nindex 0000000..0285d88\n--- /dev/null\n+++ b/Documentation/technical/index-format.txt\n@@ -0,0 +1,144 @@\n+GIT index format\n+================\n+\n+= The git index file has the following format\n+\n+ All binary numbers are in network byte order. Version 2 is described\n+ here unless stated otherwise.\n+\n+ - A 12-byte header consisting of\n+\n+ 4-byte signature:\n+ The signature is { 'D', 'I', 'R', 'C' }\n+\n+ 4-byte version number:\n+ The current supported versions are 2 and 3.\n+\n+ 32-bit number of index entries.\n+\n+ - A number of sorted index entries\n+\n+ - Extensions\n+\n+ Extensions are identified by signature. Optional extensions can\n+ be ignored if GIT does not understand them.\n+\n+ GIT currently supports tree cache and resolve undo extensions.\n+\n+ 4-byte extension signature. If the first byte is 'A'..'Z' the\n+ extension is optional and can be ignored.\n+\n+ 32-bit size of the extension\n+\n+ Extension data\n+\n+ - 160-bit SHA-1 over the content of the index file before this\n+ checksum.\n+\n+== Index entry\n+\n+ Index entries are sorted in ascending order on the name field,\n+ interpreted as a string of unsigned bytes. Entries with the same\n+ name are sorted by their stage field.\n+\n+ 32-bit ctime seconds, the last time a file's metadata changed\n+ this is stat(2) data\n+\n+ 32-bit ctime nanoseconds (modulo 1G)\n+ this is stat(2) data\n+\n+ 32-bit mtime seconds, the last time a file's data changed\n+ this is stat(2) data\n+\n+ 32-bit mtime nanoseconds (modulo 1G)\n+ this is stat(2) data\n+\n+ 32-bit dev\n+ this is stat(2) data\n+\n+ 32-bit ino\n+ this is stat(2) data\n+\n+ 32-bit mode, split into (high to low bits)\n+\n+ 4-bit object type\n+ valid values in binary are 1000 (blob), 1010 (symbolic link)\n+ and 1110 (gitlink)\n+\n+ 3-bit unused\n+\n+ 9-bit unix permission (only 0755 and 0644 are valid)\n+\n+ 32-bit uid\n+ this is stat(2) data\n+\n+ 32-bit gid\n+ this is stat(2) data\n+\n+ 32-bit file size\n+ This is the on-disk size from stat(2)\n+\n+ 160-bit SHA-1 for the represented object\n+\n+ A 16-bit field split into (high to low bits)\n+\n+ 1-bit assume-valid flag\n+\n+ 1-bit extended flag (must be zero in version 2)\n+\n+ 2-bit stage (during merge)\n+\n+ 12-bit name length if the length is less than 0x0FFF\n+\n+ (Version 3) A 16-bit field, only applicable if the \"extended flag\"\n+ above is 1, split into (high to low bits).\n+\n+ 1-bit reserved for future\n+\n+ 1-bit skip-worktree flag (used by sparse checkout)\n+\n+ 1-bit intent-to-add flag (used by \"git add -N\")\n+\n+ 13-bit unused, must be zero\n+\n+ Entry path name (variable length) relative to top level directory\n+ (without leading slash). '/' is used as path separator. The special\n+ paths \".\", \"..\" and \".git\" (without quotes) are disallowed.\n+ Trailing slash is also disallowed.\n+\n+ The exact encoding is undefined, but the '.' and '/' characters\n+ are encoded in 7-bit ASCII and the encoding cannot contain a nul\n+ byte. Generally a superset of ASCII.\n+\n+ 1-8 nul bytes as necessary to pad the entry to a multiple of eight bytes\n+ while keeping the name NUL-terminated.\n+\n+== Extensions\n+\n+=== Tree cache\n+\n+ Tree cache extension contains pre-computes hashes for all trees that\n+ can be derived from the index\n+\n+ - Extension tag { 'T', 'R', 'E', 'E' }\n+\n+ - 32-bit size\n+\n+ - A number of entries\n+\n+ NUL-terminated tree name\n+\n+ Blank-terminated ASCII decimal number of entries in this tree\n+\n+ Newline-terminated position of this tree in the parent tree. 0 for\n+ the root tree\n+\n+ 160-bit SHA-1 for this tree and it's children\n+\n+=== Resolve undo\n+\n+ TODO\n+\n+ - Extension tag { 'R', 'E', 'U', 'C' }\n+\n+ - 32-bit size\n-- \n1.7.1.rc1.69.g24c2f7\n"},{"id":"149566","messageId":"AANLkTi=uz250bEYdQssCSQar0OUJgDz3+CtYv-aVpdkh@mail.gmail.com","threadId":"24933","inReplyTo":"20100901103647.GA17260@kytes","subject":"Re: [PATCH] doc: technical details about the index file format","fromName":"Sverre Rabbelier","fromEmail":"srabbelier@gmail.com","sentAt":"2010-09-01T15:20:49Z","receivedAt":"2010-09-01T15:20:49Z","isPatch":true,"sender":{"key":"srabbelier@gmail.com","avatar":"https://avatars.githubusercontent.com/u/3098?v=4"},"body":"Heya,\n\n2010/9/1 Ramkumar Ramachandra :\n> It might be more profitable to mention the Message-ID instead.\n> <1202711335-12026-1-git-send-email-robin.rosenberg@dewire.com>\n\nIf you really want to do that, use this link:\n\nhttp://mid.gmane.org/1202711335-12026-1-git-send-email-robin.rosenberg@dewire.com\n\n-- \nCheers,\n\nSverre Rabbelier\n"},{"id":"149585","messageId":"201009012054.20482.robin.rosenberg@dewire.com","threadId":"24933","inReplyTo":"1283334825-18309-1-git-send-email-pclouds@gmail.com","subject":"Re: [PATCH] doc: technical details about the index file format","fromName":"Robin Rosenberg","fromEmail":"robin.rosenberg@dewire.com","sentAt":"2010-09-01T18:54:20Z","receivedAt":"2010-09-01T18:54:20Z","isPatch":true,"sender":{"key":"robin.rosenberg@dewire.com","avatar":"https://avatars.githubusercontent.com/u/46357?v=4"},"body":"onsdagen den 1 september 2010 11.53.45 skrev Nguyễn Thái Ngọc Duy:\n> This bases on the original work by Robin Rosenberg:\n> \n> http://thread.gmane.org/gmane.comp.version-control.git/73471\nNo need for this. My name is enough\n> \n> Signed-off-by: Nguyễn Thái Ngọc Duy \nAdd\nSigned-off-by: Nguyễn Thái Ngọc Duy \n\n> ---\n> I split index entry out so the overall format is clearer.\n> \n> Other changes:\n> - mention of version 3\n> - added ino and mode\n> - added extended flags (v3)\n> - entry sort order\n> \n> Again I don't realy know REUC extension, so only placeholder\n> \n> Documentation/technical/index-format.txt | 139\n> ++++++++++++++++++++++++++++++ 1 files changed, 139 insertions(+), 0\n> deletions(-)\n> create mode 100644 Documentation/technical/index-format.txt\n> \n> diff --git a/Documentation/technical/index-format.txt\n> b/Documentation/technical/index-format.txt new file mode 100644\n> index 0000000..3e113ca\n> --- /dev/null\n> +++ b/Documentation/technical/index-format.txt\n> @@ -0,0 +1,139 @@\n> +GIT index format\n> +================\n> +\n> += The git index file has the following format\n> +\n> + All binary numbers are in network byte order. Version 2 is described\n> + here unless stated otherwise.\n> +\n> + - A 12-byte header consisting of\n> +\n> + 4-byte signature:\n> + The signature is { 'D', 'I', 'R', 'C' }\n> +\n> + 4-byte version number:\n> + The current supported versions are 2 and 3.\n> +\n> + 32-bit number of index entries.\n> +\n> + - A number of sorted index entries\n> +\n> + - Extensions\n> +\n> + Extensions are identified by signature. Optional extensions can\n> + be ignored if GIT does not understand them.\n> +\n> + GIT currently supports tree cache and resolve undo extensions.\n> +\n> + 4-byte extension signature. If the first byte is 'A'..'Z' the\n> + extension is optional and can be ignored.\n> +\n> + 32-bit size of the extension\n> +\n> + Extension data\n> +\n> + - 160-bit SHA-1 over the content of the index file before this\n> + checksum.\n> +\n> +== Index entry\n> +\n> + Index entries are sorted with memcmp() by entry name. Entries with\n> + the same name are sorted by their stage.\nIndex entries are sorted in ascending order on the name field, interpreted as\na string of unsigned bytes.\n\n> +\n> + 32-bit ctime seconds, the last time a file's metadata changed\n> + this is stat(2) data\n> +\n> + 32-bit ctime nanoseconds (modulo 1G)\n> + this is stat(2) data\n> +\n> + 32-bit mtime seconds, the last time a file's data changed\n> + this is stat(2) data\n> +\n> + 32-bit mtime nanoseconds (modulo 1G)\n> + this is stat(2) data\n> +\n> + 32-bit dev\n> + this is stat(2) data\n> +\n> + 32-bit ino\n> + this is stat(2) data\n> +\n> + 32-bit mode, split into (high to low bits)\n> +\n> + 4-bit object type\n> + valid values in binary are 1000 (blob), 1010 (symbolic link)\n> + and 1110 (gitlink)\n> +\n> + 3-bit unused\n> +\n> + 9-bit unix permission (only 0755 and 0644 are valid)\n> +\n> + 32-bit uid\n> + this is stat(2) data\n> +\n> + 32-bit gid\n> + this is stat(2) data\n> +\n> + 32-bit file size\n> + This is the on-disk size from stat(2)\n> +\n> + 160-bit SHA-1 for the represented object\n> +\n> + A 16-bit field split into (high to low bits)\n> +\n> + 1-bit assume-valid flag\n> +\n> + 1-bit extended flag (must be zero in version 2)\n> +\n> + 2-bit stage (during merge)\n> +\n> + 12-bit name length if the length is less than 0x0FFF\n> +\n> + (Version 3) A 16-bit field, only applicable if the \"extended flag\"\n> + above is 1, split into (high to low bits).\n> +\n> + 1-bit reserved for future\n> +\n> + 1-bit skip-worktree flag (used by sparse checkout)\n> +\n> + 1-bit intent-to-add flag (used by \"git add -N\")\n> +\n> + 13-bit unused, must be zero\n> +\n> + Entry path name (variable length) relative to top-level directory\n...to the top level...\n> + (without leading slash). '/' is used as path separator. Special\nThe special...\n> + paths \".\", \"..\" and \".git\" (without quotes) are disallowed.\n> + Trailing slash is also disallowed.\nWhy would anyone even consider adding a trailing slash to a _file_ name?\n\n The exact encoding is undefined, but the '.', and '/' characters\n are encoded in 7-bit ASCII and the encoding cannot contain a nul byte.\n Generally a superset of ASCII\n\n> +\n> + 1-8 nul bytes as necessary to pad the entry to a multiple ot eight bytes\n...of eight bytes\n\nA typo of mine.\n\n> + while keeping the name NUL-terminated.\n> +\n> +== Extensions\n> +\n> +=== Tree cache\n> +\n> + Tree cache extension contains pre-computes hashes for all trees that\n> + can be derived from the index\n> +\n> + - Extension tag { 'T', 'R', 'E', 'E' }\n> +\n> + - 32-bit size\n> +\n> + - A number of entries\n> +\n> + NUL-terminated tree name\n> +\n> + Blank-terminated ASCII decimal number of entries in this tree\n> +\n> + Newline-terminated position of this tree in the parent tree. 0 for\n> + the root tree\n> +\n> + 160-bit SHA-1 for this tree and it's children\n> +\n> +=== Resolve undo\n> +\n> + TODO\n> +\n> + - Extension tag { 'R', 'E', 'U', 'C' }\n> +\n> + - 32-bit size\n"},{"id":"149610","messageId":"AANLkTikOsZa6OvWkCV27NB7k2t+kT-5wDfx5Eaymx_MW@mail.gmail.com","threadId":"24933","inReplyTo":"201009012054.20482.robin.rosenberg@dewire.com","subject":"Re: [PATCH] doc: technical details about the index file format","fromName":"Nguyen Thai Ngoc Duy","fromEmail":"pclouds@gmail.com","sentAt":"2010-09-01T23:28:44Z","receivedAt":"2010-09-01T23:28:44Z","isPatch":true,"sender":{"key":"pclouds@gmail.com","avatar":"https://avatars.githubusercontent.com/u/720?v=4"},"body":"2010/9/2 Robin Rosenberg :\n>> +  Entry path name (variable length) relative to top-level directory\n> ...to the top level...\n>> +    (without leading slash). '/' is used as path separator. Special\n> The special...\n>> +    paths \".\", \"..\" and \".git\" (without quotes) are disallowed.\n>> +    Trailing slash is also disallowed.\n> Why would anyone even consider adding a trailing slash to a _file_ name?\n\nWell, I was tempted to put directories in index more than once. And\nsubprojects are actually directories although they are treated as\nfiles in index.\n-- \nDuy\n"},{"id":"149619","messageId":"201009020759.19417.robin.rosenberg@dewire.com","threadId":"24933","inReplyTo":"201009012054.20482.robin.rosenberg@dewire.com","subject":"Re: [PATCH] doc: technical details about the index file format","fromName":"Robin Rosenberg","fromEmail":"robin.rosenberg@dewire.com","sentAt":"2010-09-02T05:59:19Z","receivedAt":"2010-09-02T05:59:19Z","isPatch":true,"sender":{"key":"robin.rosenberg@dewire.com","avatar":"https://avatars.githubusercontent.com/u/46357?v=4"},"body":"onsdagen den 1 september 2010 20.54.20 skrev Robin Rosenberg:\n> onsdagen den 1 september 2010 11.53.45 skrev Nguyễn Thái Ngọc Duy:\n> > This bases on the original work by Robin Rosenberg:\n> > \n> > http://thread.gmane.org/gmane.comp.version-control.git/73471\n> \n> No need for this. My name is enough\n> \n> > Signed-off-by: Nguyễn Thái Ngọc Duy \n> \n\nAdd this rather than then one I sent in the previus mail...\n\nSigned-off-by: Robin Rosenberg \n\n-- robin\n"},{"id":"149627","messageId":"AANLkTi=wESk38u1XSTL1rd2__eQzHfSuq-EbqooxmcVw@mail.gmail.com","threadId":"24933","inReplyTo":"1283351989-19426-1-git-send-email-pclouds@gmail.com","subject":"Re: [PATCH] doc: technical details about the index file format","fromName":"Alex Riesen","fromEmail":"raa.lkml@gmail.com","sentAt":"2010-09-02T08:56:10Z","receivedAt":"2010-09-02T08:56:10Z","isPatch":true,"sender":{"key":"raa.lkml@gmail.com","avatar":"https://avatars.githubusercontent.com/u/324101?v=4"},"body":"2010/9/1 Nguyễn Thái Ngọc Duy :\n> +== Index entry\n> +\n> +  Index entries are sorted in ascending order on the name field,\n> +  interpreted as a string of unsigned bytes. Entries with the same\n> +  name are sorted by their stage field.\n> +\n> +  32-bit ctime seconds, the last time a file's metadata changed\n> +    this is stat(2) data\n> +\n> +  32-bit ctime nanoseconds (modulo 1G)\n> +    this is stat(2) data\n\nMaybe I'm missing something, but I failed to find where \"modulo 1G\" comes from.\nAFAICS (read-cache.c), the stat data are saved almost unmodified\n(casted to unsigned int).\n(BTW, is 1G the Gravitational Constant or what?)\n\nI'm not sure it is safe to assume that every system Git will be\nported to defines\n\"unsigned int\" to be 32 bits. OTOH, never met one where it is something else.\nStill, using uint32_t (the POSIX types) in ondisk_cache_entry would be clearer\n(unlikely alignment issues aside.\n"},{"id":"149631","messageId":"E4DDD4F7-FEDB-49CE-9515-90D64B66D7D3@gmail.com","threadId":"24933","inReplyTo":"AANLkTi=wESk38u1XSTL1rd2__eQzHfSuq-EbqooxmcVw@mail.gmail.com","subject":"Re: [PATCH] doc: technical details about the index file format","fromName":"Joshua Juran","fromEmail":"jjuran@gmail.com","sentAt":"2010-09-02T09:08:57Z","receivedAt":"2010-09-02T09:08:57Z","isPatch":true,"sender":{"key":"jjuran@gmail.com","avatar":null},"body":"On Sep 2, 2010, at 1:56 AM, Alex Riesen wrote:\n\n> 2010/9/1 Nguyễn Thái Ngọc Duy :\n>> +== Index entry\n>> +\n>> + Index entries are sorted in ascending order on the name field,\n>> + interpreted as a string of unsigned bytes. Entries with the same\n>> + name are sorted by their stage field.\n>> +\n>> + 32-bit ctime seconds, the last time a file's metadata changed\n>> + this is stat(2) data\n>> +\n>> + 32-bit ctime nanoseconds (modulo 1G)\n>> + this is stat(2) data\n>\n> Maybe I'm missing something, but I failed to find where \"modulo 1G\" \n> comes from.\n> AFAICS (read-cache.c), the stat data are saved almost unmodified\n> (casted to unsigned int).\n> (BTW, is 1G the Gravitational Constant or what?)\n\nG stands for \"giga-\" meaning one billion, so 1G refers to one billion \nnanoseconds.\n\n> I'm not sure it is safe to assume that every system Git will be\n> ported to defines\n> \"unsigned int\" to be 32 bits. OTOH, never met one where it is \n> something else.\n\nDOS and early Mac compilers have used 16-bit ints, but I don't think \nanyone cares.\n\nJosh\n"},{"id":"149644","messageId":"7vlj7k42t7.fsf@alter.siamese.dyndns.org","threadId":"24933","inReplyTo":"AANLkTi=wESk38u1XSTL1rd2__eQzHfSuq-EbqooxmcVw@mail.gmail.com","subject":"Re: [PATCH] doc: technical details about the index file format","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2010-09-02T14:50:44Z","receivedAt":"2010-09-02T14:50:44Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Alex Riesen writes:\n\n>> + 32-bit ctime seconds, the last time a file's metadata changed\n>> + this is stat(2) data\n>> +\n>> + 32-bit ctime nanoseconds (modulo 1G)\n>> + this is stat(2) data\n>\n> Maybe I'm missing something, but I failed to find where \"modulo 1G\" comes from.\n\nI think the above wants to say \"seconds and sub-seconds are stored in\nseparate fields, and latter is purely sub-seconds, never reaching nor\nexceeding a whole second\" (gig == 10^-9) times nano (== 10^+9) is 1).\n\nI personally do not think it is a good idea to say \" (modulo 1G)\" there;\nit is more confusing than without.\n\nEither the reader knows, from seeing \"this is stat(2) data\", what\nseconds/nanoseconds mean, in which case the comment gives redundant\ninformation in cryptic terms, or the reader doesn't, in which case the\nconcept of storing the timestamp as a (second, subsecond) tuple needs to\nbe explained a lot better than the above to be understood.\n"},{"id":"149647","messageId":"AANLkTi=iFe=MmUiXzC_HMwueZxLJDCea+zp_-SNWvSup@mail.gmail.com","threadId":"24933","inReplyTo":"7vlj7k42t7.fsf@alter.siamese.dyndns.org","subject":"Re: [PATCH] doc: technical details about the index file format","fromName":"Erik Faye-Lund","fromEmail":"kusmabite@gmail.com","sentAt":"2010-09-02T15:11:30Z","receivedAt":"2010-09-02T15:11:30Z","isPatch":true,"sender":{"key":"kusmabite@gmail.com","avatar":"https://avatars.githubusercontent.com/u/47073?v=4"},"body":"On Thu, Sep 2, 2010 at 4:50 PM, Junio C Hamano wrote:\n> Alex Riesen writes:\n>\n>>> +  32-bit ctime seconds, the last time a file's metadata changed\n>>> +    this is stat(2) data\n>>> +\n>>> +  32-bit ctime nanoseconds (modulo 1G)\n>>> +    this is stat(2) data\n>>\n>> Maybe I'm missing something, but I failed to find where \"modulo 1G\" comes from.\n>\n> I think the above wants to say \"seconds and sub-seconds are stored in\n> separate fields, and latter is purely sub-seconds, never reaching nor\n> exceeding a whole second\" (gig == 10^-9) times nano (== 10^+9) is 1).\n>\n> I personally do not think it is a good idea to say \" (modulo 1G)\" there;\n> it is more confusing than without.\n>\n\nPerhaps \"nanosecond fractions\" would be a simple and precise description?\n"},{"id":"150146","messageId":"1283769430-9263-1-git-send-email-pclouds@gmail.com","threadId":"24933","inReplyTo":"AANLkTi=iFe=MmUiXzC_HMwueZxLJDCea+zp_-SNWvSup@mail.gmail.com","subject":"[PATCH] doc: technical details about the index file format","fromName":"Nguyễn Thái Ngọc Duy","fromEmail":"pclouds@gmail.com","sentAt":"2010-09-06T10:37:10Z","receivedAt":"2010-09-06T10:37:10Z","isPatch":true,"sender":{"key":"pclouds@gmail.com","avatar":"https://avatars.githubusercontent.com/u/720?v=4"},"body":"This bases on the original work by Robin Rosenberg.\n\nSigned-off-by: Robin Rosenberg \nSigned-off-by: Nguyễn Thái Ngọc Duy \n---\n \"nanoseconds (modulo 1G)\" is changed to \"nanosecond fractions\"\n\n Documentation/technical/index-format.txt | 144 ++++++++++++++++++++++++++++++\n 1 files changed, 144 insertions(+), 0 deletions(-)\n create mode 100644 Documentation/technical/index-format.txt\n\ndiff --git a/Documentation/technical/index-format.txt b/Documentation/technical/index-format.txt\nnew file mode 100644\nindex 0000000..5b1d70d\n--- /dev/null\n+++ b/Documentation/technical/index-format.txt\n@@ -0,0 +1,144 @@\n+GIT index format\n+================\n+\n+= The git index file has the following format\n+\n+ All binary numbers are in network byte order. Version 2 is described\n+ here unless stated otherwise.\n+\n+ - A 12-byte header consisting of\n+\n+ 4-byte signature:\n+ The signature is { 'D', 'I', 'R', 'C' }\n+\n+ 4-byte version number:\n+ The current supported versions are 2 and 3.\n+\n+ 32-bit number of index entries.\n+\n+ - A number of sorted index entries\n+\n+ - Extensions\n+\n+ Extensions are identified by signature. Optional extensions can\n+ be ignored if GIT does not understand them.\n+\n+ GIT currently supports tree cache and resolve undo extensions.\n+\n+ 4-byte extension signature. If the first byte is 'A'..'Z' the\n+ extension is optional and can be ignored.\n+\n+ 32-bit size of the extension\n+\n+ Extension data\n+\n+ - 160-bit SHA-1 over the content of the index file before this\n+ checksum.\n+\n+== Index entry\n+\n+ Index entries are sorted in ascending order on the name field,\n+ interpreted as a string of unsigned bytes. Entries with the same\n+ name are sorted by their stage field.\n+\n+ 32-bit ctime seconds, the last time a file's metadata changed\n+ this is stat(2) data\n+\n+ 32-bit ctime nanosecond fractions\n+ this is stat(2) data\n+\n+ 32-bit mtime seconds, the last time a file's data changed\n+ this is stat(2) data\n+\n+ 32-bit mtime nanosecond fractions\n+ this is stat(2) data\n+\n+ 32-bit dev\n+ this is stat(2) data\n+\n+ 32-bit ino\n+ this is stat(2) data\n+\n+ 32-bit mode, split into (high to low bits)\n+\n+ 4-bit object type\n+ valid values in binary are 1000 (blob), 1010 (symbolic link)\n+ and 1110 (gitlink)\n+\n+ 3-bit unused\n+\n+ 9-bit unix permission (only 0755 and 0644 are valid)\n+\n+ 32-bit uid\n+ this is stat(2) data\n+\n+ 32-bit gid\n+ this is stat(2) data\n+\n+ 32-bit file size\n+ This is the on-disk size from stat(2)\n+\n+ 160-bit SHA-1 for the represented object\n+\n+ A 16-bit field split into (high to low bits)\n+\n+ 1-bit assume-valid flag\n+\n+ 1-bit extended flag (must be zero in version 2)\n+\n+ 2-bit stage (during merge)\n+\n+ 12-bit name length if the length is less than 0x0FFF\n+\n+ (Version 3) A 16-bit field, only applicable if the \"extended flag\"\n+ above is 1, split into (high to low bits).\n+\n+ 1-bit reserved for future\n+\n+ 1-bit skip-worktree flag (used by sparse checkout)\n+\n+ 1-bit intent-to-add flag (used by \"git add -N\")\n+\n+ 13-bit unused, must be zero\n+\n+ Entry path name (variable length) relative to top level directory\n+ (without leading slash). '/' is used as path separator. The special\n+ paths \".\", \"..\" and \".git\" (without quotes) are disallowed.\n+ Trailing slash is also disallowed.\n+\n+ The exact encoding is undefined, but the '.' and '/' characters\n+ are encoded in 7-bit ASCII and the encoding cannot contain a nul\n+ byte. Generally a superset of ASCII.\n+\n+ 1-8 nul bytes as necessary to pad the entry to a multiple of eight bytes\n+ while keeping the name NUL-terminated.\n+\n+== Extensions\n+\n+=== Tree cache\n+\n+ Tree cache extension contains pre-computes hashes for all trees that\n+ can be derived from the index\n+\n+ - Extension tag { 'T', 'R', 'E', 'E' }\n+\n+ - 32-bit size\n+\n+ - A number of entries\n+\n+ NUL-terminated tree name\n+\n+ Blank-terminated ASCII decimal number of entries in this tree\n+\n+ Newline-terminated position of this tree in the parent tree. 0 for\n+ the root tree\n+\n+ 160-bit SHA-1 for this tree and it's children\n+\n+=== Resolve undo\n+\n+ TODO\n+\n+ - Extension tag { 'R', 'E', 'U', 'C' }\n+\n+ - 32-bit size\n-- \n1.7.1.rc1.69.g24c2f7\n"},{"id":"161717","messageId":"AANLkTi=YJkk6KHChCrrazij_ziyG-Ru7kGLWc7JnUGoN@mail.gmail.com","threadId":"24933","inReplyTo":"1283769430-9263-1-git-send-email-pclouds@gmail.com","subject":"Re: [PATCH] doc: technical details about the index file format","fromName":"Sverre Rabbelier","fromEmail":"srabbelier@gmail.com","sentAt":"2011-02-19T22:16:03Z","receivedAt":"2011-02-19T22:16:03Z","isPatch":true,"sender":{"key":"srabbelier@gmail.com","avatar":"https://avatars.githubusercontent.com/u/3098?v=4"},"body":"Heya,\n\n2010/9/6 Nguyễn Thái Ngọc Duy :\n> This bases on the original work by Robin Rosenberg.\n\nJunio, in the \"what's cooking\" you mention that you might jump in to\nimprove this? Duy, are you still interested in carrying this forward?\nThis patch [0] would be helpful to the hgit people as well :).\n\nhttp://git.kernel.org/?p=git/git.git;a=commit;h=673f3d9d4e019a15c6d3770e9f8d9b07059f16cc\n\n-- \nCheers,\n\nSverre Rabbelier\n"},{"id":"161739","messageId":"AANLkTi=hz0xRsTy5f8xhzBhu0md_iPCxvdTrEPrzYwzt@mail.gmail.com","threadId":"24933","inReplyTo":"AANLkTi=YJkk6KHChCrrazij_ziyG-Ru7kGLWc7JnUGoN@mail.gmail.com","subject":"Re: [PATCH] doc: technical details about the index file format","fromName":"Nguyen Thai Ngoc Duy","fromEmail":"pclouds@gmail.com","sentAt":"2011-02-20T09:30:07Z","receivedAt":"2011-02-20T09:30:07Z","isPatch":true,"sender":{"key":"pclouds@gmail.com","avatar":"https://avatars.githubusercontent.com/u/720?v=4"},"body":"2011/2/20 Sverre Rabbelier :\n> Heya,\n>\n> 2010/9/6 Nguyễn Thái Ngọc Duy :\n>> This bases on the original work by Robin Rosenberg.\n>\n> Junio, in the \"what's cooking\" you mention that you might jump in to\n> improve this? Duy, are you still interested in carrying this forward?\n> This patch [0] would be helpful to the hgit people as well :).\n\nI can try to study resolve undo extension next week and see if I can\nwrite it down in the document.\n-- \nDuy\n"},{"id":"162298","messageId":"20110226100310.GA21724@do","threadId":"24933","inReplyTo":"AANLkTi=hz0xRsTy5f8xhzBhu0md_iPCxvdTrEPrzYwzt@mail.gmail.com","subject":"Re: [PATCH] doc: technical details about the index file format","fromName":"Nguyen Thai Ngoc Duy","fromEmail":"pclouds@gmail.com","sentAt":"2011-02-26T10:03:10Z","receivedAt":"2011-02-26T10:03:10Z","isPatch":true,"sender":{"key":"pclouds@gmail.com","avatar":"https://avatars.githubusercontent.com/u/720?v=4"},"body":"On Sun, Feb 20, 2011 at 04:30:07PM +0700, Nguyen Thai Ngoc Duy wrote:\n> 2011/2/20 Sverre Rabbelier :\n> > Heya,\n> >\n> > 2010/9/6 Nguyễn Thái Ngọc Duy :\n> >> This bases on the original work by Robin Rosenberg.\n> >\n> > Junio, in the \"what's cooking\" you mention that you might jump in to\n> > improve this? Duy, are you still interested in carrying this forward?\n> > This patch [0] would be helpful to the hgit people as well :).\n> \n> I can try to study resolve undo extension next week and see if I can\n> write it down in the document.\n\nOK here come the missing bits on top of the previous patch. Looks good?\n\n--8<--\ndiff --git a/Documentation/technical/index-format.txt b/Documentation/technical/index-format.txt\nindex 5b1d70d..574eb3b 100644\n--- a/Documentation/technical/index-format.txt\n+++ b/Documentation/technical/index-format.txt\n@@ -118,7 +118,7 @@ GIT index format\n === Tree cache\n \n Tree cache extension contains pre-computes hashes for all trees that\n- can be derived from the index\n+ can be derived from the index.\n \n - Extension tag { 'T', 'R', 'E', 'E' }\n \n@@ -137,8 +137,20 @@ GIT index format\n \n === Resolve undo\n \n- TODO\n+ Resolve undo extension records staged entries before they are\n+ resolved and removed from index. It can be used to recreate conflicts\n+ after the conflict is incorrectly resolved.\n \n - Extension tag { 'R', 'E', 'U', 'C' }\n \n - 32-bit size\n+\n+ - A number of entries\n+\n+ NUL-terminated entry name\n+\n+ Entry mode of the entry in three stages, in increasing order from\n+ 1 to 3, in NUL-terminated ASCII octal number.\n+\n+ 160 bit SHA-1 of the entry in three stages, in increasing\n+ order from 1 to 3. A stage with zero mode will be skipped.\n-->8--\n-- \nDuy\n"},{"id":"162299","messageId":"7vsjvb6qmt.fsf@alter.siamese.dyndns.org","threadId":"24933","inReplyTo":"20110226100310.GA21724@do","subject":"Re: [PATCH] doc: technical details about the index file format","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2011-02-26T10:23:38Z","receivedAt":"2011-02-26T10:23:38Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Nguyen Thai Ngoc Duy writes:\n\n> OK here come the missing bits on top of the previous patch. Looks good?\n\nThanks.\n\n> diff --git a/Documentation/technical/index-format.txt b/Documentation/technical/index-format.txt\n> index 5b1d70d..574eb3b 100644\n> --- a/Documentation/technical/index-format.txt\n> +++ b/Documentation/technical/index-format.txt\n> @@ -118,7 +118,7 @@ GIT index format\n> === Tree cache\n> \n> Tree cache extension contains pre-computes hashes for all trees that\n> - can be derived from the index\n> + can be derived from the index.\n> \n> - Extension tag { 'T', 'R', 'E', 'E' }\n> \n> @@ -137,8 +137,20 @@ GIT index format\n> \n> === Resolve undo\n> \n> - TODO\n> + Resolve undo extension records staged entries before they are\n> + resolved and removed from index. It can be used to recreate conflicts\n> + after the conflict is incorrectly resolved.\n\nI lack energy to come up with a succinct description right now, so here is\nan undistilled version of what I would want to see the reader of the above\nparagraph understand:\n\n A set of entries for a path at higher stages (i.e. the ones that\n represent a merge conflict at the path) used to be removed from the\n index and replaced with the result of the resolution when the conflict\n is resolved (e.g. with \"git add path\"). This extension saves these\n higher stage entries away so that \"checkout -m\" and other operations\n can recreate the conflicted state, in case you botched a conflict\n resolution and want to redo it from scratch.\n\nThe description of the data contents looked fine, except that \"A number of\nentries\" felt a bit unclear (it would make the reader wonder if we record\nhow many we have at that location as an integer, which is not the case).\n\n> - Extension tag { 'R', 'E', 'U', 'C' }\n> \n> - 32-bit size\n> +\n> + - A number of entries\n> +\n> + NUL-terminated entry name\n> +\n> + Entry mode of the entry in three stages, in increasing order from\n> + 1 to 3, in NUL-terminated ASCII octal number.\n> +\n> + 160 bit SHA-1 of the entry in three stages, in increasing\n> + order from 1 to 3. A stage with zero mode will be skipped.\n"},{"id":"162314","messageId":"20110226133639.GA32442@do","threadId":"24933","inReplyTo":"7vsjvb6qmt.fsf@alter.siamese.dyndns.org","subject":"Re: [PATCH] doc: technical details about the index file format","fromName":"Nguyen Thai Ngoc Duy","fromEmail":"pclouds@gmail.com","sentAt":"2011-02-26T13:36:39Z","receivedAt":"2011-02-26T13:36:39Z","isPatch":true,"sender":{"key":"pclouds@gmail.com","avatar":"https://avatars.githubusercontent.com/u/720?v=4"},"body":"On Sat, Feb 26, 2011 at 02:23:38AM -0800, Junio C Hamano wrote:\n> I lack energy to come up with a succinct description right now, so here is\n> an undistilled version of what I would want to see the reader of the above\n> paragraph understand:\n> \n> A set of entries for a path at higher stages (i.e. the ones that\n> represent a merge conflict at the path) used to be removed from the\n> index and replaced with the result of the resolution when the conflict\n> is resolved (e.g. with \"git add path\"). This extension saves these\n> higher stage entries away so that \"checkout -m\" and other operations\n> can recreate the conflicted state, in case you botched a conflict\n> resolution and want to redo it from scratch.\n> \n> The description of the data contents looked fine, except that \"A number of\n> entries\" felt a bit unclear (it would make the reader wonder if we record\n> how many we have at that location as an integer, which is not the case).\n\nOK another try. I also add more details to tree cache. If somebody\nuses this document to create a git-compatible tool, then such a tool\nshould behave the way git expects it.\n\nA related note. Because we store SHA-1s in resolve undo ext. fsck\nshould check these for reachability as well. I see fsck checks for\ncache-tree only.\n\n--8<--\ndiff --git a/Documentation/technical/index-format.txt b/Documentation/technical/index-format.txt\nindex 5b1d70d..2a3490c 100644\n--- a/Documentation/technical/index-format.txt\n+++ b/Documentation/technical/index-format.txt\n@@ -117,8 +117,12 @@ GIT index format\n \n === Tree cache\n \n- Tree cache extension contains pre-computes hashes for all trees that\n- can be derived from the index\n+ Tree cache extension contains pre-computed hashes for trees that can\n+ be derived from the index. It helps speed up tree object generation\n+ from index for a new commit.\n+\n+ When a path is updated in index, the path must be invalidated and\n+ removed from tree cache.\n \n - Extension tag { 'T', 'R', 'E', 'E' }\n \n@@ -137,8 +141,25 @@ GIT index format\n \n === Resolve undo\n \n- TODO\n+ A conflict is represented in index as a set of higher stage entries.\n+ When a conflict is resolved (e.g. with \"git add path\"), these higher\n+ stage entries will be removed and a stage-0 entry with proper\n+ resoluton is added.\n+\n+ Resolve undo extension saves these higher stage entries so that\n+ conflicts can be recreated (e.g. with \"git checkout -m\"), in case\n+ users want to redo a conflict resolution from scratch.\n \n - Extension tag { 'R', 'E', 'U', 'C' }\n \n - 32-bit size\n+\n+ - A number of conflict entries\n+\n+ NUL-terminated conflict path\n+\n+ Three NUL-terminated ASCII octal numbers, entry mode of entries in\n+ stage 1 to 3.\n+\n+ At most three 160-bit SHA-1s of the entry in three stages from 1\n+ to 3. SHA-1 is not saved for any stage with entry mode zero.\n--8<--\n-- \nDuy\n"},{"id":"162615","messageId":"7vpqqaffy2.fsf@alter.siamese.dyndns.org","threadId":"24933","inReplyTo":"20110226133639.GA32442@do","subject":"Re: [PATCH] doc: technical details about the index file format","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2011-03-02T01:51:01Z","receivedAt":"2011-03-02T01:51:01Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Nguyen Thai Ngoc Duy writes:\n\n> OK another try. I also add more details to tree cache. If somebody\n> uses this document to create a git-compatible tool, then such a tool\n> should behave the way git expects it.\n\nThanks.\n\nHere is what I scribbled on top of yours (not quite polished).\n\n * Clarify \"string of unsigned bytes\";\n\n * Blob has two variants (regular file vs symlink), not (blob vs symlink);\n\n * Clarify permission mode bits;\n\n * Clarify ce_namelen() \"too long to fit in the length field\" case;\n\n * Clarify \".\" etc are forbidden as path components;\n\n * Match the description with the internal wording \"cache-tree\";\n\n * All types of extension begin with signature and length as explained in\n the first part. Don't repeat the \"length\" part in the description of\n each extension (can be mistaken as if there is a separate 32-bit size\n field inside the extension), but state what the signature for each\n extension is.\n\n * Don't say \"Extension tag\", as we have said \"Extension signature\" in the\n first part---be consistent;\n\n * Clarify the invalidation of cache-tree entries;\n\n * Correct description on subtree_nr field in the cache-tree;\n\n * Clarify the order of entries in cache-tree;\n\n\n\n Documentation/technical/index-format.txt | 94 ++++++++++++++++++------------\n 1 files changed, 57 insertions(+), 37 deletions(-)\n\ndiff --git a/Documentation/technical/index-format.txt b/Documentation/technical/index-format.txt\nindex 89e410a..8923f6f 100644\n--- a/Documentation/technical/index-format.txt\n+++ b/Documentation/technical/index-format.txt\n@@ -9,21 +9,21 @@ GIT index format\n - A 12-byte header consisting of\n \n 4-byte signature:\n- The signature is { 'D', 'I', 'R', 'C' }\n+ The signature is { 'D', 'I', 'R', 'C' } (stands for \"dircache\")\n \n 4-byte version number:\n The current supported versions are 2 and 3.\n \n 32-bit number of index entries.\n \n- - A number of sorted index entries\n+ - A number of sorted index entries (see below).\n \n - Extensions\n \n Extensions are identified by signature. Optional extensions can\n be ignored if GIT does not understand them.\n \n- GIT currently supports tree cache and resolve undo extensions.\n+ GIT currently supports cached tree and resolve undo extensions.\n \n 4-byte extension signature. If the first byte is 'A'..'Z' the\n extension is optional and can be ignored.\n@@ -38,8 +38,9 @@ GIT index format\n == Index entry\n \n Index entries are sorted in ascending order on the name field,\n- interpreted as a string of unsigned bytes. Entries with the same\n- name are sorted by their stage field.\n+ interpreted as a string of unsigned bytes (i.e. memcmp() order, no\n+ localization, no special casing of directory separator '/'). Entries\n+ with the same name are sorted by their stage field.\n \n 32-bit ctime seconds, the last time a file's metadata changed\n this is stat(2) data\n@@ -62,12 +63,13 @@ GIT index format\n 32-bit mode, split into (high to low bits)\n \n 4-bit object type\n- valid values in binary are 1000 (blob), 1010 (symbolic link)\n+ valid values in binary are 1000 (regular file), 1010 (symbolic link)\n and 1110 (gitlink)\n \n 3-bit unused\n \n- 9-bit unix permission (only 0755 and 0644 are valid)\n+ 9-bit unix permission. Only 0755 and 0644 are valid for regular files.\n+ Symbolic links and gitlinks have value 0 in this field.\n \n 32-bit uid\n this is stat(2) data\n@@ -76,11 +78,11 @@ GIT index format\n this is stat(2) data\n \n 32-bit file size\n- This is the on-disk size from stat(2)\n+ This is the on-disk size from stat(2), truncated to 32-bit.\n \n 160-bit SHA-1 for the represented object\n \n- A 16-bit field split into (high to low bits)\n+ A 16-bit 'flags' field split into (high to low bits)\n \n 1-bit assume-valid flag\n \n@@ -88,7 +90,8 @@ GIT index format\n \n 2-bit stage (during merge)\n \n- 12-bit name length if the length is less than 0x0FFF\n+ 12-bit name length if the length is less than 0xFFF; otherwise 0xFFF\n+ is stored in this field.\n \n (Version 3) A 16-bit field, only applicable if the \"extended flag\"\n above is 1, split into (high to low bits).\n@@ -103,63 +106,80 @@ GIT index format\n \n Entry path name (variable length) relative to top level directory\n (without leading slash). '/' is used as path separator. The special\n- paths \".\", \"..\" and \".git\" (without quotes) are disallowed.\n+ path components \".\", \"..\" and \".git\" (without quotes) are disallowed.\n Trailing slash is also disallowed.\n \n The exact encoding is undefined, but the '.' and '/' characters\n- are encoded in 7-bit ASCII and the encoding cannot contain a nul\n- byte. Generally a superset of ASCII.\n+ are encoded in 7-bit ASCII and the encoding cannot contain a NUL\n+ byte (iow, this is a UNIX pathname).\n \n 1-8 nul bytes as necessary to pad the entry to a multiple of eight bytes\n while keeping the name NUL-terminated.\n \n == Extensions\n \n-=== Tree cache\n+=== Cached tree\n \n- Tree cache extension contains pre-computed hashes for trees that can\n+ Cached tree extension contains pre-computed hashes for trees that can\n be derived from the index. It helps speed up tree object generation\n from index for a new commit.\n \n When a path is updated in index, the path must be invalidated and\n removed from tree cache.\n \n- - Extension tag { 'T', 'R', 'E', 'E' }\n+ The signature for this extension is { 'T', 'R', 'E', 'E' }.\n \n- - 32-bit size\n+ A series of entries fill the entire extension; each of which\n+ consists of:\n \n- - A number of entries\n+ - NUL-terminated path component (relative to its parent directory);\n \n- NUL-terminated tree name\n+ - ASCII decimal number of entries in the index that is covered by the\n+ tree this entry represents (entry_count);\n \n- Blank-terminated ASCII decimal number of entries in this tree\n+ - A space (ASCII 32);\n \n- Newline-terminated position of this tree in the parent tree. 0 for\n- the root tree\n+ - ASCII decimal number that represents the number of subtrees this\n+ tree has;\n \n- 160-bit SHA-1 for this tree and it's children\n+ - A newline (ASCII 10); and\n+\n+ - 160-bit object name for the object that would result from writing\n+ this span of index as a tree.\n+ \n+ An entry can be in an invalidated state and is represented by having -1\n+ in the entry_count field.\n+\n+ The entries are written out in the top-down, depth-first order. The\n+ first entry represents the root level of the repository, followed by the\n+ first subtree---let's call this A---of the root level (with its name\n+ relative to the root level), followed by the first subtree of A (with\n+ its name relative to A), ...\n \n === Resolve undo\n \n- A conflict is represented in index as a set of higher stage entries.\n+ A conflict is represented in the index as a set of higher stage entries.\n When a conflict is resolved (e.g. with \"git add path\"), these higher\n- stage entries will be removed and a stage-0 entry with proper\n- resoluton is added.\n+ stage entries will be removed and a stage-0 entry with proper resoluton\n+ is added.\n \n- Resolve undo extension saves these higher stage entries so that\n- conflicts can be recreated (e.g. with \"git checkout -m\"), in case\n- users want to redo a conflict resolution from scratch.\n+ When these higher stage entries are removed, they are saved in the\n+ resolve undo extension, so that conflicts can be recreated (e.g. with\n+ \"git checkout -m\"), in case users want to redo a conflict resolution\n+ from scratch.\n \n- - Extension tag { 'R', 'E', 'U', 'C' }\n+ The signature for this extension is { 'R', 'E', 'U', 'C' }.\n \n- - 32-bit size\n+ A series of entries fill the entire extension; each of which\n+ consists of:\n \n- - A number of conflict entries\n+ - NUL-terminated pathname the entry describes (relative to the root of\n+ the repository, i.e. full pathname);\n \n- NUL-terminated conflict path\n+ - Three NUL-terminated ASCII octal numbers, entry mode of entries in\n+ stage 1 to 3 (a missing stage is represented by \"0\" in this field);\n+ and\n \n- Three NUL-terminated ASCII octal numbers, entry mode of entries in\n- stage 1 to 3.\n+ - At most three 160-bit object names of the entry in stages from 1 to 3\n+ (nothing is written for a missing stage).\n \n- At most three 160-bit SHA-1s of the entry in three stages from 1\n- to 3. SHA-1 is not saved for any stage with entry mode zero.\n"},{"id":"162618","messageId":"AANLkTi=GhdfWCyx7MN3w0ZPhqKHcC1e6RmPeZt67OeqG@mail.gmail.com","threadId":"24933","inReplyTo":"7vpqqaffy2.fsf@alter.siamese.dyndns.org","subject":"Re: [PATCH] doc: technical details about the index file format","fromName":"Nguyen Thai Ngoc Duy","fromEmail":"pclouds@gmail.com","sentAt":"2011-03-02T03:34:23Z","receivedAt":"2011-03-02T03:34:23Z","isPatch":true,"sender":{"key":"pclouds@gmail.com","avatar":"https://avatars.githubusercontent.com/u/720?v=4"},"body":"On Wed, Mar 2, 2011 at 8:51 AM, Junio C Hamano wrote:\n> Nguyen Thai Ngoc Duy writes:\n>\n>> OK another try. I also add more details to tree cache. If somebody\n>> uses this document to create a git-compatible tool, then such a tool\n>> should behave the way git expects it.\n>\n> Thanks.\n>\n> Here is what I scribbled on top of yours (not quite polished).\n>\n>  ...\n\nLooks good. I don't really like ending a sentence with semicolon, but\nthat's just my taste.\n\nI wonder if we should also point to relevant source files, so if this\ndocument becomes out of date, the readers can jump in the source and\nverify themselves (perhaps coming up with patches to this doc)?\n-- \nDuy\n"},{"id":"162622","messageId":"7voc5ucb6b.fsf@alter.siamese.dyndns.org","threadId":"24933","inReplyTo":"AANLkTi=GhdfWCyx7MN3w0ZPhqKHcC1e6RmPeZt67OeqG@mail.gmail.com","subject":"Re: [PATCH] doc: technical details about the index file format","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2011-03-02T06:02:20Z","receivedAt":"2011-03-02T06:02:20Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Nguyen Thai Ngoc Duy writes:\n\n> Looks good. I don't really like ending a sentence with semicolon, but\n> that's just my taste.\n\nI tend to do enumerated list like \"A; B; and C.\" Perhaps just a personal\ntaste.\n\n> I wonder if we should also point to relevant source files, so if this\n> document becomes out of date, the readers can jump in the source and\n> verify themselves (perhaps coming up with patches to this doc)?\n\nI suspect that is a sure way to guarantee the document to go stale.\n\nI didn't like the way I explained the cache-tree entry order. Was it\nunderstandable?\n\nI am wondering if an illustration with an example might be in order. I\nthink anybody halfway intelligent may be able to get a fuzzy idea of what\nis going on by looking at the output from test-dump-cache-tree after\n\"reset --hard && write-tree\" and then by comparing it with the output from\ntest-dump-cache-tree after running \">t/something && git add t/something\"\n(which invalidates the top-level tree and t/ subtree). But a well written\ndocumentation should be able to help clarifying the idea obtainable that\nway. I don't think what I wrote in the previous message is sufficient\neven for that (i.e. comparing the two output would give you better\nexplanation of what is going on than what I wrote--iow, what I wrote may\nnot be very useful for people who are motivated to learn).\n"},{"id":"162630","messageId":"AANLkTimKT2dgPu_0L=K-14e6bBCgPhiQ59AmPWp7y6vg@mail.gmail.com","threadId":"24933","inReplyTo":"7voc5ucb6b.fsf@alter.siamese.dyndns.org","subject":"Re: [PATCH] doc: technical details about the index file format","fromName":"Nguyen Thai Ngoc Duy","fromEmail":"pclouds@gmail.com","sentAt":"2011-03-02T11:43:56Z","receivedAt":"2011-03-02T11:43:56Z","isPatch":true,"sender":{"key":"pclouds@gmail.com","avatar":"https://avatars.githubusercontent.com/u/720?v=4"},"body":"On Wed, Mar 2, 2011 at 1:02 PM, Junio C Hamano wrote:\n>> I wonder if we should also point to relevant source files, so if this\n>> document becomes out of date, the readers can jump in the source and\n>> verify themselves (perhaps coming up with patches to this doc)?\n>\n> I suspect that is a sure way to guarantee the document to go stale.\n\nNo it does not. The point is to make it easier for readers to help\nthemselves when they suspect the document is not entirely correct.\n\n> I didn't like the way I explained the cache-tree entry order.  Was it\n> understandable?\n\nIt is, although I'm wondering if it's just like memcmp() order with\nparent component cut out.\n\n> I am wondering if an illustration with an example might be in order.  I\n> think anybody halfway intelligent may be able to get a fuzzy idea of what\n> is going on by looking at the output from test-dump-cache-tree after\n> \"reset --hard && write-tree\" and then by comparing it with the output from\n> test-dump-cache-tree after running \">t/something && git add t/something\"\n> (which invalidates the top-level tree and t/ subtree).\n\nA short example would be great. test-dump-cache-tree might not be.\nLast time I read its output, I wasn't sure I understood. Maybe because\nI ran it on git.git and did not compare two outputs.\n\n> But a well written\n> documentation should be able to help clarifying the idea obtainable that\n> way.  I don't think what I wrote in the previous message is sufficient\n> even for that (i.e. comparing the two output would give you better\n> explanation of what is going on than what I wrote--iow, what I wrote may\n> not be very useful for people who are motivated to learn).\n-- \nDuy\n"},{"id":"162633","messageId":"1299070416.17973.29.camel@drew-northup.unet.maine.edu","threadId":"24933","inReplyTo":"7voc5ucb6b.fsf@alter.siamese.dyndns.org","subject":"Re: [PATCH] doc: technical details about the index file format","fromName":"Drew Northup","fromEmail":"drew.northup@maine.edu","sentAt":"2011-03-02T12:53:36Z","receivedAt":"2011-03-02T12:53:36Z","isPatch":true,"sender":{"key":"drew.northup@maine.edu","avatar":"https://avatars.githubusercontent.com/u/18331571?v=4"},"body":"\nOn Tue, 2011-03-01 at 22:02 -0800, Junio C Hamano wrote:\n\n> I didn't like the way I explained the cache-tree entry order. Was it\n> understandable?\n> \n> I am wondering if an illustration with an example might be in order. I\n> think anybody halfway intelligent may be able to get a fuzzy idea of what\n> is going on by looking at the output from test-dump-cache-tree after\n> \"reset --hard && write-tree\" and then by comparing it with the output from\n> test-dump-cache-tree after running \">t/something && git add t/something\"\n> (which invalidates the top-level tree and t/ subtree). But a well written\n> documentation should be able to help clarifying the idea obtainable that\n> way. I don't think what I wrote in the previous message is sufficient\n> even for that (i.e. comparing the two output would give you better\n> explanation of what is going on than what I wrote--iow, what I wrote may\n> not be very useful for people who are motivated to learn).\n\nPerhaps I'll be able to put some time into reading the work you guys are\ndoing.... I can definitely put the \"newbie goggles\" on if I do.\n\n-- \n-Drew Northup\n________________________________________________\n\"As opposed to vegetable or mineral error?\"\n-John Pescatore, SANS NewsBites Vol. 12 Num. 59\n"}]}