{"thread":{"id":"15319","subject":"[PATCH] Extend index to save more flags","startedAt":"2008-09-01T11:32:38Z","lastAt":"2008-09-02T07:33:56Z","messageCount":5,"participants":["Nguyễn Thái Ngọc Duy","Johannes Sixt","Junio C Hamano","Nguyen Thai Ngoc Duy"],"isPatch":true,"patchVersion":1,"patchTotal":null},"messages":[{"id":"89407","messageId":"1220268758-29550-1-git-send-email-pclouds@gmail.com","threadId":"15319","inReplyTo":null,"subject":"[PATCH] Extend index to save more flags","fromName":"Nguyễn Thái Ngọc Duy","fromEmail":"pclouds@gmail.com","sentAt":"2008-09-01T11:32:38Z","receivedAt":"2008-09-01T11:32:38Z","isPatch":true,"sender":{"key":"pclouds@gmail.com","avatar":"https://avatars.githubusercontent.com/u/720?v=4"},"body":"The on-disk format of index only saves 16 bit flags, nearly all have\nbeen used. The last bit (CE_EXTENDED) is used to for future extension.\n\nThis patch extends index entry format to save more flags in future.\nThe new entry format will be used when CE_EXTENDED bit is 1.\n\nBecause older implementation may not understand CE_EXTENDED bit and\nmisread the new format, if there is any extended entry in index, index\nheader version will turn 3, which makes it incompatible for older git.\nIf there is none, header version will return to 2 again.\n\nSigned-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>\n---\n > Ok.  We would need to use an extra bit for this.\n >\n > The bit 0x4000 is the last one available, so we would want to use it as\n > \"this index entry uses more bits than the traditional format\" bit, and\n > define a backward incompatible on-disk index entry format to actually\n > record CE_NO_CHECKOUT and other flags we will invent in the future.\n >\n > Perhaps ondisk_cache_entry structure will have an extra \"unsigned int\n > flags2\" after \"flags\" when that bit is on, and we can have 31 more bits in\n > flags2, with the highest bit of flags2 signalling the presense of flags3\n > word in the future, or something like that.\n\n I was about to add a new field in cache_entry in order to have 31 new bits,\n but that led to initialization nightmare because there is no common routine\n to initialize cache_entry, the new field must be initialized too many places.\n\n So new ondisk bits will stay in cache_entry.ce_flags. That means 9 bits left\n for both new on-disk bits and in-memory ones. Narrow/sparse checkout will\n use 2. The remaining 7 bits will probably be enough for coming years?\n\n cache.h      |   34 +++++++++++++++++++++++++++++++---\n read-cache.c |   49 +++++++++++++++++++++++++++++++++++++++----------\n 2 files changed, 70 insertions(+), 13 deletions(-)\n\ndiff --git a/cache.h b/cache.h\nindex f725783..120ed07 100644\n--- a/cache.h\n+++ b/cache.h\n@@ -109,6 +109,26 @@ struct ondisk_cache_entry {\n \tchar name[FLEX_ARRAY]; /* more */\n };\n \n+/*\n+ * This struct is used when CE_EXTENDED bit is 1\n+ * The struct must match ondisk_cache_entry exactly from\n+ * ctime till flags\n+ */\n+struct ondisk_cache_entry_extended {\n+\tstruct cache_time ctime;\n+\tstruct cache_time mtime;\n+\tunsigned int dev;\n+\tunsigned int ino;\n+\tunsigned int mode;\n+\tunsigned int uid;\n+\tunsigned int gid;\n+\tunsigned int size;\n+\tunsigned char sha1[20];\n+\tunsigned short flags;\n+\tunsigned short flags2;\n+\tchar name[FLEX_ARRAY]; /* more */\n+};\n+\n struct cache_entry {\n \tunsigned int ce_ctime;\n \tunsigned int ce_mtime;\n@@ -139,6 +159,10 @@ struct cache_entry {\n #define CE_HASHED    (0x100000)\n #define CE_UNHASHED  (0x200000)\n \n+/* Extended flags in ondisk_cache_entry_extended */\n+#define CE_EXTENDED2 (0x80000000)\n+#define CE_EXTENDED_MASK (CE_EXTENDED2)\n+\n /*\n  * Copy the sha1 and stat state of a cache entry from one to\n  * another. But we never change the name, or the hash state!\n@@ -171,7 +195,9 @@ static inline size_t ce_namelen(const struct cache_entry *ce)\n }\n \n #define ce_size(ce) cache_entry_size(ce_namelen(ce))\n-#define ondisk_ce_size(ce) ondisk_cache_entry_size(ce_namelen(ce))\n+#define ondisk_ce_size(ce) (((ce)->ce_flags & CE_EXTENDED) ? \\\n+\t\t\t    ondisk_cache_entry_extended_size(ce_namelen(ce)) : \\\n+\t\t\t    ondisk_cache_entry_size(ce_namelen(ce)))\n #define ce_stage(ce) ((CE_STAGEMASK & (ce)->ce_flags) >> CE_STAGESHIFT)\n #define ce_uptodate(ce) ((ce)->ce_flags & CE_UPTODATE)\n #define ce_mark_uptodate(ce) ((ce)->ce_flags |= CE_UPTODATE)\n@@ -214,8 +240,10 @@ static inline int ce_to_dtype(const struct cache_entry *ce)\n \t(S_ISREG(mode) ? (S_IFREG | ce_permissions(mode)) : \\\n \tS_ISLNK(mode) ? S_IFLNK : S_ISDIR(mode) ? S_IFDIR : S_IFGITLINK)\n \n-#define cache_entry_size(len) ((offsetof(struct cache_entry,name) + (len) + 8) & ~7)\n-#define ondisk_cache_entry_size(len) ((offsetof(struct ondisk_cache_entry,name) + (len) + 8) & ~7)\n+#define flexible_size(STRUCT,len) ((offsetof(struct STRUCT,name) + (len) + 8) & ~7)\n+#define cache_entry_size(len) flexible_size(cache_entry,len)\n+#define ondisk_cache_entry_size(len) flexible_size(ondisk_cache_entry,len)\n+#define ondisk_cache_entry_extended_size(len) flexible_size(ondisk_cache_entry_extended,len)\n \n struct index_state {\n \tstruct cache_entry **cache;\ndiff --git a/read-cache.c b/read-cache.c\nindex c5a8659..a5bfa8e 100644\n--- a/read-cache.c\n+++ b/read-cache.c\n@@ -1096,7 +1096,7 @@ static int verify_hdr(struct cache_header *hdr, unsigned long size)\n \n \tif (hdr->hdr_signature != htonl(CACHE_SIGNATURE))\n \t\treturn error(\"bad signature\");\n-\tif (hdr->hdr_version != htonl(2))\n+\tif (hdr->hdr_version != htonl(2) && hdr->hdr_version != htonl(3))\n \t\treturn error(\"bad index version\");\n \tSHA1_Init(&c);\n \tSHA1_Update(&c, hdr, size - 20);\n@@ -1131,6 +1131,7 @@ int read_index(struct index_state *istate)\n static void convert_from_disk(struct ondisk_cache_entry *ondisk, struct cache_entry *ce)\n {\n \tsize_t len;\n+\tconst char *name;\n \n \tce->ce_ctime = ntohl(ondisk->ctime.sec);\n \tce->ce_mtime = ntohl(ondisk->mtime.sec);\n@@ -1143,19 +1144,29 @@ static void convert_from_disk(struct ondisk_cache_entry *ondisk, struct cache_en\n \t/* On-disk flags are just 16 bits */\n \tce->ce_flags = ntohs(ondisk->flags);\n \n-\t/* For future extension: we do not understand this entry yet */\n-\tif (ce->ce_flags & CE_EXTENDED)\n-\t\tdie(\"Unknown index entry format\");\n \thashcpy(ce->sha1, ondisk->sha1);\n \n \tlen = ce->ce_flags & CE_NAMEMASK;\n+\n+\tif (ce->ce_flags & CE_EXTENDED) {\n+\t\tstruct ondisk_cache_entry_extended *ondisk2;\n+\t\tondisk2 = (struct ondisk_cache_entry_extended *)ondisk;\n+\t\tce->ce_flags |= (ntohs(ondisk2->flags2) << 16) & CE_EXTENDED_MASK;\n+\t\t/* For future extension: we do not understand the last bit yet */\n+\t\tif (ce->ce_flags & CE_EXTENDED2)\n+\t\t\tdie(\"Unknown index entry format\");\n+\t\tname = ondisk2->name;\n+\t}\n+\telse\n+\t\tname = ondisk->name;\n+\n \tif (len == CE_NAMEMASK)\n-\t\tlen = strlen(ondisk->name);\n+\t\tlen = strlen(name);\n \t/*\n \t * NEEDSWORK: If the original index is crafted, this copy could\n \t * go unchecked.\n \t */\n-\tmemcpy(ce->name, ondisk->name, len + 1);\n+\tmemcpy(ce->name, name, len + 1);\n }\n \n static inline size_t estimate_cache_size(size_t ondisk_size, unsigned int entries)\n@@ -1415,6 +1426,7 @@ static int ce_write_entry(SHA_CTX *c, int fd, struct cache_entry *ce)\n {\n \tint size = ondisk_ce_size(ce);\n \tstruct ondisk_cache_entry *ondisk = xcalloc(1, size);\n+\tchar *name;\n \n \tondisk->ctime.sec = htonl(ce->ce_ctime);\n \tondisk->ctime.nsec = 0;\n@@ -1428,7 +1440,15 @@ static int ce_write_entry(SHA_CTX *c, int fd, struct cache_entry *ce)\n \tondisk->size = htonl(ce->ce_size);\n \thashcpy(ondisk->sha1, ce->sha1);\n \tondisk->flags = htons(ce->ce_flags);\n-\tmemcpy(ondisk->name, ce->name, ce_namelen(ce));\n+\tif (ce->ce_flags & CE_EXTENDED) {\n+\t\tstruct ondisk_cache_entry_extended *ondisk2;\n+\t\tondisk2 = (struct ondisk_cache_entry_extended *)ondisk;\n+\t\tondisk2->flags2 = htons((ce->ce_flags & CE_EXTENDED_MASK) >> 16);\n+\t\tname = ondisk2->name;\n+\t}\n+\telse\n+\t\tname = ondisk->name;\n+\tmemcpy(name, ce->name, ce_namelen(ce));\n \n \treturn ce_write(c, fd, ondisk, size);\n }\n@@ -1437,16 +1457,25 @@ int write_index(const struct index_state *istate, int newfd)\n {\n \tSHA_CTX c;\n \tstruct cache_header hdr;\n-\tint i, err, removed;\n+\tint i, err, removed, extended;\n \tstruct cache_entry **cache = istate->cache;\n \tint entries = istate->cache_nr;\n \n-\tfor (i = removed = 0; i < entries; i++)\n+\tfor (i = removed = extended = 0; i < entries; i++) {\n \t\tif (cache[i]->ce_flags & CE_REMOVE)\n \t\t\tremoved++;\n \n+\t\t/* reduce extended entries if possible */\n+\t\tcache[i]->ce_flags &= ~CE_EXTENDED;\n+\t\tif (cache[i]->ce_flags & CE_EXTENDED_MASK) {\n+\t\t\textended++;\n+\t\t\tcache[i]->ce_flags |= CE_EXTENDED;\n+\t\t}\n+\t}\n+\n \thdr.hdr_signature = htonl(CACHE_SIGNATURE);\n-\thdr.hdr_version = htonl(2);\n+\t/* for extended format, increase version so older git won't try to read it */\n+\thdr.hdr_version = htonl(extended ? 3 : 2);\n \thdr.hdr_entries = htonl(entries - removed);\n \n \tSHA1_Init(&c);\n-- \n1.6.0.96.g2fad1.dirty\n"},{"id":"89413","messageId":"48BBE881.1050206@viscovery.net","threadId":"15319","inReplyTo":"1220268758-29550-1-git-send-email-pclouds@gmail.com","subject":"Re: [PATCH] Extend index to save more flags","fromName":"Johannes Sixt","fromEmail":"j.sixt@viscovery.net","sentAt":"2008-09-01T13:05:05Z","receivedAt":"2008-09-01T13:05:05Z","isPatch":true,"sender":{"key":"j6t@kdbg.org","avatar":"https://avatars.githubusercontent.com/u/14810926?v=4"},"body":"Nguyễn Thái Ngọc Duy schrieb:\n> +/* Extended flags in ondisk_cache_entry_extended */\n> +#define CE_EXTENDED2 (0x80000000)\n> +#define CE_EXTENDED_MASK (CE_EXTENDED2)\n\nWouldn't it be better, from a maintainance point of view, to have\n\n#define CE_EXTENDED_MASK 0xffff0000\n\nright from the beginning?\n\n> @@ -1143,19 +1144,29 @@ static void convert_from_disk(struct ondisk_cache_entry *ondisk, struct cache_en\n>  \t/* On-disk flags are just 16 bits */\n>  \tce->ce_flags = ntohs(ondisk->flags);\n>  \n> -\t/* For future extension: we do not understand this entry yet */\n> -\tif (ce->ce_flags & CE_EXTENDED)\n> -\t\tdie(\"Unknown index entry format\");\n>  \thashcpy(ce->sha1, ondisk->sha1);\n>  \n>  \tlen = ce->ce_flags & CE_NAMEMASK;\n> +\n> +\tif (ce->ce_flags & CE_EXTENDED) {\n> +\t\tstruct ondisk_cache_entry_extended *ondisk2;\n> +\t\tondisk2 = (struct ondisk_cache_entry_extended *)ondisk;\n> +\t\tce->ce_flags |= (ntohs(ondisk2->flags2) << 16) & CE_EXTENDED_MASK;\n> +\t\t/* For future extension: we do not understand the last bit yet */\n> +\t\tif (ce->ce_flags & CE_EXTENDED2)\n> +\t\t\tdie(\"Unknown index entry format\");\n\nAt this point, we do not understand the CE_EXTENDED2 flag; but we do not\nunderstand any of the other 15 extended flags, either. So, you should\nerror out if *any* of them is non-zero.\n\n> +\t\tname = ondisk2->name;\n> +\t}\n> +\telse\n> +\t\tname = ondisk->name;\n\n-- Hannes\n"},{"id":"89419","messageId":"1220278585-26969-1-git-send-email-pclouds@gmail.com","threadId":"15319","inReplyTo":"48BBE881.1050206@viscovery.net","subject":"[PATCH] Extend index to save more flags","fromName":"Nguyễn Thái Ngọc Duy","fromEmail":"pclouds@gmail.com","sentAt":"2008-09-01T14:16:25Z","receivedAt":"2008-09-01T14:16:25Z","isPatch":true,"sender":{"key":"pclouds@gmail.com","avatar":"https://avatars.githubusercontent.com/u/720?v=4"},"body":"The on-disk format of index only saves 16 bit flags, nearly all have\nbeen used. The last bit (CE_EXTENDED) is used to for future extension.\n\nThis patch extends index entry format to save more flags in future.\nThe new entry format will be used when CE_EXTENDED bit is 1.\n\nBecause older implementation may not understand CE_EXTENDED bit and\nmisread the new format, if there is any extended entry in index, index\nheader version will turn 3, which makes it incompatible for older git.\nIf there is none, header version will return to 2 again.\n\nSigned-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>\n---\n On 9/1/08, Johannes Sixt <j.sixt@viscovery.net> wrote:\n >  > +/* Extended flags in ondisk_cache_entry_extended */\n >  > +#define CE_EXTENDED2 (0x80000000)\n >  > +#define CE_EXTENDED_MASK (CE_EXTENDED2)\n >  \n >  \n > Wouldn't it be better, from a maintainance point of view, to have\n >  \n >  #define CE_EXTENDED_MASK 0xffff0000\n >  \n >  right from the beginning?\n\n Actually it's not 16 bits. We have 16 bits on disk, but will not use\n all of them until cache_entry is adjusted. But I think extending the mask\n makes sense.\n\n >  > +             /* For future extension: we do not understand the last bit yet */\n >  > +             if (ce->ce_flags & CE_EXTENDED2)\n >  > +                     die(\"Unknown index entry format\");\n >  \n >  \n > At this point, we do not understand the CE_EXTENDED2 flag; but we do not\n >  understand any of the other 15 extended flags, either. So, you should\n >  error out if *any* of them is non-zero.\n \n Fixed with CE_KNOWN_EXTENDED_FLAGS\n\n cache.h      |   54 ++++++++++++++++++++++++++++++++++++++++++++++++++----\n read-cache.c |   49 +++++++++++++++++++++++++++++++++++++++----------\n 2 files changed, 89 insertions(+), 14 deletions(-)\n\ndiff --git a/cache.h b/cache.h\nindex f725783..f8578d1 100644\n--- a/cache.h\n+++ b/cache.h\n@@ -109,6 +109,26 @@ struct ondisk_cache_entry {\n \tchar name[FLEX_ARRAY]; /* more */\n };\n \n+/*\n+ * This struct is used when CE_EXTENDED bit is 1\n+ * The struct must match ondisk_cache_entry exactly from\n+ * ctime till flags\n+ */\n+struct ondisk_cache_entry_extended {\n+\tstruct cache_time ctime;\n+\tstruct cache_time mtime;\n+\tunsigned int dev;\n+\tunsigned int ino;\n+\tunsigned int mode;\n+\tunsigned int uid;\n+\tunsigned int gid;\n+\tunsigned int size;\n+\tunsigned char sha1[20];\n+\tunsigned short flags;\n+\tunsigned short flags2;\n+\tchar name[FLEX_ARRAY]; /* more */\n+};\n+\n struct cache_entry {\n \tunsigned int ce_ctime;\n \tunsigned int ce_mtime;\n@@ -130,7 +150,15 @@ struct cache_entry {\n #define CE_VALID     (0x8000)\n #define CE_STAGESHIFT 12\n \n-/* In-memory only */\n+/*\n+ * Range 0xFFFF0000 in ce_flags is divided into\n+ * two parts: in-memory flags and on-disk ones.\n+ * Flags in CE_EXTENDED_MASK will get saved on-disk\n+ * so if you want an in-memory flag, remove it\n+ * from CE_EXTENDED_MASK.\n+ *\n+ * In-memory only flags\n+ */\n #define CE_UPDATE    (0x10000)\n #define CE_REMOVE    (0x20000)\n #define CE_UPTODATE  (0x40000)\n@@ -140,6 +168,20 @@ struct cache_entry {\n #define CE_UNHASHED  (0x200000)\n \n /*\n+ * We'll be careful on new extended on-disk flags,\n+ * Git will refuse to read unknown flags. When you\n+ * add a new flag, add it to CE_KNOWN_EXTENDED_FLAGS.\n+ * CE_EXTENDED2 is for future extension, so it will\n+ * remain unknown until future is set.\n+ *\n+ * Extended on-disk flags\n+ */\n+#define CE_EXTENDED2 (0x80000000)\n+\n+#define CE_KNOWN_EXTENDED_FLAGS (0)\n+#define CE_EXTENDED_MASK (0xFFC00000)\n+\n+/*\n  * Copy the sha1 and stat state of a cache entry from one to\n  * another. But we never change the name, or the hash state!\n  */\n@@ -171,7 +213,9 @@ static inline size_t ce_namelen(const struct cache_entry *ce)\n }\n \n #define ce_size(ce) cache_entry_size(ce_namelen(ce))\n-#define ondisk_ce_size(ce) ondisk_cache_entry_size(ce_namelen(ce))\n+#define ondisk_ce_size(ce) (((ce)->ce_flags & CE_EXTENDED) ? \\\n+\t\t\t    ondisk_cache_entry_extended_size(ce_namelen(ce)) : \\\n+\t\t\t    ondisk_cache_entry_size(ce_namelen(ce)))\n #define ce_stage(ce) ((CE_STAGEMASK & (ce)->ce_flags) >> CE_STAGESHIFT)\n #define ce_uptodate(ce) ((ce)->ce_flags & CE_UPTODATE)\n #define ce_mark_uptodate(ce) ((ce)->ce_flags |= CE_UPTODATE)\n@@ -214,8 +258,10 @@ static inline int ce_to_dtype(const struct cache_entry *ce)\n \t(S_ISREG(mode) ? (S_IFREG | ce_permissions(mode)) : \\\n \tS_ISLNK(mode) ? S_IFLNK : S_ISDIR(mode) ? S_IFDIR : S_IFGITLINK)\n \n-#define cache_entry_size(len) ((offsetof(struct cache_entry,name) + (len) + 8) & ~7)\n-#define ondisk_cache_entry_size(len) ((offsetof(struct ondisk_cache_entry,name) + (len) + 8) & ~7)\n+#define flexible_size(STRUCT,len) ((offsetof(struct STRUCT,name) + (len) + 8) & ~7)\n+#define cache_entry_size(len) flexible_size(cache_entry,len)\n+#define ondisk_cache_entry_size(len) flexible_size(ondisk_cache_entry,len)\n+#define ondisk_cache_entry_extended_size(len) flexible_size(ondisk_cache_entry_extended,len)\n \n struct index_state {\n \tstruct cache_entry **cache;\ndiff --git a/read-cache.c b/read-cache.c\nindex c5a8659..3bba6f7 100644\n--- a/read-cache.c\n+++ b/read-cache.c\n@@ -1096,7 +1096,7 @@ static int verify_hdr(struct cache_header *hdr, unsigned long size)\n \n \tif (hdr->hdr_signature != htonl(CACHE_SIGNATURE))\n \t\treturn error(\"bad signature\");\n-\tif (hdr->hdr_version != htonl(2))\n+\tif (hdr->hdr_version != htonl(2) && hdr->hdr_version != htonl(3))\n \t\treturn error(\"bad index version\");\n \tSHA1_Init(&c);\n \tSHA1_Update(&c, hdr, size - 20);\n@@ -1131,6 +1131,7 @@ int read_index(struct index_state *istate)\n static void convert_from_disk(struct ondisk_cache_entry *ondisk, struct cache_entry *ce)\n {\n \tsize_t len;\n+\tconst char *name;\n \n \tce->ce_ctime = ntohl(ondisk->ctime.sec);\n \tce->ce_mtime = ntohl(ondisk->mtime.sec);\n@@ -1143,19 +1144,29 @@ static void convert_from_disk(struct ondisk_cache_entry *ondisk, struct cache_en\n \t/* On-disk flags are just 16 bits */\n \tce->ce_flags = ntohs(ondisk->flags);\n \n-\t/* For future extension: we do not understand this entry yet */\n-\tif (ce->ce_flags & CE_EXTENDED)\n-\t\tdie(\"Unknown index entry format\");\n \thashcpy(ce->sha1, ondisk->sha1);\n \n \tlen = ce->ce_flags & CE_NAMEMASK;\n+\n+\tif (ce->ce_flags & CE_EXTENDED) {\n+\t\tstruct ondisk_cache_entry_extended *ondisk2;\n+\t\tondisk2 = (struct ondisk_cache_entry_extended *)ondisk;\n+\t\tce->ce_flags |= (ntohs(ondisk2->flags2) << 16) & CE_EXTENDED_MASK;\n+\t\t/* We do not understand any bit out of CE_KNOWN_EXTENDED_FLAGS */\n+\t\tif (ce->ce_flags & (CE_EXTENDED_MASK & ~CE_KNOWN_EXTENDED_FLAGS))\n+\t\t\tdie(\"Unknown index entry format\");\n+\t\tname = ondisk2->name;\n+\t}\n+\telse\n+\t\tname = ondisk->name;\n+\n \tif (len == CE_NAMEMASK)\n-\t\tlen = strlen(ondisk->name);\n+\t\tlen = strlen(name);\n \t/*\n \t * NEEDSWORK: If the original index is crafted, this copy could\n \t * go unchecked.\n \t */\n-\tmemcpy(ce->name, ondisk->name, len + 1);\n+\tmemcpy(ce->name, name, len + 1);\n }\n \n static inline size_t estimate_cache_size(size_t ondisk_size, unsigned int entries)\n@@ -1415,6 +1426,7 @@ static int ce_write_entry(SHA_CTX *c, int fd, struct cache_entry *ce)\n {\n \tint size = ondisk_ce_size(ce);\n \tstruct ondisk_cache_entry *ondisk = xcalloc(1, size);\n+\tchar *name;\n \n \tondisk->ctime.sec = htonl(ce->ce_ctime);\n \tondisk->ctime.nsec = 0;\n@@ -1428,7 +1440,15 @@ static int ce_write_entry(SHA_CTX *c, int fd, struct cache_entry *ce)\n \tondisk->size = htonl(ce->ce_size);\n \thashcpy(ondisk->sha1, ce->sha1);\n \tondisk->flags = htons(ce->ce_flags);\n-\tmemcpy(ondisk->name, ce->name, ce_namelen(ce));\n+\tif (ce->ce_flags & CE_EXTENDED) {\n+\t\tstruct ondisk_cache_entry_extended *ondisk2;\n+\t\tondisk2 = (struct ondisk_cache_entry_extended *)ondisk;\n+\t\tondisk2->flags2 = htons((ce->ce_flags & CE_EXTENDED_MASK) >> 16);\n+\t\tname = ondisk2->name;\n+\t}\n+\telse\n+\t\tname = ondisk->name;\n+\tmemcpy(name, ce->name, ce_namelen(ce));\n \n \treturn ce_write(c, fd, ondisk, size);\n }\n@@ -1437,16 +1457,25 @@ int write_index(const struct index_state *istate, int newfd)\n {\n \tSHA_CTX c;\n \tstruct cache_header hdr;\n-\tint i, err, removed;\n+\tint i, err, removed, extended;\n \tstruct cache_entry **cache = istate->cache;\n \tint entries = istate->cache_nr;\n \n-\tfor (i = removed = 0; i < entries; i++)\n+\tfor (i = removed = extended = 0; i < entries; i++) {\n \t\tif (cache[i]->ce_flags & CE_REMOVE)\n \t\t\tremoved++;\n \n+\t\t/* reduce extended entries if possible */\n+\t\tcache[i]->ce_flags &= ~CE_EXTENDED;\n+\t\tif (cache[i]->ce_flags & CE_EXTENDED_MASK) {\n+\t\t\textended++;\n+\t\t\tcache[i]->ce_flags |= CE_EXTENDED;\n+\t\t}\n+\t}\n+\n \thdr.hdr_signature = htonl(CACHE_SIGNATURE);\n-\thdr.hdr_version = htonl(2);\n+\t/* for extended format, increase version so older git won't try to read it */\n+\thdr.hdr_version = htonl(extended ? 3 : 2);\n \thdr.hdr_entries = htonl(entries - removed);\n \n \tSHA1_Init(&c);\n-- \n1.6.0.96.g2fad1.dirty\n"},{"id":"89490","messageId":"7vhc8zm2ka.fsf@gitster.siamese.dyndns.org","threadId":"15319","inReplyTo":"1220278585-26969-1-git-send-email-pclouds@gmail.com","subject":"Re: [PATCH] Extend index to save more flags","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2008-09-02T07:25:57Z","receivedAt":"2008-09-02T07:25:57Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Nguyễn Thái Ngọc Duy  <pclouds@gmail.com> writes:\n\n> The on-disk format of index only saves 16 bit flags, nearly all have\n> been used. The last bit (CE_EXTENDED) is used to for future extension.\n>\n> This patch extends index entry format to save more flags in future.\n> The new entry format will be used when CE_EXTENDED bit is 1.\n>\n> Because older implementation may not understand CE_EXTENDED bit and\n> misread the new format, if there is any extended entry in index, index\n> header version will turn 3, which makes it incompatible for older git.\n> If there is none, header version will return to 2 again.\n\nI think this is a good change.\n\n> diff --git a/cache.h b/cache.h\n> index f725783..f8578d1 100644\n> --- a/cache.h\n> +++ b/cache.h\n> @@ -109,6 +109,26 @@ struct ondisk_cache_entry {\n>  \tchar name[FLEX_ARRAY]; /* more */\n>  };\n>  \n> +/*\n> + * This struct is used when CE_EXTENDED bit is 1\n> + * The struct must match ondisk_cache_entry exactly from\n> + * ctime till flags\n> + */\n> +struct ondisk_cache_entry_extended {\n> +\tstruct cache_time ctime;\n> +\tstruct cache_time mtime;\n> +\tunsigned int dev;\n> +\tunsigned int ino;\n> +\tunsigned int mode;\n> +\tunsigned int uid;\n> +\tunsigned int gid;\n> +\tunsigned int size;\n> +\tunsigned char sha1[20];\n> +\tunsigned short flags;\n> +\tunsigned short flags2;\n> +\tchar name[FLEX_ARRAY]; /* more */\n> +};\n> +\n\nWe should change these to more explicitly sized uint32_t both in the\noriginal and this extended structure, but after this patch (or later\nvariant of it) lands in the tree.\n"},{"id":"89493","messageId":"fcaeb9bf0809020033i6ed8c268gac3cedcbebe57e26@mail.gmail.com","threadId":"15319","inReplyTo":"7vhc8zm2ka.fsf@gitster.siamese.dyndns.org","subject":"Re: [PATCH] Extend index to save more flags","fromName":"Nguyen Thai Ngoc Duy","fromEmail":"pclouds@gmail.com","sentAt":"2008-09-02T07:33:56Z","receivedAt":"2008-09-02T07:33:56Z","isPatch":true,"sender":{"key":"pclouds@gmail.com","avatar":"https://avatars.githubusercontent.com/u/720?v=4"},"body":"On 9/2/08, Junio C Hamano <gitster@pobox.com> wrote:\n> Nguyễn Thái Ngọc Duy  <pclouds@gmail.com> writes:\n>\n>  > The on-disk format of index only saves 16 bit flags, nearly all have\n>  > been used. The last bit (CE_EXTENDED) is used to for future extension.\n>  >\n>  > This patch extends index entry format to save more flags in future.\n>  > The new entry format will be used when CE_EXTENDED bit is 1.\n>  >\n>  > Because older implementation may not understand CE_EXTENDED bit and\n>  > misread the new format, if there is any extended entry in index, index\n>  > header version will turn 3, which makes it incompatible for older git.\n>  > If there is none, header version will return to 2 again.\n>\n>\n> I think this is a good change.\n\nJust don't apply it now. There is a bug in die() code for unknown\nflags. I'll resend with narrow checkout series, which will also be a\ntest case for these change.\n-- \nDuy\n"}]}