{"thread":{"id":"6416","subject":"[PATCH] Document pack .idx file format upgrade strategy.","startedAt":"2007-01-18T01:43:57Z","lastAt":"2007-01-18T01:43:57Z","messageCount":1,"participants":["Shawn O. Pearce"],"isPatch":true,"patchVersion":1,"patchTotal":null},"messages":[{"id":"31938","messageId":"20070118014356.GA23390@spearce.org","threadId":"6416","inReplyTo":null,"subject":"[PATCH] Document pack .idx file format upgrade strategy.","fromName":"Shawn O. Pearce","fromEmail":"spearce@spearce.org","sentAt":"2007-01-18T01:43:57Z","receivedAt":"2007-01-18T01:43:57Z","isPatch":true,"sender":{"key":"spearce@spearce.org","avatar":"https://avatars.githubusercontent.com/u/34844?v=4"},"body":"Way back when Junio developed the 64 bit index topic he came up\nwith a means of changing the .idx file format so that older Git\nclients would recognize that they don't understand the file and\nrefuse to read it, while newer clients could tell the difference\nbetween the old-style and new-style .idx files.  Unfortunately\nthis wasn't recorded anywhere.\n\nThis change documents how we might go about changing the .idx\nfile format by using a special signature in the first four bytes.\nCredit (and possible blame) goes completely to Junio for thinking\nup this technique.\n\nThe change also modifies the error message of the current Git code\nso that users get a recommendation to upgrade their Git software\nshould this version or later encounter a new-style .idx which it\ncannot process.  We already do this for the .pack files, but since\nwe usually process the .idx files first its important that these\nfiles are recognized and encourage an upgrade.\n\nSigned-off-by: Shawn O. Pearce <spearce@spearce.org>\n---\n pack.h      |   28 ++++++++++++++++++++++++++++\n sha1_file.c |   17 ++++++++++++++---\n 2 files changed, 42 insertions(+), 3 deletions(-)\n\ndiff --git a/pack.h b/pack.h\nindex 4814800..790ff4a 100644\n--- a/pack.h\n+++ b/pack.h\n@@ -15,5 +15,33 @@ struct pack_header {\n \tunsigned int hdr_entries;\n };\n \n+/*\n+ * Packed object index header\n+ *\n+ * struct pack_idx_header {\n+ * \tuint32_t idx_signature;\n+ *\tuint32_t idx_version;\n+ * };\n+ *\n+ * Note: this header isn't active yet.  In future versions of git\n+ * we may change the index file format.  At that time we would start\n+ * the first four bytes of the new index format with this signature,\n+ * as all older git binaries would find this value illegal and abort\n+ * reading the file.\n+ *\n+ * This is the case because the number of objects in a packfile\n+ * cannot exceed 1,431,660,000 as every object would need at least\n+ * 3 bytes of data and the overall packfile cannot exceed 4 GiB due\n+ * to the 32 bit offsets used by the index.  Clearly the signature\n+ * exceeds this maximum.\n+ *\n+ * Very old git binaries will also compare the first 4 bytes to the\n+ * next 4 bytes in the index and abort with a \"non-monotonic index\"\n+ * error if the second 4 byte word is smaller than the first 4\n+ * byte word.  This would be true in the proposed future index\n+ * format as idx_signature would be greater than idx_version.\n+ */\n+#define PACK_IDX_SIGNATURE 0xff744f63\t/* \"\\377tOc\" */\n+\n extern int verify_pack(struct packed_git *, int);\n #endif\ndiff --git a/sha1_file.c b/sha1_file.c\nindex 1b1c0f7..0b70545 100644\n--- a/sha1_file.c\n+++ b/sha1_file.c\n@@ -456,12 +456,23 @@ static int check_packed_git_idx(const char *path, unsigned long *idx_size_,\n \n \t/* check index map */\n \tif (idx_size < 4*256 + 20 + 20)\n-\t\treturn error(\"index file too small\");\n+\t\treturn error(\"index file %s is too small\", path);\n+\n+\t/* a future index format would start with this, as older git\n+\t * binaries would fail the non-monotonic index check below.\n+\t * give a nicer warning to the user if we can.\n+\t */\n+\tif (index[0] == htonl(PACK_IDX_SIGNATURE))\n+\t\treturn error(\"index file %s is a newer version\"\n+\t\t\t\" and is not supported by this binary\"\n+\t\t\t\" (try upgrading GIT to a newer version)\",\n+\t\t\tpath);\n+\n \tnr = 0;\n \tfor (i = 0; i < 256; i++) {\n \t\tunsigned int n = ntohl(index[i]);\n \t\tif (n < nr)\n-\t\t\treturn error(\"non-monotonic index\");\n+\t\t\treturn error(\"non-monotonic index %s\", path);\n \t\tnr = n;\n \t}\n \n@@ -473,7 +484,7 @@ static int check_packed_git_idx(const char *path, unsigned long *idx_size_,\n \t *  - 20-byte SHA1 file checksum\n \t */\n \tif (idx_size != 4*256 + nr * 24 + 20 + 20)\n-\t\treturn error(\"wrong index file size\");\n+\t\treturn error(\"wrong index file size in %s\", path);\n \n \treturn 0;\n }\n-- \n1.5.0.rc1.g556b\n"}]}