{"thread":{"id":"35023","subject":"[PATCH 00/10] pack v4 UI support","startedAt":"2013-09-26T02:26:39Z","lastAt":"2013-10-17T17:26:21Z","messageCount":20,"participants":["Nguyễn Thái Ngọc Duy","Nicolas Pitre","Duy Nguyen","Stefan Beller","Felipe Contreras","Junio C Hamano"],"isPatch":true,"patchVersion":1,"patchTotal":10},"messages":[{"id":"228245","messageId":"1380162409-18224-1-git-send-email-pclouds@gmail.com","threadId":"35023","inReplyTo":null,"subject":"[PATCH 00/10] pack v4 UI support","fromName":"Nguyễn Thái Ngọc Duy","fromEmail":"pclouds@gmail.com","sentAt":"2013-09-26T02:26:39Z","receivedAt":"2013-09-26T02:26:39Z","isPatch":true,"sender":{"key":"pclouds@gmail.com","avatar":"https://avatars.githubusercontent.com/u/720?v=4"},"body":"This adds\n\n - clone, fetch and repack learn --pack-version (backends also learn\n new options, check out the patches for details)\n\n - core.preferredPackVersion to specify the default pack version for\n fetch, clone, repack and the receiver of push.\n \n This config is supposed to be used by porcelain commands only (with\n the exception of receive-pack). All other plumbing will default to\n pack v2 forever.\n\n - git and remote helper protocols learn about packv4 capabilities\n\nI think the UI changes for pack v4 are done with this series. Well,\ngit-bundle needs --pack-version too but it's not part of core\nuser interaction. The remaining work for v4 would be optimization\n(that includes multi-base tree support) and more v4 tests. Let me know\nif I missed anything here.\n\nI'm aware that repack is being rewritten in C, so it'll cause\nconflicts when np/pack-v4 is re-pulled to 'pu' again. I suggest that\nthe new tests in t7700 are marked failed at the merge. We can add\n--pack-version to the C-based repack and enable the tests later.\n\nAbout the \"packv4\" capability in git protocol. I considered a more\ngeneric cap \"packver=ver[,ver[,ver...]]\" that represents all supported\nversions, with the order of preference from the first ver (most preferred)\nto the last (least preferred). But it adds more parsing code\nand frankly I don't think we'll have pack v5 in the next five years.\nSo I dropped it.\n\nMulti-base tree support is not part of \"packv4\" capability. Let's see\nif such support comes before the series is merged to master. If so we\ncan drop that line from protocol-capabilities.txt. Otherwise a new\ncapability can be added for multi-base trees.\n\nAnother capability could be added for sending the actual number of\nobjects in a thin pack for more accurate display in index-pack. Low\npriority in my opinion.\n\nThere's also the pack conversion issue. Suppose the client requests v4\nbut, for performance purposes, the server sends v2. Should index-pack\nconvert the received pack to v4? My answer is no because there's no\nway we can do it without saving v2 first. And if we have to save v2\nanyway, pack-objects (or repack) will do the conversion better. We can\nadjust git-gc to maybe repack more often when the number of v2 packs\nis over a limit, or just repack v2 packs into one v4 pack.\n\nNguyễn Thái Ngọc Duy (10):\n test-dump: new test program to examine binary data\n config: add core.preferredPackVersion\n upload-pack: new capability to send pack v4\n fetch: new option to set preferred pack version for transfer\n clone: new option to set preferred pack version for transfer\n fetch: pack v4 support on smart http\n receive-pack: request for packv4 if it's the preferred version\n send-pack: support pack v4\n repack: add --pack-version and fall back to core.preferredPackVersion\n count-objects: report pack v4 usage\n\n .gitignore | 1 +\n Documentation/config.txt | 5 ++++\n Documentation/fetch-options.txt | 5 ++++\n Documentation/git-clone.txt | 4 +++\n Documentation/git-count-objects.txt | 4 +++\n Documentation/git-fetch-pack.txt | 4 +++\n Documentation/git-repack.txt | 6 +++-\n Documentation/gitremote-helpers.txt | 3 ++\n Documentation/technical/protocol-capabilities.txt | 14 +++++++++\n Makefile | 1 +\n builtin/clone.c | 19 ++++++++++++\n builtin/count-objects.c | 23 +++++++++++++--\n builtin/fetch-pack.c | 7 +++++\n builtin/fetch.c | 10 +++++++\n builtin/receive-pack.c | 3 +-\n cache.h | 1 +\n config.c | 5 ++++\n environment.c | 1 +\n fetch-pack.c | 3 ++\n fetch-pack.h | 1 +\n git-repack.sh | 8 +++++-\n remote-curl.c | 14 ++++++++-\n send-pack.c | 5 ++++\n send-pack.h | 1 +\n t/t5510-fetch.sh | 13 +++++++++\n t/t5516-fetch-push.sh | 12 ++++++++\n t/t5551-http-fetch.sh | 9 ++++++\n t/t5601-clone.sh | 12 ++++++++\n t/t7700-repack.sh | 35 +++++++++++++++++++++++\n test-dump.c (new) | 27 +++++++++++++++++\n transport-helper.c | 3 ++\n transport.c | 4 +++\n transport.h | 5 ++++\n upload-pack.c | 16 +++++++++--\n 34 files changed, 274 insertions(+), 10 deletions(-)\n create mode 100644 test-dump.c\n\n-- \n1.8.2.82.gc24b958\n"},{"id":"228246","messageId":"1380162409-18224-2-git-send-email-pclouds@gmail.com","threadId":"35023","inReplyTo":"1380162409-18224-1-git-send-email-pclouds@gmail.com","subject":"[PATCH 01/10] test-dump: new test program to examine binary data","fromName":"Nguyễn Thái Ngọc Duy","fromEmail":"pclouds@gmail.com","sentAt":"2013-09-26T02:26:40Z","receivedAt":"2013-09-26T02:26:40Z","isPatch":true,"sender":{"key":"pclouds@gmail.com","avatar":"https://avatars.githubusercontent.com/u/720?v=4"},"body":"This will come handy for verifying binary files like .pack, .idx or\n$GIT_DIR/index. For now the only supported command is \"ntohl\". This\ncommand looks into the given file at the given offset, do ntohl() and\nreturn the value in decimal.\n\nMore commands may be added later to decode varint, or decompress a\nzlib stream and so on...\n\nSigned-off-by: Nguyễn Thái Ngọc Duy \n---\n .gitignore | 1 +\n Makefile | 1 +\n test-dump.c (new) | 27 +++++++++++++++++++++++++++\n 3 files changed, 29 insertions(+)\n create mode 100644 test-dump.c\n\ndiff --git a/.gitignore b/.gitignore\nindex 2e2ae2b..6d835e2 100644\n--- a/.gitignore\n+++ b/.gitignore\n@@ -182,6 +182,7 @@\n /test-ctype\n /test-date\n /test-delta\n+/test-dump\n /test-dump-cache-tree\n /test-scrap-cache-tree\n /test-genrandom\ndiff --git a/Makefile b/Makefile\nindex af2e3e3..a07a194 100644\n--- a/Makefile\n+++ b/Makefile\n@@ -560,6 +560,7 @@ TEST_PROGRAMS_NEED_X += test-chmtime\n TEST_PROGRAMS_NEED_X += test-ctype\n TEST_PROGRAMS_NEED_X += test-date\n TEST_PROGRAMS_NEED_X += test-delta\n+TEST_PROGRAMS_NEED_X += test-dump\n TEST_PROGRAMS_NEED_X += test-dump-cache-tree\n TEST_PROGRAMS_NEED_X += test-genrandom\n TEST_PROGRAMS_NEED_X += test-index-version\ndiff --git a/test-dump.c b/test-dump.c\nnew file mode 100644\nindex 0000000..71c6f8f\n--- /dev/null\n+++ b/test-dump.c\n@@ -0,0 +1,27 @@\n+#include \"cache.h\"\n+\n+static inline uint32_t ntoh_l_force_align(void *p)\n+{\n+\tuint32_t x;\n+\tmemcpy(&x, p, sizeof(x));\n+\treturn ntohl(x);\n+}\n+\n+int main(int ac, char **av)\n+{\n+\tunsigned char *p;\n+\tint fd = open(av[2], O_RDONLY);\n+\tstruct stat st;\n+\tif (lstat(av[2], &st))\n+\t\treturn 1;\n+\tp = mmap(0, st.st_size, PROT_READ, MAP_PRIVATE, fd, 0);\n+\tif (p == (unsigned char*)MAP_FAILED)\n+\t\treturn 2;\n+\tif (!strcmp(av[1], \"ntohl\"))\n+\t\tprintf(\"%u\\n\", ntoh_l_force_align(p + atoi(av[3])));\n+\telse {\n+\t\tfprintf(stderr, \"unknown command %s\\n\", av[1]);\n+\t\treturn 3;\n+\t}\n+\treturn 0;\n+}\n-- \n1.8.2.82.gc24b958\n"},{"id":"228247","messageId":"1380162409-18224-3-git-send-email-pclouds@gmail.com","threadId":"35023","inReplyTo":"1380162409-18224-1-git-send-email-pclouds@gmail.com","subject":"[PATCH 02/10] config: add core.preferredPackVersion","fromName":"Nguyễn Thái Ngọc Duy","fromEmail":"pclouds@gmail.com","sentAt":"2013-09-26T02:26:41Z","receivedAt":"2013-09-26T02:26:41Z","isPatch":true,"sender":{"key":"pclouds@gmail.com","avatar":"https://avatars.githubusercontent.com/u/720?v=4"},"body":"\nSigned-off-by: Nguyễn Thái Ngọc Duy \n---\n Documentation/config.txt | 5 +++++\n cache.h | 1 +\n config.c | 5 +++++\n environment.c | 1 +\n 4 files changed, 12 insertions(+)\n\ndiff --git a/Documentation/config.txt b/Documentation/config.txt\nindex ec57a15..6e7037b 100644\n--- a/Documentation/config.txt\n+++ b/Documentation/config.txt\n@@ -644,6 +644,11 @@ core.abbrev::\n \tfor abbreviated object names to stay unique for sufficiently long\n \ttime.\n \n+core.preferredPackVersion::\n+\tThis is the preferred pack format version for various operations\n+\tthat may produce pack files such as clone, fetch, push or repack.\n+\tValid values are 2 and 4. The default value is 2.\n+\n add.ignore-errors::\n add.ignoreErrors::\n \tTells 'git add' to continue adding files when some files cannot be\ndiff --git a/cache.h b/cache.h\nindex b68ad5a..20c2d6d 100644\n--- a/cache.h\n+++ b/cache.h\n@@ -597,6 +597,7 @@ extern int fsync_object_files;\n extern int core_preload_index;\n extern int core_apply_sparse_checkout;\n extern int precomposed_unicode;\n+extern int core_default_pack_version;\n \n /*\n * The character that begins a commented line in user-editable file\ndiff --git a/config.c b/config.c\nindex e13a7b6..02af0d1 100644\n--- a/config.c\n+++ b/config.c\n@@ -831,6 +831,11 @@ static int git_default_core_config(const char *var, const char *value)\n \t\treturn 0;\n \t}\n \n+\tif (!strcmp(var, \"core.preferredpackversion\")) {\n+\t\tcore_default_pack_version = git_config_int(var, value);\n+\t\treturn 0;\n+\t}\n+\n \t/* Add other config variables here and to Documentation/config.txt. */\n \treturn 0;\n }\ndiff --git a/environment.c b/environment.c\nindex 5398c36..24c43ba 100644\n--- a/environment.c\n+++ b/environment.c\n@@ -62,6 +62,7 @@ int merge_log_config = -1;\n int precomposed_unicode = -1; /* see probe_utf8_pathname_composition() */\n struct startup_info *startup_info;\n unsigned long pack_size_limit_cfg;\n+int core_default_pack_version = 2;\n \n /*\n * The character that begins a commented line in user-editable file\n-- \n1.8.2.82.gc24b958\n"},{"id":"228248","messageId":"1380162409-18224-4-git-send-email-pclouds@gmail.com","threadId":"35023","inReplyTo":"1380162409-18224-1-git-send-email-pclouds@gmail.com","subject":"[PATCH 03/10] upload-pack: new capability to send pack v4","fromName":"Nguyễn Thái Ngọc Duy","fromEmail":"pclouds@gmail.com","sentAt":"2013-09-26T02:26:42Z","receivedAt":"2013-09-26T02:26:42Z","isPatch":true,"sender":{"key":"pclouds@gmail.com","avatar":"https://avatars.githubusercontent.com/u/720?v=4"},"body":"\nSigned-off-by: Nguyễn Thái Ngọc Duy \n---\n Documentation/technical/protocol-capabilities.txt | 10 ++++++++++\n upload-pack.c | 16 +++++++++++++---\n 2 files changed, 23 insertions(+), 3 deletions(-)\n\ndiff --git a/Documentation/technical/protocol-capabilities.txt b/Documentation/technical/protocol-capabilities.txt\nindex fd8ffa5..be09792 100644\n--- a/Documentation/technical/protocol-capabilities.txt\n+++ b/Documentation/technical/protocol-capabilities.txt\n@@ -217,3 +217,13 @@ allow-tip-sha1-in-want\n If the upload-pack server advertises this capability, fetch-pack may\n send \"want\" lines with SHA-1s that exist at the server but are not\n advertised by upload-pack.\n+\n+packv4\n+------\n+\n+The upload-server can generate .pack version 4. If the client accepts\n+this capability, the server may send a pack version 4. The server can\n+choose to send pack version 2 even if the client accepts this\n+capability.\n+\n+This capability does not include multi-base tree support.\ndiff --git a/upload-pack.c b/upload-pack.c\nindex ccf76d9..291d366 100644\n--- a/upload-pack.c\n+++ b/upload-pack.c\n@@ -32,7 +32,7 @@ static unsigned long oldest_have;\n \n static int multi_ack;\n static int no_done;\n-static int use_thin_pack, use_ofs_delta, use_include_tag;\n+static int use_thin_pack, use_ofs_delta, use_include_tag, packv4_ok;\n static int no_progress, daemon_mode;\n static int allow_tip_sha1_in_want;\n static int shallow_nr;\n@@ -140,7 +140,7 @@ static void create_pack_file(void)\n \t\t\"corruption on the remote side.\";\n \tint buffered = -1;\n \tssize_t sz;\n-\tconst char *argv[10];\n+\tconst char *argv[11];\n \tint arg = 0;\n \n \targv[arg++] = \"pack-objects\";\n@@ -157,6 +157,14 @@ static void create_pack_file(void)\n \t\targv[arg++] = \"--delta-base-offset\";\n \tif (use_include_tag)\n \t\targv[arg++] = \"--include-tag\";\n+\t/*\n+\t * The client may have requested pack v4. But if this is a v2\n+\t * repo on a busy machine, we may want to send v2 and let the\n+\t * client do the v2->v4 conversion to reduce the cost on the\n+\t * server side.\n+\t */\n+\tif (packv4_ok)\n+\t\targv[arg++] = \"--version=4\";\n \targv[arg++] = NULL;\n \n \tmemset(&pack_objects, 0, sizeof(pack_objects));\n@@ -633,6 +641,8 @@ static void receive_needs(void)\n \t\t\tno_progress = 1;\n \t\tif (parse_feature_request(features, \"include-tag\"))\n \t\t\tuse_include_tag = 1;\n+\t\tif (parse_feature_request(features, \"packv4\"))\n+\t\t\tpackv4_ok = 1;\n \n \t\to = parse_object(sha1_buf);\n \t\tif (!o)\n@@ -738,7 +748,7 @@ static int send_ref(const char *refname, const unsigned char *sha1, int flag, vo\n {\n \tstatic const char *capabilities = \"multi_ack thin-pack side-band\"\n \t\t\" side-band-64k ofs-delta shallow no-progress\"\n-\t\t\" include-tag multi_ack_detailed\";\n+\t\t\" include-tag multi_ack_detailed packv4\";\n \tconst char *refname_nons = strip_namespace(refname);\n \tunsigned char peeled[20];\n \n-- \n1.8.2.82.gc24b958\n"},{"id":"228249","messageId":"1380162409-18224-5-git-send-email-pclouds@gmail.com","threadId":"35023","inReplyTo":"1380162409-18224-1-git-send-email-pclouds@gmail.com","subject":"[PATCH 04/10] fetch: new option to set preferred pack version for transfer","fromName":"Nguyễn Thái Ngọc Duy","fromEmail":"pclouds@gmail.com","sentAt":"2013-09-26T02:26:43Z","receivedAt":"2013-09-26T02:26:43Z","isPatch":true,"sender":{"key":"pclouds@gmail.com","avatar":"https://avatars.githubusercontent.com/u/720?v=4"},"body":"\nSigned-off-by: Nguyễn Thái Ngọc Duy \n---\n Documentation/fetch-options.txt | 5 +++++\n builtin/fetch.c | 10 ++++++++++\n fetch-pack.c | 3 +++\n fetch-pack.h | 1 +\n t/t5510-fetch.sh | 13 +++++++++++++\n transport.c | 4 ++++\n transport.h | 5 +++++\n 7 files changed, 41 insertions(+)\n\ndiff --git a/Documentation/fetch-options.txt b/Documentation/fetch-options.txt\nindex ba1fe49..47d55e5 100644\n--- a/Documentation/fetch-options.txt\n+++ b/Documentation/fetch-options.txt\n@@ -17,6 +17,11 @@\n \tConvert a shallow repository to a complete one, removing all\n \tthe limitations imposed by shallow repositories.\n \n+--pack-version=::\n+\tDefine the preferred pack format version for data transfer.\n+\tValid values are 2 and 4. Default value is specified by\n+\tcore.preferredPackVersion setting. See linkgit:git-config[1].\n+\n ifndef::git-pull[]\n --dry-run::\n \tShow what would be done, without making any changes.\ndiff --git a/builtin/fetch.c b/builtin/fetch.c\nindex d784b2e..695fbf1 100644\n--- a/builtin/fetch.c\n+++ b/builtin/fetch.c\n@@ -31,6 +31,7 @@ enum {\n };\n \n static int all, append, dry_run, force, keep, multiple, prune, update_head_ok, verbosity;\n+static int pack_version;\n static int progress = -1, recurse_submodules = RECURSE_SUBMODULES_DEFAULT;\n static int tags = TAGS_DEFAULT, unshallow;\n static const char *depth;\n@@ -90,6 +91,8 @@ static struct option builtin_fetch_options[] = {\n \t{ OPTION_STRING, 0, \"recurse-submodules-default\",\n \t\t &recurse_submodules_default, NULL,\n \t\t N_(\"default mode for recursion\"), PARSE_OPT_HIDDEN },\n+\tOPT_INTEGER(0, \"pack-version\", &pack_version,\n+\t\t N_(\"preferred pack version for transfer\")),\n \tOPT_END()\n };\n \n@@ -957,6 +960,8 @@ static int fetch_one(struct remote *remote, int argc, const char **argv)\n \t\tset_option(TRANS_OPT_KEEP, \"yes\");\n \tif (depth)\n \t\tset_option(TRANS_OPT_DEPTH, depth);\n+\tif (pack_version == 4)\n+\t\tset_option(TRANS_OPT_PACKV4, \"yes\");\n \n \tif (argc > 0) {\n \t\tint j = 0;\n@@ -1010,6 +1015,11 @@ int cmd_fetch(int argc, const char **argv, const char *prefix)\n \targc = parse_options(argc, argv, prefix,\n \t\t\t builtin_fetch_options, builtin_fetch_usage, 0);\n \n+\tif (!pack_version)\n+\t\tpack_version = core_default_pack_version;\n+\tif (pack_version != 2 && pack_version != 4)\n+\t\tdie(_(\"invalid pack version %d\"), pack_version);\n+\n \tif (unshallow) {\n \t\tif (depth)\n \t\t\tdie(_(\"--depth and --unshallow cannot be used together\"));\ndiff --git a/fetch-pack.c b/fetch-pack.c\nindex 6684348..16aa3d0 100644\n--- a/fetch-pack.c\n+++ b/fetch-pack.c\n@@ -324,6 +324,7 @@ static int find_common(struct fetch_pack_args *args,\n \t\t\tif (args->no_progress) strbuf_addstr(&c, \" no-progress\");\n \t\t\tif (args->include_tag) strbuf_addstr(&c, \" include-tag\");\n \t\t\tif (prefer_ofs_delta) strbuf_addstr(&c, \" ofs-delta\");\n+\t\t\tif (args->packv4) strbuf_addstr(&c, \" packv4\");\n \t\t\tif (agent_supported) strbuf_addf(&c, \" agent=%s\",\n \t\t\t\t\t\t\t git_user_agent_sanitized());\n \t\t\tpacket_buf_write(&req_buf, \"want %s%s\\n\", remote_hex, c.buf);\n@@ -869,6 +870,8 @@ static struct ref *do_fetch_pack(struct fetch_pack_args *args,\n \t\targs->no_progress = 0;\n \tif (!server_supports(\"include-tag\"))\n \t\targs->include_tag = 0;\n+\tif (!server_supports(\"packv4\"))\n+\t\targs->packv4 = 0;\n \tif (server_supports(\"ofs-delta\")) {\n \t\tif (args->verbose)\n \t\t\tfprintf(stderr, \"Server supports ofs-delta\\n\");\ndiff --git a/fetch-pack.h b/fetch-pack.h\nindex 40f08ba..5f03739 100644\n--- a/fetch-pack.h\n+++ b/fetch-pack.h\n@@ -17,6 +17,7 @@ struct fetch_pack_args {\n \t\tno_progress:1,\n \t\tinclude_tag:1,\n \t\tstateless_rpc:1,\n+\t\tpackv4:1,\n \t\tcheck_self_contained_and_connected:1,\n \t\tself_contained_and_connected:1;\n };\ndiff --git a/t/t5510-fetch.sh b/t/t5510-fetch.sh\nindex fde6891..d3da088 100755\n--- a/t/t5510-fetch.sh\n+++ b/t/t5510-fetch.sh\n@@ -512,4 +512,17 @@ test_expect_success 'all boundary commits are excluded' '\n \ttest_bundle_object_count .git/objects/pack/pack-${pack##pack\t}.pack 3\n '\n \n+test_expect_success 'fetch --pack-version=4' '\n+\tgit init pv4 &&\n+\t(\n+\t\tcd pv4 &&\n+\t\tgit fetch --pack-version=4 --keep file://\"$D\"/.git &&\n+\t\tP=`ls .git/objects/pack/pack-*.pack` &&\n+\t\t# Offset 4 is pack version\n+\t\ttest-dump ntohl \"$P\" 4 >ver.actual &&\n+\t\techo 4 >ver.expected &&\n+\t\ttest_cmp ver.expected ver.actual\n+\t)\n+'\n+\n test_done\ndiff --git a/transport.c b/transport.c\nindex e15db98..ad5a4f1 100644\n--- a/transport.c\n+++ b/transport.c\n@@ -473,6 +473,9 @@ static int set_git_option(struct git_transport_options *opts,\n \t} else if (!strcmp(name, TRANS_OPT_KEEP)) {\n \t\topts->keep = !!value;\n \t\treturn 0;\n+\t} else if (!strcmp(name, TRANS_OPT_PACKV4)) {\n+\t\topts->packv4 = !!value;\n+\t\treturn 0;\n \t} else if (!strcmp(name, TRANS_OPT_DEPTH)) {\n \t\tif (!value)\n \t\t\topts->depth = 0;\n@@ -534,6 +537,7 @@ static int fetch_refs_via_pack(struct transport *transport,\n \targs.quiet = (transport->verbose < 0);\n \targs.no_progress = !transport->progress;\n \targs.depth = data->options.depth;\n+\targs.packv4 = data->options.packv4;\n \targs.check_self_contained_and_connected =\n \t\tdata->options.check_self_contained_and_connected;\n \ndiff --git a/transport.h b/transport.h\nindex ea70ea7..59785c0 100644\n--- a/transport.h\n+++ b/transport.h\n@@ -8,6 +8,7 @@ struct git_transport_options {\n \tunsigned thin : 1;\n \tunsigned keep : 1;\n \tunsigned followtags : 1;\n+\tunsigned packv4 : 1;\n \tunsigned check_self_contained_and_connected : 1;\n \tunsigned self_contained_and_connected : 1;\n \tint depth;\n@@ -135,6 +136,10 @@ struct transport *transport_get(struct remote *, const char *);\n /* Aggressively fetch annotated tags if possible */\n #define TRANS_OPT_FOLLOWTAGS \"followtags\"\n \n+/* Prefer pack version 4 as the transport format */\n+#define TRANS_OPT_PACKV4 \"packv4\"\n+\n+\n /**\n * Returns 0 if the option was used, non-zero otherwise. Prints a\n * message to stderr if the option is not used.\n-- \n1.8.2.82.gc24b958\n"},{"id":"228250","messageId":"1380162409-18224-6-git-send-email-pclouds@gmail.com","threadId":"35023","inReplyTo":"1380162409-18224-1-git-send-email-pclouds@gmail.com","subject":"[PATCH 05/10] clone: new option to set preferred pack version for transfer","fromName":"Nguyễn Thái Ngọc Duy","fromEmail":"pclouds@gmail.com","sentAt":"2013-09-26T02:26:44Z","receivedAt":"2013-09-26T02:26:44Z","isPatch":true,"sender":{"key":"pclouds@gmail.com","avatar":"https://avatars.githubusercontent.com/u/720?v=4"},"body":"\nSigned-off-by: Nguyễn Thái Ngọc Duy \n---\n Documentation/git-clone.txt | 4 ++++\n builtin/clone.c | 19 +++++++++++++++++++\n t/t5601-clone.sh | 12 ++++++++++++\n 3 files changed, 35 insertions(+)\n\ndiff --git a/Documentation/git-clone.txt b/Documentation/git-clone.txt\nindex 450f158..6299a70 100644\n--- a/Documentation/git-clone.txt\n+++ b/Documentation/git-clone.txt\n@@ -217,6 +217,10 @@ objects from the source repository into a pack in the cloned repository.\n \tThe result is Git repository can be separated from working\n \ttree.\n \n+--pack-version=::\n+\tDefine the preferred pack format version for data transfer.\n+\tValid values are 2 and 4. Default value is specified by\n+\tcore.preferredPackVersion setting. See linkgit:git-config[1].\n \n ::\n \tThe (possibly remote) repository to clone from. See the\ndiff --git a/builtin/clone.c b/builtin/clone.c\nindex 430307b..9c0e296 100644\n--- a/builtin/clone.c\n+++ b/builtin/clone.c\n@@ -46,6 +46,7 @@ static const char *real_git_dir;\n static char *option_upload_pack = \"git-upload-pack\";\n static int option_verbosity;\n static int option_progress = -1;\n+static int pack_version;\n static struct string_list option_config;\n static struct string_list option_reference;\n \n@@ -98,6 +99,8 @@ static struct option builtin_clone_options[] = {\n \t\t N_(\"separate git dir from working tree\")),\n \tOPT_STRING_LIST('c', \"config\", &option_config, N_(\"key=value\"),\n \t\t\tN_(\"set config inside the new repository\")),\n+\tOPT_INTEGER(0, \"pack-version\", &pack_version,\n+\t\t N_(\"preferred pack version for transfer\")),\n \tOPT_END()\n };\n \n@@ -764,6 +767,11 @@ int cmd_clone(int argc, const char **argv, const char *prefix)\n \t\tusage_msg_opt(_(\"You must specify a repository to clone.\"),\n \t\t\tbuiltin_clone_usage, builtin_clone_options);\n \n+\tif (!pack_version)\n+\t\tpack_version = core_default_pack_version;\n+\tif (pack_version != 2 && pack_version != 4)\n+\t\tdie(_(\"invalid pack version %d\"), pack_version);\n+\n \tif (option_single_branch == -1)\n \t\toption_single_branch = option_depth ? 1 : 0;\n \n@@ -794,6 +802,8 @@ int cmd_clone(int argc, const char **argv, const char *prefix)\n \tis_local = option_local != 0 && path && !is_bundle;\n \tif (is_local && option_depth)\n \t\twarning(_(\"--depth is ignored in local clones; use file:// instead.\"));\n+\tif (is_local && pack_version == 4)\n+\t\twarning(_(\"--pack-version is ignored in local clones; use file:// instead.\"));\n \tif (option_local > 0 && !is_local)\n \t\twarning(_(\"--local is ignored\"));\n \n@@ -868,6 +878,12 @@ int cmd_clone(int argc, const char **argv, const char *prefix)\n \t} else {\n \t\tstrbuf_addf(&branch_top, \"refs/remotes/%s/\", option_origin);\n \t}\n+\tif (pack_version != core_default_pack_version) {\n+\t\tstruct strbuf sb = STRBUF_INIT;\n+\t\tstrbuf_addf(&sb, \"%d\", pack_version);\n+\t\tgit_config_set(\"core.preferredPackVersion\", sb.buf);\n+\t\tstrbuf_release(&sb);\n+\t}\n \n \tstrbuf_addf(&value, \"+%s*:%s*\", src_ref_prefix, branch_top.buf);\n \tstrbuf_addf(&key, \"remote.%s.url\", option_origin);\n@@ -903,6 +919,9 @@ int cmd_clone(int argc, const char **argv, const char *prefix)\n \t\t\ttransport_set_option(transport, TRANS_OPT_UPLOADPACK,\n \t\t\t\t\t option_upload_pack);\n \n+\t\tif (pack_version == 4)\n+\t\t\ttransport_set_option(transport, TRANS_OPT_PACKV4, \"yes\");\n+\n \t\tif (transport->smart_options && !option_depth)\n \t\t\ttransport->smart_options->check_self_contained_and_connected = 1;\n \t}\ndiff --git a/t/t5601-clone.sh b/t/t5601-clone.sh\nindex 0629149..8118524 100755\n--- a/t/t5601-clone.sh\n+++ b/t/t5601-clone.sh\n@@ -285,4 +285,16 @@ test_expect_success NOT_MINGW,NOT_CYGWIN 'clone local path foo:bar' '\n \tgit clone \"./foo:bar\" foobar\n '\n \n+test_expect_success 'clone --pack-version=4' '\n+\tgit clone --pack-version=4 --no-local src pv4 &&\n+\tP=`ls pv4/.git/objects/pack/pack-*.pack` &&\n+\t# Offset 4 is pack version\n+\ttest-dump ntohl \"$P\" 4 >ver.actual &&\n+\techo 4 >ver.expected &&\n+\ttest_cmp ver.expected ver.actual &&\n+\tgit --git-dir=pv4/.git config --int core.preferredPackVersion >cfgver.actual &&\n+\techo 4 >cfgver.expected &&\n+\ttest_cmp cfgver.expected cfgver.actual\n+'\n+\n test_done\n-- \n1.8.2.82.gc24b958\n"},{"id":"228251","messageId":"1380162409-18224-7-git-send-email-pclouds@gmail.com","threadId":"35023","inReplyTo":"1380162409-18224-1-git-send-email-pclouds@gmail.com","subject":"[PATCH 06/10] fetch: pack v4 support on smart http","fromName":"Nguyễn Thái Ngọc Duy","fromEmail":"pclouds@gmail.com","sentAt":"2013-09-26T02:26:45Z","receivedAt":"2013-09-26T02:26:45Z","isPatch":true,"sender":{"key":"pclouds@gmail.com","avatar":"https://avatars.githubusercontent.com/u/720?v=4"},"body":"\nSigned-off-by: Nguyễn Thái Ngọc Duy \n---\n Documentation/git-fetch-pack.txt | 4 ++++\n Documentation/gitremote-helpers.txt | 3 +++\n builtin/fetch-pack.c | 7 +++++++\n remote-curl.c | 14 +++++++++++++-\n t/t5551-http-fetch.sh | 9 +++++++++\n transport-helper.c | 3 +++\n 6 files changed, 39 insertions(+), 1 deletion(-)\n\ndiff --git a/Documentation/git-fetch-pack.txt b/Documentation/git-fetch-pack.txt\nindex 1e71754..b68cd45 100644\n--- a/Documentation/git-fetch-pack.txt\n+++ b/Documentation/git-fetch-pack.txt\n@@ -87,6 +87,10 @@ be in a separate packet, and the list must end with a flush packet.\n \t'git-upload-pack' treats the special depth 2147483647 as\n \tinfinite even if there is an ancestor-chain that long.\n \n+--pack-version=::\n+\tDefine the preferred pack format version for data transfer.\n+\tValid values are 2 and 4. Default is 2.\n+\n --no-progress::\n \tDo not show the progress.\n \ndiff --git a/Documentation/gitremote-helpers.txt b/Documentation/gitremote-helpers.txt\nindex 0827f69..90dfc2e 100644\n--- a/Documentation/gitremote-helpers.txt\n+++ b/Documentation/gitremote-helpers.txt\n@@ -416,6 +416,9 @@ set by Git if the remote helper has the 'option' capability.\n \tmust not rely on this option being set before\n \tconnect request occurs.\n \n+'option packv4 \\{'true'|'false'\\}::\n+\tPrefer pack version 4 (true) or 2 (false) for data transfer.\n+\n SEE ALSO\n --------\n linkgit:git-remote[1]\ndiff --git a/builtin/fetch-pack.c b/builtin/fetch-pack.c\nindex aba4465..bfe940a 100644\n--- a/builtin/fetch-pack.c\n+++ b/builtin/fetch-pack.c\n@@ -100,6 +100,13 @@ int cmd_fetch_pack(int argc, const char **argv, const char *prefix)\n \t\t\tpack_lockfile_ptr = &pack_lockfile;\n \t\t\tcontinue;\n \t\t}\n+\t\tif (!prefixcmp(arg, \"--pack-version=\")) {\n+\t\t\tint ver = strtol(arg + 15, NULL, 0);\n+\t\t\tif (ver != 2 && ver != 4)\n+\t\t\t\tdie(_(\"invalid pack version %d\"), ver);\n+\t\t\targs.packv4 = ver == 4;\n+\t\t\tcontinue;\n+\t\t}\n \t\tusage(fetch_pack_usage);\n \t}\n \ndiff --git a/remote-curl.c b/remote-curl.c\nindex 5b3ce9e..1a3d215 100644\n--- a/remote-curl.c\n+++ b/remote-curl.c\n@@ -18,6 +18,7 @@ struct options {\n \tunsigned progress : 1,\n \t\tfollowtags : 1,\n \t\tdry_run : 1,\n+\t\tpackv4 : 1,\n \t\tthin : 1;\n };\n static struct options options;\n@@ -67,6 +68,15 @@ static int set_option(const char *name, const char *value)\n \t\t\treturn -1;\n \t\treturn 0;\n \t}\n+\telse if (!strcmp(name, \"packv4\")) {\n+\t\tif (!strcmp(value, \"true\"))\n+\t\t\toptions.packv4 = 1;\n+\t\telse if (!strcmp(value, \"false\"))\n+\t\t\toptions.packv4 = 0;\n+\t\telse\n+\t\t\treturn -1;\n+\t\treturn 0;\n+\t}\n \telse {\n \t\treturn 1 /* unsupported */;\n \t}\n@@ -654,7 +664,7 @@ static int fetch_git(struct discovery *heads,\n \tstruct strbuf preamble = STRBUF_INIT;\n \tchar *depth_arg = NULL;\n \tint argc = 0, i, err;\n-\tconst char *argv[15];\n+\tconst char *argv[16];\n \n \targv[argc++] = \"fetch-pack\";\n \targv[argc++] = \"--stateless-rpc\";\n@@ -676,6 +686,8 @@ static int fetch_git(struct discovery *heads,\n \t\tdepth_arg = strbuf_detach(&buf, NULL);\n \t\targv[argc++] = depth_arg;\n \t}\n+\tif (options.packv4)\n+\t\targv[argc++] = \"--pack-version=4\";\n \targv[argc++] = url;\n \targv[argc++] = NULL;\n \ndiff --git a/t/t5551-http-fetch.sh b/t/t5551-http-fetch.sh\nindex 55a866a..5b4e6aa 100755\n--- a/t/t5551-http-fetch.sh\n+++ b/t/t5551-http-fetch.sh\n@@ -222,5 +222,14 @@ test_expect_success EXPENSIVE 'clone the 50,000 tag repo to check OS command lin\n \t)\n '\n \n+test_expect_success 'clone http repository with pack v4' '\n+\tgit -c transfer.unpackLimit=1 clone --pack-version=4 $HTTPD_URL/smart/repo.git pv4 &&\n+\tP=`ls pv4/.git/objects/pack/pack-*.pack` &&\n+\t# Offset 4 is pack version\n+\ttest-dump ntohl \"$P\" 4 >ver.actual &&\n+\techo 4 >ver.expected &&\n+\ttest_cmp ver.expected ver.actual\n+'\n+\n stop_httpd\n test_done\ndiff --git a/transport-helper.c b/transport-helper.c\nindex 63cabc3..07fbf6e 100644\n--- a/transport-helper.c\n+++ b/transport-helper.c\n@@ -350,6 +350,9 @@ static int fetch_with_fetch(struct transport *transport,\n \n \tstandard_options(transport);\n \n+\tset_helper_option(transport, \"packv4\",\n+\t\t\t data->transport_options.packv4 ? \"true\" : \"false\");\n+\n \tfor (i = 0; i < nr_heads; i++) {\n \t\tconst struct ref *posn = to_fetch[i];\n \t\tif (posn->status & REF_STATUS_UPTODATE)\n-- \n1.8.2.82.gc24b958\n"},{"id":"228252","messageId":"1380162409-18224-8-git-send-email-pclouds@gmail.com","threadId":"35023","inReplyTo":"1380162409-18224-1-git-send-email-pclouds@gmail.com","subject":"[PATCH 07/10] receive-pack: request for packv4 if it's the preferred version","fromName":"Nguyễn Thái Ngọc Duy","fromEmail":"pclouds@gmail.com","sentAt":"2013-09-26T02:26:46Z","receivedAt":"2013-09-26T02:26:46Z","isPatch":true,"sender":{"key":"pclouds@gmail.com","avatar":"https://avatars.githubusercontent.com/u/720?v=4"},"body":"This is the only plumbing command that is controlled by\ncore.preferredPackVersion so far.\n\nSigned-off-by: Nguyễn Thái Ngọc Duy \n---\n Documentation/technical/protocol-capabilities.txt | 4 ++++\n builtin/receive-pack.c | 3 ++-\n 2 files changed, 6 insertions(+), 1 deletion(-)\n\ndiff --git a/Documentation/technical/protocol-capabilities.txt b/Documentation/technical/protocol-capabilities.txt\nindex be09792..32153cd 100644\n--- a/Documentation/technical/protocol-capabilities.txt\n+++ b/Documentation/technical/protocol-capabilities.txt\n@@ -226,4 +226,8 @@ this capability, the server may send a pack version 4. The server can\n choose to send pack version 2 even if the client accepts this\n capability.\n \n+The receive-pack server advertises this capability if it wants to\n+receive the pack in format version 4 and the client should send in\n+this format.\n+\n This capability does not include multi-base tree support.\ndiff --git a/builtin/receive-pack.c b/builtin/receive-pack.c\nindex e3eb5fc..288b0bc 100644\n--- a/builtin/receive-pack.c\n+++ b/builtin/receive-pack.c\n@@ -130,10 +130,11 @@ static void show_ref(const char *path, const unsigned char *sha1)\n \tif (sent_capabilities)\n \t\tpacket_write(1, \"%s %s\\n\", sha1_to_hex(sha1), path);\n \telse\n-\t\tpacket_write(1, \"%s %s%c%s%s agent=%s\\n\",\n+\t\tpacket_write(1, \"%s %s%c%s%s%s agent=%s\\n\",\n \t\t\t sha1_to_hex(sha1), path, 0,\n \t\t\t \" report-status delete-refs side-band-64k quiet\",\n \t\t\t prefer_ofs_delta ? \" ofs-delta\" : \"\",\n+\t\t\t core_default_pack_version == 4 ? \" packv4\" : \"\",\n \t\t\t git_user_agent_sanitized());\n \tsent_capabilities = 1;\n }\n-- \n1.8.2.82.gc24b958\n"},{"id":"228253","messageId":"1380162409-18224-9-git-send-email-pclouds@gmail.com","threadId":"35023","inReplyTo":"1380162409-18224-1-git-send-email-pclouds@gmail.com","subject":"[PATCH 08/10] send-pack: support pack v4","fromName":"Nguyễn Thái Ngọc Duy","fromEmail":"pclouds@gmail.com","sentAt":"2013-09-26T02:26:47Z","receivedAt":"2013-09-26T02:26:47Z","isPatch":true,"sender":{"key":"pclouds@gmail.com","avatar":"https://avatars.githubusercontent.com/u/720?v=4"},"body":"Contrary to the fetch direction, whether send-pack sends packv4 is\ntotally controlled by the server. This is in favor of lowering load at\nthe server side. More logic may be added later to allow the client to\nstick to v2 even if the server requests v4.\n\nSigned-off-by: Nguyễn Thái Ngọc Duy \n---\n send-pack.c | 5 +++++\n send-pack.h | 1 +\n t/t5516-fetch-push.sh | 12 ++++++++++++\n 3 files changed, 18 insertions(+)\n\ndiff --git a/send-pack.c b/send-pack.c\nindex 7d172ef..977c14b 100644\n--- a/send-pack.c\n+++ b/send-pack.c\n@@ -44,6 +44,7 @@ static int pack_objects(int fd, struct ref *refs, struct extra_have_objects *ext\n \t\tNULL,\n \t\tNULL,\n \t\tNULL,\n+\t\tNULL,\n \t};\n \tstruct child_process po;\n \tint i;\n@@ -57,6 +58,8 @@ static int pack_objects(int fd, struct ref *refs, struct extra_have_objects *ext\n \t\targv[i++] = \"-q\";\n \tif (args->progress)\n \t\targv[i++] = \"--progress\";\n+\tif (args->packv4)\n+\t\targv[i++] = \"--version=4\";\n \tmemset(&po, 0, sizeof(po));\n \tpo.argv = argv;\n \tpo.in = -1;\n@@ -205,6 +208,8 @@ int send_pack(struct send_pack_args *args,\n \t\tquiet_supported = 1;\n \tif (server_supports(\"agent\"))\n \t\tagent_supported = 1;\n+\tif (server_supports(\"packv4\"))\n+\t\targs->packv4 = 1;\n \n \tif (!remote_refs) {\n \t\tfprintf(stderr, \"No refs in common and none specified; doing nothing.\\n\"\ndiff --git a/send-pack.h b/send-pack.h\nindex 05d7ab1..cda770c 100644\n--- a/send-pack.h\n+++ b/send-pack.h\n@@ -11,6 +11,7 @@ struct send_pack_args {\n \t\tuse_thin_pack:1,\n \t\tuse_ofs_delta:1,\n \t\tdry_run:1,\n+\t\tpackv4:1,\n \t\tstateless_rpc:1;\n };\n \ndiff --git a/t/t5516-fetch-push.sh b/t/t5516-fetch-push.sh\nindex 4691d51..d0c116f 100755\n--- a/t/t5516-fetch-push.sh\n+++ b/t/t5516-fetch-push.sh\n@@ -1172,4 +1172,16 @@ test_expect_success 'push --follow-tag only pushes relevant tags' '\n \ttest_cmp expect actual\n '\n \n+test_expect_success 'push pack v4' '\n+\tgit init pv4 &&\n+\tgit --git-dir pv4/.git config core.preferredPackVersion 4 &&\n+\tgit --git-dir pv4/.git config transfer.unpackLimit 1 &&\n+\tgit push pv4 HEAD:refs/heads/head &&\n+\tP=`ls pv4/.git/objects/pack/pack-*.pack` &&\n+\t# Offset 4 is pack version\n+\ttest-dump ntohl \"$P\" 4 >ver.actual &&\n+\techo 4 >ver.expected &&\n+\ttest_cmp ver.expected ver.actual\n+'\n+\n test_done\n-- \n1.8.2.82.gc24b958\n"},{"id":"228254","messageId":"1380162409-18224-10-git-send-email-pclouds@gmail.com","threadId":"35023","inReplyTo":"1380162409-18224-1-git-send-email-pclouds@gmail.com","subject":"[PATCH 09/10] repack: add --pack-version and fall back to core.preferredPackVersion","fromName":"Nguyễn Thái Ngọc Duy","fromEmail":"pclouds@gmail.com","sentAt":"2013-09-26T02:26:48Z","receivedAt":"2013-09-26T02:26:48Z","isPatch":true,"sender":{"key":"pclouds@gmail.com","avatar":"https://avatars.githubusercontent.com/u/720?v=4"},"body":"\nSigned-off-by: Nguyễn Thái Ngọc Duy \n---\n Documentation/git-repack.txt | 6 +++++-\n git-repack.sh | 8 +++++++-\n t/t7700-repack.sh | 35 +++++++++++++++++++++++++++++++++++\n 3 files changed, 47 insertions(+), 2 deletions(-)\n\ndiff --git a/Documentation/git-repack.txt b/Documentation/git-repack.txt\nindex 4c1aff6..c43eb4a 100644\n--- a/Documentation/git-repack.txt\n+++ b/Documentation/git-repack.txt\n@@ -9,7 +9,7 @@ git-repack - Pack unpacked objects in a repository\n SYNOPSIS\n --------\n [verse]\n-'git repack' [-a] [-A] [-d] [-f] [-F] [-l] [-n] [-q] [--window=] [--depth=]\n+'git repack' [options]\n \n DESCRIPTION\n -----------\n@@ -110,6 +110,10 @@ other objects in that pack they already have locally.\n \tThe default is unlimited, unless the config variable\n \t`pack.packSizeLimit` is set.\n \n+--pack-version=::\n+\tForce the version for the generated pack.\n+\tValid values are 2 and 4. Default value is specified by\n+\tcore.preferredPackVersion setting. See linkgit:git-config[1].\n \n Configuration\n -------------\ndiff --git a/git-repack.sh b/git-repack.sh\nindex 7579331..0d898eb 100755\n--- a/git-repack.sh\n+++ b/git-repack.sh\n@@ -21,12 +21,13 @@ window= size of the window used for delta compression\n window-memory= same as the above, but limit memory size instead of entries count\n depth= limits the maximum delta depth\n max-pack-size= maximum size of each packfile\n+pack-version= format version of the output pack\n \"\n SUBDIRECTORY_OK='Yes'\n . git-sh-setup\n \n no_update_info= all_into_one= remove_redundant= unpack_unreachable=\n-local= no_reuse= extra=\n+local= no_reuse= extra= packver=\n while test $# != 0\n do\n \tcase \"$1\" in\n@@ -43,6 +44,8 @@ do\n \t-l)\tlocal=--local ;;\n \t--max-pack-size|--window|--window-memory|--depth)\n \t\textra=\"$extra $1=$2\"; shift ;;\n+\t--pack-version)\n+\t\tpackver=\"$2\"; shift ;;\n \t--) shift; break;;\n \t*)\tusage ;;\n \tesac\n@@ -92,6 +95,9 @@ esac\n \n mkdir -p \"$PACKDIR\" || exit\n \n+[ -n \"$packver\" ] || packver=\"`git config --int core.preferredPackVersion`\"\n+[ -n \"$packver\" ] && args=\"$args --version=$packver\"\n+\n args=\"$args $local ${GIT_QUIET:+-q} $no_reuse$extra\"\n names=$(git pack-objects --keep-true-parents --honor-pack-keep --non-empty --all --reflog $args ver.actual &&\n+\t\techo 4 >ver.expected &&\n+\t\ttest_cmp ver.expected ver.actual\n+\t)\n+'\n+\n+test_expect_success 'repack --pack-version=4' '\n+\tgit init pv4.2 &&\n+\t(\n+\t\tunset GIT_TEST_PACKV4 &&\n+\t\tcd pv4.2 &&\n+\t\ttest_commit one &&\n+\t\ttest_commit two &&\n+\t\ttest_commit three &&\n+\t\tgit repack -ad --pack-version=4 &&\n+\t\tP=`ls .git/objects/pack/pack-*.pack` &&\n+\t\t# Offset 4 is pack version\n+\t\ttest-dump ntohl \"$P\" 4 >ver.actual &&\n+\t\techo 4 >ver.expected &&\n+\t\ttest_cmp ver.expected ver.actual\n+\t)\n+'\n+\n test_done\n \n-- \n1.8.2.82.gc24b958\n"},{"id":"228255","messageId":"1380162409-18224-11-git-send-email-pclouds@gmail.com","threadId":"35023","inReplyTo":"1380162409-18224-1-git-send-email-pclouds@gmail.com","subject":"[PATCH 10/10] count-objects: report pack v4 usage","fromName":"Nguyễn Thái Ngọc Duy","fromEmail":"pclouds@gmail.com","sentAt":"2013-09-26T02:26:49Z","receivedAt":"2013-09-26T02:26:49Z","isPatch":true,"sender":{"key":"pclouds@gmail.com","avatar":"https://avatars.githubusercontent.com/u/720?v=4"},"body":"An ideal repository is v4 only, no v2 packs. Show pack v4 statistics\nso people know if the repository is mixed with v2 and repack it. Note\nthat \"in-pack\" and \"size-pack\" include all pack versions, not just v2.\n\nOnly display v4 info when there are v4 packs. It's still experimental\nso don't polute the output with v4 that's never used by 99.99% of\nusers. We can make it unconditional later when v4 is officially\nsupported and recommended.\n\nSigned-off-by: Nguyễn Thái Ngọc Duy \n---\n Documentation/git-count-objects.txt | 4 ++++\n builtin/count-objects.c | 23 ++++++++++++++++++++---\n 2 files changed, 24 insertions(+), 3 deletions(-)\n\ndiff --git a/Documentation/git-count-objects.txt b/Documentation/git-count-objects.txt\nindex b300e84..d15b4dc 100644\n--- a/Documentation/git-count-objects.txt\n+++ b/Documentation/git-count-objects.txt\n@@ -28,8 +28,12 @@ size: disk space consumed by loose objects, in KiB (unless -H is specified)\n +\n in-pack: the number of in-pack objects\n +\n+in-packv4: the number of objects in packs version 4\n++\n size-pack: disk space consumed by the packs, in KiB (unless -H is specified)\n +\n+size-packv4: disk space consumed by the packs version 4\n++\n prune-packable: the number of loose objects that are also present in\n the packs. These objects could be pruned using `git prune-packed`.\n +\ndiff --git a/builtin/count-objects.c b/builtin/count-objects.c\nindex a7f70cb..db73ce7 100644\n--- a/builtin/count-objects.c\n+++ b/builtin/count-objects.c\n@@ -89,7 +89,7 @@ int cmd_count_objects(int argc, const char **argv, const char *prefix)\n \tconst char *objdir = get_object_directory();\n \tint len = strlen(objdir);\n \tchar *path = xmalloc(len + 50);\n-\tunsigned long loose = 0, packed = 0, packed_loose = 0;\n+\tunsigned long loose = 0, packed_loose = 0;\n \toff_t loose_size = 0;\n \tstruct option opts[] = {\n \t\tOPT__VERBOSE(&verbose, N_(\"be verbose\")),\n@@ -119,10 +119,12 @@ int cmd_count_objects(int argc, const char **argv, const char *prefix)\n \t}\n \tif (verbose) {\n \t\tstruct packed_git *p;\n-\t\tunsigned long num_pack = 0;\n-\t\toff_t size_pack = 0;\n+\t\tunsigned long num_pack = 0, num_pack_v4 = 0;\n+\t\tunsigned long packed = 0, packed_v4 = 0;\n+\t\toff_t size_pack = 0, size_pack_v4 = 0;\n \t\tstruct strbuf loose_buf = STRBUF_INIT;\n \t\tstruct strbuf pack_buf = STRBUF_INIT;\n+\t\tstruct strbuf pack_buf_v4 = STRBUF_INIT;\n \t\tstruct strbuf garbage_buf = STRBUF_INIT;\n \t\tif (!packed_git)\n \t\t\tprepare_packed_git();\n@@ -134,17 +136,25 @@ int cmd_count_objects(int argc, const char **argv, const char *prefix)\n \t\t\tpacked += p->num_objects;\n \t\t\tsize_pack += p->pack_size + p->index_size;\n \t\t\tnum_pack++;\n+\t\t\tif (p->version == 4) {\n+\t\t\t\tpacked_v4 += p->num_objects;\n+\t\t\t\tsize_pack_v4 += p->pack_size + p->index_size;\n+\t\t\t\tnum_pack_v4++;\n+\t\t\t}\n \t\t}\n \n \t\tif (human_readable) {\n \t\t\tstrbuf_humanise_bytes(&loose_buf, loose_size);\n \t\t\tstrbuf_humanise_bytes(&pack_buf, size_pack);\n+\t\t\tstrbuf_humanise_bytes(&pack_buf_v4, size_pack_v4);\n \t\t\tstrbuf_humanise_bytes(&garbage_buf, size_garbage);\n \t\t} else {\n \t\t\tstrbuf_addf(&loose_buf, \"%lu\",\n \t\t\t\t (unsigned long)(loose_size / 1024));\n \t\t\tstrbuf_addf(&pack_buf, \"%lu\",\n \t\t\t\t (unsigned long)(size_pack / 1024));\n+\t\t\tstrbuf_addf(&pack_buf_v4, \"%lu\",\n+\t\t\t\t (unsigned long)(size_pack_v4 / 1024));\n \t\t\tstrbuf_addf(&garbage_buf, \"%lu\",\n \t\t\t\t (unsigned long)(size_garbage / 1024));\n \t\t}\n@@ -152,13 +162,20 @@ int cmd_count_objects(int argc, const char **argv, const char *prefix)\n \t\tprintf(\"count: %lu\\n\", loose);\n \t\tprintf(\"size: %s\\n\", loose_buf.buf);\n \t\tprintf(\"in-pack: %lu\\n\", packed);\n+\t\tif (num_pack_v4)\n+\t\t\tprintf(\"in-packv4: %lu\\n\", packed_v4);\n \t\tprintf(\"packs: %lu\\n\", num_pack);\n+\t\tif (num_pack_v4)\n+\t\t\tprintf(\"v4-packs: %lu\\n\", num_pack_v4);\n \t\tprintf(\"size-pack: %s\\n\", pack_buf.buf);\n+\t\tif (num_pack_v4)\n+\t\t\tprintf(\"size-packv4: %s\\n\", pack_buf_v4.buf);\n \t\tprintf(\"prune-packable: %lu\\n\", packed_loose);\n \t\tprintf(\"garbage: %lu\\n\", garbage);\n \t\tprintf(\"size-garbage: %s\\n\", garbage_buf.buf);\n \t\tstrbuf_release(&loose_buf);\n \t\tstrbuf_release(&pack_buf);\n+\t\tstrbuf_release(&pack_buf_v4);\n \t\tstrbuf_release(&garbage_buf);\n \t} else {\n \t\tstruct strbuf buf = STRBUF_INIT;\n-- \n1.8.2.82.gc24b958\n"},{"id":"228257","messageId":"alpine.LFD.2.03.1309260031290.312@syhkavp.arg","threadId":"35023","inReplyTo":"1380162409-18224-1-git-send-email-pclouds@gmail.com","subject":"Re: [PATCH 00/10] pack v4 UI support","fromName":"Nicolas Pitre","fromEmail":"nico@fluxnic.net","sentAt":"2013-09-26T04:51:34Z","receivedAt":"2013-09-26T04:51:34Z","isPatch":true,"sender":{"key":"nico@fluxnic.net","avatar":"https://avatars.githubusercontent.com/u/702790?v=4"},"body":"On Thu, 26 Sep 2013, Nguyễn Thái Ngọc Duy wrote:\n\n> About the \"packv4\" capability in git protocol. I considered a more\n> generic cap \"packver=ver[,ver[,ver...]]\" that represents all supported\n> versions, with the order of preference from the first ver (most preferred)\n> to the last (least preferred). But it adds more parsing code\n> and frankly I don't think we'll have pack v5 in the next five years.\n> So I dropped it.\n\nAgreed. In fact the receiver should only indicate the highest pack \nversion it is ready to accept. We'll have to support the sending and \nreceiving of any previous pack versions forever anyway.\n\n> Multi-base tree support is not part of \"packv4\" capability. Let's see\n> if such support comes before the series is merged to master. If so we\n> can drop that line from protocol-capabilities.txt. Otherwise a new\n> capability can be added for multi-base trees.\n\nWhat is that for? Multi-base trees are not created yet, but the code to \nparse them is already there. So I don't see the point of having a \ncapability in the protocol for this.\n\n> Another capability could be added for sending the actual number of\n> objects in a thin pack for more accurate display in index-pack. Low\n> priority in my opinion.\n\nThat just cannot be communicated during capability exchange. This \nnumber is known only after object enumeration. Hence my suggestion of a \n['T', 'H', 'I', 'N', htonl()] special header \nprepended to the actual pack on the wire. And that has to be decided \nbefore formalizing the pack v4 capability. That makes it a somewhat \nhigher priority.\n\n> There's also the pack conversion issue. Suppose the client requests v4\n> but, for performance purposes, the server sends v2. Should index-pack\n> convert the received pack to v4? My answer is no because there's no\n> way we can do it without saving v2 first. And if we have to save v2\n> anyway, pack-objects (or repack) will do the conversion better. We can\n> adjust git-gc to maybe repack more often when the number of v2 packs\n> is over a limit, or just repack v2 packs into one v4 pack.\n\nYeah... those are questions that certainly can wait. Only \nexperimentation will tell what is best, and that won't be conclusive \nuntil the core git code learns to parse pack v4 data natively.\n\nI'll try to review your patches soon.\n\n\nNicolas\n"},{"id":"228258","messageId":"CACsJy8De9mMvSpnzHuq+XG+Pfi5QD1fY44uHT8hNQZN6sU-hgg@mail.gmail.com","threadId":"35023","inReplyTo":"alpine.LFD.2.03.1309260031290.312@syhkavp.arg","subject":"Re: [PATCH 00/10] pack v4 UI support","fromName":"Duy Nguyen","fromEmail":"pclouds@gmail.com","sentAt":"2013-09-26T05:09:49Z","receivedAt":"2013-09-26T05:09:49Z","isPatch":true,"sender":{"key":"pclouds@gmail.com","avatar":"https://avatars.githubusercontent.com/u/720?v=4"},"body":"On Thu, Sep 26, 2013 at 11:51 AM, Nicolas Pitre wrote:\n>> Multi-base tree support is not part of \"packv4\" capability. Let's see\n>> if such support comes before the series is merged to master. If so we\n>> can drop that line from protocol-capabilities.txt. Otherwise a new\n>> capability can be added for multi-base trees.\n>\n> What is that for? Multi-base trees are not created yet, but the code to\n> parse them is already there. So I don't see the point of having a\n> capability in the protocol for this.\n\npv4_get_tree() can. index-pack cannot.\n\n>> Another capability could be added for sending the actual number of\n>> objects in a thin pack for more accurate display in index-pack. Low\n>> priority in my opinion.\n>\n> That just cannot be communicated during capability exchange. This\n> number is known only after object enumeration. Hence my suggestion of a\n> ['T', 'H', 'I', 'N', htonl()] special header\n> prepended to the actual pack on the wire. And that has to be decided\n> before formalizing the pack v4 capability. That makes it a somewhat\n> higher priority.\n\nThe capability is to let the server know the client understands ['T',\n'H', 'I', 'N' ..]. The server can choose not to send it later, but\nthat's ok. Hence the new capability. I'm somewhat reluctant to do it\nbecause of peeking code in fetch-pack and receive-pack and some\nrefactoring may be needed first. But I could certainly put it on\nhigher priority.\n-- \nDuy\n"},{"id":"228262","messageId":"1380184349-26380-1-git-send-email-stefanbeller@googlemail.com","threadId":"35023","inReplyTo":"1380162409-18224-10-git-send-email-pclouds@gmail.com","subject":"[PATCH] repack: Add --version parameter","fromName":"Stefan Beller","fromEmail":"stefanbeller@googlemail.com","sentAt":"2013-09-26T08:32:29Z","receivedAt":"2013-09-26T08:32:29Z","isPatch":true,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"This is just a direct translation of\nhttp://article.gmane.org/gmane.comp.version-control.git/235396\nSo I don't consider this is ready for inclusion.\n\nSome notes:\nWe need to have more error checking, repack shall be 0, 2 or 4 but nothing\nelse. If 0 is given, no argument is passed to pack-objects, in case of\n2 or 4 --version= is passed.\n\nDo we really want to call it \"--version\"? This parameter sounds so much\nlike questioning for the program version, similar to\n\tgit --version\n\t1.8.4\nSo I'd rather use \"--repack-version\".\n---\n builtin/repack.c | 8 ++++++++\n 1 file changed, 8 insertions(+)\n\ndiff --git a/builtin/repack.c b/builtin/repack.c\nindex 3e56614..fd05e9a 100644\n--- a/builtin/repack.c\n+++ b/builtin/repack.c\n@@ -9,6 +9,7 @@\n #include \"argv-array.h\"\n \n static int delta_base_offset = 1;\n+static int pack_version;\n static char *packdir, *packtmp;\n \n static const char *const git_repack_usage[] = {\n@@ -22,6 +23,9 @@ static int repack_config(const char *var, const char *value, void *cb)\n \t\tdelta_base_offset = git_config_bool(var, value);\n \t\treturn 0;\n \t}\n+\tif (!strcmp(var, \"core.preferredPackVersion\")) {\n+\t\tpack_version = git_config_int(var, value);\n+\t}\n \treturn git_default_config(var, value, cb);\n }\n \n@@ -165,6 +169,8 @@ int cmd_repack(int argc, const char **argv, const char *prefix)\n \t\t\t\tN_(\"limits the maximum delta depth\")),\n \t\tOPT_INTEGER(0, \"max-pack-size\", &max_pack_size,\n \t\t\t\tN_(\"maximum size of each packfile\")),\n+\t\tOPT_INTEGER(0, \"pack-version\", &pack_version,\n+\t\t\t\tN_(\"format version of the output pack\")),\n \t\tOPT_END()\n \t};\n \n@@ -220,6 +226,8 @@ int cmd_repack(int argc, const char **argv, const char *prefix)\n \t\targv_array_push(&cmd_args, \"--quiet\");\n \tif (delta_base_offset)\n \t\targv_array_push(&cmd_args, \"--delta-base-offset\");\n+\tif (pack_version)\n+\t\targv_array_pushf(&cmd_args, \"--version=%u\", pack_version);\n \n \targv_array_push(&cmd_args, packtmp);\n \n-- \n1.8.4.474.g128a96c\n"},{"id":"228271","messageId":"CAMP44s0E7=fupwfe=rm9yETmzdjFyH8vuyWY=Ce4wKg=g4-iJg@mail.gmail.com","threadId":"35023","inReplyTo":"1380184349-26380-1-git-send-email-stefanbeller@googlemail.com","subject":"Re: [PATCH] repack: Add --version parameter","fromName":"Felipe Contreras","fromEmail":"felipe.contreras@gmail.com","sentAt":"2013-09-26T10:17:42Z","receivedAt":"2013-09-26T10:17:42Z","isPatch":true,"sender":{"key":"felipe.contreras@gmail.com","avatar":"https://avatars.githubusercontent.com/u/8358?v=4"},"body":"On Thu, Sep 26, 2013 at 3:32 AM, Stefan Beller\n wrote:\n> This is just a direct translation of\n> http://article.gmane.org/gmane.comp.version-control.git/235396\n> So I don't consider this is ready for inclusion.\n>\n> Some notes:\n> We need to have more error checking, repack shall be 0, 2 or 4 but nothing\n> else. If 0 is given, no argument is passed to pack-objects, in case of\n> 2 or 4 --version= is passed.\n>\n> Do we really want to call it \"--version\"? This parameter sounds so much\n> like questioning for the program version, similar to\n> git --version\n> 1.8.4\n> So I'd rather use \"--repack-version\".\n> ---\n> builtin/repack.c | 8 ++++++++\n> 1 file changed, 8 insertions(+)\n>\n> diff --git a/builtin/repack.c b/builtin/repack.c\n> index 3e56614..fd05e9a 100644\n> --- a/builtin/repack.c\n> +++ b/builtin/repack.c\n> @@ -9,6 +9,7 @@\n> #include \"argv-array.h\"\n>\n> static int delta_base_offset = 1;\n> +static int pack_version;\n> static char *packdir, *packtmp;\n>\n> static const char *const git_repack_usage[] = {\n> @@ -22,6 +23,9 @@ static int repack_config(const char *var, const char *value, void *cb)\n> delta_base_offset = git_config_bool(var, value);\n> return 0;\n> }\n> + if (!strcmp(var, \"core.preferredPackVersion\")) {\n> + pack_version = git_config_int(var, value);\n> + }\n\nThe style is without braces if the block is a single line.\n\n-- \nFelipe Contreras\n"},{"id":"228275","messageId":"CACsJy8BjTQhA6rDWk6FboxU-b6d93Jn+ZNvQ0DJYXehgrpSmwg@mail.gmail.com","threadId":"35023","inReplyTo":"1380184349-26380-1-git-send-email-stefanbeller@googlemail.com","subject":"Re: [PATCH] repack: Add --version parameter","fromName":"Duy Nguyen","fromEmail":"pclouds@gmail.com","sentAt":"2013-09-26T11:42:10Z","receivedAt":"2013-09-26T11:42:10Z","isPatch":true,"sender":{"key":"pclouds@gmail.com","avatar":"https://avatars.githubusercontent.com/u/720?v=4"},"body":"On Thu, Sep 26, 2013 at 3:32 PM, Stefan Beller\n wrote:\n> This is just a direct translation of\n> http://article.gmane.org/gmane.comp.version-control.git/235396\n> So I don't consider this is ready for inclusion.\n>\n> Some notes:\n> We need to have more error checking, repack shall be 0, 2 or 4 but nothing\n> else. If 0 is given, no argument is passed to pack-objects, in case of\n> 2 or 4 --version= is passed.\n\nIt's not that bad. If you don't catch it, pack-objects will.\n\n>\n> Do we really want to call it \"--version\"? This parameter sounds so much\n> like questioning for the program version, similar to\n> git --version\n> 1.8.4\n> So I'd rather use \"--repack-version\".\n\nHmm.. I think it's \"git repack --pack-version\"? Or if you meant \"git\npack-objects --version\", I drop the \"pack-\" out because there's\nalready \"pack\" in \"pack-objects\". But I'm OK renaming --version to\n--pack-version too. Maybe later.\n\n> @@ -22,6 +23,9 @@ static int repack_config(const char *var, const char *value, void *cb)\n> delta_base_offset = git_config_bool(var, value);\n> return 0;\n> }\n> + if (!strcmp(var, \"core.preferredPackVersion\")) {\n> + pack_version = git_config_int(var, value);\n> + }\n> return git_default_config(var, value, cb);\n\nIn np/pack-v4 series (not the one on 'pu' yet) git_default_config will\ndo this and save the value in core_default_pack_format. So you don't\nneed to set it here.\n\n> @@ -220,6 +226,8 @@ int cmd_repack(int argc, const char **argv, const char *prefix)\n> argv_array_push(&cmd_args, \"--quiet\");\n> if (delta_base_offset)\n> argv_array_push(&cmd_args, \"--delta-base-offset\");\n> + if (pack_version)\n> + argv_array_pushf(&cmd_args, \"--version=%u\", pack_version);\n\nbut then you may need \"if (!pack_version) pack_version =\ncore_defaul_pack_format;\" before this \"if\".\n\n>\n> argv_array_push(&cmd_args, packtmp);\n-- \nDuy\n"},{"id":"228298","messageId":"alpine.LFD.2.03.1309262150190.312@syhkavp.arg","threadId":"35023","inReplyTo":"CACsJy8De9mMvSpnzHuq+XG+Pfi5QD1fY44uHT8hNQZN6sU-hgg@mail.gmail.com","subject":"Re: [PATCH 00/10] pack v4 UI support","fromName":"Nicolas Pitre","fromEmail":"nico@fluxnic.net","sentAt":"2013-09-27T02:59:08Z","receivedAt":"2013-09-27T02:59:08Z","isPatch":true,"sender":{"key":"nico@fluxnic.net","avatar":"https://avatars.githubusercontent.com/u/702790?v=4"},"body":"On Thu, 26 Sep 2013, Duy Nguyen wrote:\n\n> On Thu, Sep 26, 2013 at 11:51 AM, Nicolas Pitre wrote:\n> >> Multi-base tree support is not part of \"packv4\" capability. Let's see\n> >> if such support comes before the series is merged to master. If so we\n> >> can drop that line from protocol-capabilities.txt. Otherwise a new\n> >> capability can be added for multi-base trees.\n> >\n> > What is that for? Multi-base trees are not created yet, but the code to\n> > parse them is already there. So I don't see the point of having a\n> > capability in the protocol for this.\n> \n> pv4_get_tree() can. index-pack cannot.\n\nHmmm... right.\n\nI think this ought to be part of the \"packv4\" capability nevertheless. \nThis is not going into the official git branch right away so there is \nstill time to implement it.\n\n> >> Another capability could be added for sending the actual number of\n> >> objects in a thin pack for more accurate display in index-pack. Low\n> >> priority in my opinion.\n> >\n> > That just cannot be communicated during capability exchange. This\n> > number is known only after object enumeration. Hence my suggestion of a\n> > ['T', 'H', 'I', 'N', htonl()] special header\n> > prepended to the actual pack on the wire. And that has to be decided\n> > before formalizing the pack v4 capability. That makes it a somewhat\n> > higher priority.\n> \n> The capability is to let the server know the client understands ['T',\n> 'H', 'I', 'N' ..]. The server can choose not to send it later, but\n> that's ok. Hence the new capability. I'm somewhat reluctant to do it\n> because of peeking code in fetch-pack and receive-pack and some\n> refactoring may be needed first. But I could certainly put it on\n> higher priority.\n\nHere's what I'm suggesting as a start (untested):\n\ndiff --git a/builtin/receive-pack.c b/builtin/receive-pack.c\nindex e3eb5fc..04e5ae1 100644\n--- a/builtin/receive-pack.c\n+++ b/builtin/receive-pack.c\n@@ -792,9 +792,10 @@ static struct command *read_head_info(void)\n \treturn commands;\n }\n \n-static const char *parse_pack_header(struct pack_header *hdr)\n+static const char *parse_pack_header(struct pack_header *hdr,\n+\t\t\t\t struct thin_header *thin_hdr)\n {\n-\tswitch (read_pack_header(0, hdr)) {\n+\tswitch (read_pack_header(0, hdr, thin_hdr)) {\n \tcase PH_ERROR_EOF:\n \t\treturn \"eof before pack header was fully read\";\n \n@@ -817,23 +818,25 @@ static const char *pack_lockfile;\n static const char *unpack(int err_fd)\n {\n \tstruct pack_header hdr;\n+\tstruct thin_header thin_hdr;\n \tconst char *hdr_err;\n-\tchar hdr_arg[38];\n+\tchar hdr_arg[50];\n \tint fsck_objects = (receive_fsck_objects >= 0\n \t\t\t ? receive_fsck_objects\n \t\t\t : transfer_fsck_objects >= 0\n \t\t\t ? transfer_fsck_objects\n \t\t\t : 0);\n \n-\thdr_err = parse_pack_header(&hdr);\n+\thdr_err = parse_pack_header(&hdr, &thin_hdr);\n \tif (hdr_err) {\n \t\tif (err_fd > 0)\n \t\t\tclose(err_fd);\n \t\treturn hdr_err;\n \t}\n \tsnprintf(hdr_arg, sizeof(hdr_arg),\n-\t\t\t\"--pack_header=%\"PRIu32\",%\"PRIu32,\n-\t\t\tntohl(hdr.hdr_version), ntohl(hdr.hdr_entries));\n+\t\t\t\"--pack_header=%\"PRIu32\",%\"PRIu32\",%\"PRIu32,\n+\t\t\tntohl(hdr.hdr_version), ntohl(hdr.hdr_entries),\n+\t\t\tntohl(thin_hdr.hdr_entries));\n \n \tif (ntohl(hdr.hdr_entries) < unpack_limit) {\n \t\tint code, i = 0;\ndiff --git a/fetch-pack.c b/fetch-pack.c\nindex 6684348..d86e5d1 100644\n--- a/fetch-pack.c\n+++ b/fetch-pack.c\n@@ -715,12 +715,14 @@ static int get_pack(struct fetch_pack_args *args,\n \t*hdr_arg = 0;\n \tif (!args->keep_pack && unpack_limit) {\n \t\tstruct pack_header header;\n+\t\tstruct thin_header thin_hdr;\n \n-\t\tif (read_pack_header(demux.out, &header))\n+\t\tif (read_pack_header(demux.out, &header, &thin_hdr))\n \t\t\tdie(\"protocol error: bad pack header\");\n \t\tsnprintf(hdr_arg, sizeof(hdr_arg),\n-\t\t\t \"--pack_header=%\"PRIu32\",%\"PRIu32,\n-\t\t\t ntohl(header.hdr_version), ntohl(header.hdr_entries));\n+\t\t\t \"--pack_header=%\"PRIu32\",%\"PRIu32\",%\"PRIu32,\n+\t\t\t ntohl(header.hdr_version), ntohl(header.hdr_entries),\n+\t\t\t ntohl(thin_hdr.hdr_entries));\n \t\tif (ntohl(header.hdr_entries) < unpack_limit)\n \t\t\tdo_keep = 0;\n \t\telse\ndiff --git a/pack.h b/pack.h\nindex ccefdbe..974b860 100644\n--- a/pack.h\n+++ b/pack.h\n@@ -16,6 +16,18 @@ struct pack_header {\n };\n \n /*\n+ * Pack v4 header prefix for thin packs, indicating the actual number\n+ * of objects being transmitted. Expected before the actual pack header\n+ * on the wire and not stored on disk upon reception. This is not\n+ * included in the computation of the pack trailing SHA1.\n+ */\n+#define THIN_SIGNATURE 0x5448494e\t/* \"THIN\" */\n+struct thin_header {\n+\tuint32_t hdr_signature;\n+\tuint32_t hdr_entries;\n+};\n+\n+/*\n * The first four bytes of index formats later than version 1 should\n * start with this signature, as all older git binaries would find this\n * value illegal and abort reading the file.\n@@ -88,7 +100,7 @@ extern int pv4_encode_object_header(enum object_type, uintmax_t, unsigned char *\n #define PH_ERROR_EOF\t\t(-1)\n #define PH_ERROR_PACK_SIGNATURE\t(-2)\n #define PH_ERROR_PROTOCOL\t(-3)\n-extern int read_pack_header(int fd, struct pack_header *);\n+extern int read_pack_header(int fd, struct pack_header *, struct thin_header *);\n \n extern struct sha1file *create_tmp_packfile(char **pack_tmp_name);\n extern void finish_tmp_packfile(char *name_buffer, const char *pack_tmp_name, struct pack_idx_entry **written_list, uint32_t nr_written, struct pack_idx_option *pack_idx_opts, unsigned char sha1[]);\ndiff --git a/sha1_file.c b/sha1_file.c\nindex ef6ecc8..5f00c15 100644\n--- a/sha1_file.c\n+++ b/sha1_file.c\n@@ -3180,12 +3180,26 @@ int index_path(unsigned char *sha1, const char *path, struct stat *st, unsigned\n \treturn 0;\n }\n \n-int read_pack_header(int fd, struct pack_header *header)\n+int read_pack_header(int fd, struct pack_header *header,\n+\t\t struct thin_header *thin_hdr)\n {\n \tif (read_in_full(fd, header, sizeof(*header)) < sizeof(*header))\n \t\t/* \"eof before pack header was fully read\" */\n \t\treturn PH_ERROR_EOF;\n \n+\tif (header->hdr_signature == htonl(THIN_SIGNATURE)) {\n+\t\tchar *thin_hdr_end = (char *)header + sizeof(*thin_hdr);\n+\t\tint leftover = sizeof(*header) - sizeof(*thin_hdr);\n+\t\tif (!thin_hdr)\n+\t\t\treturn PH_ERROR_PROTOCOL;\n+\t\tmemcpy(thin_hdr, header, sizeof(*thin_hdr));\n+\t\tmemcpy(header, thin_hdr_end, leftover);\n+\t\tif (read_in_full(fd, thin_hdr_end, sizeof(*header) - leftover)\n+\t\t\t\t< sizeof(*header) - leftover)\n+\t\t\treturn PH_ERROR_EOF;\n+\t} else if (thin_hdr)\n+\t\tmemset(thin_hdr, 0, sizeof(*thin_hdr));\n+\n \tif (header->hdr_signature != htonl(PACK_SIGNATURE))\n \t\t/* \"protocol error (pack signature mismatch detected)\" */\n \t\treturn PH_ERROR_PACK_SIGNATURE;\n\n\nNicolas\n"},{"id":"228356","messageId":"524698FF.4020602@googlemail.com","threadId":"35023","inReplyTo":"CAMP44s0E7=fupwfe=rm9yETmzdjFyH8vuyWY=Ce4wKg=g4-iJg@mail.gmail.com","subject":"Re: [PATCH] repack: Add --version parameter","fromName":"Stefan Beller","fromEmail":"stefanbeller@googlemail.com","sentAt":"2013-09-28T08:53:19Z","receivedAt":"2013-09-28T08:53:19Z","isPatch":true,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"On 09/26/2013 12:17 PM, Felipe Contreras wrote:\n> On Thu, Sep 26, 2013 at 3:32 AM, Stefan Beller\n\n>> static const char *const git_repack_usage[] = {\n>> @@ -22,6 +23,9 @@ static int repack_config(const char *var, const char *value, void *cb)\n>> delta_base_offset = git_config_bool(var, value);\n>> return 0;\n>> }\n>> + if (!strcmp(var, \"core.preferredPackVersion\")) {\n>> + pack_version = git_config_int(var, value);\n>> + }\n> \n> The style is without braces if the block is a single line.\n> \n\nThanks for spotting, I should have put a \"return 0;\" there.\n\nStefan\n"},{"id":"228357","messageId":"5246994C.3020709@googlemail.com","threadId":"35023","inReplyTo":"CACsJy8BjTQhA6rDWk6FboxU-b6d93Jn+ZNvQ0DJYXehgrpSmwg@mail.gmail.com","subject":"Re: [PATCH] repack: Add --version parameter","fromName":"Stefan Beller","fromEmail":"stefanbeller@googlemail.com","sentAt":"2013-09-28T08:54:36Z","receivedAt":"2013-09-28T08:54:36Z","isPatch":true,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"On 09/26/2013 01:42 PM, Duy Nguyen wrote:\n> On Thu, Sep 26, 2013 at 3:32 PM, Stefan Beller\n> wrote:\n>> This is just a direct translation of\n>> http://article.gmane.org/gmane.comp.version-control.git/235396\n>> So I don't consider this is ready for inclusion.\n>>\n>> Some notes:\n>> We need to have more error checking, repack shall be 0, 2 or 4 but\nnothing\n>> else. If 0 is given, no argument is passed to pack-objects, in case of\n>> 2 or 4 --version= is passed.\n>\n> It's not that bad. If you don't catch it, pack-objects will.\n\nOk, noted.\n\n>\n>>\n>> Do we really want to call it \"--version\"? This parameter sounds so much\n>> like questioning for the program version, similar to\n>> git --version\n>> 1.8.4\n>> So I'd rather use \"--repack-version\".\n>\n> Hmm.. I think it's \"git repack --pack-version\"? Or if you meant \"git\n> pack-objects --version\", I drop the \"pack-\" out because there's\n> already \"pack\" in \"pack-objects\". But I'm OK renaming --version to\n> --pack-version too. Maybe later.\n>\n>> @@ -22,6 +23,9 @@ static int repack_config(const char *var, const\nchar *value, void *cb)\n>> delta_base_offset = git_config_bool(var, value);\n>> return 0;\n>> }\n>> + if (!strcmp(var, \"core.preferredPackVersion\")) {\n>> + pack_version = git_config_int(var, value);\n>> + }\n>> return git_default_config(var, value, cb);\n>\n> In np/pack-v4 series (not the one on 'pu' yet) git_default_config will\n> do this and save the value in core_default_pack_format. So you don't\n> need to set it here.\n>\n>> @@ -220,6 +226,8 @@ int cmd_repack(int argc, const char **argv, const\nchar *prefix)\n>> argv_array_push(&cmd_args, \"--quiet\");\n>> if (delta_base_offset)\n>> argv_array_push(&cmd_args, \"--delta-base-offset\");\n>> + if (pack_version)\n>> + argv_array_pushf(&cmd_args, \"--version=%u\",\npack_version);\n>\n> but then you may need \"if (!pack_version) pack_version =\n> core_defaul_pack_format;\" before this \"if\".\n\nThe reason I put the pack_version not here is for structural clarity.\n(\"All config is done in either the parse_options section or in the\nrepack_config function\"). This may help having a the actual core logic\neasier and more understandable?\nIf you feel otherwise, I'd change it to your proposal.\n\nThanks,\nStefan\n"},{"id":"229124","messageId":"xmqqzjq724hu.fsf@gitster.dls.corp.google.com","threadId":"35023","inReplyTo":"1380162409-18224-8-git-send-email-pclouds@gmail.com","subject":"Re: [PATCH 07/10] receive-pack: request for packv4 if it's the preferred version","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2013-10-17T17:26:21Z","receivedAt":"2013-10-17T17:26:21Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Nguyễn Thái Ngọc Duy writes:\n\n> This is the only plumbing command that is controlled by\n> core.preferredPackVersion so far.\n>\n> Signed-off-by: Nguyễn Thái Ngọc Duy \n> ---\n> Documentation/technical/protocol-capabilities.txt | 4 ++++\n> builtin/receive-pack.c | 3 ++-\n> 2 files changed, 6 insertions(+), 1 deletion(-)\n>\n> diff --git a/Documentation/technical/protocol-capabilities.txt b/Documentation/technical/protocol-capabilities.txt\n> index be09792..32153cd 100644\n> --- a/Documentation/technical/protocol-capabilities.txt\n> +++ b/Documentation/technical/protocol-capabilities.txt\n> @@ -226,4 +226,8 @@ this capability, the server may send a pack version 4. The server can\n> choose to send pack version 2 even if the client accepts this\n> capability.\n> \n> +The receive-pack server advertises this capability if it wants to\n> +receive the pack in format version 4 and the client should send in\n> +this format.\n\nTechnically, \"if it can and if it wants to receive\" is more correct,\nas a v4 capable receiving end can choose to pretend it does not\nunderstand v4 by not sending this capability. Also a v4 incapable\nreceiver would not advertise it even if it _wants_ to receive. So\nin practice, we see this header only from a receiver that wants to\nreceive v4, which makes the above statement accurate in a twisted\nway.\n\nThere needs a bit more explanation on the \"should\" part, especially\nbecause this is very unusual and unlike all the other capabilities,\nwhich are offered as more freedom of choices without preference on\nthe advertising side. Rationale (i.e. reduce load on the receiving\nend) and ramifications of non-compliance (e.g. the receiver may\nchoose to fail the push when its load is too high) are good things\nto guide third-party implementors to do the right thing.\n\n> This capability does not include multi-base tree support.\n> diff --git a/builtin/receive-pack.c b/builtin/receive-pack.c\n> index e3eb5fc..288b0bc 100644\n> --- a/builtin/receive-pack.c\n> +++ b/builtin/receive-pack.c\n> @@ -130,10 +130,11 @@ static void show_ref(const char *path, const unsigned char *sha1)\n> \tif (sent_capabilities)\n> \t\tpacket_write(1, \"%s %s\\n\", sha1_to_hex(sha1), path);\n> \telse\n> -\t\tpacket_write(1, \"%s %s%c%s%s agent=%s\\n\",\n> +\t\tpacket_write(1, \"%s %s%c%s%s%s agent=%s\\n\",\n> \t\t\t sha1_to_hex(sha1), path, 0,\n> \t\t\t \" report-status delete-refs side-band-64k quiet\",\n> \t\t\t prefer_ofs_delta ? \" ofs-delta\" : \"\",\n> +\t\t\t core_default_pack_version == 4 ? \" packv4\" : \"\",\n> \t\t\t git_user_agent_sanitized());\n> \tsent_capabilities = 1;\n> }\n"}]}