{"thread":{"id":"38233","subject":"[PATCH v3 3/4] pack-objects: use --objects-edge-aggressive only for shallow repos","startedAt":"2014-12-23T12:01:18Z","lastAt":"2014-12-24T21:31:40Z","messageCount":11,"participants":["brian m. carlson","Michael Blume","Junio C Hamano","Jeff King"],"isPatch":true,"patchVersion":3,"patchTotal":4},"messages":[{"id":"254003","messageId":"1419336082-283091-1-git-send-email-sandals@crustytoothpaste.net","threadId":"38233","inReplyTo":null,"subject":"[PATCH v3 0/4] Improve push performance with lots of refs","fromName":"brian m. carlson","fromEmail":"sandals@crustytoothpaste.net","sentAt":"2014-12-23T12:01:18Z","receivedAt":"2014-12-23T12:01:18Z","isPatch":true,"sender":{"key":"sandals@crustytoothpaste.net","avatar":"https://avatars.githubusercontent.com/u/497054?v=4"},"body":"This series contains patches to address a significant push performance\nregression in repositories with large amounts of refs. It avoids\nperforming expensive edge marking unless the repository is shallow.\n\nThe first patch in the series is a fix for a minor typo I discovered\nwhen editing the documentation. The second patch implements git\nrev-list --objects-edge-aggressive, and the third patch ensures it's\nused for shallow repos only. The final patch ensures that sending packs\nto shallow clients use --object-edge-aggressive as well.\n\nThe only change from v2 is the addition of a fourth patch, which fixes\nt5500. It's necessary because the test wants packs for fetches to\nshallow clones to be minimal.\n\nI'm not especially thrilled with having to provide a --shallow command\nline argument, but the alternative is to buffer a potentially large\namount of data in order to determine whether the remote side is shallow.\n\nThe original fix was suggested by Duy Nguyen.\n\nbrian m. carlson (4):\n Documentation: add missing article in rev-list-options.txt\n rev-list: add an option to mark fewer edges as uninteresting\n pack-objects: use --objects-edge-aggressive only for shallow repos\n upload-pack: use --objects-edge-aggressive for shallow fetches\n\n Documentation/git-pack-objects.txt | 7 ++++++-\n Documentation/git-rev-list.txt | 3 ++-\n Documentation/rev-list-options.txt | 7 ++++++-\n builtin/pack-objects.c | 7 ++++++-\n list-objects.c | 4 ++--\n revision.c | 6 ++++++\n revision.h | 1 +\n upload-pack.c | 4 +++-\n 8 files changed, 32 insertions(+), 7 deletions(-)\n\n-- \n2.2.1.209.g41e5f3a\n"},{"id":"254000","messageId":"1419336082-283091-2-git-send-email-sandals@crustytoothpaste.net","threadId":"38233","inReplyTo":"1419336082-283091-1-git-send-email-sandals@crustytoothpaste.net","subject":"[PATCH v3 1/4] Documentation: add missing article in rev-list-options.txt","fromName":"brian m. carlson","fromEmail":"sandals@crustytoothpaste.net","sentAt":"2014-12-23T12:01:19Z","receivedAt":"2014-12-23T12:01:19Z","isPatch":true,"sender":{"key":"sandals@crustytoothpaste.net","avatar":"https://avatars.githubusercontent.com/u/497054?v=4"},"body":"Add the missing article \"a\".\n\nSigned-off-by: brian m. carlson \n---\n Documentation/rev-list-options.txt | 2 +-\n 1 file changed, 1 insertion(+), 1 deletion(-)\n\ndiff --git a/Documentation/rev-list-options.txt b/Documentation/rev-list-options.txt\nindex afccfdc..2277fcb 100644\n--- a/Documentation/rev-list-options.txt\n+++ b/Documentation/rev-list-options.txt\n@@ -653,7 +653,7 @@ These options are mostly targeted for packing of Git repositories.\n --objects-edge::\n \tSimilar to `--objects`, but also print the IDs of excluded\n \tcommits prefixed with a ``-'' character. This is used by\n-\tlinkgit:git-pack-objects[1] to build ``thin'' pack, which records\n+\tlinkgit:git-pack-objects[1] to build a ``thin'' pack, which records\n \tobjects in deltified form based on objects contained in these\n \texcluded commits to reduce network traffic.\n \n-- \n2.2.1.209.g41e5f3a\n"},{"id":"254002","messageId":"1419336082-283091-3-git-send-email-sandals@crustytoothpaste.net","threadId":"38233","inReplyTo":"1419336082-283091-1-git-send-email-sandals@crustytoothpaste.net","subject":"[PATCH v3 2/4] rev-list: add an option to mark fewer edges as uninteresting","fromName":"brian m. carlson","fromEmail":"sandals@crustytoothpaste.net","sentAt":"2014-12-23T12:01:20Z","receivedAt":"2014-12-23T12:01:20Z","isPatch":true,"sender":{"key":"sandals@crustytoothpaste.net","avatar":"https://avatars.githubusercontent.com/u/497054?v=4"},"body":"In commit fbd4a70 (list-objects: mark more commits as edges in\nmark_edges_uninteresting - 2013-08-16), we marked an increasing number\nof edges uninteresting. This change, and the subsequent change to make\nthis conditional on --objects-edge, are used by --thin to make much\nsmaller packs for shallow clones.\n\nUnfortunately, they cause a significant performance regression when\npushing non-shallow clones with lots of refs (23.322 seconds vs.\n4.785 seconds with 22400 refs). Add an option to git rev-list,\n--objects-edge-aggressive, that preserves this more aggressive behavior,\nwhile leaving --objects-edge to provide more performant behavior.\n\nSigned-off-by: brian m. carlson \n---\n Documentation/git-rev-list.txt | 3 ++-\n Documentation/rev-list-options.txt | 4 ++++\n list-objects.c | 4 ++--\n revision.c | 6 ++++++\n revision.h | 1 +\n 5 files changed, 15 insertions(+), 3 deletions(-)\n\ndiff --git a/Documentation/git-rev-list.txt b/Documentation/git-rev-list.txt\nindex fd7f8b5..5b11922 100644\n--- a/Documentation/git-rev-list.txt\n+++ b/Documentation/git-rev-list.txt\n@@ -46,7 +46,8 @@ SYNOPSIS\n \t [ \\--extended-regexp | -E ]\n \t [ \\--fixed-strings | -F ]\n \t [ \\--date=(local|relative|default|iso|iso-strict|rfc|short) ]\n-\t [ [\\--objects | \\--objects-edge] [ \\--unpacked ] ]\n+\t [ [ \\--objects | \\--objects-edge | \\--objects-edge-aggressive ]\n+\t [ \\--unpacked ] ]\n \t [ \\--pretty | \\--header ]\n \t [ \\--bisect ]\n \t [ \\--bisect-vars ]\ndiff --git a/Documentation/rev-list-options.txt b/Documentation/rev-list-options.txt\nindex 2277fcb..8cb6f92 100644\n--- a/Documentation/rev-list-options.txt\n+++ b/Documentation/rev-list-options.txt\n@@ -657,6 +657,10 @@ These options are mostly targeted for packing of Git repositories.\n \tobjects in deltified form based on objects contained in these\n \texcluded commits to reduce network traffic.\n \n+--objects-edge-aggressive::\n+\tSimilar to `--objects-edge`, but it tries harder to find excluded\n+\tcommits at the cost of increased time.\n+\n --unpacked::\n \tOnly useful with `--objects`; print the object IDs that are not\n \tin packs.\ndiff --git a/list-objects.c b/list-objects.c\nindex 2910bec..2a139b6 100644\n--- a/list-objects.c\n+++ b/list-objects.c\n@@ -157,7 +157,7 @@ void mark_edges_uninteresting(struct rev_info *revs, show_edge_fn show_edge)\n \n \t\tif (commit->object.flags & UNINTERESTING) {\n \t\t\tmark_tree_uninteresting(commit->tree);\n-\t\t\tif (revs->edge_hint && !(commit->object.flags & SHOWN)) {\n+\t\t\tif (revs->edge_hint_aggressive && !(commit->object.flags & SHOWN)) {\n \t\t\t\tcommit->object.flags |= SHOWN;\n \t\t\t\tshow_edge(commit);\n \t\t\t}\n@@ -165,7 +165,7 @@ void mark_edges_uninteresting(struct rev_info *revs, show_edge_fn show_edge)\n \t\t}\n \t\tmark_edge_parents_uninteresting(commit, revs, show_edge);\n \t}\n-\tif (revs->edge_hint) {\n+\tif (revs->edge_hint_aggressive) {\n \t\tfor (i = 0; i < revs->cmdline.nr; i++) {\n \t\t\tstruct object *obj = revs->cmdline.rev[i].item;\n \t\t\tstruct commit *commit = (struct commit *)obj;\ndiff --git a/revision.c b/revision.c\nindex 75dda92..753dd2f 100644\n--- a/revision.c\n+++ b/revision.c\n@@ -1853,6 +1853,12 @@ static int handle_revision_opt(struct rev_info *revs, int argc, const char **arg\n \t\trevs->tree_objects = 1;\n \t\trevs->blob_objects = 1;\n \t\trevs->edge_hint = 1;\n+\t} else if (!strcmp(arg, \"--objects-edge-aggressive\")) {\n+\t\trevs->tag_objects = 1;\n+\t\trevs->tree_objects = 1;\n+\t\trevs->blob_objects = 1;\n+\t\trevs->edge_hint = 1;\n+\t\trevs->edge_hint_aggressive = 1;\n \t} else if (!strcmp(arg, \"--verify-objects\")) {\n \t\trevs->tag_objects = 1;\n \t\trevs->tree_objects = 1;\ndiff --git a/revision.h b/revision.h\nindex 9cb5adc..033a244 100644\n--- a/revision.h\n+++ b/revision.h\n@@ -93,6 +93,7 @@ struct rev_info {\n \t\t\tblob_objects:1,\n \t\t\tverify_objects:1,\n \t\t\tedge_hint:1,\n+\t\t\tedge_hint_aggressive:1,\n \t\t\tlimited:1,\n \t\t\tunpacked:1,\n \t\t\tboundary:2,\n-- \n2.2.1.209.g41e5f3a\n"},{"id":"253999","messageId":"1419336082-283091-4-git-send-email-sandals@crustytoothpaste.net","threadId":"38233","inReplyTo":"1419336082-283091-1-git-send-email-sandals@crustytoothpaste.net","subject":"[PATCH v3 3/4] pack-objects: use --objects-edge-aggressive only for shallow repos","fromName":"brian m. carlson","fromEmail":"sandals@crustytoothpaste.net","sentAt":"2014-12-23T12:01:21Z","receivedAt":"2014-12-23T12:01:21Z","isPatch":true,"sender":{"key":"sandals@crustytoothpaste.net","avatar":"https://avatars.githubusercontent.com/u/497054?v=4"},"body":"Using --objects-edge-aggressive is important for shallow repos, as it\ncan result in a much smaller pack (in some cases, 10% of the size).\nHowever, it performs poorly on large non-shallow repositories with many\nrefs. Since shallow repositories are less likely to have many refs (due\nto having less history), the smaller pack size is advantageous there.\nAdjust pack-objects to use --objects-edge-aggressive only for shallow\nrepositories.\n\nSigned-off-by: brian m. carlson \n---\n Documentation/rev-list-options.txt | 3 ++-\n builtin/pack-objects.c | 4 +++-\n 2 files changed, 5 insertions(+), 2 deletions(-)\n\ndiff --git a/Documentation/rev-list-options.txt b/Documentation/rev-list-options.txt\nindex 8cb6f92..2984f40 100644\n--- a/Documentation/rev-list-options.txt\n+++ b/Documentation/rev-list-options.txt\n@@ -659,7 +659,8 @@ These options are mostly targeted for packing of Git repositories.\n \n --objects-edge-aggressive::\n \tSimilar to `--objects-edge`, but it tries harder to find excluded\n-\tcommits at the cost of increased time.\n+\tcommits at the cost of increased time. This is used instead of\n+\t`--objects-edge` to build ``thin'' packs for shallow repositories.\n \n --unpacked::\n \tOnly useful with `--objects`; print the object IDs that are not\ndiff --git a/builtin/pack-objects.c b/builtin/pack-objects.c\nindex 3f9f5c7..f3ba861 100644\n--- a/builtin/pack-objects.c\n+++ b/builtin/pack-objects.c\n@@ -2711,7 +2711,9 @@ int cmd_pack_objects(int argc, const char **argv, const char *prefix)\n \targv_array_push(&rp, \"pack-objects\");\n \tif (thin) {\n \t\tuse_internal_rev_list = 1;\n-\t\targv_array_push(&rp, \"--objects-edge\");\n+\t\targv_array_push(&rp, is_repository_shallow()\n+\t\t\t\t? \"--objects-edge-aggressive\"\n+\t\t\t\t: \"--objects-edge\");\n \t} else\n \t\targv_array_push(&rp, \"--objects\");\n \n-- \n2.2.1.209.g41e5f3a\n"},{"id":"254001","messageId":"1419336082-283091-5-git-send-email-sandals@crustytoothpaste.net","threadId":"38233","inReplyTo":"1419336082-283091-1-git-send-email-sandals@crustytoothpaste.net","subject":"[PATCH v3 4/4] upload-pack: use --objects-edge-aggressive for shallow fetches","fromName":"brian m. carlson","fromEmail":"sandals@crustytoothpaste.net","sentAt":"2014-12-23T12:01:22Z","receivedAt":"2014-12-23T12:01:22Z","isPatch":true,"sender":{"key":"sandals@crustytoothpaste.net","avatar":"https://avatars.githubusercontent.com/u/497054?v=4"},"body":"When fetching into a shallow repository, we want to aggressively mark\nedges as uninteresting, since this decreases the pack size. However,\nis_shallow_repository() returns false on the server side, since the\nserver is not shallow.\n\nAs the server, we get an indication of whether the client is shallow\nbased on stdin; however, by the time we get to parsing stdin, we already\nwill have called setup_revisions and won't be able to change our\ndecision anymore. Teach pack-objects a --shallow option to indicate\nthat the remote side is shallow and use it in upload-pack.\n\nSigned-off-by: brian m. carlson \n---\n Documentation/git-pack-objects.txt | 7 ++++++-\n builtin/pack-objects.c | 5 ++++-\n upload-pack.c | 4 +++-\n 3 files changed, 13 insertions(+), 3 deletions(-)\n\ndiff --git a/Documentation/git-pack-objects.txt b/Documentation/git-pack-objects.txt\nindex d2d8f47..c2f76fb 100644\n--- a/Documentation/git-pack-objects.txt\n+++ b/Documentation/git-pack-objects.txt\n@@ -13,7 +13,7 @@ SYNOPSIS\n \t[--no-reuse-delta] [--delta-base-offset] [--non-empty]\n \t[--local] [--incremental] [--window=] [--depth=]\n \t[--revs [--unpacked | --all]] [--stdout | base-name]\n-\t[--keep-true-parents] < object-list\n+\t[--shallow] [--keep-true-parents] < object-list\n \n \n DESCRIPTION\n@@ -190,6 +190,11 @@ required objects and is thus unusable by Git without making it\n self-contained. Use `git index-pack --fix-thin`\n (see linkgit:git-index-pack[1]) to restore the self-contained property.\n \n+--shallow::\n+\tOptimize a pack that will be provided to a client with a shallow\n+\trepository. This option, combined with \\--thin, can result in a\n+\tsmaller pack at the cost of speed.\n+\n --delta-base-offset::\n \tA packed archive can express the base object of a delta as\n \teither a 20-byte object name or as an offset in the\ndiff --git a/builtin/pack-objects.c b/builtin/pack-objects.c\nindex f3ba861..6bfbd3d 100644\n--- a/builtin/pack-objects.c\n+++ b/builtin/pack-objects.c\n@@ -2613,6 +2613,7 @@ int cmd_pack_objects(int argc, const char **argv, const char *prefix)\n {\n \tint use_internal_rev_list = 0;\n \tint thin = 0;\n+\tint shallow = 0;\n \tint all_progress_implied = 0;\n \tstruct argv_array rp = ARGV_ARRAY_INIT;\n \tint rev_list_unpacked = 0, rev_list_all = 0, rev_list_reflog = 0;\n@@ -2677,6 +2678,8 @@ int cmd_pack_objects(int argc, const char **argv, const char *prefix)\n \t\t PARSE_OPT_OPTARG, option_parse_unpack_unreachable },\n \t\tOPT_BOOL(0, \"thin\", &thin,\n \t\t\t N_(\"create thin packs\")),\n+\t\tOPT_BOOL(0, \"shallow\", &shallow,\n+\t\t\t N_(\"create packs suitable for shallow fetches\")),\n \t\tOPT_BOOL(0, \"honor-pack-keep\", &ignore_packed_keep,\n \t\t\t N_(\"ignore packs that have companion .keep file\")),\n \t\tOPT_INTEGER(0, \"compression\", &pack_compression_level,\n@@ -2711,7 +2714,7 @@ int cmd_pack_objects(int argc, const char **argv, const char *prefix)\n \targv_array_push(&rp, \"pack-objects\");\n \tif (thin) {\n \t\tuse_internal_rev_list = 1;\n-\t\targv_array_push(&rp, is_repository_shallow()\n+\t\targv_array_push(&rp, is_repository_shallow() || shallow\n \t\t\t\t? \"--objects-edge-aggressive\"\n \t\t\t\t: \"--objects-edge\");\n \t} else\ndiff --git a/upload-pack.c b/upload-pack.c\nindex ac9ac15..b531a32 100644\n--- a/upload-pack.c\n+++ b/upload-pack.c\n@@ -86,7 +86,7 @@ static void create_pack_file(void)\n \t\t\"corruption on the remote side.\";\n \tint buffered = -1;\n \tssize_t sz;\n-\tconst char *argv[12];\n+\tconst char *argv[13];\n \tint i, arg = 0;\n \tFILE *pipe_fd;\n \n@@ -100,6 +100,8 @@ static void create_pack_file(void)\n \t\targv[arg++] = \"--thin\";\n \n \targv[arg++] = \"--stdout\";\n+\tif (shallow_nr)\n+\t\targv[arg++] = \"--shallow\";\n \tif (!no_progress)\n \t\targv[arg++] = \"--progress\";\n \tif (use_ofs_delta)\n-- \n2.2.1.209.g41e5f3a\n"},{"id":"254026","messageId":"CAO2U3Qjbmz+fP-SLSeq1S+BDi4PSTSLf_TdqW-ik8GLV7=nUmA@mail.gmail.com","threadId":"38233","inReplyTo":"1419336082-283091-3-git-send-email-sandals@crustytoothpaste.net","subject":"Re: [PATCH v3 2/4] rev-list: add an option to mark fewer edges as uninteresting","fromName":"Michael Blume","fromEmail":"blume.mike@gmail.com","sentAt":"2014-12-23T17:55:48Z","receivedAt":"2014-12-23T17:55:48Z","isPatch":true,"sender":{"key":"blume.mike@gmail.com","avatar":"https://gravatar.com/avatar/1a7b440e1d942425ff4098ac7fc15b86b30cecaa56e1692a7ef8b5939ba25ea7?d=mp&s=160"},"body":"This patch causes an error on my mac, test 5500 fetch-pack errors on\npart 44 - fetch creating new shallow root. It looks for \"remote: Total\n1\" in the fetch output and gets 3 instead.\n\nOn Tue, Dec 23, 2014 at 7:01 AM, brian m. carlson\n wrote:\n> In commit fbd4a70 (list-objects: mark more commits as edges in\n> mark_edges_uninteresting - 2013-08-16), we marked an increasing number\n> of edges uninteresting. This change, and the subsequent change to make\n> this conditional on --objects-edge, are used by --thin to make much\n> smaller packs for shallow clones.\n>\n> Unfortunately, they cause a significant performance regression when\n> pushing non-shallow clones with lots of refs (23.322 seconds vs.\n> 4.785 seconds with 22400 refs). Add an option to git rev-list,\n> --objects-edge-aggressive, that preserves this more aggressive behavior,\n> while leaving --objects-edge to provide more performant behavior.\n>\n> Signed-off-by: brian m. carlson \n> ---\n> Documentation/git-rev-list.txt | 3 ++-\n> Documentation/rev-list-options.txt | 4 ++++\n> list-objects.c | 4 ++--\n> revision.c | 6 ++++++\n> revision.h | 1 +\n> 5 files changed, 15 insertions(+), 3 deletions(-)\n>\n> diff --git a/Documentation/git-rev-list.txt b/Documentation/git-rev-list.txt\n> index fd7f8b5..5b11922 100644\n> --- a/Documentation/git-rev-list.txt\n> +++ b/Documentation/git-rev-list.txt\n> @@ -46,7 +46,8 @@ SYNOPSIS\n> [ \\--extended-regexp | -E ]\n> [ \\--fixed-strings | -F ]\n> [ \\--date=(local|relative|default|iso|iso-strict|rfc|short) ]\n> - [ [\\--objects | \\--objects-edge] [ \\--unpacked ] ]\n> + [ [ \\--objects | \\--objects-edge | \\--objects-edge-aggressive ]\n> + [ \\--unpacked ] ]\n> [ \\--pretty | \\--header ]\n> [ \\--bisect ]\n> [ \\--bisect-vars ]\n> diff --git a/Documentation/rev-list-options.txt b/Documentation/rev-list-options.txt\n> index 2277fcb..8cb6f92 100644\n> --- a/Documentation/rev-list-options.txt\n> +++ b/Documentation/rev-list-options.txt\n> @@ -657,6 +657,10 @@ These options are mostly targeted for packing of Git repositories.\n> objects in deltified form based on objects contained in these\n> excluded commits to reduce network traffic.\n>\n> +--objects-edge-aggressive::\n> + Similar to `--objects-edge`, but it tries harder to find excluded\n> + commits at the cost of increased time.\n> +\n> --unpacked::\n> Only useful with `--objects`; print the object IDs that are not\n> in packs.\n> diff --git a/list-objects.c b/list-objects.c\n> index 2910bec..2a139b6 100644\n> --- a/list-objects.c\n> +++ b/list-objects.c\n> @@ -157,7 +157,7 @@ void mark_edges_uninteresting(struct rev_info *revs, show_edge_fn show_edge)\n>\n> if (commit->object.flags & UNINTERESTING) {\n> mark_tree_uninteresting(commit->tree);\n> - if (revs->edge_hint && !(commit->object.flags & SHOWN)) {\n> + if (revs->edge_hint_aggressive && !(commit->object.flags & SHOWN)) {\n> commit->object.flags |= SHOWN;\n> show_edge(commit);\n> }\n> @@ -165,7 +165,7 @@ void mark_edges_uninteresting(struct rev_info *revs, show_edge_fn show_edge)\n> }\n> mark_edge_parents_uninteresting(commit, revs, show_edge);\n> }\n> - if (revs->edge_hint) {\n> + if (revs->edge_hint_aggressive) {\n> for (i = 0; i < revs->cmdline.nr; i++) {\n> struct object *obj = revs->cmdline.rev[i].item;\n> struct commit *commit = (struct commit *)obj;\n> diff --git a/revision.c b/revision.c\n> index 75dda92..753dd2f 100644\n> --- a/revision.c\n> +++ b/revision.c\n> @@ -1853,6 +1853,12 @@ static int handle_revision_opt(struct rev_info *revs, int argc, const char **arg\n> revs->tree_objects = 1;\n> revs->blob_objects = 1;\n> revs->edge_hint = 1;\n> + } else if (!strcmp(arg, \"--objects-edge-aggressive\")) {\n> + revs->tag_objects = 1;\n> + revs->tree_objects = 1;\n> + revs->blob_objects = 1;\n> + revs->edge_hint = 1;\n> + revs->edge_hint_aggressive = 1;\n> } else if (!strcmp(arg, \"--verify-objects\")) {\n> revs->tag_objects = 1;\n> revs->tree_objects = 1;\n> diff --git a/revision.h b/revision.h\n> index 9cb5adc..033a244 100644\n> --- a/revision.h\n> +++ b/revision.h\n> @@ -93,6 +93,7 @@ struct rev_info {\n> blob_objects:1,\n> verify_objects:1,\n> edge_hint:1,\n> + edge_hint_aggressive:1,\n> limited:1,\n> unpacked:1,\n> boundary:2,\n> --\n> 2.2.1.209.g41e5f3a\n>\n> --\n> To unsubscribe from this list: send the line \"unsubscribe git\" in\n> the body of a message to majordomo@vger.kernel.org\n> More majordomo info at http://vger.kernel.org/majordomo-info.html\n"},{"id":"254036","messageId":"xmqqd27a2oqx.fsf@gitster.dls.corp.google.com","threadId":"38233","inReplyTo":"1419336082-283091-1-git-send-email-sandals@crustytoothpaste.net","subject":"Re: [PATCH v3 0/4] Improve push performance with lots of refs","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2014-12-23T18:40:54Z","receivedAt":"2014-12-23T18:40:54Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"\"brian m. carlson\" writes:\n\n> The only change from v2 is the addition of a fourth patch, which fixes\n> t5500. It's necessary because the test wants packs for fetches to\n> shallow clones to be minimal.\n>\n> I'm not especially thrilled with having to provide a --shallow command\n> line argument, but the alternative is to buffer a potentially large\n> amount of data in order to determine whether the remote side is shallow.\n\nYou spell \"--thin-aggressive\" as two words, \"--thin\" \"--shallow\", in\nthis series, essentially, no?\n\nI think this is going in the right direction. The \"shallow\"\npropagated on the wire from the fetcher is the right thing to use\nto make this decision.\n\nI wonder if the call to is_repository_shallow() is still necessary\n(read: I would prefer to see it go away) where we decide between\n\"--objects-edge\" and \"--objects-edge-aggressive\".\n\nHere is the relevant part from 4/4:\n\n@@ -2711,7 +2714,7 @@ int cmd_pack_objects(int argc, const char **argv, const char *prefix)\n \targv_array_push(&rp, \"pack-objects\");\n \tif (thin) {\n \t\tuse_internal_rev_list = 1;\n-\t\targv_array_push(&rp, is_repository_shallow()\n+\t\targv_array_push(&rp, is_repository_shallow() || shallow\n \t\t\t\t? \"--objects-edge-aggressive\"\n \t\t\t\t: \"--objects-edge\");\n \t} else\n"},{"id":"254039","messageId":"20141223185142.GA27954@peff.net","threadId":"38233","inReplyTo":"CAO2U3Qjbmz+fP-SLSeq1S+BDi4PSTSLf_TdqW-ik8GLV7=nUmA@mail.gmail.com","subject":"Re: [PATCH v3 2/4] rev-list: add an option to mark fewer edges as uninteresting","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2014-12-23T18:51:42Z","receivedAt":"2014-12-23T18:51:42Z","isPatch":true,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Tue, Dec 23, 2014 at 12:55:48PM -0500, Michael Blume wrote:\n\n> This patch causes an error on my mac, test 5500 fetch-pack errors on\n> part 44 - fetch creating new shallow root. It looks for \"remote: Total\n> 1\" in the fetch output and gets 3 instead.\n\nIt fails for me on Linux, too. Interestingly the tip of the series\npasses. I didn't investigate further.\n\n-Peff\n"},{"id":"254043","messageId":"xmqqvbl218rf.fsf@gitster.dls.corp.google.com","threadId":"38233","inReplyTo":"20141223185142.GA27954@peff.net","subject":"Re: [PATCH v3 2/4] rev-list: add an option to mark fewer edges as uninteresting","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2014-12-23T19:11:32Z","receivedAt":"2014-12-23T19:11:32Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Jeff King writes:\n\n> On Tue, Dec 23, 2014 at 12:55:48PM -0500, Michael Blume wrote:\n>\n>> This patch causes an error on my mac, test 5500 fetch-pack errors on\n>> part 44 - fetch creating new shallow root. It looks for \"remote: Total\n>> 1\" in the fetch output and gets 3 instead.\n>\n> It fails for me on Linux, too. Interestingly the tip of the series\n> passes. I didn't investigate further.\n\nNot surprised, as the first three are the same from the yesterday's\none that failed for everybody.\n"},{"id":"254087","messageId":"20141224212639.GC170128@vauxhall.crustytoothpaste.net","threadId":"38233","inReplyTo":"20141223185142.GA27954@peff.net","subject":"Re: [PATCH v3 2/4] rev-list: add an option to mark fewer edges as uninteresting","fromName":"brian m. carlson","fromEmail":"sandals@crustytoothpaste.net","sentAt":"2014-12-24T21:26:40Z","receivedAt":"2014-12-24T21:26:40Z","isPatch":true,"sender":{"key":"sandals@crustytoothpaste.net","avatar":"https://avatars.githubusercontent.com/u/497054?v=4"},"body":"On Tue, Dec 23, 2014 at 01:51:42PM -0500, Jeff King wrote:\n> On Tue, Dec 23, 2014 at 12:55:48PM -0500, Michael Blume wrote:\n> \n> > This patch causes an error on my mac, test 5500 fetch-pack errors on\n> > part 44 - fetch creating new shallow root. It looks for \"remote: Total\n> > 1\" in the fetch output and gets 3 instead.\n> \n> It fails for me on Linux, too. Interestingly the tip of the series\n> passes. I didn't investigate further.\n\nYes, the tip of the series fixes that issue. I had trouble coming up\nwith a good technique to split the patches in a way such that they were\nlogically distinct yet addressed the failure. I had an epiphany on how\nto do that last night, so I'll reroll sometime today.\n-- \nbrian m. carlson / brian with sandals: Houston, Texas, US\n+1 832 623 2791 | http://www.crustytoothpaste.net/~bmc | My opinion only\nOpenPGP: RSA v4 4096b: 88AC E9B2 9196 305B A994 7552 F1BA 225C 0223 B187\n"},{"id":"254088","messageId":"20141224213140.GD170128@vauxhall.crustytoothpaste.net","threadId":"38233","inReplyTo":"xmqqd27a2oqx.fsf@gitster.dls.corp.google.com","subject":"Re: [PATCH v3 0/4] Improve push performance with lots of refs","fromName":"brian m. carlson","fromEmail":"sandals@crustytoothpaste.net","sentAt":"2014-12-24T21:31:40Z","receivedAt":"2014-12-24T21:31:40Z","isPatch":true,"sender":{"key":"sandals@crustytoothpaste.net","avatar":"https://avatars.githubusercontent.com/u/497054?v=4"},"body":"On Tue, Dec 23, 2014 at 10:40:54AM -0800, Junio C Hamano wrote:\n> You spell \"--thin-aggressive\" as two words, \"--thin\" \"--shallow\", in\n> this series, essentially, no?\n\nEssentially, yes. It became obligatory after I noticed the test\nfailure, since that test actually checks whether the remote side sends\na shallow-optimized pack.\n\n> I think this is going in the right direction. The \"shallow\"\n> propagated on the wire from the fetcher is the right thing to use\n> to make this decision.\n> \n> I wonder if the call to is_repository_shallow() is still necessary\n> (read: I would prefer to see it go away) where we decide between\n> \"--objects-edge\" and \"--objects-edge-aggressive\".\n\nOkay. I'll try to push it up the stack a little bit.\n-- \nbrian m. carlson / brian with sandals: Houston, Texas, US\n+1 832 623 2791 | http://www.crustytoothpaste.net/~bmc | My opinion only\nOpenPGP: RSA v4 4096b: 88AC E9B2 9196 305B A994 7552 F1BA 225C 0223 B187\n"}]}