{"thread":{"id":"41092","subject":"[PATCH 00/20] More flexibility in making shallow clones","startedAt":"2015-12-29T12:10:23Z","lastAt":"2016-01-04T09:45:05Z","messageCount":24,"participants":["Nguyễn Thái Ngọc Duy","Duy Nguyen","Junio C Hamano","Eric Sunshine"],"isPatch":true,"patchVersion":1,"patchTotal":20},"messages":[{"id":"275112","messageId":"1451391043-28093-1-git-send-email-pclouds@gmail.com","threadId":"41092","inReplyTo":null,"subject":"[PATCH 00/20] More flexibility in making shallow clones","fromName":"Nguyễn Thái Ngọc Duy","fromEmail":"pclouds@gmail.com","sentAt":"2015-12-29T12:10:23Z","receivedAt":"2015-12-29T12:10:23Z","isPatch":true,"sender":{"key":"pclouds@gmail.com","avatar":"https://avatars.githubusercontent.com/u/720?v=4"},"body":"This series brings three new options to shallow clone/fetch. --since\nlets you specify cut point by time. --not cuts by excluding specified\nrefs. And --deepen= extends shallow boundary in a more predictable\nway. Some of these were requested in the past.\n\nAn important point of this series is it starts to use rev-list behind\nthe scene to define shallow boundary, which opens doors to even more\nways of cutting a repository.\n\nThe series is good enough to look at but I don't think I have stared\nlong enough to spot all the bugs. This mail is mostly about checking\nif the design (at both code and protocol levels) and the UI make\nsense. I'm thinking --since and --not may be too generic, but I don't\nsee any better names, for example.\n\nRefactor/cleanup patches are sprinkled throughout (bad?). The meat is\nin 05, 09, 12, 13, 16, 17, 18 and 20.\n\nHTTP support is not in this series (and I don't intend to do soon).\nThe amount of filler code just to pass some info from UI down to the\nprotocol seems too much, and it's even more so when HTTP support is\nadded. But I don't see anyway around it. Maybe we can share some code\nbetween git-clone, git-fetch and git-fetch-pack, at least the argument\nparsing..\n\nNguyễn Thái Ngọc Duy (20):\n upload-pack: move shallow deepen code out of receive_needs()\n upload-pack: move \"shallow\" sending code out of deepen()\n upload-pack: remove unused variable \"backup\"\n upload-pack: move \"unshallow\" sending code out of deepen()\n shallow.c: implement a generic shallow boundary finder based on rev-list\n upload-pack: glue code to use get_shallow_commits_by_rev_list\n upload-pack: use skip_prefix() instead of starts_with() when possible\n upload-pack: tighten number parsing at \"deepen\" lines\n upload-pack: add deepen-since to cut shallow repos based on time\n fetch-pack: use a common function for verbose printing\n fetch-pack: use a separate flag for fetch in deepening mode\n fetch: define shallow boundary with --since\n clone: define shallow clone boundary based on time with --since\n Add test_repo_expect_success for running tests in a new repository\n t5500: test for shallow depth since a specific date\n upload-pack: support define shallow boundary by excluding revisions\n fetch: define shallow boundary with --not\n clone: define shallow clone boundary with --not\n t5500: test for shallow depth excluding a ref\n fetch: add --deepen= to extend shallow boundary by commits\n\n Documentation/fetch-options.txt | 14 ++\n Documentation/git-clone.txt | 8 +\n Documentation/technical/pack-protocol.txt | 4 +-\n Documentation/technical/protocol-capabilities.txt | 25 +++\n builtin/clone.c | 32 +++-\n builtin/fetch.c | 44 ++++-\n commit.h | 2 +\n fetch-pack.c | 128 ++++++++------\n fetch-pack.h | 4 +\n shallow.c | 92 ++++++++++\n t/README | 15 ++\n t/t5500-fetch-pack.sh | 36 ++++\n t/t5510-fetch.sh | 12 ++\n t/test-lib-functions.sh | 20 +++\n transport.c | 11 ++\n transport.h | 14 ++\n upload-pack.c | 206 ++++++++++++++++------\n 17 files changed, 550 insertions(+), 117 deletions(-)\n\n-- \n2.3.0.rc1.137.g477eb31\n"},{"id":"275113","messageId":"1451391043-28093-2-git-send-email-pclouds@gmail.com","threadId":"41092","inReplyTo":"1451391043-28093-1-git-send-email-pclouds@gmail.com","subject":"[PATCH 01/20] upload-pack: move shallow deepen code out of receive_needs()","fromName":"Nguyễn Thái Ngọc Duy","fromEmail":"pclouds@gmail.com","sentAt":"2015-12-29T12:10:24Z","receivedAt":"2015-12-29T12:10:24Z","isPatch":true,"sender":{"key":"pclouds@gmail.com","avatar":"https://avatars.githubusercontent.com/u/720?v=4"},"body":"This is a prep step for further refactoring. Besides reindentation and\ns/shallows\\./shallows->/g, no other changes are expected.\n\nSigned-off-by: Nguyễn Thái Ngọc Duy \n---\n upload-pack.c | 99 +++++++++++++++++++++++++++++++----------------------------\n 1 file changed, 52 insertions(+), 47 deletions(-)\n\ndiff --git a/upload-pack.c b/upload-pack.c\nindex b3f6653..97ed620 100644\n--- a/upload-pack.c\n+++ b/upload-pack.c\n@@ -538,6 +538,55 @@ error:\n \t}\n }\n \n+static void deepen(int depth, const struct object_array *shallows)\n+{\n+\tstruct commit_list *result = NULL, *backup = NULL;\n+\tint i;\n+\tif (depth == INFINITE_DEPTH && !is_repository_shallow())\n+\t\tfor (i = 0; i < shallows->nr; i++) {\n+\t\t\tstruct object *object = shallows->objects[i].item;\n+\t\t\tobject->flags |= NOT_SHALLOW;\n+\t\t}\n+\telse\n+\t\tbackup = result =\n+\t\t\tget_shallow_commits(&want_obj, depth,\n+\t\t\t\t\t SHALLOW, NOT_SHALLOW);\n+\twhile (result) {\n+\t\tstruct object *object = &result->item->object;\n+\t\tif (!(object->flags & (CLIENT_SHALLOW|NOT_SHALLOW))) {\n+\t\t\tpacket_write(1, \"shallow %s\",\n+\t\t\t\t oid_to_hex(&object->oid));\n+\t\t\tregister_shallow(object->oid.hash);\n+\t\t\tshallow_nr++;\n+\t\t}\n+\t\tresult = result->next;\n+\t}\n+\tfree_commit_list(backup);\n+\tfor (i = 0; i < shallows->nr; i++) {\n+\t\tstruct object *object = shallows->objects[i].item;\n+\t\tif (object->flags & NOT_SHALLOW) {\n+\t\t\tstruct commit_list *parents;\n+\t\t\tpacket_write(1, \"unshallow %s\",\n+\t\t\t\t oid_to_hex(&object->oid));\n+\t\t\tobject->flags &= ~CLIENT_SHALLOW;\n+\t\t\t/* make sure the real parents are parsed */\n+\t\t\tunregister_shallow(object->oid.hash);\n+\t\t\tobject->parsed = 0;\n+\t\t\tparse_commit_or_die((struct commit *)object);\n+\t\t\tparents = ((struct commit *)object)->parents;\n+\t\t\twhile (parents) {\n+\t\t\t\tadd_object_array(&parents->item->object,\n+\t\t\t\t\t\t NULL, &want_obj);\n+\t\t\t\tparents = parents->next;\n+\t\t\t}\n+\t\t\tadd_object_array(object, NULL, &extra_edge_obj);\n+\t\t}\n+\t\t/* make sure commit traversal conforms to client */\n+\t\tregister_shallow(object->oid.hash);\n+\t}\n+\tpacket_flush(1);\n+}\n+\n static void receive_needs(void)\n {\n \tstruct object_array shallows = OBJECT_ARRAY_INIT;\n@@ -630,53 +679,9 @@ static void receive_needs(void)\n \n \tif (depth == 0 && shallows.nr == 0)\n \t\treturn;\n-\tif (depth > 0) {\n-\t\tstruct commit_list *result = NULL, *backup = NULL;\n-\t\tint i;\n-\t\tif (depth == INFINITE_DEPTH && !is_repository_shallow())\n-\t\t\tfor (i = 0; i < shallows.nr; i++) {\n-\t\t\t\tstruct object *object = shallows.objects[i].item;\n-\t\t\t\tobject->flags |= NOT_SHALLOW;\n-\t\t\t}\n-\t\telse\n-\t\t\tbackup = result =\n-\t\t\t\tget_shallow_commits(&want_obj, depth,\n-\t\t\t\t\t\t SHALLOW, NOT_SHALLOW);\n-\t\twhile (result) {\n-\t\t\tstruct object *object = &result->item->object;\n-\t\t\tif (!(object->flags & (CLIENT_SHALLOW|NOT_SHALLOW))) {\n-\t\t\t\tpacket_write(1, \"shallow %s\",\n-\t\t\t\t\t\toid_to_hex(&object->oid));\n-\t\t\t\tregister_shallow(object->oid.hash);\n-\t\t\t\tshallow_nr++;\n-\t\t\t}\n-\t\t\tresult = result->next;\n-\t\t}\n-\t\tfree_commit_list(backup);\n-\t\tfor (i = 0; i < shallows.nr; i++) {\n-\t\t\tstruct object *object = shallows.objects[i].item;\n-\t\t\tif (object->flags & NOT_SHALLOW) {\n-\t\t\t\tstruct commit_list *parents;\n-\t\t\t\tpacket_write(1, \"unshallow %s\",\n-\t\t\t\t\toid_to_hex(&object->oid));\n-\t\t\t\tobject->flags &= ~CLIENT_SHALLOW;\n-\t\t\t\t/* make sure the real parents are parsed */\n-\t\t\t\tunregister_shallow(object->oid.hash);\n-\t\t\t\tobject->parsed = 0;\n-\t\t\t\tparse_commit_or_die((struct commit *)object);\n-\t\t\t\tparents = ((struct commit *)object)->parents;\n-\t\t\t\twhile (parents) {\n-\t\t\t\t\tadd_object_array(&parents->item->object,\n-\t\t\t\t\t\t\tNULL, &want_obj);\n-\t\t\t\t\tparents = parents->next;\n-\t\t\t\t}\n-\t\t\t\tadd_object_array(object, NULL, &extra_edge_obj);\n-\t\t\t}\n-\t\t\t/* make sure commit traversal conforms to client */\n-\t\t\tregister_shallow(object->oid.hash);\n-\t\t}\n-\t\tpacket_flush(1);\n-\t} else\n+\tif (depth > 0)\n+\t\tdeepen(depth, &shallows);\n+\telse\n \t\tif (shallows.nr > 0) {\n \t\t\tint i;\n \t\t\tfor (i = 0; i < shallows.nr; i++)\n-- \n2.3.0.rc1.137.g477eb31\n"},{"id":"275114","messageId":"1451391043-28093-3-git-send-email-pclouds@gmail.com","threadId":"41092","inReplyTo":"1451391043-28093-1-git-send-email-pclouds@gmail.com","subject":"[PATCH 02/20] upload-pack: move \"shallow\" sending code out of deepen()","fromName":"Nguyễn Thái Ngọc Duy","fromEmail":"pclouds@gmail.com","sentAt":"2015-12-29T12:10:25Z","receivedAt":"2015-12-29T12:10:25Z","isPatch":true,"sender":{"key":"pclouds@gmail.com","avatar":"https://avatars.githubusercontent.com/u/720?v=4"},"body":"Signed-off-by: Nguyễn Thái Ngọc Duy \n---\n upload-pack.c | 25 +++++++++++++++----------\n 1 file changed, 15 insertions(+), 10 deletions(-)\n\ndiff --git a/upload-pack.c b/upload-pack.c\nindex 97ed620..0eb9a0b 100644\n--- a/upload-pack.c\n+++ b/upload-pack.c\n@@ -538,6 +538,20 @@ error:\n \t}\n }\n \n+static void send_shallow(struct commit_list *result)\n+{\n+\twhile (result) {\n+\t\tstruct object *object = &result->item->object;\n+\t\tif (!(object->flags & (CLIENT_SHALLOW|NOT_SHALLOW))) {\n+\t\t\tpacket_write(1, \"shallow %s\",\n+\t\t\t\t oid_to_hex(&object->oid));\n+\t\t\tregister_shallow(object->oid.hash);\n+\t\t\tshallow_nr++;\n+\t\t}\n+\t\tresult = result->next;\n+\t}\n+}\n+\n static void deepen(int depth, const struct object_array *shallows)\n {\n \tstruct commit_list *result = NULL, *backup = NULL;\n@@ -551,16 +565,7 @@ static void deepen(int depth, const struct object_array *shallows)\n \t\tbackup = result =\n \t\t\tget_shallow_commits(&want_obj, depth,\n \t\t\t\t\t SHALLOW, NOT_SHALLOW);\n-\twhile (result) {\n-\t\tstruct object *object = &result->item->object;\n-\t\tif (!(object->flags & (CLIENT_SHALLOW|NOT_SHALLOW))) {\n-\t\t\tpacket_write(1, \"shallow %s\",\n-\t\t\t\t oid_to_hex(&object->oid));\n-\t\t\tregister_shallow(object->oid.hash);\n-\t\t\tshallow_nr++;\n-\t\t}\n-\t\tresult = result->next;\n-\t}\n+\tsend_shallow(result);\n \tfree_commit_list(backup);\n \tfor (i = 0; i < shallows->nr; i++) {\n \t\tstruct object *object = shallows->objects[i].item;\n-- \n2.3.0.rc1.137.g477eb31\n"},{"id":"275116","messageId":"1451391043-28093-4-git-send-email-pclouds@gmail.com","threadId":"41092","inReplyTo":"1451391043-28093-1-git-send-email-pclouds@gmail.com","subject":"[PATCH 03/20] upload-pack: remove unused variable \"backup\"","fromName":"Nguyễn Thái Ngọc Duy","fromEmail":"pclouds@gmail.com","sentAt":"2015-12-29T12:10:26Z","receivedAt":"2015-12-29T12:10:26Z","isPatch":true,"sender":{"key":"pclouds@gmail.com","avatar":"https://avatars.githubusercontent.com/u/720?v=4"},"body":"After the last patch, \"result\" and \"backup\" are the same. \"result\" used\nto move, but the movement is now contained in send_shallow(). Delete\nthis redundant variable.\n\nSigned-off-by: Nguyễn Thái Ngọc Duy \n---\n upload-pack.c | 6 +++---\n 1 file changed, 3 insertions(+), 3 deletions(-)\n\ndiff --git a/upload-pack.c b/upload-pack.c\nindex 0eb9a0b..4774f78 100644\n--- a/upload-pack.c\n+++ b/upload-pack.c\n@@ -554,7 +554,7 @@ static void send_shallow(struct commit_list *result)\n \n static void deepen(int depth, const struct object_array *shallows)\n {\n-\tstruct commit_list *result = NULL, *backup = NULL;\n+\tstruct commit_list *result = NULL;\n \tint i;\n \tif (depth == INFINITE_DEPTH && !is_repository_shallow())\n \t\tfor (i = 0; i < shallows->nr; i++) {\n@@ -562,11 +562,11 @@ static void deepen(int depth, const struct object_array *shallows)\n \t\t\tobject->flags |= NOT_SHALLOW;\n \t\t}\n \telse\n-\t\tbackup = result =\n+\t\tresult =\n \t\t\tget_shallow_commits(&want_obj, depth,\n \t\t\t\t\t SHALLOW, NOT_SHALLOW);\n \tsend_shallow(result);\n-\tfree_commit_list(backup);\n+\tfree_commit_list(result);\n \tfor (i = 0; i < shallows->nr; i++) {\n \t\tstruct object *object = shallows->objects[i].item;\n \t\tif (object->flags & NOT_SHALLOW) {\n-- \n2.3.0.rc1.137.g477eb31\n"},{"id":"275115","messageId":"1451391043-28093-5-git-send-email-pclouds@gmail.com","threadId":"41092","inReplyTo":"1451391043-28093-1-git-send-email-pclouds@gmail.com","subject":"[PATCH 04/20] upload-pack: move \"unshallow\" sending code out of deepen()","fromName":"Nguyễn Thái Ngọc Duy","fromEmail":"pclouds@gmail.com","sentAt":"2015-12-29T12:10:27Z","receivedAt":"2015-12-29T12:10:27Z","isPatch":true,"sender":{"key":"pclouds@gmail.com","avatar":"https://avatars.githubusercontent.com/u/720?v=4"},"body":"Signed-off-by: Nguyễn Thái Ngọc Duy \n---\n upload-pack.c | 33 ++++++++++++++++++++-------------\n 1 file changed, 20 insertions(+), 13 deletions(-)\n\ndiff --git a/upload-pack.c b/upload-pack.c\nindex 4774f78..4bb104c 100644\n--- a/upload-pack.c\n+++ b/upload-pack.c\n@@ -552,21 +552,10 @@ static void send_shallow(struct commit_list *result)\n \t}\n }\n \n-static void deepen(int depth, const struct object_array *shallows)\n+static void send_unshallow(const struct object_array *shallows)\n {\n-\tstruct commit_list *result = NULL;\n \tint i;\n-\tif (depth == INFINITE_DEPTH && !is_repository_shallow())\n-\t\tfor (i = 0; i < shallows->nr; i++) {\n-\t\t\tstruct object *object = shallows->objects[i].item;\n-\t\t\tobject->flags |= NOT_SHALLOW;\n-\t\t}\n-\telse\n-\t\tresult =\n-\t\t\tget_shallow_commits(&want_obj, depth,\n-\t\t\t\t\t SHALLOW, NOT_SHALLOW);\n-\tsend_shallow(result);\n-\tfree_commit_list(result);\n+\n \tfor (i = 0; i < shallows->nr; i++) {\n \t\tstruct object *object = shallows->objects[i].item;\n \t\tif (object->flags & NOT_SHALLOW) {\n@@ -589,6 +578,24 @@ static void deepen(int depth, const struct object_array *shallows)\n \t\t/* make sure commit traversal conforms to client */\n \t\tregister_shallow(object->oid.hash);\n \t}\n+}\n+\n+static void deepen(int depth, const struct object_array *shallows)\n+{\n+\tstruct commit_list *result = NULL;\n+\tint i;\n+\tif (depth == INFINITE_DEPTH && !is_repository_shallow())\n+\t\tfor (i = 0; i < shallows->nr; i++) {\n+\t\t\tstruct object *object = shallows->objects[i].item;\n+\t\t\tobject->flags |= NOT_SHALLOW;\n+\t\t}\n+\telse\n+\t\tresult =\n+\t\t\tget_shallow_commits(&want_obj, depth,\n+\t\t\t\t\t SHALLOW, NOT_SHALLOW);\n+\tsend_shallow(result);\n+\tfree_commit_list(result);\n+\tsend_unshallow(shallows);\n \tpacket_flush(1);\n }\n \n-- \n2.3.0.rc1.137.g477eb31\n"},{"id":"275118","messageId":"1451391043-28093-6-git-send-email-pclouds@gmail.com","threadId":"41092","inReplyTo":"1451391043-28093-1-git-send-email-pclouds@gmail.com","subject":"[PATCH 05/20] shallow.c: implement a generic shallow boundary finder based on rev-list","fromName":"Nguyễn Thái Ngọc Duy","fromEmail":"pclouds@gmail.com","sentAt":"2015-12-29T12:10:28Z","receivedAt":"2015-12-29T12:10:28Z","isPatch":true,"sender":{"key":"pclouds@gmail.com","avatar":"https://avatars.githubusercontent.com/u/720?v=4"},"body":"Instead of a custom commit walker like get_shallow_commits(), this new\nfunction uses rev-list to mark NOT_SHALLOW to all reachable commits,\nexcept borders. The definition of reachable is to be defined by the\nprotocol later. This makes it more flexible to define shallow boundary.\n\nNote: if a commit has one NOT_SHALLOW parent and one SHALLOW parent,\nthen it's considered the boundary. Which means in the client side, this\ncommit has _no_ parents. This could lead to surprising cuts if we're not\ncareful.\n\nAnother option is to include more commits and only mark commits whose\nall parents are SHALLOW as boundary.\n\nSigned-off-by: Nguyễn Thái Ngọc Duy \n---\n commit.h | 2 ++\n shallow.c | 92 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++\n 2 files changed, 94 insertions(+)\n\ndiff --git a/commit.h b/commit.h\nindex 5d58be0..b717be1 100644\n--- a/commit.h\n+++ b/commit.h\n@@ -258,6 +258,8 @@ extern int for_each_commit_graft(each_commit_graft_fn, void *);\n extern int is_repository_shallow(void);\n extern struct commit_list *get_shallow_commits(struct object_array *heads,\n \t\tint depth, int shallow_flag, int not_shallow_flag);\n+extern struct commit_list *get_shallow_commits_by_rev_list(\n+\t\tint ac, const char **av, int shallow_flag, int not_shallow_flag);\n extern void set_alternate_shallow_file(const char *path, int override);\n extern int write_shallow_commits(struct strbuf *out, int use_pack_protocol,\n \t\t\t\t const struct sha1_array *extra);\ndiff --git a/shallow.c b/shallow.c\nindex 60f1505..45dffaa 100644\n--- a/shallow.c\n+++ b/shallow.c\n@@ -10,6 +10,8 @@\n #include \"diff.h\"\n #include \"revision.h\"\n #include \"commit-slab.h\"\n+#include \"revision.h\"\n+#include \"list-objects.h\"\n \n static int is_shallow = -1;\n static struct stat_validity shallow_stat;\n@@ -137,6 +139,96 @@ struct commit_list *get_shallow_commits(struct object_array *heads, int depth,\n \treturn result;\n }\n \n+static void show_commit(struct commit *commit, void *data)\n+{\n+\tcommit->object.flags |= *(int *)data;\n+}\n+\n+/*\n+ * Given rev-list arguments, run rev-list. All reachable commits\n+ * except border ones are marked with not_shallow_flag. Border commits\n+ * are marked with shallow_flag. The list of border/shallow commits\n+ * are also returned.\n+ */\n+struct commit_list *get_shallow_commits_by_rev_list(int ac, const char **av,\n+\t\t\t\t\t\t int shallow_flag,\n+\t\t\t\t\t\t int not_shallow_flag)\n+{\n+\tstruct commit_list *result = NULL, *cl;\n+\tstruct rev_info revs;\n+\tunsigned int i, nr;\n+\n+\t/*\n+\t * SHALLOW and NOT_SHALLOW should not be set at this\n+\t * point. But better be safe than sorry.\n+\t */\n+\tnr = get_max_object_index();\n+\tfor (i = 0; i < nr; i++) {\n+\t\tstruct object *o = get_indexed_object(i);\n+\t\tif (!o)\n+\t\t\tcontinue;\n+\t\to->flags &= ~(shallow_flag | not_shallow_flag);\n+\t}\n+\n+\tis_repository_shallow(); /* make sure shallows are read */\n+\n+\tinit_revisions(&revs, NULL);\n+\tsave_commit_buffer = 0;\n+\tsetup_revisions(ac, av, &revs, NULL);\n+\n+\t/* Mark all reachable commits as NOT_SHALLOW */\n+\tif (prepare_revision_walk(&revs))\n+\t\tdie(\"revision walk setup failed\");\n+\ttraverse_commit_list(&revs, show_commit, NULL, ¬_shallow_flag);\n+\n+\t/*\n+\t * mark border commits SHALLOW + NOT_SHALLOW.\n+\t * We cannot clear NOT_SHALLOW right now. Imagine border\n+\t * commit A is processed first, then commit B, whose parent is\n+\t * A, later. If NOT_SHALLOW on A is cleared at step 1, B\n+\t * itself is considered border at step 2, which is incorrect.\n+\t */\n+\tnr = get_max_object_index();\n+\tfor (i = 0; i < nr; i++) {\n+\t\tstruct object *o = get_indexed_object(i);\n+\t\tstruct commit *c = (struct commit *)o;\n+\t\tstruct commit_list *p;\n+\t\tint parent_is_shallow = 0;\n+\n+\t\tif (!o || o->type != OBJ_COMMIT || !(o->flags & not_shallow_flag))\n+\t\t\tcontinue;\n+\n+\t\tif (parse_commit(c))\n+\t\t\tdie(\"unable to parse commit %s\",\n+\t\t\t oid_to_hex(&c->object.oid));\n+\n+\t\tfor (p = c->parents; p; p = p->next) {\n+\t\t\tif (p->item->object.flags & not_shallow_flag)\n+\t\t\t\tcontinue;\n+\t\t\tparent_is_shallow = 1;\n+\t\t\tif (p->item->object.flags & shallow_flag)\n+\t\t\t\tcontinue;\n+\t\t\tp->item->object.flags |= shallow_flag;\n+\t\t\tcommit_list_insert(p->item, &result);\n+\t\t}\n+\n+\t\tif (parent_is_shallow) {\n+\t\t\to->flags |= shallow_flag;\n+\t\t\tcommit_list_insert(c, &result);\n+\t\t}\n+\t}\n+\t/* clean up NOT_SHALLOW on border commits */\n+\tcl = result;\n+\twhile (cl) {\n+\t\tstruct object *object = &result->item->object;\n+\t\tif ((object->flags & not_shallow_flag) &&\n+\t\t (object->flags & shallow_flag))\n+\t\t\tobject->flags &= ~not_shallow_flag;\n+\t\tcl = cl->next;\n+\t}\n+\treturn result;\n+}\n+\n static void check_shallow_file_for_update(void)\n {\n \tif (is_shallow == -1)\n-- \n2.3.0.rc1.137.g477eb31\n"},{"id":"275117","messageId":"1451391043-28093-7-git-send-email-pclouds@gmail.com","threadId":"41092","inReplyTo":"1451391043-28093-1-git-send-email-pclouds@gmail.com","subject":"[PATCH 06/20] upload-pack: glue code to use get_shallow_commits_by_rev_list","fromName":"Nguyễn Thái Ngọc Duy","fromEmail":"pclouds@gmail.com","sentAt":"2015-12-29T12:10:29Z","receivedAt":"2015-12-29T12:10:29Z","isPatch":true,"sender":{"key":"pclouds@gmail.com","avatar":"https://avatars.githubusercontent.com/u/720?v=4"},"body":"The remaining thing to do is protocol extensions and translate the\nrequests to rev-list options.\n\nSigned-off-by: Nguyễn Thái Ngọc Duy \n---\n upload-pack.c | 12 ++++++++++++\n 1 file changed, 12 insertions(+)\n\ndiff --git a/upload-pack.c b/upload-pack.c\nindex 4bb104c..9ae09a0 100644\n--- a/upload-pack.c\n+++ b/upload-pack.c\n@@ -599,6 +599,18 @@ static void deepen(int depth, const struct object_array *shallows)\n \tpacket_flush(1);\n }\n \n+static void deepen_by_rev_list(int ac, const char **av,\n+\t\t\t struct object_array *shallows)\n+{\n+\tstruct commit_list *result;\n+\n+\tresult = get_shallow_commits_by_rev_list(ac, av, SHALLOW, NOT_SHALLOW);\n+\tsend_shallow(result);\n+\tfree_commit_list(result);\n+\tsend_unshallow(shallows);\n+\tpacket_flush(1);\n+}\n+\n static void receive_needs(void)\n {\n \tstruct object_array shallows = OBJECT_ARRAY_INIT;\n-- \n2.3.0.rc1.137.g477eb31\n"},{"id":"275119","messageId":"1451391043-28093-8-git-send-email-pclouds@gmail.com","threadId":"41092","inReplyTo":"1451391043-28093-1-git-send-email-pclouds@gmail.com","subject":"[PATCH 07/20] upload-pack: use skip_prefix() instead of starts_with() when possible","fromName":"Nguyễn Thái Ngọc Duy","fromEmail":"pclouds@gmail.com","sentAt":"2015-12-29T12:10:30Z","receivedAt":"2015-12-29T12:10:30Z","isPatch":true,"sender":{"key":"pclouds@gmail.com","avatar":"https://avatars.githubusercontent.com/u/720?v=4"},"body":"Signed-off-by: Nguyễn Thái Ngọc Duy \n---\n upload-pack.c | 26 +++++++++++++++-----------\n 1 file changed, 15 insertions(+), 11 deletions(-)\n\ndiff --git a/upload-pack.c b/upload-pack.c\nindex 9ae09a0..e9594d1 100644\n--- a/upload-pack.c\n+++ b/upload-pack.c\n@@ -276,7 +276,7 @@ static void create_pack_file(void)\n \tdie(\"git upload-pack: %s\", abort_msg);\n }\n \n-static int got_sha1(char *hex, unsigned char *sha1)\n+static int got_sha1(const char *hex, unsigned char *sha1)\n {\n \tstruct object *o;\n \tint we_knew_they_have = 0;\n@@ -382,6 +382,8 @@ static int get_common_commits(void)\n \n \tfor (;;) {\n \t\tchar *line = packet_read_line(0, NULL);\n+\t\tconst char *arg;\n+\n \t\treset_timeout();\n \n \t\tif (!line) {\n@@ -403,8 +405,8 @@ static int get_common_commits(void)\n \t\t\tgot_other = 0;\n \t\t\tcontinue;\n \t\t}\n-\t\tif (starts_with(line, \"have \")) {\n-\t\t\tswitch (got_sha1(line+5, sha1)) {\n+\t\tif (skip_prefix(line, \"have \", &arg)) {\n+\t\t\tswitch (got_sha1(arg, sha1)) {\n \t\t\tcase -1: /* they have what we do not */\n \t\t\t\tgot_other = 1;\n \t\t\t\tif (multi_ack && ok_to_give_up()) {\n@@ -623,14 +625,16 @@ static void receive_needs(void)\n \t\tconst char *features;\n \t\tunsigned char sha1_buf[20];\n \t\tchar *line = packet_read_line(0, NULL);\n+\t\tconst char *arg;\n+\n \t\treset_timeout();\n \t\tif (!line)\n \t\t\tbreak;\n \n-\t\tif (starts_with(line, \"shallow \")) {\n+\t\tif (skip_prefix(line, \"shallow \", &arg)) {\n \t\t\tunsigned char sha1[20];\n \t\t\tstruct object *object;\n-\t\t\tif (get_sha1_hex(line + 8, sha1))\n+\t\t\tif (get_sha1_hex(arg, sha1))\n \t\t\t\tdie(\"invalid shallow line: %s\", line);\n \t\t\tobject = parse_object(sha1);\n \t\t\tif (!object)\n@@ -643,19 +647,19 @@ static void receive_needs(void)\n \t\t\t}\n \t\t\tcontinue;\n \t\t}\n-\t\tif (starts_with(line, \"deepen \")) {\n+\t\tif (skip_prefix(line, \"deepen \", &arg)) {\n \t\t\tchar *end;\n-\t\t\tdepth = strtol(line + 7, &end, 0);\n-\t\t\tif (end == line + 7 || depth <= 0)\n+\t\t\tdepth = strtol(arg, &end, 0);\n+\t\t\tif (end == arg || depth <= 0)\n \t\t\t\tdie(\"Invalid deepen: %s\", line);\n \t\t\tcontinue;\n \t\t}\n-\t\tif (!starts_with(line, \"want \") ||\n-\t\t get_sha1_hex(line+5, sha1_buf))\n+\t\tif (!skip_prefix(line, \"want \", &arg) ||\n+\t\t get_sha1_hex(arg, sha1_buf))\n \t\t\tdie(\"git upload-pack: protocol error, \"\n \t\t\t \"expected to get sha, not '%s'\", line);\n \n-\t\tfeatures = line + 45;\n+\t\tfeatures = arg + 40;\n \n \t\tif (parse_feature_request(features, \"multi_ack_detailed\"))\n \t\t\tmulti_ack = 2;\n-- \n2.3.0.rc1.137.g477eb31\n"},{"id":"275120","messageId":"1451391043-28093-9-git-send-email-pclouds@gmail.com","threadId":"41092","inReplyTo":"1451391043-28093-1-git-send-email-pclouds@gmail.com","subject":"[PATCH 08/20] upload-pack: tighten number parsing at \"deepen\" lines","fromName":"Nguyễn Thái Ngọc Duy","fromEmail":"pclouds@gmail.com","sentAt":"2015-12-29T12:10:31Z","receivedAt":"2015-12-29T12:10:31Z","isPatch":true,"sender":{"key":"pclouds@gmail.com","avatar":"https://avatars.githubusercontent.com/u/720?v=4"},"body":"Signed-off-by: Nguyễn Thái Ngọc Duy \n---\n upload-pack.c | 4 ++--\n 1 file changed, 2 insertions(+), 2 deletions(-)\n\ndiff --git a/upload-pack.c b/upload-pack.c\nindex e9594d1..573ffa2 100644\n--- a/upload-pack.c\n+++ b/upload-pack.c\n@@ -648,9 +648,9 @@ static void receive_needs(void)\n \t\t\tcontinue;\n \t\t}\n \t\tif (skip_prefix(line, \"deepen \", &arg)) {\n-\t\t\tchar *end;\n+\t\t\tchar *end = NULL;\n \t\t\tdepth = strtol(arg, &end, 0);\n-\t\t\tif (end == arg || depth <= 0)\n+\t\t\tif (!end || *end || depth <= 0)\n \t\t\t\tdie(\"Invalid deepen: %s\", line);\n \t\t\tcontinue;\n \t\t}\n-- \n2.3.0.rc1.137.g477eb31\n"},{"id":"275121","messageId":"1451391043-28093-10-git-send-email-pclouds@gmail.com","threadId":"41092","inReplyTo":"1451391043-28093-1-git-send-email-pclouds@gmail.com","subject":"[PATCH 09/20] upload-pack: add deepen-since to cut shallow repos based on time","fromName":"Nguyễn Thái Ngọc Duy","fromEmail":"pclouds@gmail.com","sentAt":"2015-12-29T12:10:32Z","receivedAt":"2015-12-29T12:10:32Z","isPatch":true,"sender":{"key":"pclouds@gmail.com","avatar":"https://avatars.githubusercontent.com/u/720?v=4"},"body":"This should allow the user to say \"create a shallow clone containing the\nwork from last year\" (once the client side is fixed up, of course).\n\nIn theory deepen-since and deepen (aka --depth) can be used together to\ndraw the shallow boundary (whether it's intersection or union is up to\ndiscussion, but if rev-list is used, it's likely intersection). However,\nbecause deepen goes with a custom commit walker, we can't mix the two\nyet.\n\nSigned-off-by: Nguyễn Thái Ngọc Duy \n---\n Documentation/technical/pack-protocol.txt | 3 ++-\n Documentation/technical/protocol-capabilities.txt | 9 +++++++\n upload-pack.c | 33 +++++++++++++++++++++--\n 3 files changed, 42 insertions(+), 3 deletions(-)\n\ndiff --git a/Documentation/technical/pack-protocol.txt b/Documentation/technical/pack-protocol.txt\nindex c6977bb..9251df1 100644\n--- a/Documentation/technical/pack-protocol.txt\n+++ b/Documentation/technical/pack-protocol.txt\n@@ -219,7 +219,8 @@ out of what the server said it could do with the first 'want' line.\n \n shallow-line = PKT-LINE(\"shallow\" SP obj-id)\n \n- depth-request = PKT-LINE(\"deepen\" SP depth)\n+ depth-request = PKT-LINE(\"deepen\" SP depth) /\n+\t\t PKT-LINE(\"deepen-since\" SP timestamp)\n \n first-want = PKT-LINE(\"want\" SP obj-id SP capability-list)\n additional-want = PKT-LINE(\"want\" SP obj-id)\ndiff --git a/Documentation/technical/protocol-capabilities.txt b/Documentation/technical/protocol-capabilities.txt\nindex eaab6b4..f08cc4e 100644\n--- a/Documentation/technical/protocol-capabilities.txt\n+++ b/Documentation/technical/protocol-capabilities.txt\n@@ -179,6 +179,15 @@ This capability adds \"deepen\", \"shallow\" and \"unshallow\" commands to\n the fetch-pack/upload-pack protocol so clients can request shallow\n clones.\n \n+deepen-since\n+------------\n+\n+This capability adds \"deepen-since\" command to fetch-pack/upload-pack\n+protocol so the client can request shallow clones that are cut at a\n+specific time, instead of depth. Internally it's equivalent of doing\n+\"rev-list --max-age=\" on the server side. \"deepen-since\"\n+cannot be used with \"deepen\".\n+\n no-progress\n -----------\n \ndiff --git a/upload-pack.c b/upload-pack.c\nindex 573ffa2..100b604 100644\n--- a/upload-pack.c\n+++ b/upload-pack.c\n@@ -14,6 +14,7 @@\n #include \"sigchain.h\"\n #include \"version.h\"\n #include \"string-list.h\"\n+#include \"argv-array.h\"\n \n static const char upload_pack_usage[] = \"git upload-pack [--strict] [--timeout=] \";\n \n@@ -618,6 +619,8 @@ static void receive_needs(void)\n \tstruct object_array shallows = OBJECT_ARRAY_INIT;\n \tint depth = 0;\n \tint has_non_tip = 0;\n+\tunsigned long deepen_since = 0;\n+\tint deepen_rev_list = 0;\n \n \tshallow_nr = 0;\n \tfor (;;) {\n@@ -654,6 +657,16 @@ static void receive_needs(void)\n \t\t\t\tdie(\"Invalid deepen: %s\", line);\n \t\t\tcontinue;\n \t\t}\n+\t\tif (skip_prefix(line, \"deepen-since \", &arg)) {\n+\t\t\tchar *end = NULL;\n+\t\t\tdeepen_since = strtoul(arg, &end, 0);\n+\t\t\tif (!end || *end || !deepen_since ||\n+\t\t\t /* revisions.c's max_age -1 is special */\n+\t\t\t deepen_since == -1)\n+\t\t\t\tdie(\"Invalid deepen-since: %s\", line);\n+\t\t\tdeepen_rev_list = 1;\n+\t\t\tcontinue;\n+\t\t}\n \t\tif (!skip_prefix(line, \"want \", &arg) ||\n \t\t get_sha1_hex(arg, sha1_buf))\n \t\t\tdie(\"git upload-pack: protocol error, \"\n@@ -705,10 +718,26 @@ static void receive_needs(void)\n \tif (!use_sideband && daemon_mode)\n \t\tno_progress = 1;\n \n-\tif (depth == 0 && shallows.nr == 0)\n+\tif (depth == 0 && !deepen_rev_list && shallows.nr == 0)\n \t\treturn;\n+\tif (depth > 0 && deepen_rev_list)\n+\t\tdie(\"--depth and --since cannot be used together\");\n \tif (depth > 0)\n \t\tdeepen(depth, &shallows);\n+\telse if (deepen_rev_list) {\n+\t\tstruct argv_array av = ARGV_ARRAY_INIT;\n+\t\tint i;\n+\n+\t\targv_array_push(&av, \"rev-list\");\n+\t\tif (deepen_since)\n+\t\t\targv_array_pushf(&av, \"--max-age=%lu\", deepen_since);\n+\t\tfor (i = 0; i < want_obj.nr; i++) {\n+\t\t\tstruct object *o = want_obj.objects[i].item;\n+\t\t\targv_array_push(&av, oid_to_hex(&o->oid));\n+\t\t}\n+\t\tdeepen_by_rev_list(av.argc, av.argv, &shallows);\n+\t\targv_array_clear(&av);\n+\t}\n \telse\n \t\tif (shallows.nr > 0) {\n \t\t\tint i;\n@@ -757,7 +786,7 @@ static int send_ref(const char *refname, const struct object_id *oid,\n \t\t int flag, void *cb_data)\n {\n \tstatic const char *capabilities = \"multi_ack thin-pack side-band\"\n-\t\t\" side-band-64k ofs-delta shallow no-progress\"\n+\t\t\" side-band-64k ofs-delta shallow deepen-since no-progress\"\n \t\t\" include-tag multi_ack_detailed\";\n \tconst char *refname_nons = strip_namespace(refname);\n \tstruct object_id peeled;\n-- \n2.3.0.rc1.137.g477eb31\n"},{"id":"275122","messageId":"1451391043-28093-11-git-send-email-pclouds@gmail.com","threadId":"41092","inReplyTo":"1451391043-28093-1-git-send-email-pclouds@gmail.com","subject":"[PATCH 10/20] fetch-pack: use a common function for verbose printing","fromName":"Nguyễn Thái Ngọc Duy","fromEmail":"pclouds@gmail.com","sentAt":"2015-12-29T12:10:33Z","receivedAt":"2015-12-29T12:10:33Z","isPatch":true,"sender":{"key":"pclouds@gmail.com","avatar":"https://avatars.githubusercontent.com/u/720?v=4"},"body":"This reduces the number of \"if (verbose)\" which makes it a bit easier\nto read imo. It also makes it easier to redirect all these printouts,\nto a file for example.\n\nSigned-off-by: Nguyễn Thái Ngọc Duy \n---\n fetch-pack.c | 88 +++++++++++++++++++++++++++++-------------------------------\n 1 file changed, 42 insertions(+), 46 deletions(-)\n\ndiff --git a/fetch-pack.c b/fetch-pack.c\nindex 01e34b6..16917f9 100644\n--- a/fetch-pack.c\n+++ b/fetch-pack.c\n@@ -50,6 +50,21 @@ static int non_common_revs, multi_ack, use_sideband;\n #define ALLOW_REACHABLE_SHA1\t02\n static unsigned int allow_unadvertised_object_request;\n \n+__attribute__((format (printf, 2, 3)))\n+static inline void print_verbose(const struct fetch_pack_args *args,\n+\t\t\t\t const char *fmt, ...)\n+{\n+\tva_list params;\n+\n+\tif (!args->verbose)\n+\t\treturn;\n+\n+\tva_start(params, fmt);\n+\tvfprintf(stderr, fmt, params);\n+\tva_end(params);\n+\tfputc('\\n', stderr);\n+}\n+\n static void rev_list_push(struct commit *commit, int mark)\n {\n \tif (!(commit->object.flags & mark)) {\n@@ -375,8 +390,7 @@ static int find_common(struct fetch_pack_args *args,\n \tretval = -1;\n \twhile ((sha1 = get_rev())) {\n \t\tpacket_buf_write(&req_buf, \"have %s\\n\", sha1_to_hex(sha1));\n-\t\tif (args->verbose)\n-\t\t\tfprintf(stderr, \"have %s\\n\", sha1_to_hex(sha1));\n+\t\tprint_verbose(args, \"have %s\", sha1_to_hex(sha1));\n \t\tin_vain++;\n \t\tif (flush_at <= ++count) {\n \t\t\tint ack;\n@@ -397,9 +411,9 @@ static int find_common(struct fetch_pack_args *args,\n \t\t\tconsume_shallow_list(args, fd[0]);\n \t\t\tdo {\n \t\t\t\tack = get_ack(fd[0], result_sha1);\n-\t\t\t\tif (args->verbose && ack)\n-\t\t\t\t\tfprintf(stderr, \"got ack %d %s\\n\", ack,\n-\t\t\t\t\t\t\tsha1_to_hex(result_sha1));\n+\t\t\t\tif (ack)\n+\t\t\t\t\tprint_verbose(args, \"got ack %d %s\", ack,\n+\t\t\t\t\t\t sha1_to_hex(result_sha1));\n \t\t\t\tswitch (ack) {\n \t\t\t\tcase ACK:\n \t\t\t\t\tflushes = 0;\n@@ -438,8 +452,7 @@ static int find_common(struct fetch_pack_args *args,\n \t\t\t} while (ack);\n \t\t\tflushes--;\n \t\t\tif (got_continue && MAX_IN_VAIN < in_vain) {\n-\t\t\t\tif (args->verbose)\n-\t\t\t\t\tfprintf(stderr, \"giving up\\n\");\n+\t\t\t\tprint_verbose(args, \"giving up\");\n \t\t\t\tbreak; /* give up */\n \t\t\t}\n \t\t}\n@@ -449,8 +462,7 @@ done:\n \t\tpacket_buf_write(&req_buf, \"done\\n\");\n \t\tsend_request(args, fd[1], &req_buf);\n \t}\n-\tif (args->verbose)\n-\t\tfprintf(stderr, \"done\\n\");\n+\tprint_verbose(args, \"done\");\n \tif (retval != 0) {\n \t\tmulti_ack = 0;\n \t\tflushes++;\n@@ -462,9 +474,8 @@ done:\n \twhile (flushes || multi_ack) {\n \t\tint ack = get_ack(fd[0], result_sha1);\n \t\tif (ack) {\n-\t\t\tif (args->verbose)\n-\t\t\t\tfprintf(stderr, \"got ack (%d) %s\\n\", ack,\n-\t\t\t\t\tsha1_to_hex(result_sha1));\n+\t\t\tprint_verbose(args, \"got ack (%d) %s\", ack,\n+\t\t\t\t sha1_to_hex(result_sha1));\n \t\t\tif (ack == ACK)\n \t\t\t\treturn 0;\n \t\t\tmulti_ack = 1;\n@@ -509,9 +520,8 @@ static void mark_recent_complete_commits(struct fetch_pack_args *args,\n \t\t\t\t\t unsigned long cutoff)\n {\n \twhile (complete && cutoff <= complete->item->date) {\n-\t\tif (args->verbose)\n-\t\t\tfprintf(stderr, \"Marking %s as complete\\n\",\n-\t\t\t\toid_to_hex(&complete->item->object.oid));\n+\t\tprint_verbose(args, \"Marking %s as complete\",\n+\t\t\t oid_to_hex(&complete->item->object.oid));\n \t\tpop_most_recent_commit(&complete, COMPLETE);\n \t}\n }\n@@ -652,18 +662,12 @@ static int everything_local(struct fetch_pack_args *args,\n \t\to = lookup_object(remote);\n \t\tif (!o || !(o->flags & COMPLETE)) {\n \t\t\tretval = 0;\n-\t\t\tif (!args->verbose)\n-\t\t\t\tcontinue;\n-\t\t\tfprintf(stderr,\n-\t\t\t\t\"want %s (%s)\\n\", sha1_to_hex(remote),\n-\t\t\t\tref->name);\n+\t\t\tprint_verbose(args, \"want %s (%s)\", sha1_to_hex(remote),\n+\t\t\t\t ref->name);\n \t\t\tcontinue;\n \t\t}\n-\t\tif (!args->verbose)\n-\t\t\tcontinue;\n-\t\tfprintf(stderr,\n-\t\t\t\"already have %s (%s)\\n\", sha1_to_hex(remote),\n-\t\t\tref->name);\n+\t\tprint_verbose(args, \"already have %s (%s)\", sha1_to_hex(remote),\n+\t\t\t ref->name);\n \t}\n \treturn retval;\n }\n@@ -810,39 +814,32 @@ static struct ref *do_fetch_pack(struct fetch_pack_args *args,\n \tif ((args->depth > 0 || is_repository_shallow()) && !server_supports(\"shallow\"))\n \t\tdie(\"Server does not support shallow clients\");\n \tif (server_supports(\"multi_ack_detailed\")) {\n-\t\tif (args->verbose)\n-\t\t\tfprintf(stderr, \"Server supports multi_ack_detailed\\n\");\n+\t\tprint_verbose(args, \"Server supports multi_ack_detailed\");\n \t\tmulti_ack = 2;\n \t\tif (server_supports(\"no-done\")) {\n-\t\t\tif (args->verbose)\n-\t\t\t\tfprintf(stderr, \"Server supports no-done\\n\");\n+\t\t\tprint_verbose(args, \"Server supports no-done\");\n \t\t\tif (args->stateless_rpc)\n \t\t\t\tno_done = 1;\n \t\t}\n \t}\n \telse if (server_supports(\"multi_ack\")) {\n-\t\tif (args->verbose)\n-\t\t\tfprintf(stderr, \"Server supports multi_ack\\n\");\n+\t\tprint_verbose(args, \"Server supports multi_ack\");\n \t\tmulti_ack = 1;\n \t}\n \tif (server_supports(\"side-band-64k\")) {\n-\t\tif (args->verbose)\n-\t\t\tfprintf(stderr, \"Server supports side-band-64k\\n\");\n+\t\tprint_verbose(args, \"Server supports side-band-64k\");\n \t\tuse_sideband = 2;\n \t}\n \telse if (server_supports(\"side-band\")) {\n-\t\tif (args->verbose)\n-\t\t\tfprintf(stderr, \"Server supports side-band\\n\");\n+\t\tprint_verbose(args, \"Server supports side-band\");\n \t\tuse_sideband = 1;\n \t}\n \tif (server_supports(\"allow-tip-sha1-in-want\")) {\n-\t\tif (args->verbose)\n-\t\t\tfprintf(stderr, \"Server supports allow-tip-sha1-in-want\\n\");\n+\t\tprint_verbose(args, \"Server supports allow-tip-sha1-in-want\");\n \t\tallow_unadvertised_object_request |= ALLOW_TIP_SHA1;\n \t}\n \tif (server_supports(\"allow-reachable-sha1-in-want\")) {\n-\t\tif (args->verbose)\n-\t\t\tfprintf(stderr, \"Server supports allow-reachable-sha1-in-want\\n\");\n+\t\tprint_verbose(args, \"Server supports allow-reachable-sha1-in-want\\n\");\n \t\tallow_unadvertised_object_request |= ALLOW_REACHABLE_SHA1;\n \t}\n \tif (!server_supports(\"thin-pack\"))\n@@ -851,17 +848,16 @@ static struct ref *do_fetch_pack(struct fetch_pack_args *args,\n \t\targs->no_progress = 0;\n \tif (!server_supports(\"include-tag\"))\n \t\targs->include_tag = 0;\n-\tif (server_supports(\"ofs-delta\")) {\n-\t\tif (args->verbose)\n-\t\t\tfprintf(stderr, \"Server supports ofs-delta\\n\");\n-\t} else\n+\tif (server_supports(\"ofs-delta\"))\n+\t\tprint_verbose(args, \"Server supports ofs-delta\");\n+\telse\n \t\tprefer_ofs_delta = 0;\n \n \tif ((agent_feature = server_feature_value(\"agent\", &agent_len))) {\n \t\tagent_supported = 1;\n-\t\tif (args->verbose && agent_len)\n-\t\t\tfprintf(stderr, \"Server version is %.*s\\n\",\n-\t\t\t\tagent_len, agent_feature);\n+\t\tif (agent_len)\n+\t\t\tprint_verbose(args, \"Server version is %.*s\",\n+\t\t\t\t agent_len, agent_feature);\n \t}\n \n \tif (everything_local(args, &ref, sought, nr_sought)) {\n-- \n2.3.0.rc1.137.g477eb31\n"},{"id":"275123","messageId":"1451391043-28093-12-git-send-email-pclouds@gmail.com","threadId":"41092","inReplyTo":"1451391043-28093-1-git-send-email-pclouds@gmail.com","subject":"[PATCH 11/20] fetch-pack: use a separate flag for fetch in deepening mode","fromName":"Nguyễn Thái Ngọc Duy","fromEmail":"pclouds@gmail.com","sentAt":"2015-12-29T12:10:34Z","receivedAt":"2015-12-29T12:10:34Z","isPatch":true,"sender":{"key":"pclouds@gmail.com","avatar":"https://avatars.githubusercontent.com/u/720?v=4"},"body":"The shallow repo could be deepened or shortened when then user gives\n--depth. But in future that won't be the only way to deepen/shorten a\nrepo. Stop relying on args->depth in this mode. Future deepening\nmethods can simply set this flag on instead of updating all these if\nexpressions.\n\nThe new name \"deepen\" was chosen after the command to define shallow\nboundary in pack protocol. New commands also follow this tradition.\n\nSigned-off-by: Nguyễn Thái Ngọc Duy \n---\n fetch-pack.c | 14 ++++++++------\n fetch-pack.h | 1 +\n 2 files changed, 9 insertions(+), 6 deletions(-)\n\ndiff --git a/fetch-pack.c b/fetch-pack.c\nindex 16917f9..e947514 100644\n--- a/fetch-pack.c\n+++ b/fetch-pack.c\n@@ -197,7 +197,7 @@ enum ack_type {\n \n static void consume_shallow_list(struct fetch_pack_args *args, int fd)\n {\n-\tif (args->stateless_rpc && args->depth > 0) {\n+\tif (args->stateless_rpc && args->deepen) {\n \t\t/* If we sent a depth we will get back \"duplicate\"\n \t\t * shallow and unshallow commands every time there\n \t\t * is a block of have lines exchanged.\n@@ -348,7 +348,7 @@ static int find_common(struct fetch_pack_args *args,\n \tpacket_buf_flush(&req_buf);\n \tstate_len = req_buf.len;\n \n-\tif (args->depth > 0) {\n+\tif (args->deepen) {\n \t\tchar *line;\n \t\tconst char *arg;\n \t\tunsigned char sha1[20];\n@@ -557,7 +557,7 @@ static void filter_refs(struct fetch_pack_args *args,\n \t\t}\n \n \t\tif (!keep && args->fetch_all &&\n-\t\t (!args->depth || !starts_with(ref->name, \"refs/tags/\")))\n+\t\t (!args->deepen || !starts_with(ref->name, \"refs/tags/\")))\n \t\t\tkeep = 1;\n \n \t\tif (keep) {\n@@ -627,7 +627,7 @@ static int everything_local(struct fetch_pack_args *args,\n \t\t}\n \t}\n \n-\tif (!args->depth) {\n+\tif (!args->deepen) {\n \t\tfor_each_ref(mark_complete_oid, NULL);\n \t\tfor_each_alternate_ref(mark_alternate_complete, NULL);\n \t\tcommit_list_sort_by_date(&complete);\n@@ -813,6 +813,8 @@ static struct ref *do_fetch_pack(struct fetch_pack_args *args,\n \n \tif ((args->depth > 0 || is_repository_shallow()) && !server_supports(\"shallow\"))\n \t\tdie(\"Server does not support shallow clients\");\n+\tif (args->depth > 0)\n+\t\targs->deepen = 1;\n \tif (server_supports(\"multi_ack_detailed\")) {\n \t\tprint_verbose(args, \"Server supports multi_ack_detailed\");\n \t\tmulti_ack = 2;\n@@ -873,7 +875,7 @@ static struct ref *do_fetch_pack(struct fetch_pack_args *args,\n \n \tif (args->stateless_rpc)\n \t\tpacket_flush(fd[1]);\n-\tif (args->depth > 0)\n+\tif (args->deepen)\n \t\tsetup_alternate_shallow(&shallow_lock, &alternate_shallow_file,\n \t\t\t\t\tNULL);\n \telse if (si->nr_ours || si->nr_theirs)\n@@ -940,7 +942,7 @@ static void update_shallow(struct fetch_pack_args *args,\n \tint *status;\n \tint i;\n \n-\tif (args->depth > 0 && alternate_shallow_file) {\n+\tif (args->deepen && alternate_shallow_file) {\n \t\tif (*alternate_shallow_file == '\\0') { /* --unshallow */\n \t\t\tunlink_or_warn(git_path_shallow());\n \t\t\trollback_lock_file(&shallow_lock);\ndiff --git a/fetch-pack.h b/fetch-pack.h\nindex bb7fd76..4d0adb0 100644\n--- a/fetch-pack.h\n+++ b/fetch-pack.h\n@@ -25,6 +25,7 @@ struct fetch_pack_args {\n \tunsigned self_contained_and_connected:1;\n \tunsigned cloning:1;\n \tunsigned update_shallow:1;\n+\tunsigned deepen:1;\n };\n \n /*\n-- \n2.3.0.rc1.137.g477eb31\n"},{"id":"275124","messageId":"1451391043-28093-13-git-send-email-pclouds@gmail.com","threadId":"41092","inReplyTo":"1451391043-28093-1-git-send-email-pclouds@gmail.com","subject":"[PATCH 12/20] fetch: define shallow boundary with --since","fromName":"Nguyễn Thái Ngọc Duy","fromEmail":"pclouds@gmail.com","sentAt":"2015-12-29T12:10:35Z","receivedAt":"2015-12-29T12:10:35Z","isPatch":true,"sender":{"key":"pclouds@gmail.com","avatar":"https://avatars.githubusercontent.com/u/720?v=4"},"body":"Signed-off-by: Nguyễn Thái Ngọc Duy \n---\n Documentation/fetch-options.txt | 4 ++++\n builtin/fetch.c | 12 ++++++++++--\n fetch-pack.c | 12 +++++++++++-\n fetch-pack.h | 1 +\n transport.c | 4 ++++\n transport.h | 4 ++++\n 6 files changed, 34 insertions(+), 3 deletions(-)\n\ndiff --git a/Documentation/fetch-options.txt b/Documentation/fetch-options.txt\nindex 45583d8..ff6e6ad 100644\n--- a/Documentation/fetch-options.txt\n+++ b/Documentation/fetch-options.txt\n@@ -13,6 +13,10 @@\n \tto the specified number of commits from the tip of each remote\n \tbranch history. Tags for the deepened commits are not fetched.\n \n+--since=::\n+\tDeepen or shorten the history of a 'shallow' repository to\n+\tinclude all reachable commits after .\n+\n --unshallow::\n \tIf the source repository is complete, convert a shallow\n \trepository to a complete one, removing all the limitations\ndiff --git a/builtin/fetch.c b/builtin/fetch.c\nindex c85f347..3317152 100644\n--- a/builtin/fetch.c\n+++ b/builtin/fetch.c\n@@ -36,8 +36,9 @@ static int prune = -1; /* unspecified */\n \n static int all, append, dry_run, force, keep, multiple, update_head_ok, verbosity;\n static int progress = -1, recurse_submodules = RECURSE_SUBMODULES_DEFAULT;\n-static int tags = TAGS_DEFAULT, unshallow, update_shallow;\n+static int tags = TAGS_DEFAULT, unshallow, update_shallow, deepen;\n static const char *depth;\n+static const char *deepen_since;\n static const char *upload_pack;\n static struct strbuf default_rla = STRBUF_INIT;\n static struct transport *gtransport;\n@@ -112,6 +113,8 @@ static struct option builtin_fetch_options[] = {\n \tOPT_BOOL(0, \"progress\", &progress, N_(\"force progress reporting\")),\n \tOPT_STRING(0, \"depth\", &depth, N_(\"depth\"),\n \t\t N_(\"deepen history of shallow clone\")),\n+\tOPT_STRING(0, \"since\", &deepen_since, N_(\"time\"),\n+\t\t N_(\"deepen history of shallow clone based on time\")),\n \t{ OPTION_SET_INT, 0, \"unshallow\", &unshallow, NULL,\n \t\t N_(\"convert to a complete repository\"),\n \t\t PARSE_OPT_NONEG | PARSE_OPT_NOARG, NULL, 1 },\n@@ -751,7 +754,7 @@ static int quickfetch(struct ref *ref_map)\n \t * really need to perform. Claiming failure now will ensure\n \t * we perform the network exchange to deepen our history.\n \t */\n-\tif (depth)\n+\tif (deepen)\n \t\treturn -1;\n \treturn check_everything_connected(iterate_ref_map, 1, &rm);\n }\n@@ -867,6 +870,8 @@ static struct transport *prepare_transport(struct remote *remote)\n \t\tset_option(transport, TRANS_OPT_KEEP, \"yes\");\n \tif (depth)\n \t\tset_option(transport, TRANS_OPT_DEPTH, depth);\n+\tif (deepen_since)\n+\t\tset_option(transport, TRANS_OPT_DEEPEN_SINCE, deepen_since);\n \tif (update_shallow)\n \t\tset_option(transport, TRANS_OPT_UPDATE_SHALLOW, \"yes\");\n \treturn transport;\n@@ -881,6 +886,7 @@ static void backfill_tags(struct transport *transport, struct ref *ref_map)\n \n \ttransport_set_option(transport, TRANS_OPT_FOLLOWTAGS, NULL);\n \ttransport_set_option(transport, TRANS_OPT_DEPTH, \"0\");\n+\ttransport_set_option(transport, TRANS_OPT_DEEPEN_SINCE, NULL);\n \tfetch_refs(transport, ref_map);\n \n \tif (gsecondary) {\n@@ -1164,6 +1170,8 @@ int cmd_fetch(int argc, const char **argv, const char *prefix)\n \t/* no need to be strict, transport_set_option() will validate it again */\n \tif (depth && atoi(depth) < 1)\n \t\tdie(_(\"depth %s is not a positive number\"), depth);\n+\tif (depth || deepen_since)\n+\t\tdeepen = 1;\n \n \tif (recurse_submodules != RECURSE_SUBMODULES_OFF) {\n \t\tif (recurse_submodules_default) {\ndiff --git a/fetch-pack.c b/fetch-pack.c\nindex e947514..f26cef4 100644\n--- a/fetch-pack.c\n+++ b/fetch-pack.c\n@@ -21,6 +21,7 @@ static int fetch_unpack_limit = -1;\n static int unpack_limit = 100;\n static int prefer_ofs_delta = 1;\n static int no_done;\n+static int deepen_since_ok;\n static int fetch_fsck_objects = -1;\n static int transfer_fsck_objects = -1;\n static int agent_supported;\n@@ -326,6 +327,7 @@ static int find_common(struct fetch_pack_args *args,\n \t\t\tif (args->no_progress) strbuf_addstr(&c, \" no-progress\");\n \t\t\tif (args->include_tag) strbuf_addstr(&c, \" include-tag\");\n \t\t\tif (prefer_ofs_delta) strbuf_addstr(&c, \" ofs-delta\");\n+\t\t\tif (deepen_since_ok) strbuf_addstr(&c, \" deepen-since\");\n \t\t\tif (agent_supported) strbuf_addf(&c, \" agent=%s\",\n \t\t\t\t\t\t\t git_user_agent_sanitized());\n \t\t\tpacket_buf_write(&req_buf, \"want %s%s\\n\", remote_hex, c.buf);\n@@ -345,6 +347,10 @@ static int find_common(struct fetch_pack_args *args,\n \t\twrite_shallow_commits(&req_buf, 1, NULL);\n \tif (args->depth > 0)\n \t\tpacket_buf_write(&req_buf, \"deepen %d\", args->depth);\n+\tif (args->deepen_since) {\n+\t\tunsigned long max_age = approxidate(args->deepen_since);\n+\t\tpacket_buf_write(&req_buf, \"deepen-since %lu\", max_age);\n+\t}\n \tpacket_buf_flush(&req_buf);\n \tstate_len = req_buf.len;\n \n@@ -813,7 +819,7 @@ static struct ref *do_fetch_pack(struct fetch_pack_args *args,\n \n \tif ((args->depth > 0 || is_repository_shallow()) && !server_supports(\"shallow\"))\n \t\tdie(\"Server does not support shallow clients\");\n-\tif (args->depth > 0)\n+\tif (args->depth > 0 || args->deepen_since)\n \t\targs->deepen = 1;\n \tif (server_supports(\"multi_ack_detailed\")) {\n \t\tprint_verbose(args, \"Server supports multi_ack_detailed\");\n@@ -861,6 +867,10 @@ static struct ref *do_fetch_pack(struct fetch_pack_args *args,\n \t\t\tprint_verbose(args, \"Server version is %.*s\",\n \t\t\t\t agent_len, agent_feature);\n \t}\n+\tif (server_supports(\"deepen-since\"))\n+\t\tdeepen_since_ok = 1;\n+\telse if (args->deepen_since)\n+\t\tdie(\"Server does not support --since\");\n \n \tif (everything_local(args, &ref, sought, nr_sought)) {\n \t\tpacket_flush(fd[1]);\ndiff --git a/fetch-pack.h b/fetch-pack.h\nindex 4d0adb0..f7eadb2 100644\n--- a/fetch-pack.h\n+++ b/fetch-pack.h\n@@ -10,6 +10,7 @@ struct fetch_pack_args {\n \tconst char *uploadpack;\n \tint unpacklimit;\n \tint depth;\n+\tconst char *deepen_since;\n \tunsigned quiet:1;\n \tunsigned keep_pack:1;\n \tunsigned lock_pack:1;\ndiff --git a/transport.c b/transport.c\nindex 67f3666..4902036 100644\n--- a/transport.c\n+++ b/transport.c\n@@ -477,6 +477,9 @@ static int set_git_option(struct git_transport_options *opts,\n \t\t\t\tdie(\"transport: invalid depth option '%s'\", value);\n \t\t}\n \t\treturn 0;\n+\t} else if (!strcmp(name, TRANS_OPT_DEEPEN_SINCE)) {\n+\t\topts->deepen_since = value;\n+\t\treturn 0;\n \t}\n \treturn 1;\n }\n@@ -530,6 +533,7 @@ static int fetch_refs_via_pack(struct transport *transport,\n \targs.quiet = (transport->verbose < 0);\n \targs.no_progress = !transport->progress;\n \targs.depth = data->options.depth;\n+\targs.deepen_since = data->options.deepen_since;\n \targs.check_self_contained_and_connected =\n \t\tdata->options.check_self_contained_and_connected;\n \targs.cloning = transport->cloning;\ndiff --git a/transport.h b/transport.h\nindex 8ebaaf2..9c10a44 100644\n--- a/transport.h\n+++ b/transport.h\n@@ -13,6 +13,7 @@ struct git_transport_options {\n \tunsigned self_contained_and_connected : 1;\n \tunsigned update_shallow : 1;\n \tint depth;\n+\tconst char *deepen_since;\n \tconst char *uploadpack;\n \tconst char *receivepack;\n \tstruct push_cas_option *cas;\n@@ -171,6 +172,9 @@ int transport_restrict_protocols(void);\n /* Limit the depth of the fetch if not null */\n #define TRANS_OPT_DEPTH \"depth\"\n \n+/* Limit the depth of the fetch based on time if not null */\n+#define TRANS_OPT_DEEPEN_SINCE \"deepen-since\"\n+\n /* Aggressively fetch annotated tags if possible */\n #define TRANS_OPT_FOLLOWTAGS \"followtags\"\n \n-- \n2.3.0.rc1.137.g477eb31\n"},{"id":"275125","messageId":"1451391043-28093-14-git-send-email-pclouds@gmail.com","threadId":"41092","inReplyTo":"1451391043-28093-1-git-send-email-pclouds@gmail.com","subject":"[PATCH 13/20] clone: define shallow clone boundary based on time with --since","fromName":"Nguyễn Thái Ngọc Duy","fromEmail":"pclouds@gmail.com","sentAt":"2015-12-29T12:10:36Z","receivedAt":"2015-12-29T12:10:36Z","isPatch":true,"sender":{"key":"pclouds@gmail.com","avatar":"https://avatars.githubusercontent.com/u/720?v=4"},"body":"Signed-off-by: Nguyễn Thái Ngọc Duy \n---\n Documentation/git-clone.txt | 3 +++\n builtin/clone.c | 16 +++++++++++++---\n 2 files changed, 16 insertions(+), 3 deletions(-)\n\ndiff --git a/Documentation/git-clone.txt b/Documentation/git-clone.txt\nindex 6bf000d..28993c6 100644\n--- a/Documentation/git-clone.txt\n+++ b/Documentation/git-clone.txt\n@@ -192,6 +192,9 @@ objects from the source repository into a pack in the cloned repository.\n \tCreate a 'shallow' clone with a history truncated to the\n \tspecified number of revisions.\n \n+--since=::\n+\tCreate a 'shallow' clone with a history after the specified time.\n+\n --[no-]single-branch::\n \tClone only the history leading to the tip of a single branch,\n \teither specified by the `--branch` option or the primary\ndiff --git a/builtin/clone.c b/builtin/clone.c\nindex a0b3cd9..94bbfef 100644\n--- a/builtin/clone.c\n+++ b/builtin/clone.c\n@@ -40,7 +40,8 @@ static const char * const builtin_clone_usage[] = {\n \n static int option_no_checkout, option_bare, option_mirror, option_single_branch = -1;\n static int option_local = -1, option_no_hardlinks, option_shared, option_recursive;\n-static char *option_template, *option_depth;\n+static int deepen;\n+static char *option_template, *option_depth, *option_since;\n static char *option_origin = NULL;\n static char *option_branch = NULL;\n static const char *real_git_dir;\n@@ -86,6 +87,8 @@ static struct option builtin_clone_options[] = {\n \t\t N_(\"path to git-upload-pack on the remote\")),\n \tOPT_STRING(0, \"depth\", &option_depth, N_(\"depth\"),\n \t\t N_(\"create a shallow clone of that depth\")),\n+\tOPT_STRING(0, \"since\", &option_since, N_(\"time\"),\n+\t\t N_(\"create a shallow clone since a specific time\")),\n \tOPT_BOOL(0, \"single-branch\", &option_single_branch,\n \t\t N_(\"clone only one branch, HEAD or --branch\")),\n \tOPT_STRING(0, \"separate-git-dir\", &real_git_dir, N_(\"gitdir\"),\n@@ -846,8 +849,10 @@ int cmd_clone(int argc, const char **argv, const char *prefix)\n \t\tusage_msg_opt(_(\"You must specify a repository to clone.\"),\n \t\t\tbuiltin_clone_usage, builtin_clone_options);\n \n+\tif (option_depth || option_since)\n+\t\tdeepen = 1;\n \tif (option_single_branch == -1)\n-\t\toption_single_branch = option_depth ? 1 : 0;\n+\t\toption_single_branch = deepen ? 1 : 0;\n \n \tif (option_mirror)\n \t\toption_bare = 1;\n@@ -973,6 +978,8 @@ int cmd_clone(int argc, const char **argv, const char *prefix)\n \tif (is_local) {\n \t\tif (option_depth)\n \t\t\twarning(_(\"--depth is ignored in local clones; use file:// instead.\"));\n+\t\tif (option_since)\n+\t\t\twarning(_(\"--since is ignored in local clones; use file:// instead.\"));\n \t\tif (!access(mkpath(\"%s/shallow\", path), F_OK)) {\n \t\t\tif (option_local > 0)\n \t\t\t\twarning(_(\"source repository is shallow, ignoring --local\"));\n@@ -991,6 +998,9 @@ int cmd_clone(int argc, const char **argv, const char *prefix)\n \tif (option_depth)\n \t\ttransport_set_option(transport, TRANS_OPT_DEPTH,\n \t\t\t\t option_depth);\n+\tif (option_since)\n+\t\ttransport_set_option(transport, TRANS_OPT_DEEPEN_SINCE,\n+\t\t\t\t option_since);\n \tif (option_single_branch)\n \t\ttransport_set_option(transport, TRANS_OPT_FOLLOWTAGS, \"1\");\n \n@@ -998,7 +1008,7 @@ int cmd_clone(int argc, const char **argv, const char *prefix)\n \t\ttransport_set_option(transport, TRANS_OPT_UPLOADPACK,\n \t\t\t\t option_upload_pack);\n \n-\tif (transport->smart_options && !option_depth)\n+\tif (transport->smart_options && !deepen)\n \t\ttransport->smart_options->check_self_contained_and_connected = 1;\n \n \trefs = transport_get_remote_refs(transport);\n-- \n2.3.0.rc1.137.g477eb31\n"},{"id":"275126","messageId":"1451391043-28093-15-git-send-email-pclouds@gmail.com","threadId":"41092","inReplyTo":"1451391043-28093-1-git-send-email-pclouds@gmail.com","subject":"[PATCH 14/20] Add test_repo_expect_success for running tests in a new repository","fromName":"Nguyễn Thái Ngọc Duy","fromEmail":"pclouds@gmail.com","sentAt":"2015-12-29T12:10:37Z","receivedAt":"2015-12-29T12:10:37Z","isPatch":true,"sender":{"key":"pclouds@gmail.com","avatar":"https://avatars.githubusercontent.com/u/720?v=4"},"body":"This could be convenient when tests are independent from the rest in the\nsame file. Normally we would do this\n\ntest_expect_success '...' '\n\tgit init foo &&\n\t(\n\t\tcd foo &&\n\t\t