{"thread":{"id":"43557","subject":"[PATCH 2/5] support fetching into a shallow repository","startedAt":"2006-10-30T19:09:06Z","lastAt":"2006-11-14T17:42:34Z","messageCount":4,"participants":["Johannes Schindelin","Junio C Hamano"],"isPatch":true,"patchVersion":1,"patchTotal":5},"messages":[{"id":"295494","messageId":"Pine.LNX.4.63.0610302008520.26682@wbgn013.biozentrum.uni-wuerzburg.de","threadId":"43557","inReplyTo":null,"subject":"[PATCH 2/5] support fetching into a shallow repository","fromName":"Johannes Schindelin","fromEmail":"johannes.schindelin@gmx.de","sentAt":"2006-10-30T19:09:06Z","receivedAt":"2006-10-30T19:09:06Z","isPatch":true,"sender":{"key":"johannes.schindelin@gmx.de","avatar":"https://avatars.githubusercontent.com/u/127790?v=4"},"body":"\nA shallow commit is a commit which has parents, which in turn are\n\"grafted away\", i.e. the commit appears as if it were a root.\n\nSince these shallow commits should not be edited by the user, but\nonly by core git, they are recorded in the file $GIT_DIR/shallow.\n\nA repository containing shallow commits is called shallow.\n\nThe advantage of a shallow repository is that even if the upstream\ncontains lots of history, your local (shallow) repository needs not\noccupy much disk space.\n\nThe disadvantage is that you might miss a merge base when pulling\nsome remote branch.\n\nSigned-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>\n---\n Makefile      |    2 +-\n commit.c      |   21 ++++++++++++\n commit.h      |    8 ++++-\n fetch-pack.c  |    4 ++\n shallow.c     |   97 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++\n upload-pack.c |   22 ++++++++++++-\n 6 files changed, 150 insertions(+), 4 deletions(-)\n\ndiff --git a/Makefile b/Makefile\nindex 28a5d91..5f57aad 100644\n--- a/Makefile\n+++ b/Makefile\n@@ -263,7 +263,7 @@ LIB_OBJS = \\\n \tfetch-clone.o revision.o pager.o tree-walk.o xdiff-interface.o \\\n \twrite_or_die.o trace.o list-objects.o grep.o \\\n \talloc.o merge-file.o path-list.o help.o unpack-trees.o $(DIFF_OBJS) \\\n-\tcolor.o wt-status.o archive-zip.o archive-tar.o\n+\tcolor.o wt-status.o archive-zip.o archive-tar.o shallow.o\n \n BUILTIN_OBJS = \\\n \tbuiltin-add.o \\\ndiff --git a/commit.c b/commit.c\nindex a6d543e..bffa278 100644\n--- a/commit.c\n+++ b/commit.c\n@@ -1,6 +1,7 @@\n #include \"cache.h\"\n #include \"tag.h\"\n #include \"commit.h\"\n+#include \"pkt-line.h\"\n \n int save_commit_buffer = 1;\n \n@@ -221,6 +222,8 @@ static void prepare_commit_graft(void)\n \t\treturn;\n \tgraft_file = get_graft_file();\n \tread_graft_file(graft_file);\n+\t/* make sure shallows are read */\n+\tis_repository_shallow();\n \tcommit_graft_prepared = 1;\n }\n \n@@ -234,6 +237,24 @@ static struct commit_graft *lookup_commi\n \treturn commit_graft[pos];\n }\n \n+int write_shallow_commits(int fd, int use_pack_protocol)\n+{\n+\tint i, count = 0;\n+\tfor (i = 0; i < commit_graft_nr; i++)\n+\t\tif (commit_graft[i]->nr_parent < 0) {\n+\t\t\tconst char *hex =\n+\t\t\t\tsha1_to_hex(commit_graft[i]->sha1);\n+\t\t\tcount++;\n+\t\t\tif (use_pack_protocol)\n+\t\t\t\tpacket_write(fd, \"shallow %s\", hex);\n+\t\t\telse {\n+\t\t\t\twrite(fd, hex,  40);\n+\t\t\t\twrite(fd, \"\\n\", 1);\n+\t\t\t}\n+\t\t}\n+\treturn count;\n+}\n+\n int parse_commit_buffer(struct commit *item, void *buffer, unsigned long size)\n {\n \tchar *tail = buffer;\ndiff --git a/commit.h b/commit.h\nindex fc13de9..c559510 100644\n--- a/commit.h\n+++ b/commit.h\n@@ -97,7 +97,7 @@ void sort_in_topological_order_fn(struct\n \n struct commit_graft {\n \tunsigned char sha1[20];\n-\tint nr_parent;\n+\tint nr_parent; /* < 0 if shallow commit */\n \tunsigned char parent[FLEX_ARRAY][20]; /* more */\n };\n \n@@ -107,4 +107,10 @@ int read_graft_file(const char *graft_fi\n \n extern struct commit_list *get_merge_bases(struct commit *rev1, struct commit *rev2, int cleanup);\n \n+extern int register_shallow(const unsigned char *sha1);\n+extern int write_shallow_commits(int fd, int use_pack_protocol);\n+extern int is_repository_shallow();\n+extern struct commit_list *get_shallow_commits(struct object_array *heads,\n+\t\tint depth);\n+\n #endif /* COMMIT_H */\ndiff --git a/fetch-pack.c b/fetch-pack.c\nindex 36ea092..488adc9 100644\n--- a/fetch-pack.c\n+++ b/fetch-pack.c\n@@ -177,6 +177,8 @@ static int find_common(int fd[2], unsign\n \t\t\tpacket_write(fd[1], \"want %s\\n\", sha1_to_hex(remote));\n \t\tfetching++;\n \t}\n+\tif (is_repository_shallow())\n+\t\twrite_shallow_commits(fd[1], 1);\n \tpacket_flush(fd[1]);\n \tif (!fetching)\n \t\treturn 1;\n@@ -423,6 +425,8 @@ static int fetch_pack(int fd[2], int nr_\n \tint status;\n \n \tget_remote_heads(fd[0], &ref, 0, NULL, 0);\n+\tif (is_repository_shallow() && !server_supports(\"shallow\"))\n+\t\tdie(\"Server does not support shallow clients\");\n \tif (server_supports(\"multi_ack\")) {\n \t\tif (verbose)\n \t\t\tfprintf(stderr, \"Server supports multi_ack\\n\");\ndiff --git a/shallow.c b/shallow.c\nnew file mode 100644\nindex 0000000..3cf2127\n--- /dev/null\n+++ b/shallow.c\n@@ -0,0 +1,97 @@\n+#include \"cache.h\"\n+#include \"commit.h\"\n+\n+static int is_shallow = -1;\n+\n+int register_shallow(const unsigned char *sha1)\n+{\n+\tstruct commit_graft *graft =\n+\t\txmalloc(sizeof(struct commit_graft));\n+\tstruct commit *commit = lookup_commit(sha1);\n+\n+\thashcpy(graft->sha1, sha1);\n+\tgraft->nr_parent = -1;\n+\tif (commit && commit->object.parsed)\n+\t\tcommit->parents = NULL;\n+\treturn register_commit_graft(graft, 0);\n+}\n+\n+int is_repository_shallow()\n+{\n+\tFILE *fp;\n+\tchar buf[1024];\n+\n+\tif (is_shallow >= 0)\n+\t\treturn is_shallow;\n+\n+\tfp = fopen(git_path(\"shallow\"), \"r\");\n+\tif (!fp) {\n+\t\tis_shallow = 0;\n+\t\treturn is_shallow;\n+\t}\n+\tis_shallow = 1;\n+\n+\twhile (fgets(buf, sizeof(buf), fp)) {\n+\t\tunsigned char sha1[20];\n+\t\tif (get_sha1_hex(buf, sha1))\n+\t\t\tdie(\"bad shallow line: %s\", buf);\n+\t\tregister_shallow(sha1);\n+\t}\n+\tfclose(fp);\n+\treturn is_shallow;\n+}\n+\n+struct commit_list *get_shallow_commits(struct object_array *heads, int depth)\n+{\n+\tint i = 0, cur_depth = 0;\n+\tstruct commit_list *result = NULL;\n+\tstruct object_array stack = {0, 0, NULL};\n+\tstruct commit *commit = NULL;\n+\n+\twhile (commit || i < heads->nr || stack.nr) {\n+\t\tstruct commit_list *p;\n+\t\tif (!commit) {\n+\t\t\tif (i < heads->nr) {\n+\t\t\t\tcommit = (struct commit *)\n+\t\t\t\t\theads->objects[i++].item;\n+\t\t\t\tif (commit->object.type != OBJ_COMMIT) {\n+\t\t\t\t\tcommit = NULL;\n+\t\t\t\t\tcontinue;\n+\t\t\t\t}\n+\t\t\t\tcommit->util = xcalloc(1, sizeof(int));\n+\t\t\t\tcur_depth = 0;\n+\t\t\t} else {\n+\t\t\t\tcommit = (struct commit *)\n+\t\t\t\t\tstack.objects[--stack.nr].item;\n+\t\t\t\tcur_depth = *(int *)commit->util;\n+\t\t\t}\n+\t\t}\n+\t\tparse_commit(commit);\n+\t\tcur_depth++;\n+\t\tfor (p = commit->parents, commit = NULL; p; p = p->next) {\n+\t\t\tif (!p->item->util) {\n+\t\t\t\tint *pointer = xmalloc(sizeof(int));\n+\t\t\t\tp->item->util = pointer;\n+\t\t\t\t*pointer =  cur_depth;\n+\t\t\t} else {\n+\t\t\t\tint *pointer = p->item->util;\n+\t\t\t\tif (cur_depth >= *pointer)\n+\t\t\t\t\tcontinue;\n+\t\t\t\t*pointer = cur_depth;\n+\t\t\t}\n+\t\t\tif (cur_depth < depth) {\n+\t\t\t\tif (p->next)\n+\t\t\t\t\tadd_object_array(&p->item->object,\n+\t\t\t\t\t\t\tNULL, &stack);\n+\t\t\t\telse {\n+\t\t\t\t\tcommit = p->item;\n+\t\t\t\t\tcur_depth = *(int *)commit->util;\n+\t\t\t\t}\n+\t\t\t} else\n+\t\t\t\tcommit_list_insert(p->item, &result);\n+\t\t}\n+\t}\n+\n+\treturn result;\n+}\n+\ndiff --git a/upload-pack.c b/upload-pack.c\nindex 7f7df2a..8dd6121 100644\n--- a/upload-pack.c\n+++ b/upload-pack.c\n@@ -486,6 +486,7 @@ static int get_common_commits(void)\n \n static void receive_needs(void)\n {\n+\tstruct object_array shallows = {0, 0, NULL};\n \tstatic char line[1000];\n \tint len;\n \n@@ -495,8 +496,19 @@ static void receive_needs(void)\n \t\tlen = packet_read_line(0, line, sizeof(line));\n \t\treset_timeout();\n \t\tif (!len)\n-\t\t\treturn;\n+\t\t\tbreak;\n \n+\t\tif (!strncmp(\"shallow \", line, 8)) {\n+\t\t\tunsigned char sha1[20];\n+\t\t\tstruct object *object;\n+\t\t\tif (get_sha1(line + 8, sha1))\n+\t\t\t\tdie(\"invalid shallow line: %s\", line);\n+\t\t\tobject = parse_object(sha1);\n+\t\t\tif (!object)\n+\t\t\t\tdie(\"did not find object for %s\", line);\n+\t\t\tadd_object_array(object, NULL, &shallows);\n+\t\t\tcontinue;\n+\t\t}\n \t\tif (strncmp(\"want \", line, 5) ||\n \t\t    get_sha1_hex(line+5, sha1_buf))\n \t\t\tdie(\"git-upload-pack: protocol error, \"\n@@ -528,11 +540,17 @@ static void receive_needs(void)\n \t\t\tadd_object_array(o, NULL, &want_obj);\n \t\t}\n \t}\n+\tif (shallows.nr > 0) {\n+\t\tint i;\n+\t\tfor (i = 0; i < shallows.nr; i++)\n+\t\t\tregister_shallow(shallows.objects[i].item->sha1);\n+\t}\n }\n \n static int send_ref(const char *refname, const unsigned char *sha1, int flag, void *cb_data)\n {\n-\tstatic const char *capabilities = \"multi_ack thin-pack side-band side-band-64k ofs-delta\";\n+\tstatic const char *capabilities = \"multi_ack thin-pack side-band\"\n+\t\t\" side-band-64k ofs-delta shallow\";\n \tstruct object *o = parse_object(sha1);\n \n \tif (!o)\n-- \n1.4.3.3.gca42\n"},{"id":"296843","messageId":"7vac2u1oee.fsf@assigned-by-dhcp.cox.net","threadId":"43557","inReplyTo":"Pine.LNX.4.63.0610302008520.26682@wbgn013.biozentrum.uni-wuerzburg.de","subject":"Re: [PATCH 2/5] support fetching into a shallow repository","fromName":"Junio C Hamano","fromEmail":"junkio@cox.net","sentAt":"2006-11-14T07:24:41Z","receivedAt":"2006-11-14T07:24:41Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Johannes Schindelin <Johannes.Schindelin@gmx.de> writes:\n\n> A shallow commit is a commit which has parents, which in turn are\n> \"grafted away\", i.e. the commit appears as if it were a root.\n>\n> Since these shallow commits should not be edited by the user, but\n> only by core git, they are recorded in the file $GIT_DIR/shallow.\n>\n> A repository containing shallow commits is called shallow.\n>\n> The advantage of a shallow repository is that even if the upstream\n> contains lots of history, your local (shallow) repository needs not\n> occupy much disk space.\n>\n> The disadvantage is that you might miss a merge base when pulling\n> some remote branch.\n>\n> Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>\n> ---\n> diff --git a/shallow.c b/shallow.c\n> new file mode 100644\n> index 0000000..3cf2127\n> --- /dev/null\n> +++ b/shallow.c\n>...\n> +struct commit_list *get_shallow_commits(struct object_array *heads, int depth)\n> +{\n> +\tint i = 0, cur_depth = 0;\n> +\tstruct commit_list *result = NULL;\n> +\tstruct object_array stack = {0, 0, NULL};\n> +\tstruct commit *commit = NULL;\n> +\n> +\twhile (commit || i < heads->nr || stack.nr) {\n> +\t\tstruct commit_list *p;\n> +\t\tif (!commit) {\n>...\n> +\t\t}\n> +\t\tparse_commit(commit);\n> +\t\tcur_depth++;\n> +\t\tfor (p = commit->parents, commit = NULL; p; p = p->next) {\n> +\t\t\tif (!p->item->util) {\n>...\n> +\t\t\t} else {\n>...\n> +\t\t\t}\n> +\t\t\tif (cur_depth < depth) {\n> +\t\t\t\tif (p->next)\n> +\t\t\t\t\tadd_object_array(&p->item->object,\n> +\t\t\t\t\t\t\tNULL, &stack);\n> +\t\t\t\telse {\n> +\t\t\t\t\tcommit = p->item;\n> +\t\t\t\t\tcur_depth = *(int *)commit->util;\n> +\t\t\t\t}\n> +\t\t\t} else\n> +\t\t\t\tcommit_list_insert(p->item, &result);\n> +\t\t}\n> +\t}\n> +\n> +\treturn result;\n> +}\n\nI think the \"commit = p->item\" part is trying to do a tail\nrecursion optimization, but this is a bit too clever to my\nliking (at first I mistook that the code forgot to re-point p at\nits parents list and incrementing cur_depth).\n"},{"id":"296041","messageId":"Pine.LNX.4.63.0611141140530.13772@wbgn013.biozentrum.uni-wuerzburg.de","threadId":"43557","inReplyTo":"7vac2u1oee.fsf@assigned-by-dhcp.cox.net","subject":"Re: [PATCH 2/5] support fetching into a shallow repository","fromName":"Johannes Schindelin","fromEmail":"johannes.schindelin@gmx.de","sentAt":"2006-11-14T10:43:31Z","receivedAt":"2006-11-14T10:43:31Z","isPatch":true,"sender":{"key":"johannes.schindelin@gmx.de","avatar":"https://avatars.githubusercontent.com/u/127790?v=4"},"body":"Hi,\n\nOn Mon, 13 Nov 2006, Junio C Hamano wrote:\n\n> Johannes Schindelin <Johannes.Schindelin@gmx.de> writes:\n> \n> > diff --git a/shallow.c b/shallow.c\n> > new file mode 100644\n> > index 0000000..3cf2127\n> > --- /dev/null\n> > +++ b/shallow.c\n> >...\n> > +struct commit_list *get_shallow_commits(struct object_array *heads, int depth)\n> > +{\n> > +\tint i = 0, cur_depth = 0;\n> > +\tstruct commit_list *result = NULL;\n> > +\tstruct object_array stack = {0, 0, NULL};\n> > +\tstruct commit *commit = NULL;\n> > +\n> > +\twhile (commit || i < heads->nr || stack.nr) {\n> > +\t\tstruct commit_list *p;\n> > +\t\tif (!commit) {\n> >...\n> > +\t\t}\n> > +\t\tparse_commit(commit);\n> > +\t\tcur_depth++;\n> > +\t\tfor (p = commit->parents, commit = NULL; p; p = p->next) {\n> > +\t\t\tif (!p->item->util) {\n> >...\n> > +\t\t\t} else {\n> >...\n> > +\t\t\t}\n> > +\t\t\tif (cur_depth < depth) {\n> > +\t\t\t\tif (p->next)\n> > +\t\t\t\t\tadd_object_array(&p->item->object,\n> > +\t\t\t\t\t\t\tNULL, &stack);\n> > +\t\t\t\telse {\n> > +\t\t\t\t\tcommit = p->item;\n> > +\t\t\t\t\tcur_depth = *(int *)commit->util;\n> > +\t\t\t\t}\n> > +\t\t\t} else\n> > +\t\t\t\tcommit_list_insert(p->item, &result);\n> > +\t\t}\n> > +\t}\n> > +\n> > +\treturn result;\n> > +}\n> \n> I think the \"commit = p->item\" part is trying to do a tail\n> recursion optimization, but this is a bit too clever to my\n> liking (at first I mistook that the code forgot to re-point p at\n> its parents list and incrementing cur_depth).\n\nI take it as a compliment ;-)\n\nSeriously again, would you like me to add a comment, or rather do away \nwith the tail recursion optimization? It is not a huge optimization \nanyway. Maybe a cleverer way would be to use an object_array instead of a \ncommit_list?\n\nCiao,\nDscho\n"},{"id":"296769","messageId":"7vu011zzzp.fsf@assigned-by-dhcp.cox.net","threadId":"43557","inReplyTo":"Pine.LNX.4.63.0611141140530.13772@wbgn013.biozentrum.uni-wuerzburg.de","subject":"Re: [PATCH 2/5] support fetching into a shallow repository","fromName":"Junio C Hamano","fromEmail":"junkio@cox.net","sentAt":"2006-11-14T17:42:34Z","receivedAt":"2006-11-14T17:42:34Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Johannes Schindelin <Johannes.Schindelin@gmx.de> writes:\n\n>> I think the \"commit = p->item\" part is trying to do a tail\n>> recursion optimization, but this is a bit too clever to my\n>> liking (at first I mistook that the code forgot to re-point p at\n>> its parents list and incrementing cur_depth).\n>\n> I take it as a compliment ;-)\n>\n> Seriously again, would you like me to add a comment, or rather do away \n> with the tail recursion optimization? It is not a huge optimization \n> anyway. Maybe a cleverer way would be to use an object_array instead of a \n> commit_list?\n\nLeaving it as a compliment is just fine.\n"}]}