{"thread":{"id":"24851","subject":"[RFD PATCH 00/32] subtree clone v2","startedAt":"2010-08-24T22:19:50Z","lastAt":"2010-08-26T04:45:35Z","messageCount":66,"participants":["Nguyễn Thái Ngọc Duy","Jonathan Nieder","Nguyen Thai Ngoc Duy","Elijah Newren","Junio C Hamano"],"isPatch":true,"patchVersion":1,"patchTotal":32},"messages":[{"id":"148867","messageId":"1282688422-7738-1-git-send-email-pclouds@gmail.com","threadId":"24851","inReplyTo":null,"subject":"[RFD PATCH 00/32] subtree clone v2","fromName":"Nguyễn Thái Ngọc Duy","fromEmail":"pclouds@gmail.com","sentAt":"2010-08-24T22:19:50Z","receivedAt":"2010-08-24T22:19:50Z","isPatch":true,"sender":{"key":"pclouds@gmail.com","avatar":"https://avatars.githubusercontent.com/u/720?v=4"},"body":"This is a follow-up from the subtree clone proof of concept [1] I\nposted a few weeks ago. I think it's getting closer to an acceptable\ndesign. Hence the purpose of this post: to ask for opinions about the\ndesign issues I'm facing.\n\nThe good news is that tree rewrite/replace has gone. The repo now is\nperfectly \"normal\". Git client has to be modified to live peacefully\nwithin a narrow tree. How much modification is still a question ahead.\n\nOther things from the previous series are pretty much unchanged:\nsingle narrow tree, only send necessary trees, can be used together\nwith shallow clone, can't be fetch source.\n\nA new thing is the index is now narrowed down, because I don't have\nall trees to fully populate it, and tree rewrite mechanism is now\ngone. One item for RFD.\n\nAnother major thing (you guys are going to hate me) is merge could not\nalways be done locally. So I introduce \"trivial 3-way remote\nmerge\". RFD material (any other aproaches?). More details in commit\nmessages.\n\nIs there any other full-tree operations, like merge, that I should pay\nattention to? What commands can be troublesome?\n\nLast note, please don't look too closely. The code is full of bugs :-)\n\n\nPatches 1..7\n add const to ce_write()\n cache-tree: abstract out write_sha1_file from cache_tree_update()\n cache-tree: ignore CE_REMOVE entries in verify_cache()\n move do_compress() from pack-objects.c to pack-write.c\n pack-write: add functions for creating simple packs\n tree.c: Add {set,clear}_tree_marks\n tree.c: find_subtree() to search for a tree\n\nHelper patches. You can skip these if you are only interested in the\nnarrow ideas.\n\n\nPatches 8..14\n Add $GIT_DIR/narrow check\n index: make narrow index incompatible with older git\n rev-list: support traversing in narrow repository mode\n rev-list: support --narrow-tree\n pack-objects: support --narrow-tree\n upload-pack: support narrow-tree capability\n fetch-pack: support --narrow-tree\n\nNarrow clone. traverse_commit_list() does not change much since the\nlast series.\n\n\nPatches 15..19\n unpack_trees: only unpack $GIT_DIR/narrow subtree in narrow\n repository\n cache-tree: only cache tree within narrow area\n tree-diff: add narrow versions of diff_{root_,}tree_sha1\n log-tree: use narrow version of diff_tree_sha1\n clone: support --narrow option\n\nClient support so that you can check files out, diff them... This is WIP.\nBasically if you stay inside narrow tree, you'd be safe :-) Elijah may\nwant to reuse some of these code in his sparse clone, if he can extend\nit to support multiple narrow trees, I think.\n\n\nPatches 20..25\n narrow-tree: add join_narrow_tree to do tree fixup for commits\n commit: add narrow's commit_tree version\n commit: use commit_narrow_tree() to support narrow repo\n commit-tree: require --narrow-base in narrow repo\n merge: refuse to merge if narrow bases are different\n merge: prepare commit properly in narrow mode\n\nCommit part, how to use narrow index and make proper commits. This is\nwhere merge issue arises. Because a merge can have many parents, which\nparent can be used as a base to create new commits?\n\n\nPatches 26..30\n Add upload-narrow-base command\n rev-list: traverse some more trees to make upload-narrow-base happy\n narrow-tree: add oldest_narrow_base()\n Add command fetch-narrow-base\n merge: support merging when narrow bases are different\n\nRemote merge part.\n\nSplit a merge operation into two parts, the real merge will be done\nwithin narrow tree. Conflicts can happen and be resolved in the narrow\nindex, locally.\n\nEverything outside narrow tree will be merged (trivially) by\nserver. Then server sends the base tree back, so join_narrow_tree() in\npatch 20 can be used to create proper commit.\n\nServer can disable this remote merge feature, which means users are\nforced to do rebase/fast-forward. Not too bad.\n\n\nPatches 31..32\n send-pack: do not use thin pack in narrow mode\n daemon: support upload-narrow-base\n\nMisc stuff..\n\n[1] http://mid.gmane.org/1280593105-22015-1-git-send-email-pclouds@gmail.com\n\n Makefile | 4 +\n builtin.h | 2 +\n builtin/clone.c | 15 +++-\n builtin/commit-tree.c | 20 ++++-\n builtin/commit.c | 10 ++-\n builtin/fetch-narrow-base.c | 89 +++++++++++++++++\n builtin/fetch-pack.c | 11 ++\n builtin/merge.c | 45 +++++++++-\n builtin/pack-objects.c | 33 +++----\n builtin/send-pack.c | 4 +\n builtin/upload-narrow-base.c | 215 ++++++++++++++++++++++++++++++++++++++++++\n cache-tree.c | 37 ++++++--\n cache-tree.h | 3 +\n cache.h | 4 +\n commit.c | 21 ++++\n commit.h | 5 +\n daemon.c | 7 ++\n diff.h | 5 +\n environment.c | 2 +\n git.c | 2 +\n list-objects.c | 91 +++++++++++++++++-\n log-tree.c | 23 ++++-\n narrow-tree.c | 200 +++++++++++++++++++++++++++++++++++++++\n narrow-tree.h | 9 ++\n pack-write.c | 112 ++++++++++++++++++++++\n pack.h | 18 ++++\n read-cache.c | 37 ++++++-\n revision.c | 5 +\n revision.h | 4 +-\n t/t1013-read-tree-narrow.sh | 72 ++++++++++++++\n t/t6060-narrow-tree.sh | 28 ++++++\n t/t6061-rev-list-narrow.sh | 185 ++++++++++++++++++++++++++++++++++++\n tree-diff.c | 74 ++++++++++++++\n tree.c | 79 +++++++++++++++\n tree.h | 4 +\n unpack-trees.c | 70 ++++++++++++++-\n upload-pack.c | 31 ++++++-\n 37 files changed, 1528 insertions(+), 48 deletions(-)\n create mode 100644 builtin/fetch-narrow-base.c\n create mode 100644 builtin/upload-narrow-base.c\n create mode 100644 narrow-tree.c\n create mode 100644 narrow-tree.h\n create mode 100755 t/t1013-read-tree-narrow.sh\n create mode 100755 t/t6060-narrow-tree.sh\n create mode 100755 t/t6061-rev-list-narrow.sh\n"},{"id":"148870","messageId":"1282688422-7738-2-git-send-email-pclouds@gmail.com","threadId":"24851","inReplyTo":"1282688422-7738-1-git-send-email-pclouds@gmail.com","subject":"[PATCH 01/32] add const to ce_write()","fromName":"Nguyễn Thái Ngọc Duy","fromEmail":"pclouds@gmail.com","sentAt":"2010-08-24T22:19:51Z","receivedAt":"2010-08-24T22:19:51Z","isPatch":true,"sender":{"key":"pclouds@gmail.com","avatar":"https://avatars.githubusercontent.com/u/720?v=4"},"body":"\nSigned-off-by: Nguyễn Thái Ngọc Duy \n---\n read-cache.c | 2 +-\n 1 files changed, 1 insertions(+), 1 deletions(-)\n\ndiff --git a/read-cache.c b/read-cache.c\nindex 1f42473..881dd93 100644\n--- a/read-cache.c\n+++ b/read-cache.c\n@@ -1409,7 +1409,7 @@ static int ce_write_flush(git_SHA_CTX *context, int fd)\n \treturn 0;\n }\n \n-static int ce_write(git_SHA_CTX *context, int fd, void *data, unsigned int len)\n+static int ce_write(git_SHA_CTX *context, int fd, const void *data, unsigned int len)\n {\n \twhile (len) {\n \t\tunsigned int buffered = write_buffer_len;\n-- \n1.7.1.rc1.69.g24c2f7\n"},{"id":"148875","messageId":"1282688422-7738-3-git-send-email-pclouds@gmail.com","threadId":"24851","inReplyTo":"1282688422-7738-1-git-send-email-pclouds@gmail.com","subject":"[PATCH 02/32] cache-tree: abstract out write_sha1_file from cache_tree_update()","fromName":"Nguyễn Thái Ngọc Duy","fromEmail":"pclouds@gmail.com","sentAt":"2010-08-24T22:19:52Z","receivedAt":"2010-08-24T22:19:52Z","isPatch":true,"sender":{"key":"pclouds@gmail.com","avatar":"https://avatars.githubusercontent.com/u/720?v=4"},"body":"This allows cache-tree to produce trees directly in a pack, for instance.\n\nSigned-off-by: Nguyễn Thái Ngọc Duy \n---\n cache-tree.c | 18 +++++++++++++++---\n cache-tree.h | 3 +++\n 2 files changed, 18 insertions(+), 3 deletions(-)\n\ndiff --git a/cache-tree.c b/cache-tree.c\nindex c60cf91..7fd7203 100644\n--- a/cache-tree.c\n+++ b/cache-tree.c\n@@ -237,6 +237,7 @@ static int update_one(struct cache_tree *it,\n \t\t int entries,\n \t\t const char *base,\n \t\t int baselen,\n+\t\t write_tree_fn write_tree,\n \t\t int missing_ok,\n \t\t int dryrun)\n {\n@@ -284,6 +285,7 @@ static int update_one(struct cache_tree *it,\n \t\t\t\t cache + i, entries - i,\n \t\t\t\t path,\n \t\t\t\t baselen + sublen + 1,\n+\t\t\t\t write_tree,\n \t\t\t\t missing_ok,\n \t\t\t\t dryrun);\n \t\tif (subcnt < 0)\n@@ -349,7 +351,7 @@ static int update_one(struct cache_tree *it,\n \n \tif (dryrun)\n \t\thash_sha1_file(buffer.buf, buffer.len, tree_type, it->sha1);\n-\telse if (write_sha1_file(buffer.buf, buffer.len, tree_type, it->sha1)) {\n+\telse if (write_tree(buffer.buf, buffer.len, tree_type, it->sha1)) {\n \t\tstrbuf_release(&buffer);\n \t\treturn -1;\n \t}\n@@ -364,9 +366,10 @@ static int update_one(struct cache_tree *it,\n \treturn i;\n }\n \n-int cache_tree_update(struct cache_tree *it,\n+int cache_tree_update_fn(struct cache_tree *it,\n \t\t struct cache_entry **cache,\n \t\t int entries,\n+\t\t write_tree_fn write_tree,\n \t\t int missing_ok,\n \t\t int dryrun)\n {\n@@ -374,12 +377,21 @@ int cache_tree_update(struct cache_tree *it,\n \ti = verify_cache(cache, entries);\n \tif (i)\n \t\treturn i;\n-\ti = update_one(it, cache, entries, \"\", 0, missing_ok, dryrun);\n+\ti = update_one(it, cache, entries, \"\", 0, write_tree, missing_ok, dryrun);\n \tif (i < 0)\n \t\treturn i;\n \treturn 0;\n }\n \n+int cache_tree_update(struct cache_tree *it,\n+\t\t struct cache_entry **cache,\n+\t\t int entries,\n+\t\t int missing_ok,\n+\t\t int dryrun)\n+{\n+\treturn cache_tree_update_fn(it, cache, entries, write_sha1_file, missing_ok, dryrun);\n+}\n+\n static void write_one(struct strbuf *buffer, struct cache_tree *it,\n const char *path, int pathlen)\n {\ndiff --git a/cache-tree.h b/cache-tree.h\nindex 3df641f..b32ae41 100644\n--- a/cache-tree.h\n+++ b/cache-tree.h\n@@ -20,6 +20,8 @@ struct cache_tree {\n \tstruct cache_tree_sub **down;\n };\n \n+typedef int (*write_tree_fn)(const void *, unsigned long, const char *, unsigned char *);\n+\n struct cache_tree *cache_tree(void);\n void cache_tree_free(struct cache_tree **);\n void cache_tree_invalidate_path(struct cache_tree *, const char *);\n@@ -29,6 +31,7 @@ void cache_tree_write(struct strbuf *, struct cache_tree *root);\n struct cache_tree *cache_tree_read(const char *buffer, unsigned long size);\n \n int cache_tree_fully_valid(struct cache_tree *);\n+int cache_tree_update_fn(struct cache_tree *, struct cache_entry **, int, write_tree_fn, int, int);\n int cache_tree_update(struct cache_tree *, struct cache_entry **, int, int, int);\n \n /* bitmasks to write_cache_as_tree flags */\n-- \n1.7.1.rc1.69.g24c2f7\n"},{"id":"148868","messageId":"1282688422-7738-4-git-send-email-pclouds@gmail.com","threadId":"24851","inReplyTo":"1282688422-7738-1-git-send-email-pclouds@gmail.com","subject":"[PATCH 03/32] cache-tree: ignore CE_REMOVE entries in verify_cache()","fromName":"Nguyễn Thái Ngọc Duy","fromEmail":"pclouds@gmail.com","sentAt":"2010-08-24T22:19:53Z","receivedAt":"2010-08-24T22:19:53Z","isPatch":true,"sender":{"key":"pclouds@gmail.com","avatar":"https://avatars.githubusercontent.com/u/720?v=4"},"body":"Those entries are not to be written out. MAY HAVE NEGATIVE IMPACT!!\n\nSigned-off-by: Nguyễn Thái Ngọc Duy \n---\n cache-tree.c | 2 ++\n 1 files changed, 2 insertions(+), 0 deletions(-)\n\ndiff --git a/cache-tree.c b/cache-tree.c\nindex 7fd7203..03952c7 100644\n--- a/cache-tree.c\n+++ b/cache-tree.c\n@@ -156,6 +156,8 @@ static int verify_cache(struct cache_entry **cache,\n \tfunny = 0;\n \tfor (i = 0; i < entries; i++) {\n \t\tstruct cache_entry *ce = cache[i];\n+\t\tif (ce->ce_flags & CE_REMOVE)\n+\t\t\tcontinue;\n \t\tif (ce_stage(ce) || (ce->ce_flags & CE_INTENT_TO_ADD)) {\n \t\t\tif (10 < ++funny) {\n \t\t\t\tfprintf(stderr, \"...\\n\");\n-- \n1.7.1.rc1.69.g24c2f7\n"},{"id":"148869","messageId":"1282688422-7738-5-git-send-email-pclouds@gmail.com","threadId":"24851","inReplyTo":"1282688422-7738-1-git-send-email-pclouds@gmail.com","subject":"[PATCH 04/32] move do_compress() from pack-objects.c to pack-write.c","fromName":"Nguyễn Thái Ngọc Duy","fromEmail":"pclouds@gmail.com","sentAt":"2010-08-24T22:19:54Z","receivedAt":"2010-08-24T22:19:54Z","isPatch":true,"sender":{"key":"pclouds@gmail.com","avatar":"https://avatars.githubusercontent.com/u/720?v=4"},"body":"Signed-off-by: Nguyễn Thái Ngọc Duy \n---\n builtin/pack-objects.c | 24 +++---------------------\n pack-write.c | 25 +++++++++++++++++++++++++\n pack.h | 1 +\n 3 files changed, 29 insertions(+), 21 deletions(-)\n\ndiff --git a/builtin/pack-objects.c b/builtin/pack-objects.c\nindex 0e81673..a664223 100644\n--- a/builtin/pack-objects.c\n+++ b/builtin/pack-objects.c\n@@ -130,28 +130,10 @@ static void *get_delta(struct object_entry *entry)\n \n static unsigned long do_compress(void **pptr, unsigned long size)\n {\n-\tz_stream stream;\n-\tvoid *in, *out;\n-\tunsigned long maxsize;\n-\n-\tmemset(&stream, 0, sizeof(stream));\n-\tdeflateInit(&stream, pack_compression_level);\n-\tmaxsize = deflateBound(&stream, size);\n-\n-\tin = *pptr;\n-\tout = xmalloc(maxsize);\n-\t*pptr = out;\n-\n-\tstream.next_in = in;\n-\tstream.avail_in = size;\n-\tstream.next_out = out;\n-\tstream.avail_out = maxsize;\n-\twhile (deflate(&stream, Z_FINISH) == Z_OK)\n-\t\t; /* nothing */\n-\tdeflateEnd(&stream);\n-\n+\tvoid **in = *pptr;\n+\tunsigned total_out = compress_object(pptr, size);\n \tfree(in);\n-\treturn stream.total_out;\n+\treturn total_out;\n }\n \n /*\ndiff --git a/pack-write.c b/pack-write.c\nindex a905ca4..cc7761e 100644\n--- a/pack-write.c\n+++ b/pack-write.c\n@@ -280,3 +280,28 @@ int encode_in_pack_object_header(enum object_type type, uintmax_t size, unsigned\n \t*hdr = c;\n \treturn n;\n }\n+\n+unsigned long compress_object(void **pptr, unsigned long size)\n+{\n+\tz_stream stream;\n+\tvoid *in, *out;\n+\tunsigned long maxsize;\n+\n+\tmemset(&stream, 0, sizeof(stream));\n+\tdeflateInit(&stream, Z_DEFAULT_COMPRESSION);\n+\tmaxsize = deflateBound(&stream, size);\n+\n+\tin = *pptr;\n+\tout = xmalloc(maxsize);\n+\t*pptr = out;\n+\n+\tstream.next_in = in;\n+\tstream.avail_in = size;\n+\tstream.next_out = out;\n+\tstream.avail_out = maxsize;\n+\twhile (deflate(&stream, Z_FINISH) == Z_OK)\n+\t\t; /* nothing */\n+\tdeflateEnd(&stream);\n+\n+\treturn stream.total_out;\n+}\ndiff --git a/pack.h b/pack.h\nindex bb27576..28a966b 100644\n--- a/pack.h\n+++ b/pack.h\n@@ -62,6 +62,7 @@ extern int verify_pack(struct packed_git *);\n extern void fixup_pack_header_footer(int, unsigned char *, const char *, uint32_t, unsigned char *, off_t);\n extern char *index_pack_lockfile(int fd);\n extern int encode_in_pack_object_header(enum object_type, uintmax_t, unsigned char *);\n+extern unsigned long compress_object(void **, unsigned long size);\n \n #define PH_ERROR_EOF\t\t(-1)\n #define PH_ERROR_PACK_SIGNATURE\t(-2)\n-- \n1.7.1.rc1.69.g24c2f7\n"},{"id":"148871","messageId":"1282688422-7738-6-git-send-email-pclouds@gmail.com","threadId":"24851","inReplyTo":"1282688422-7738-1-git-send-email-pclouds@gmail.com","subject":"[PATCH 05/32] pack-write: add functions for creating simple packs","fromName":"Nguyễn Thái Ngọc Duy","fromEmail":"pclouds@gmail.com","sentAt":"2010-08-24T22:19:55Z","receivedAt":"2010-08-24T22:19:55Z","isPatch":true,"sender":{"key":"pclouds@gmail.com","avatar":"https://avatars.githubusercontent.com/u/720?v=4"},"body":"These are ripoffs from pack-objects and do not do deltas at all.\n\nSigned-off-by: Nguyễn Thái Ngọc Duy \n---\n pack-write.c | 87 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++\n pack.h | 17 +++++++++++\n 2 files changed, 104 insertions(+), 0 deletions(-)\n\ndiff --git a/pack-write.c b/pack-write.c\nindex cc7761e..2b71bd2 100644\n--- a/pack-write.c\n+++ b/pack-write.c\n@@ -305,3 +305,90 @@ unsigned long compress_object(void **pptr, unsigned long size)\n \n \treturn stream.total_out;\n }\n+\n+int create_pack(struct simple_pack *pack)\n+{\n+\tchar tmpname[PATH_MAX];\n+\tstruct pack_header hdr;\n+\n+\tif (pack->fd <= 0) {\n+\t\tpack->fd = odb_mkstemp(tmpname, sizeof(tmpname), \"pack/tmp_pack_XXXXXX\");\n+\t\tpack->pack_tmp_name = xstrdup(tmpname);\n+\t}\n+\tpack->f = sha1fd(pack->fd, pack->pack_tmp_name ? pack->pack_tmp_name : \"\");\n+\n+\tif (pack->nr_objects) {\n+\t\tint pointer_size = sizeof(struct pack_idx_entry*) * pack->nr_objects;\n+\t\tint item_size = sizeof(struct pack_idx_entry) * pack->nr_objects;\n+\t\tvoid *buf = xmalloc(item_size + pointer_size);\n+\t\tpack->written_list = buf;\n+\t\tpack->written_items = (void*)(((char*)buf) + pointer_size);\n+\t}\n+\n+\thdr.hdr_signature = htonl(PACK_SIGNATURE);\n+\thdr.hdr_version = htonl(PACK_VERSION);\n+\thdr.hdr_entries = htonl(pack->nr_objects);\n+\tsha1write(pack->f, &hdr, sizeof(hdr));\n+\tif (pack->written_list)\n+\t\tpack->offset = sizeof(hdr);\n+\n+\treturn 0;\n+}\n+\n+int write_object_to_pack(struct simple_pack *pack,\n+\t\t\t const unsigned char *sha1,\n+\t\t\t void *buf,\n+\t\t\t unsigned long size,\n+\t\t\t enum object_type type)\n+{\n+\tunsigned long datalen;\n+\tunsigned hdrlen;\n+\tunsigned char header[10];\n+\n+\tdatalen = compress_object(&buf, size);\n+\thdrlen = encode_in_pack_object_header(type, size, header);\n+\tif (pack->written_list)\n+\t\tcrc32_begin(pack->f);\n+\tsha1write(pack->f, header, hdrlen);\n+\tsha1write(pack->f, buf, datalen);\n+\tfree(buf);\n+\n+\tif (pack->written_list) {\n+\t\tstruct pack_idx_entry *e;\n+\n+\t\te = pack->written_list[pack->nr_written] = pack->written_items;\n+\t\te->crc32 = crc32_end(pack->f);\n+\t\thashcpy(e->sha1, sha1);\n+\t\te->offset = pack->offset;\n+\t\tpack->offset += datalen+hdrlen;\n+\t}\n+\tpack->nr_written++;\n+\tpack->written_items++;\n+\n+\treturn 0;\n+}\n+\n+int close_pack(struct simple_pack *pack)\n+{\n+\tsha1close(pack->f, pack->sha1, CSUM_CLOSE);\n+\tif (pack->written_list) {\n+\t\tchar tmpname[PATH_MAX];\n+\t\tconst char *idx_tmp_name;\n+\n+\t\tidx_tmp_name = write_idx_file(NULL, pack->written_list, pack->nr_written, pack->sha1);\n+\n+\t\tsnprintf(tmpname, sizeof(tmpname), \"%s/pack/pack-%s.pack\", get_object_directory(), sha1_to_hex(pack->sha1));\n+\t\tfree_pack_by_name(tmpname);\n+\t\tif (adjust_shared_perm(pack->pack_tmp_name))\n+\t\t\tdie_errno(\"unable to make temporary pack file readable\");\n+\t\tif (rename(pack->pack_tmp_name, tmpname))\n+\t\t\tdie_errno(\"unable to rename temporary pack file\");\n+\t\tsnprintf(tmpname, sizeof(tmpname), \"%s/pack/pack-%s.idx\", get_object_directory(), sha1_to_hex(pack->sha1));\n+\t\tif (rename(idx_tmp_name, tmpname))\n+\t\t\tdie_errno(\"unable to rename temporary pack file\");\n+\t\tfree(pack->written_list);\n+\t}\n+\tif (pack->pack_tmp_name)\n+\t\tfree(pack->pack_tmp_name);\n+\treturn 0;\n+}\ndiff --git a/pack.h b/pack.h\nindex 28a966b..a2e32c7 100644\n--- a/pack.h\n+++ b/pack.h\n@@ -55,6 +55,19 @@ struct pack_idx_entry {\n \toff_t offset;\n };\n \n+struct simple_pack {\n+\tuint32_t nr_objects;\n+\tunsigned char sha1[20];\n+\n+\tint fd;\n+\tstruct sha1file *f;\n+\tuint32_t nr_written;\n+\toff_t offset;\n+\tchar *pack_tmp_name;\n+\tstruct pack_idx_entry **written_list;\n+\tstruct pack_idx_entry *written_items;\n+};\n+\n extern const char *write_idx_file(const char *index_name, struct pack_idx_entry **objects, int nr_objects, unsigned char *sha1);\n extern int check_pack_crc(struct packed_git *p, struct pack_window **w_curs, off_t offset, off_t len, unsigned int nr);\n extern int verify_pack_index(struct packed_git *);\n@@ -64,6 +77,10 @@ extern char *index_pack_lockfile(int fd);\n extern int encode_in_pack_object_header(enum object_type, uintmax_t, unsigned char *);\n extern unsigned long compress_object(void **, unsigned long size);\n \n+extern int create_pack(struct simple_pack *);\n+extern int write_object_to_pack(struct simple_pack *, const unsigned char *, void *, unsigned long, enum object_type);\n+extern int close_pack(struct simple_pack *);\n+\n #define PH_ERROR_EOF\t\t(-1)\n #define PH_ERROR_PACK_SIGNATURE\t(-2)\n #define PH_ERROR_PROTOCOL\t(-3)\n-- \n1.7.1.rc1.69.g24c2f7\n"},{"id":"148874","messageId":"1282688422-7738-7-git-send-email-pclouds@gmail.com","threadId":"24851","inReplyTo":"1282688422-7738-1-git-send-email-pclouds@gmail.com","subject":"[PATCH 06/32] tree.c: Add {set,clear}_tree_marks","fromName":"Nguyễn Thái Ngọc Duy","fromEmail":"pclouds@gmail.com","sentAt":"2010-08-24T22:19:56Z","receivedAt":"2010-08-24T22:19:56Z","isPatch":true,"sender":{"key":"pclouds@gmail.com","avatar":"https://avatars.githubusercontent.com/u/720?v=4"},"body":"These function traverse the given tree and set/clear object.flags\n\nSigned-off-by: Nguyễn Thái Ngọc Duy \n---\n tree.c | 44 ++++++++++++++++++++++++++++++++++++++++++++\n tree.h | 3 +++\n 2 files changed, 47 insertions(+), 0 deletions(-)\n\ndiff --git a/tree.c b/tree.c\nindex 5ab90af..434bf56 100644\n--- a/tree.c\n+++ b/tree.c\n@@ -284,3 +284,47 @@ struct tree *parse_tree_indirect(const unsigned char *sha1)\n \t\t\tparse_object(obj->sha1);\n \t} while (1);\n }\n+\n+void set_tree_marks(struct tree *tree, unsigned mark)\n+{\n+\tstruct name_entry entry;\n+\tstruct tree_desc desc;\n+\tenum object_type type;\n+\tunsigned long size;\n+\tvoid *buffer;\n+\n+\ttree->object.flags |= mark;\n+\n+\tbuffer = read_sha1_file(tree->object.sha1, &type, &size);\n+\tif (!buffer || type != OBJ_TREE)\n+\t\tdie(\"%s is not a tree\", sha1_to_hex(tree->object.sha1));\n+\n+\tinit_tree_desc(&desc, buffer, size);\n+\twhile (tree_entry(&desc, &entry)) {\n+\t\tif (S_ISDIR(entry.mode))\n+\t\t\tset_tree_marks(lookup_tree(entry.sha1), mark);\n+\t}\n+\tfree(buffer);\n+}\n+\n+void clear_tree_marks(struct tree *tree, unsigned mark)\n+{\n+\tstruct name_entry entry;\n+\tstruct tree_desc desc;\n+\tenum object_type type;\n+\tunsigned long size;\n+\tvoid *buffer;\n+\n+\ttree->object.flags &= ~mark;\n+\n+\tbuffer = read_sha1_file(tree->object.sha1, &type, &size);\n+\tif (!buffer || type != OBJ_TREE)\n+\t\tdie(\"%s is not a tree\", sha1_to_hex(tree->object.sha1));\n+\n+\tinit_tree_desc(&desc, buffer, size);\n+\twhile (tree_entry(&desc, &entry)) {\n+\t\tif (S_ISDIR(entry.mode))\n+\t\t\tclear_tree_marks(lookup_tree(entry.sha1), mark);\n+\t}\n+\tfree(buffer);\n+}\ndiff --git a/tree.h b/tree.h\nindex 2ff01a4..4376a3a 100644\n--- a/tree.h\n+++ b/tree.h\n@@ -30,4 +30,7 @@ extern int read_tree_recursive(struct tree *tree,\n \n extern int read_tree(struct tree *tree, int stage, const char **paths);\n \n+extern void set_tree_marks(struct tree *tree, unsigned mark);\n+extern void clear_tree_marks(struct tree *tree, unsigned mark);\n+\n #endif /* TREE_H */\n-- \n1.7.1.rc1.69.g24c2f7\n"},{"id":"148873","messageId":"1282688422-7738-8-git-send-email-pclouds@gmail.com","threadId":"24851","inReplyTo":"1282688422-7738-1-git-send-email-pclouds@gmail.com","subject":"[PATCH 07/32] tree.c: find_subtree() to search for a tree","fromName":"Nguyễn Thái Ngọc Duy","fromEmail":"pclouds@gmail.com","sentAt":"2010-08-24T22:19:57Z","receivedAt":"2010-08-24T22:19:57Z","isPatch":true,"sender":{"key":"pclouds@gmail.com","avatar":"https://avatars.githubusercontent.com/u/720?v=4"},"body":"\nSigned-off-by: Nguyễn Thái Ngọc Duy \n---\n tree.c | 35 +++++++++++++++++++++++++++++++++++\n tree.h | 1 +\n 2 files changed, 36 insertions(+), 0 deletions(-)\n\ndiff --git a/tree.c b/tree.c\nindex 434bf56..684afb5 100644\n--- a/tree.c\n+++ b/tree.c\n@@ -328,3 +328,38 @@ void clear_tree_marks(struct tree *tree, unsigned mark)\n \t}\n \tfree(buffer);\n }\n+\n+int find_tree(const unsigned char *sha1, unsigned char *newsha1, const char *path)\n+{\n+\tstruct name_entry entry;\n+\tstruct tree_desc desc;\n+\tenum object_type type;\n+\tunsigned long size;\n+\tvoid *buffer;\n+\tconst char *slash;\n+\tint len;\n+\n+\tbuffer = read_sha1_file(sha1, &type, &size);\n+\tif (!buffer || type != OBJ_TREE)\n+\t\tdie(\"%s is not a tree\", sha1_to_hex(sha1));\n+\n+\tslash = strchr(path, '/');\n+\tlen = slash ? slash - path : strlen(path);\n+\n+\tinit_tree_desc(&desc, buffer, size);\n+\twhile (tree_entry(&desc, &entry)) {\n+\t\tif (!S_ISDIR(entry.mode))\n+\t\t\tcontinue;\n+\t\tif (!strncmp(entry.path, path, len)) {\n+\t\t\tfree(buffer);\n+\t\t\tif (slash)\n+\t\t\t\treturn find_tree(entry.sha1, newsha1, slash+1);\n+\t\t\telse {\n+\t\t\t\thashcpy(newsha1, entry.sha1);\n+\t\t\t\treturn 1;\n+\t\t\t}\n+\t\t}\n+\t}\n+\tfree(buffer);\n+\treturn 0;\n+}\ndiff --git a/tree.h b/tree.h\nindex 4376a3a..6ba1034 100644\n--- a/tree.h\n+++ b/tree.h\n@@ -32,5 +32,6 @@ extern int read_tree(struct tree *tree, int stage, const char **paths);\n \n extern void set_tree_marks(struct tree *tree, unsigned mark);\n extern void clear_tree_marks(struct tree *tree, unsigned mark);\n+extern int find_tree(const unsigned char *sha1, unsigned char *newsha1, const char *path);\n \n #endif /* TREE_H */\n-- \n1.7.1.rc1.69.g24c2f7\n"},{"id":"148872","messageId":"1282688422-7738-9-git-send-email-pclouds@gmail.com","threadId":"24851","inReplyTo":"1282688422-7738-1-git-send-email-pclouds@gmail.com","subject":"[PATCH 08/32] Add $GIT_DIR/narrow check","fromName":"Nguyễn Thái Ngọc Duy","fromEmail":"pclouds@gmail.com","sentAt":"2010-08-24T22:19:58Z","receivedAt":"2010-08-24T22:19:58Z","isPatch":true,"sender":{"key":"pclouds@gmail.com","avatar":"https://avatars.githubusercontent.com/u/720?v=4"},"body":"This file contains the narrow prefix. With get_narrow_prefix() being\nnon-empty behavior of git may be totally different.\n\nSigned-off-by: Nguyễn Thái Ngọc Duy \n---\n Makefile | 2 ++\n cache.h | 3 +++\n environment.c | 2 ++\n narrow-tree.c | 31 +++++++++++++++++++++++++++++++\n narrow-tree.h | 1 +\n 5 files changed, 39 insertions(+), 0 deletions(-)\n create mode 100644 narrow-tree.c\n create mode 100644 narrow-tree.h\n\ndiff --git a/Makefile b/Makefile\nindex 1f11618..54c435e 100644\n--- a/Makefile\n+++ b/Makefile\n@@ -525,6 +525,7 @@ LIB_H += sigchain.h\n LIB_H += strbuf.h\n LIB_H += string-list.h\n LIB_H += submodule.h\n+LIB_H += narrow-tree.h\n LIB_H += tag.h\n LIB_H += transport.h\n LIB_H += tree.h\n@@ -629,6 +630,7 @@ LIB_OBJS += sigchain.o\n LIB_OBJS += strbuf.o\n LIB_OBJS += string-list.o\n LIB_OBJS += submodule.o\n+LIB_OBJS += narrow-tree.o\n LIB_OBJS += symlinks.o\n LIB_OBJS += tag.o\n LIB_OBJS += trace.o\ndiff --git a/cache.h b/cache.h\nindex 37ef9d8..ff401ec 100644\n--- a/cache.h\n+++ b/cache.h\n@@ -1101,4 +1101,7 @@ int split_cmdline(char *cmdline, const char ***argv);\n /* builtin/merge.c */\n int checkout_fast_forward(const unsigned char *from, const unsigned char *to);\n \n+/* narrow-tree.c */\n+extern const char *get_narrow_prefix();\n+\n #endif /* CACHE_H */\ndiff --git a/environment.c b/environment.c\nindex 83d38d3..41fcbd4 100644\n--- a/environment.c\n+++ b/environment.c\n@@ -8,6 +8,7 @@\n * are.\n */\n #include \"cache.h\"\n+#include \"narrow-tree.h\"\n \n char git_default_email[MAX_GITNAME];\n char git_default_name[MAX_GITNAME];\n@@ -105,6 +106,7 @@ static void setup_git_env(void)\n \t\tgit_graft_file = git_pathdup(\"info/grafts\");\n \tif (getenv(NO_REPLACE_OBJECTS_ENVIRONMENT))\n \t\tread_replace_refs = 0;\n+\tcheck_narrow_prefix();\n }\n \n int is_bare_repository(void)\ndiff --git a/narrow-tree.c b/narrow-tree.c\nnew file mode 100644\nindex 0000000..46e913d\n--- /dev/null\n+++ b/narrow-tree.c\n@@ -0,0 +1,31 @@\n+#include \"cache.h\"\n+#include \"narrow-tree.h\"\n+\n+static const char *narrow_prefix;\n+\n+int check_narrow_prefix()\n+{\n+\tint fd;\n+\tstatic char buf[PATH_MAX];\n+\tint ret;\n+\n+\tfd = open(git_path(\"narrow\"), O_RDONLY);\n+\tif (fd == -1)\n+\t\treturn 0;\n+\n+\tret = read(fd, buf, PATH_MAX);\n+\tif (ret > 0) {\n+\t\t/* $GIT_DIR/narrow is not meant for manual editing, anyway.. */\n+\t\twhile (ret && buf[ret-1] == '\\n')\n+\t\t\tret--;\n+\t\tbuf[ret] = '\\0';\n+\t\tnarrow_prefix = buf;\n+\t}\n+\tclose(fd);\n+\treturn 0;\n+}\n+\n+const char *get_narrow_prefix()\n+{\n+\treturn narrow_prefix;\n+}\ndiff --git a/narrow-tree.h b/narrow-tree.h\nnew file mode 100644\nindex 0000000..aa8c94f\n--- /dev/null\n+++ b/narrow-tree.h\n@@ -0,0 +1 @@\n+extern int check_narrow_prefix();\n-- \n1.7.1.rc1.69.g24c2f7\n"},{"id":"148876","messageId":"1282688422-7738-10-git-send-email-pclouds@gmail.com","threadId":"24851","inReplyTo":"1282688422-7738-1-git-send-email-pclouds@gmail.com","subject":"[PATCH 09/32] index: make narrow index incompatible with older git","fromName":"Nguyễn Thái Ngọc Duy","fromEmail":"pclouds@gmail.com","sentAt":"2010-08-24T22:19:59Z","receivedAt":"2010-08-24T22:19:59Z","isPatch":true,"sender":{"key":"pclouds@gmail.com","avatar":"https://avatars.githubusercontent.com/u/720?v=4"},"body":"Index in narrow repos is not a full index and should not be used\nto create commits without modification (to be explained later on).\n\nAlso save narrow prefix inside index and check against\n$GIT_DIR/narrow, just in case.\n\nSigned-off-by: Nguyễn Thái Ngọc Duy \n---\n cache.h | 1 +\n read-cache.c | 35 +++++++++++++++++++++++++++++++----\n t/t6060-narrow-tree.sh | 28 ++++++++++++++++++++++++++++\n 3 files changed, 60 insertions(+), 4 deletions(-)\n create mode 100755 t/t6060-narrow-tree.sh\n\ndiff --git a/cache.h b/cache.h\nindex ff401ec..6bc80b3 100644\n--- a/cache.h\n+++ b/cache.h\n@@ -293,6 +293,7 @@ struct index_state {\n \tstruct string_list *resolve_undo;\n \tstruct cache_tree *cache_tree;\n \tstruct cache_time timestamp;\n+\tchar *narrow_prefix;\n \tvoid *alloc;\n \tunsigned name_hash_initialized : 1,\n \t\t initialized : 1;\ndiff --git a/read-cache.c b/read-cache.c\nindex 881dd93..20b619c 100644\n--- a/read-cache.c\n+++ b/read-cache.c\n@@ -25,8 +25,9 @@ static struct cache_entry *refresh_cache_entry(struct cache_entry *ce, int reall\n */\n \n #define CACHE_EXT(s) ( (s[0]<<24)|(s[1]<<16)|(s[2]<<8)|(s[3]) )\n-#define CACHE_EXT_TREE 0x54524545\t/* \"TREE\" */\n+#define CACHE_EXT_TREE 0x54524545\t /* \"TREE\" */\n #define CACHE_EXT_RESOLVE_UNDO 0x52455543 /* \"REUC\" */\n+#define CACHE_EXT_NARROW 0x4e415257\t /* \"NARW\" */\n \n struct index_state the_index;\n \n@@ -1168,7 +1169,9 @@ static int verify_hdr(struct cache_header *hdr, unsigned long size)\n \n \tif (hdr->hdr_signature != htonl(CACHE_SIGNATURE))\n \t\treturn error(\"bad signature\");\n-\tif (hdr->hdr_version != htonl(2) && hdr->hdr_version != htonl(3))\n+\tif (hdr->hdr_version != htonl(2) &&\n+\t hdr->hdr_version != htonl(3) &&\n+\t hdr->hdr_version != htonl(4))\n \t\treturn error(\"bad index version\");\n \tgit_SHA1_Init(&c);\n \tgit_SHA1_Update(&c, hdr, size - 20);\n@@ -1188,6 +1191,9 @@ static int read_index_extension(struct index_state *istate,\n \tcase CACHE_EXT_RESOLVE_UNDO:\n \t\tistate->resolve_undo = resolve_undo_read(data, sz);\n \t\tbreak;\n+\tcase CACHE_EXT_NARROW:\n+\t\tistate->narrow_prefix = xstrdup(data);\n+\t\tbreak;\n \tdefault:\n \t\tif (*ext < 'A' || 'Z' < *ext)\n \t\t\treturn error(\"index uses %.4s extension, which we do not understand\",\n@@ -1352,6 +1358,16 @@ int read_index_from(struct index_state *istate, const char *path)\n \t\tsrc_offset += extsize;\n \t}\n \tmunmap(mmap, mmap_size);\n+\n+\tif ((!get_narrow_prefix() && !istate->narrow_prefix) ||\n+\t (get_narrow_prefix() && istate->narrow_prefix &&\n+\t !strcmp(get_narrow_prefix(), istate->narrow_prefix)))\n+\t\t;\t\t/* good */\n+\telse\n+\t\tdie(\"Incompatible index and narrow mode (%s/%s)\",\n+\t\t get_narrow_prefix(),\n+\t\t istate->narrow_prefix);\n+\n \treturn istate->cache_nr;\n \n unmap:\n@@ -1549,7 +1565,7 @@ int write_index(struct index_state *istate, int newfd)\n {\n \tgit_SHA_CTX c;\n \tstruct cache_header hdr;\n-\tint i, err, removed, extended;\n+\tint i, err, removed, extended, ver;\n \tstruct cache_entry **cache = istate->cache;\n \tint entries = istate->cache_nr;\n \tstruct stat st;\n@@ -1568,7 +1584,11 @@ int write_index(struct index_state *istate, int newfd)\n \n \thdr.hdr_signature = htonl(CACHE_SIGNATURE);\n \t/* for extended format, increase version so older git won't try to read it */\n-\thdr.hdr_version = htonl(extended ? 3 : 2);\n+\tver = extended ? 3 : 2;\n+\tif (get_narrow_prefix() && ver < 4)\n+\t\tver = 4;\t/* narrow-unaware git should to touch this index */\n+\n+\thdr.hdr_version = htonl(ver);\n \thdr.hdr_entries = htonl(entries - removed);\n \n \tgit_SHA1_Init(&c);\n@@ -1607,6 +1627,13 @@ int write_index(struct index_state *istate, int newfd)\n \t\tif (err)\n \t\t\treturn -1;\n \t}\n+\tif (get_narrow_prefix()) {\n+\t\tint len = strlen(get_narrow_prefix())+1;\n+\t\terr = write_index_ext_header(&c, newfd, CACHE_EXT_NARROW, len) < 0 ||\n+\t\t\tce_write(&c, newfd, get_narrow_prefix(), len) < 0;\n+\t\tif (err)\n+\t\t\treturn -1;\n+\t}\n \n \tif (ce_flush(&c, newfd) || fstat(newfd, &st))\n \t\treturn -1;\ndiff --git a/t/t6060-narrow-tree.sh b/t/t6060-narrow-tree.sh\nnew file mode 100755\nindex 0000000..0e561c8\n--- /dev/null\n+++ b/t/t6060-narrow-tree.sh\n@@ -0,0 +1,28 @@\n+#!/bin/sh\n+\n+test_description='narrow-tree tests'\n+\n+. ./test-lib.sh\n+\n+test_expect_success setup '\n+\t: >test\n+'\n+\n+test_expect_success 'index ver 4 in narrow mode' '\n+\techo foo >.git/narrow &&\n+\tgit update-index --add test &&\n+\techo 4 >expected &&\n+\ttest-index-version < .git/index >result &&\n+\ttest_cmp expected result\n+'\n+\n+test_expect_success 'index ver < 4 when narrow mode is gone' '\n+\trm .git/narrow &&\n+\trm .git/index &&\n+\tgit update-index --add test &&\n+\techo 2 >expected &&\n+\ttest-index-version < .git/index >result &&\n+\ttest_cmp expected result\n+'\n+\n+test_done\n-- \n1.7.1.rc1.69.g24c2f7\n"},{"id":"148878","messageId":"1282688422-7738-11-git-send-email-pclouds@gmail.com","threadId":"24851","inReplyTo":"1282688422-7738-1-git-send-email-pclouds@gmail.com","subject":"[PATCH 10/32] rev-list: support traversing in narrow repository mode","fromName":"Nguyễn Thái Ngọc Duy","fromEmail":"pclouds@gmail.com","sentAt":"2010-08-24T22:20:00Z","receivedAt":"2010-08-24T22:20:00Z","isPatch":true,"sender":{"key":"pclouds@gmail.com","avatar":"https://avatars.githubusercontent.com/u/720?v=4"},"body":"In this mode, only trees within revs->narrow_prefix is traversed. This\nnarrows down the whole repository to the given subtree. This mode will\nbe used by upload-pack/pack-objects to create a narrow pack, and by\nall git operations in narrow repository (i.e. $GIT_DIR/narrow exists).\n\nSigned-off-by: Nguyễn Thái Ngọc Duy \n---\n list-objects.c | 36 +++++++++++++++++++++++++++++++++---\n revision.h | 4 +++-\n 2 files changed, 36 insertions(+), 4 deletions(-)\n\ndiff --git a/list-objects.c b/list-objects.c\nindex 8953548..7014926 100644\n--- a/list-objects.c\n+++ b/list-objects.c\n@@ -61,12 +61,15 @@ static void process_tree(struct rev_info *revs,\n \t\t\t struct tree *tree,\n \t\t\t show_object_fn show,\n \t\t\t struct name_path *path,\n-\t\t\t const char *name)\n+\t\t\t const char *name,\n+\t\t\t const char *subtree)\n {\n \tstruct object *obj = &tree->object;\n \tstruct tree_desc desc;\n \tstruct name_entry entry;\n \tstruct name_path me;\n+\tconst char *slash = NULL;\n+\tint subtree_len;\n \n \tif (!revs->tree_objects)\n \t\treturn;\n@@ -82,13 +85,34 @@ static void process_tree(struct rev_info *revs,\n \tme.elem = name;\n \tme.elem_len = strlen(name);\n \n+\tif (subtree) {\n+\t\tslash = strchr(subtree, '/');\n+\t\tsubtree_len = slash ? slash - subtree : strlen(subtree);\n+\t}\n+\n \tinit_tree_desc(&desc, tree->buffer, tree->size);\n \n \twhile (tree_entry(&desc, &entry)) {\n+\t\t/*\n+\t\t * FIXME, does not follow tree rename\n+\t\t * some sort of rename hints would be nice\n+\t\t * (because diff machinery should not be used\n+\t\t * here for detecting renames)\n+\t\t */\n+\t\tif (!subtree)\n+\t\t\t; \t/* no subtree restriction, go on */\n+\t\telse if (S_ISDIR(entry.mode) &&\n+\t\t\t !strncmp(entry.path, subtree, subtree_len) &&\n+\t\t\t entry.path[subtree_len] == '\\0')\n+\t\t\t;\t/* inside subtree, go on */\n+\t\telse\n+\t\t\tcontinue; /* stop */\n+\n \t\tif (S_ISDIR(entry.mode))\n \t\t\tprocess_tree(revs,\n \t\t\t\t lookup_tree(entry.sha1),\n-\t\t\t\t show, &me, entry.path);\n+\t\t\t\t show, &me, entry.path,\n+\t\t\t\t slash ? slash+1 : NULL);\n \t\telse if (S_ISGITLINK(entry.mode))\n \t\t\tprocess_gitlink(revs, entry.sha1,\n \t\t\t\t\tshow, &me, entry.path);\n@@ -147,6 +171,11 @@ void traverse_commit_list(struct rev_info *revs,\n \tint i;\n \tstruct commit *commit;\n \n+\tif (get_narrow_prefix() && !revs->narrow_tree) {\n+\t\trevs->narrow_tree = 1;\n+\t\trevs->narrow_prefix = get_narrow_prefix();\n+\t}\n+\n \twhile ((commit = get_revision(revs)) != NULL) {\n \t\tadd_pending_tree(revs, commit->tree);\n \t\tshow_commit(commit, data);\n@@ -164,7 +193,8 @@ void traverse_commit_list(struct rev_info *revs,\n \t\t}\n \t\tif (obj->type == OBJ_TREE) {\n \t\t\tprocess_tree(revs, (struct tree *)obj, show_object,\n-\t\t\t\t NULL, name);\n+\t\t\t\t NULL, name,\n+\t\t\t\t revs->narrow_tree ? revs->narrow_prefix : NULL);\n \t\t\tcontinue;\n \t\t}\n \t\tif (obj->type == OBJ_BLOB) {\ndiff --git a/revision.h b/revision.h\nindex 36fdf22..ccdf28a 100644\n--- a/revision.h\n+++ b/revision.h\n@@ -33,6 +33,7 @@ struct rev_info {\n \n \t/* Basic information */\n \tconst char *prefix;\n+\tconst char *narrow_prefix;\n \tconst char *def;\n \tvoid *prune_data;\n \tunsigned int early_output;\n@@ -68,7 +69,8 @@ struct rev_info {\n \t\t\tcherry_pick:1,\n \t\t\tbisect:1,\n \t\t\tancestry_path:1,\n-\t\t\tfirst_parent_only:1;\n+\t\t\tfirst_parent_only:1,\n+\t\t\tnarrow_tree:1;\n \n \t/* Diff flags */\n \tunsigned int\tdiff:1,\n-- \n1.7.1.rc1.69.g24c2f7\n"},{"id":"148880","messageId":"1282688422-7738-12-git-send-email-pclouds@gmail.com","threadId":"24851","inReplyTo":"1282688422-7738-1-git-send-email-pclouds@gmail.com","subject":"[PATCH 11/32] rev-list: support --narrow-tree","fromName":"Nguyễn Thái Ngọc Duy","fromEmail":"pclouds@gmail.com","sentAt":"2010-08-24T22:20:01Z","receivedAt":"2010-08-24T22:20:01Z","isPatch":true,"sender":{"key":"pclouds@gmail.com","avatar":"https://avatars.githubusercontent.com/u/720?v=4"},"body":"These options allow plumbing to access narrow tree traverse modes.\nAs a consequence, tests can now be written to test these modes.\n\nSigned-off-by: Nguyễn Thái Ngọc Duy \n---\n revision.c | 5 ++\n t/t6061-rev-list-narrow.sh | 158 ++++++++++++++++++++++++++++++++++++++++++++\n 2 files changed, 163 insertions(+), 0 deletions(-)\n create mode 100755 t/t6061-rev-list-narrow.sh\n\ndiff --git a/revision.c b/revision.c\nindex 7e82efd..a4eb11f 100644\n--- a/revision.c\n+++ b/revision.c\n@@ -1562,6 +1562,11 @@ int setup_revisions(int argc, const char **argv, struct rev_info *revs, struct s\n \t\t\t\tread_revisions_from_stdin(revs, &prune_data);\n \t\t\t\tcontinue;\n \t\t\t}\n+\t\t\tif (!prefixcmp(arg, \"--narrow-tree=\")) {\n+\t\t\t\trevs->narrow_prefix = arg + 14;\n+\t\t\t\trevs->narrow_tree = 1;\n+\t\t\t\tcontinue;\n+\t\t\t}\n \n \t\t\topts = handle_revision_opt(revs, argc - i, argv + i, &left, argv);\n \t\t\tif (opts > 0) {\ndiff --git a/t/t6061-rev-list-narrow.sh b/t/t6061-rev-list-narrow.sh\nnew file mode 100755\nindex 0000000..e489347\n--- /dev/null\n+++ b/t/t6061-rev-list-narrow.sh\n@@ -0,0 +1,158 @@\n+#!/bin/sh\n+\n+test_description='rev-list in narrow tree mode'\n+\n+. ./test-lib.sh\n+\n+# Normal tree\n+cat <revlist.expected\n+ba7a30916c792624a8372cb26da50f5594098222\n+3b52e9990a52d9ea46b25199b5810566324759f4 \n+573541ac9702dd3969c9bc859d2b91ec1f7e6e56 f0\n+fa0bba1767985729582729a12a5459cd041d6cf6 t1\n+f599e28b8ab0d8c9c57a486c89c4a5132dcbd3b2 t1/f10\n+d81015b2a1c40fc6cbc5bf5a8b16949018c3aede t1/t12\n+52bd8e43afb01d0c9747f1fedf2fc94684ee4cc4 t1/t12/f120\n+74a398027f0b59183db54ca8c67dc30b5aeed0ff t2\n+209e3ef4b6247ce746048d5711befda46206d235 t2/f20\n+EOF\n+\n+# narrowed down tree\n+cat <t12.expected\n+ba7a30916c792624a8372cb26da50f5594098222\n+3b52e9990a52d9ea46b25199b5810566324759f4 \n+fa0bba1767985729582729a12a5459cd041d6cf6 t1\n+d81015b2a1c40fc6cbc5bf5a8b16949018c3aede t1/t12\n+52bd8e43afb01d0c9747f1fedf2fc94684ee4cc4 t1/t12/f120\n+EOF\n+\n+# local tree\n+cat <t12-subtree.expected\n+ba7a30916c792624a8372cb26da50f5594098222\n+c0167b5ce78e9fb801ef96dcc319f3411d35278b \n+e9a12654c859aff51978b25b2a6c5eddf6c1e13c t1\n+d81015b2a1c40fc6cbc5bf5a8b16949018c3aede t1/t12\n+52bd8e43afb01d0c9747f1fedf2fc94684ee4cc4 t1/t12/f120\n+EOF\n+\n+cat <t2.expected\n+ba7a30916c792624a8372cb26da50f5594098222\n+3b52e9990a52d9ea46b25199b5810566324759f4 \n+74a398027f0b59183db54ca8c67dc30b5aeed0ff t2\n+209e3ef4b6247ce746048d5711befda46206d235 t2/f20\n+EOF\n+\n+test_expect_success setup '\n+\ttest_tick &&\n+\tmkdir t1 t2 t1/t12 &&\n+\techo 0 > f0 &&\n+\techo 10 > t1/f10 &&\n+\techo 120 > t1/t12/f120 &&\n+\techo 20 > t2/f20\n+\tgit add t1 t2 f0 && git commit -m initial &&\n+\tC1=`git rev-parse HEAD` &&\n+\tT2=`git rev-parse HEAD^{tree}` &&\n+\tgit rev-list --objects HEAD >result &&\n+\ttest_cmp revlist.expected result\n+'\n+\n+cat <branch1.expected\n+7475cb8a389b36ce238b9ee6cbfdfa26a1b67e35\n+ba7a30916c792624a8372cb26da50f5594098222\n+aa0602ee56ea4cb5e5bfff8713bb8a9a6ab4303e \n+407c07594c8083c3d0521a1cb463b1204d880c13 f0\n+d77d258d806b2aa7cdb5a301e6a54f023d9bdcfe t1\n+737483f30ba682e414d715f7976fe9a9904fdbc0 t1/f10\n+ab246f7eb05e94f695d2a0e30093d94fde7b837e t1/t12\n+04156ae83e615fa5b6019170928bc131539f9996 t1/t12/f120\n+1da4df952e59f2fa7ff211a8ee9f72b3174bbfb7 t2\n+9aa2407894b849226773016ef5d68c5b87a23926 t2/f20\n+3b52e9990a52d9ea46b25199b5810566324759f4 \n+573541ac9702dd3969c9bc859d2b91ec1f7e6e56 f0\n+fa0bba1767985729582729a12a5459cd041d6cf6 t1\n+f599e28b8ab0d8c9c57a486c89c4a5132dcbd3b2 t1/f10\n+d81015b2a1c40fc6cbc5bf5a8b16949018c3aede t1/t12\n+52bd8e43afb01d0c9747f1fedf2fc94684ee4cc4 t1/t12/f120\n+74a398027f0b59183db54ca8c67dc30b5aeed0ff t2\n+209e3ef4b6247ce746048d5711befda46206d235 t2/f20\n+EOF\n+\n+test_expect_success 'setup branch1' '\n+\techo common >> t1/t12/f120 &&\n+\techo branch1 >> t1/f10 &&\n+\techo branch1 >> t2/f20 &&\n+\techo branch1 >> f0 &&\n+\tgit add -u && git commit -m branch1 &&\n+\tBRANCH1=`git rev-parse HEAD` &&\n+\tgit rev-list --objects $BRANCH1 > result &&\n+\ttest_cmp branch1.expected result\n+'\n+\n+cat <branch2.expected\n+af9c31c0e217154296d93d66b9a5a41892c7b321\n+ba7a30916c792624a8372cb26da50f5594098222\n+72e05e456b9ff4c01ccf6178ce60d9b52b41aae4 \n+573541ac9702dd3969c9bc859d2b91ec1f7e6e56 f0\n+4fc1656b01ce8b21987c55a2870b8c9a431ec772 t1\n+f599e28b8ab0d8c9c57a486c89c4a5132dcbd3b2 t1/f10\n+ab246f7eb05e94f695d2a0e30093d94fde7b837e t1/t12\n+04156ae83e615fa5b6019170928bc131539f9996 t1/t12/f120\n+cef4db52388f5e0853acc772faa68058436a95ff t2\n+7319a2144592e50a564d49902302a8b1bad2f49c t2/f20\n+3b52e9990a52d9ea46b25199b5810566324759f4 \n+fa0bba1767985729582729a12a5459cd041d6cf6 t1\n+d81015b2a1c40fc6cbc5bf5a8b16949018c3aede t1/t12\n+52bd8e43afb01d0c9747f1fedf2fc94684ee4cc4 t1/t12/f120\n+74a398027f0b59183db54ca8c67dc30b5aeed0ff t2\n+209e3ef4b6247ce746048d5711befda46206d235 t2/f20\n+EOF\n+\n+test_expect_success 'setup branch2' '\n+\tgit reset --hard HEAD^ &&\n+\techo common >> t1/t12/f120 &&\n+\techo branch2 >> t2/f20 &&\n+\tgit add -u && git commit -m branch2 &&\n+\tBRANCH2=`git rev-parse HEAD` &&\n+\tgit rev-list --objects $BRANCH2 > result &&\n+\ttest_cmp branch2.expected result\n+'\n+\n+test_expect_success 'setup merge' '\n+\tgit reset --hard HEAD^\n+\tMERGE=`echo merge | git commit-tree $T2 -p $BRANCH1 -p $BRANCH2`\n+'\n+\n+test_expect_success 'rev-list --narrow-tree=t2' '\n+\tgit rev-list --objects --narrow-tree=t2 HEAD >result &&\n+\ttest_cmp t2.expected result\n+'\n+\n+test_expect_success 'rev-list --narrow-tree=t1/t12' '\n+\tgit rev-list --objects --narrow-tree=t1/t12 HEAD >result &&\n+\ttest_cmp t12.expected result\n+'\n+\n+cat <merge.expected\n+0da28f8308e336bd4b2c26b61c7fc44e41a30e49\n+7475cb8a389b36ce238b9ee6cbfdfa26a1b67e35\n+af9c31c0e217154296d93d66b9a5a41892c7b321\n+ba7a30916c792624a8372cb26da50f5594098222\n+74a398027f0b59183db54ca8c67dc30b5aeed0ff t2\n+3b52e9990a52d9ea46b25199b5810566324759f4 \n+fa0bba1767985729582729a12a5459cd041d6cf6 t1\n+d81015b2a1c40fc6cbc5bf5a8b16949018c3aede t1/t12\n+52bd8e43afb01d0c9747f1fedf2fc94684ee4cc4 t1/t12/f120\n+aa0602ee56ea4cb5e5bfff8713bb8a9a6ab4303e \n+d77d258d806b2aa7cdb5a301e6a54f023d9bdcfe t1\n+ab246f7eb05e94f695d2a0e30093d94fde7b837e t1/t12\n+04156ae83e615fa5b6019170928bc131539f9996 t1/t12/f120\n+72e05e456b9ff4c01ccf6178ce60d9b52b41aae4 \n+4fc1656b01ce8b21987c55a2870b8c9a431ec772 t1\n+EOF\n+\n+test_expect_success 'rev-list --narrow-tree=t1/t12 with merges' '\n+\tgit rev-list --objects --narrow-tree=t1/t12 $MERGE >result &&\n+\ttest_cmp merge.expected result\n+'\n+\n+test_done\n-- \n1.7.1.rc1.69.g24c2f7\n"},{"id":"148879","messageId":"1282688422-7738-13-git-send-email-pclouds@gmail.com","threadId":"24851","inReplyTo":"1282688422-7738-1-git-send-email-pclouds@gmail.com","subject":"[PATCH 12/32] pack-objects: support --narrow-tree","fromName":"Nguyễn Thái Ngọc Duy","fromEmail":"pclouds@gmail.com","sentAt":"2010-08-24T22:20:02Z","receivedAt":"2010-08-24T22:20:02Z","isPatch":true,"sender":{"key":"pclouds@gmail.com","avatar":"https://avatars.githubusercontent.com/u/720?v=4"},"body":"\nSigned-off-by: Nguyễn Thái Ngọc Duy \n---\n builtin/pack-objects.c | 9 +++++++++\n t/t6061-rev-list-narrow.sh | 27 +++++++++++++++++++++++++++\n 2 files changed, 36 insertions(+), 0 deletions(-)\n\ndiff --git a/builtin/pack-objects.c b/builtin/pack-objects.c\nindex a664223..36ab350 100644\n--- a/builtin/pack-objects.c\n+++ b/builtin/pack-objects.c\n@@ -106,6 +106,7 @@ static int object_ix_hashsz;\n static uint32_t written, written_delta;\n static uint32_t reused, reused_delta;\n \n+const char *narrow_prefix;\n \n static void *get_delta(struct object_entry *entry)\n {\n@@ -2055,6 +2056,10 @@ static void get_object_list(int ac, const char **av)\n \tinit_revisions(&revs, NULL);\n \tsave_commit_buffer = 0;\n \tsetup_revisions(ac, av, &revs, NULL);\n+\tif (narrow_prefix) {\n+\t\trevs.narrow_tree = 1;\n+\t\trevs.narrow_prefix = narrow_prefix;\n+\t}\n \n \twhile (fgets(line, sizeof(line), stdin) != NULL) {\n \t\tint len = strlen(line);\n@@ -2259,6 +2264,10 @@ int cmd_pack_objects(int argc, const char **argv, const char *prefix)\n \t\t\tgrafts_replace_parents = 0;\n \t\t\tcontinue;\n \t\t}\n+\t\tif (!prefixcmp(arg, \"--narrow-tree=\")) {\n+\t\t\tnarrow_prefix = arg + 14;\n+\t\t\tcontinue;\n+\t\t}\n \t\tusage(pack_usage);\n \t}\n \ndiff --git a/t/t6061-rev-list-narrow.sh b/t/t6061-rev-list-narrow.sh\nindex e489347..30ef46f 100755\n--- a/t/t6061-rev-list-narrow.sh\n+++ b/t/t6061-rev-list-narrow.sh\n@@ -155,4 +155,31 @@ test_expect_success 'rev-list --narrow-tree=t1/t12 with merges' '\n \ttest_cmp merge.expected result\n '\n \n+cat <pack.expected\n+04156ae83e615fa5b6019170928bc131539f9996\n+0da28f8308e336bd4b2c26b61c7fc44e41a30e49\n+3b52e9990a52d9ea46b25199b5810566324759f4\n+4fc1656b01ce8b21987c55a2870b8c9a431ec772\n+52bd8e43afb01d0c9747f1fedf2fc94684ee4cc4\n+72e05e456b9ff4c01ccf6178ce60d9b52b41aae4\n+7475cb8a389b36ce238b9ee6cbfdfa26a1b67e35\n+74a398027f0b59183db54ca8c67dc30b5aeed0ff\n+aa0602ee56ea4cb5e5bfff8713bb8a9a6ab4303e\n+ab246f7eb05e94f695d2a0e30093d94fde7b837e\n+af9c31c0e217154296d93d66b9a5a41892c7b321\n+ba7a30916c792624a8372cb26da50f5594098222\n+d77d258d806b2aa7cdb5a301e6a54f023d9bdcfe\n+d81015b2a1c40fc6cbc5bf5a8b16949018c3aede\n+fa0bba1767985729582729a12a5459cd041d6cf6\n+EOF\n+\n+test_expect_success 'pack-object --narrow-tree=t1/t12' '\n+\techo $MERGE|git pack-objects --revs --narrow-tree=t1/t12 --stdout >p.pack &&\n+\tgit index-pack -o p.idx p.pack &&\n+\tgit verify-pack -v p.pack |\n+\t\tgrep \"^[0-9a-f][0-9a-f][0-9a-f][0-9a-f]\" |\n+\t\tcut -d \" \" -f 1 >result &&\n+\ttest_cmp pack.expected result\n+'\n+\n test_done\n-- \n1.7.1.rc1.69.g24c2f7\n"},{"id":"148877","messageId":"1282688422-7738-14-git-send-email-pclouds@gmail.com","threadId":"24851","inReplyTo":"1282688422-7738-1-git-send-email-pclouds@gmail.com","subject":"[PATCH 13/32] upload-pack: support narrow-tree capability","fromName":"Nguyễn Thái Ngọc Duy","fromEmail":"pclouds@gmail.com","sentAt":"2010-08-24T22:20:03Z","receivedAt":"2010-08-24T22:20:03Z","isPatch":true,"sender":{"key":"pclouds@gmail.com","avatar":"https://avatars.githubusercontent.com/u/720?v=4"},"body":"If narrow-tree line is sent from fetch-pack, rev-walk will be put in\nnarrow mode.\n\nCurrently only one narrow-tree is supported.\n\nSigned-off-by: Nguyễn Thái Ngọc Duy \n---\n upload-pack.c | 31 +++++++++++++++++++++++++++++--\n 1 files changed, 29 insertions(+), 2 deletions(-)\n\ndiff --git a/upload-pack.c b/upload-pack.c\nindex fc79dde..571fcb6 100644\n--- a/upload-pack.c\n+++ b/upload-pack.c\n@@ -41,6 +41,7 @@ static int use_sideband;\n static int debug_fd;\n static int advertise_refs;\n static int stateless_rpc;\n+char *narrow_prefix;\n \n static void reset_timeout(void)\n {\n@@ -117,6 +118,10 @@ static int do_rev_list(int in, int out, void *user_data)\n \trevs.blob_objects = 1;\n \tif (use_thin_pack)\n \t\trevs.edge_hint = 1;\n+\tif (narrow_prefix) {\n+\t\trevs.narrow_tree = 1;\n+\t\trevs.narrow_prefix = narrow_prefix;\n+\t}\n \n \tfor (i = 0; i < want_obj.nr; i++) {\n \t\tstruct object *o = want_obj.objects[i].item;\n@@ -153,7 +158,8 @@ static void create_pack_file(void)\n \t\t\"corruption on the remote side.\";\n \tint buffered = -1;\n \tssize_t sz;\n-\tconst char *argv[10];\n+\tconst char *argv[11];\n+\tchar arg_narrow[PATH_MAX];\n \tint arg = 0;\n \n \tif (shallow_nr) {\n@@ -173,6 +179,10 @@ static void create_pack_file(void)\n \t}\n \n \targv[arg++] = \"--stdout\";\n+\tif (narrow_prefix) {\n+\t\tsprintf(arg_narrow, \"--narrow-tree=%s\", narrow_prefix);\n+\t\targv[arg++] = arg_narrow;\n+\t}\n \tif (!no_progress)\n \t\targv[arg++] = \"--progress\";\n \tif (use_ofs_delta)\n@@ -499,6 +509,18 @@ static void receive_needs(void)\n \t\tif (debug_fd)\n \t\t\twrite_in_full(debug_fd, line, len);\n \n+\t\tif (!prefixcmp(line, \"narrow-tree \")) {\n+\t\t\tint len;\n+\t\t\tif (narrow_prefix)\n+\t\t\t\tdie(\"sorry, only one narrow prefix is supported\");\n+\t\t\tlen = strlen(line+12);\n+\t\t\tnarrow_prefix = malloc(len+1);\n+\t\t\tmemcpy(narrow_prefix, line+12, len-1);\n+\t\t\tnarrow_prefix[len-1] = '\\0'; /* \\n */\n+\t\t\tif (narrow_prefix[len-2] == '/')\n+\t\t\t\tdie(\"trailing slash in narrow prefix not allowed\");\n+\t\t\tcontinue;\n+\t\t}\n \t\tif (!prefixcmp(line, \"shallow \")) {\n \t\t\tunsigned char sha1[20];\n \t\t\tstruct object *object;\n@@ -562,6 +584,9 @@ static void receive_needs(void)\n \tif (!use_sideband && daemon_mode)\n \t\tno_progress = 1;\n \n+\tif (narrow_prefix)\n+\t\tuse_thin_pack = 0;\n+\n \tif (depth == 0 && shallows.nr == 0)\n \t\treturn;\n \tif (depth > 0) {\n@@ -619,7 +644,7 @@ static int send_ref(const char *refname, const unsigned char *sha1, int flag, vo\n {\n \tstatic const char *capabilities = \"multi_ack thin-pack side-band\"\n \t\t\" side-band-64k ofs-delta shallow no-progress\"\n-\t\t\" include-tag multi_ack_detailed\";\n+\t\t\" include-tag multi_ack_detailed narrow-tree\";\n \tstruct object *o = parse_object(sha1);\n \n \tif (!o)\n@@ -724,6 +749,8 @@ int main(int argc, char **argv)\n \t\tdie(\"'%s' does not appear to be a git repository\", dir);\n \tif (is_repository_shallow())\n \t\tdie(\"attempt to fetch/clone from a shallow repository\");\n+\tif (get_narrow_prefix())\n+\t\tdie(\"attempt to fetch/clone from a narrow repository\");\n \tif (getenv(\"GIT_DEBUG_SEND_PACK\"))\n \t\tdebug_fd = atoi(getenv(\"GIT_DEBUG_SEND_PACK\"));\n \tupload_pack();\n-- \n1.7.1.rc1.69.g24c2f7\n"},{"id":"148883","messageId":"1282688422-7738-15-git-send-email-pclouds@gmail.com","threadId":"24851","inReplyTo":"1282688422-7738-1-git-send-email-pclouds@gmail.com","subject":"[PATCH 14/32] fetch-pack: support --narrow-tree","fromName":"Nguyễn Thái Ngọc Duy","fromEmail":"pclouds@gmail.com","sentAt":"2010-08-24T22:20:04Z","receivedAt":"2010-08-24T22:20:04Z","isPatch":true,"sender":{"key":"pclouds@gmail.com","avatar":"https://avatars.githubusercontent.com/u/720?v=4"},"body":"This options requires narrow-tree capability from upload-pack. It\nsimply sends narrow-tree line from argument to to upload-pack.\n\nSigned-off-by: Nguyễn Thái Ngọc Duy \n---\n builtin/fetch-pack.c | 11 +++++++++++\n 1 files changed, 11 insertions(+), 0 deletions(-)\n\ndiff --git a/builtin/fetch-pack.c b/builtin/fetch-pack.c\nindex dbd8b7b..821b713 100644\n--- a/builtin/fetch-pack.c\n+++ b/builtin/fetch-pack.c\n@@ -37,6 +37,7 @@ static int marked;\n \n static struct commit_list *rev_list;\n static int non_common_revs, multi_ack, use_sideband;\n+const char *narrow_prefix;\n \n static void rev_list_push(struct commit *commit, int mark)\n {\n@@ -237,6 +238,10 @@ static int find_common(int fd[2], unsigned char *result_sha1,\n \tfor_each_ref(rev_list_insert_ref, NULL);\n \n \tfetching = 0;\n+\tif (get_narrow_prefix() && !narrow_prefix)\n+\t\tnarrow_prefix = get_narrow_prefix();\n+\tif (narrow_prefix)\n+\t\tpacket_buf_write(&req_buf, \"narrow-tree %s\\n\", narrow_prefix);\n \tfor ( ; refs ; refs = refs->next) {\n \t\tunsigned char *remote = refs->old_sha1;\n \t\tconst char *remote_hex;\n@@ -692,6 +697,8 @@ static struct ref *do_fetch_pack(int fd[2],\n \n \tif (is_repository_shallow() && !server_supports(\"shallow\"))\n \t\tdie(\"Server does not support shallow clients\");\n+\tif (narrow_prefix && !server_supports(\"narrow-tree\"))\n+\t\tdie(\"Server does not support narrow-tree\");\n \tif (server_supports(\"multi_ack_detailed\")) {\n \t\tif (args.verbose)\n \t\t\tfprintf(stderr, \"Server supports multi_ack_detailed\\n\");\n@@ -860,6 +867,10 @@ int cmd_fetch_pack(int argc, const char **argv, const char *prefix)\n \t\t\t\tpack_lockfile_ptr = &pack_lockfile;\n \t\t\t\tcontinue;\n \t\t\t}\n+\t\t\tif (!prefixcmp(arg, \"--narrow-tree=\")) {\n+\t\t\t\tnarrow_prefix = arg + 14;\n+\t\t\t\tcontinue;\n+\t\t\t}\n \t\t\tusage(fetch_pack_usage);\n \t\t}\n \t\tdest = (char *)arg;\n-- \n1.7.1.rc1.69.g24c2f7\n"},{"id":"148884","messageId":"1282688422-7738-16-git-send-email-pclouds@gmail.com","threadId":"24851","inReplyTo":"1282688422-7738-1-git-send-email-pclouds@gmail.com","subject":"[PATCH 15/32] unpack_trees: only unpack $GIT_DIR/narrow subtree in narrow repository","fromName":"Nguyễn Thái Ngọc Duy","fromEmail":"pclouds@gmail.com","sentAt":"2010-08-24T22:20:05Z","receivedAt":"2010-08-24T22:20:05Z","isPatch":true,"sender":{"key":"pclouds@gmail.com","avatar":"https://avatars.githubusercontent.com/u/720?v=4"},"body":"By definition, narrow repository is incomplete. It does not even have\nenough tree for a single commit. So populating a full index is\nimpossible.\n\nBecause of this, unpack_trees() is modified to only unpack trees\nwithin $GIT_DIR/narrow, which narrow repo has all needed trees. This\nmakes the resulting index unsuitable for creating commits later on.\nThis is the reason index version is increased to 4, to avoid older\ngit from using it.\n\nThe resulting tree objects created from the index is only part of the\nfull tree. Manipulation will be needed at commit time to create proper\ntree for commits.\n\nSigned-off-by: Nguyễn Thái Ngọc Duy \n---\n t/t1013-read-tree-narrow.sh | 72 +++++++++++++++++++++++++++++++++++++++++++\n unpack-trees.c | 70 +++++++++++++++++++++++++++++++++++++++++-\n 2 files changed, 141 insertions(+), 1 deletions(-)\n create mode 100755 t/t1013-read-tree-narrow.sh\n\ndiff --git a/t/t1013-read-tree-narrow.sh b/t/t1013-read-tree-narrow.sh\nnew file mode 100755\nindex 0000000..1921985\n--- /dev/null\n+++ b/t/t1013-read-tree-narrow.sh\n@@ -0,0 +1,72 @@\n+#!/bin/sh\n+\n+test_description='read-tree in narrow mode'\n+\n+. ./test-lib.sh\n+\n+test_expect_success setup '\n+\ttest_tick &&\n+\ttest_commit 1 &&\n+\tmkdir t1 t2 t1/t12 &&\n+\techo 0 > f0 &&\n+\techo 10 > t1/f10 &&\n+\techo 120 > t1/t12/f120 &&\n+\techo 20 > t2/f20\n+\tgit add t1 t2 f0 && git commit -m initial &&\n+\tHEAD=`git rev-parse HEAD` &&\n+\tgit rev-parse HEAD | git pack-objects --revs --narrow-tree=t1/t12 pack &&\n+\ttest_create_repo narrow &&\n+\tmv pack-* narrow/.git/objects/pack &&\n+\tcd narrow &&\n+\techo $HEAD >.git/refs/heads/master &&\n+\techo \"ref: refs/heads/master\" >.git/HEAD &&\n+\techo t1/t12 >.git/narrow\n+'\n+\n+test_expect_failure ls-tree '\n+\tgit ls-tree -r HEAD &&\n+\tgit ls-files --stage >result &&\n+\techo \"100644 blob 52bd8e43afb01d0c9747f1fedf2fc94684ee4cc4\tt1/t12/f120\" >expected &&\n+\ttest_cmp expected result\n+'\n+\n+test_expect_success read-tree '\n+\tgit read-tree HEAD &&\n+\tgit ls-files --stage >result &&\n+\techo \"100644 52bd8e43afb01d0c9747f1fedf2fc94684ee4cc4 0\tt1/t12/f120\" >expected &&\n+\ttest_cmp expected result\n+'\n+\n+test_expect_success checkout '\n+\tgit checkout . &&\n+\ttest_cmp ../t1/t12/f120 t1/t12/f120\n+'\n+\n+cat <diff.expected\n+diff --git a/t1/t12/f120 b/t1/t12/f120\n+index 52bd8e4..645fb94 100644\n+--- a/t1/t12/f120\n++++ b/t1/t12/f120\n+@@ -1 +1,2 @@\n+ 120\n++modified\n+EOF\n+\n+test_expect_success diff '\n+\techo modified >> t1/t12/f120 &&\n+\tgit diff >result &&\n+\ttest_cmp diff.expected result\n+'\n+\n+test_expect_success 'diff HEAD' '\n+\tgit diff HEAD >result &&\n+\ttest_cmp diff.expected result\n+'\n+\n+test_expect_success 'diff --cached' '\n+\tgit add -u . &&\n+\tgit diff --cached >result &&\n+\ttest_cmp diff.expected result\n+'\n+\n+test_done\ndiff --git a/unpack-trees.c b/unpack-trees.c\nindex f561d88..661fcb7 100644\n--- a/unpack-trees.c\n+++ b/unpack-trees.c\n@@ -713,7 +713,7 @@ static int unpack_callback(int n, unsigned long mask, unsigned long dirmask, str\n * N-way merge \"len\" trees. Returns 0 on success, -1 on failure to manipulate the\n * resulting index, -2 on failure to reflect the changes to the work tree.\n */\n-int unpack_trees(unsigned len, struct tree_desc *t, struct unpack_trees_options *o)\n+static int unpack_trees_1(unsigned len, struct tree_desc *t, struct unpack_trees_options *o)\n {\n \tint i, ret;\n \tstatic struct cache_entry *dfc;\n@@ -839,6 +839,74 @@ return_failed:\n \tgoto done;\n }\n \n+static int find_tree_desc(struct tree_desc *desc, unsigned char *newsha1, const char *path)\n+{\n+\tstruct name_entry entry;\n+\tconst char *slash;\n+\tint len;\n+\n+\tslash = strchr(path, '/');\n+\tlen = slash ? slash - path : strlen(path);\n+\n+\twhile (tree_entry(desc, &entry)) {\n+\t\tif (!S_ISDIR(entry.mode))\n+\t\t\tcontinue;\n+\t\tif (!strncmp(entry.path, path, len)) {\n+\t\t\tif (slash)\n+\t\t\t\treturn find_tree(entry.sha1, newsha1, slash+1);\n+\t\t\telse {\n+\t\t\t\thashcpy(newsha1, entry.sha1);\n+\t\t\t\treturn 1;\n+\t\t\t}\n+\t\t}\n+\t}\n+\treturn 0;\n+}\n+\n+int unpack_trees(unsigned len, struct tree_desc *t, struct unpack_trees_options *o)\n+{\n+\tif (!get_narrow_prefix())\n+\t\treturn unpack_trees_1(len, t, o);\n+\telse {\n+\t\tstruct tree_desc *t2;\n+\t\tstruct unpack_trees_options o2;\n+\t\tchar *prefix = NULL;\n+\t\tvoid **buf;\n+\t\tint i, ret;\n+\t\tt2 = xmalloc(sizeof(*t2)*len);\n+\t\tbuf = xmalloc(sizeof(*buf)*len);\n+\t\tfor (i = 0; i < len; i++) {\n+\t\t\tunsigned char sha1[20];\n+\t\t\tunsigned long size;\n+\t\t\tenum object_type type;\n+\n+\t\t\tif (!find_tree_desc(t+i, sha1, get_narrow_prefix()))\n+\t\t\t\treturn -1;\n+\t\t\tbuf[i] = read_sha1_file(sha1, &type, &size);\n+\t\t\tif (type != OBJ_TREE)\n+\t\t\t\treturn -1;\n+\t\t\tinit_tree_desc(t2+i, buf[i], size);\n+\t\t}\n+\t\to2 = *o;\n+\t\tif (!o->prefix)\n+\t\t\to2.prefix = xstrdup(get_narrow_prefix());\n+\t\telse {\n+\t\t\tprefix = xmalloc(strlen(get_narrow_prefix()) + strlen(o->prefix)+1);\n+\t\t\tstrcpy(prefix, o->prefix);\n+\t\t\tstrcat(prefix, get_narrow_prefix());\n+\t\t\to2.prefix = prefix;\n+\t\t}\n+\t\tret = unpack_trees_1(len, t2, &o2);\n+\t\tfor (i = 0; i < len; i++) {\n+\t\t\tfree(buf[i]);\n+\t\t}\n+\t\tfree(prefix);\n+\t\tfree(t2);\n+\t\tfree(buf);\n+\t\treturn ret;\n+\t}\n+}\n+\n /* Here come the merge functions */\n \n static int reject_merge(struct cache_entry *ce, struct unpack_trees_options *o)\n-- \n1.7.1.rc1.69.g24c2f7\n"},{"id":"148882","messageId":"1282688422-7738-17-git-send-email-pclouds@gmail.com","threadId":"24851","inReplyTo":"1282688422-7738-1-git-send-email-pclouds@gmail.com","subject":"[PATCH 16/32] cache-tree: only cache tree within narrow area","fromName":"Nguyễn Thái Ngọc Duy","fromEmail":"pclouds@gmail.com","sentAt":"2010-08-24T22:20:06Z","receivedAt":"2010-08-24T22:20:06Z","isPatch":true,"sender":{"key":"pclouds@gmail.com","avatar":"https://avatars.githubusercontent.com/u/720?v=4"},"body":"\nSigned-off-by: Nguyễn Thái Ngọc Duy \n---\n cache-tree.c | 17 +++++++++++++----\n 1 files changed, 13 insertions(+), 4 deletions(-)\n\ndiff --git a/cache-tree.c b/cache-tree.c\nindex 03952c7..69ce0a2 100644\n--- a/cache-tree.c\n+++ b/cache-tree.c\n@@ -616,11 +616,17 @@ int write_cache_as_tree(unsigned char *sha1, int flags, const char *prefix)\n \treturn 0;\n }\n \n-static void prime_cache_tree_rec(struct cache_tree *it, struct tree *tree)\n+static void prime_cache_tree_rec(struct cache_tree *it, struct tree *tree, const char *path)\n {\n \tstruct tree_desc desc;\n \tstruct name_entry entry;\n-\tint cnt;\n+\tconst char *slash;\n+\tint len = 0, cnt;\n+\n+\tif (path) {\n+\t\tslash = strchr(path, '/');\n+\t\tlen = slash ? slash - path : strlen(path);\n+\t}\n \n \thashcpy(it->sha1, tree->object.sha1);\n \tinit_tree_desc(&desc, tree->buffer, tree->size);\n@@ -628,6 +634,8 @@ static void prime_cache_tree_rec(struct cache_tree *it, struct tree *tree)\n \twhile (tree_entry(&desc, &entry)) {\n \t\tif (!S_ISDIR(entry.mode))\n \t\t\tcnt++;\n+\t\telse if (get_narrow_prefix() && len && strncmp(entry.path, path, len))\n+\t\t\tcnt++;\n \t\telse {\n \t\t\tstruct cache_tree_sub *sub;\n \t\t\tstruct tree *subtree = lookup_tree(entry.sha1);\n@@ -635,7 +643,8 @@ static void prime_cache_tree_rec(struct cache_tree *it, struct tree *tree)\n \t\t\t\tparse_tree(subtree);\n \t\t\tsub = cache_tree_sub(it, entry.path);\n \t\t\tsub->cache_tree = cache_tree();\n-\t\t\tprime_cache_tree_rec(sub->cache_tree, subtree);\n+\t\t\tprime_cache_tree_rec(sub->cache_tree, subtree,\n+\t\t\t\t\t slash ? slash+1 : NULL);\n \t\t\tcnt += sub->cache_tree->entry_count;\n \t\t}\n \t}\n@@ -646,7 +655,7 @@ void prime_cache_tree(struct cache_tree **it, struct tree *tree)\n {\n \tcache_tree_free(it);\n \t*it = cache_tree();\n-\tprime_cache_tree_rec(*it, tree);\n+\tprime_cache_tree_rec(*it, tree, get_narrow_prefix());\n }\n \n /*\n-- \n1.7.1.rc1.69.g24c2f7\n"},{"id":"148885","messageId":"1282688422-7738-18-git-send-email-pclouds@gmail.com","threadId":"24851","inReplyTo":"1282688422-7738-1-git-send-email-pclouds@gmail.com","subject":"[PATCH 17/32] tree-diff: add narrow versions of diff_{root_,}tree_sha1","fromName":"Nguyễn Thái Ngọc Duy","fromEmail":"pclouds@gmail.com","sentAt":"2010-08-24T22:20:07Z","receivedAt":"2010-08-24T22:20:07Z","isPatch":true,"sender":{"key":"pclouds@gmail.com","avatar":"https://avatars.githubusercontent.com/u/720?v=4"},"body":"These versions assume that the given trees for diff are toplevel\ntrees. They simply go down to find $GIT_DIR/narrow subtree and do diff\non those trees instead. diff_tree_sha1 callers must be adapted to use\nthis version to support narrow repos.\n\nSigned-off-by: Nguyễn Thái Ngọc Duy \n---\n diff.h | 5 ++++\n tree-diff.c | 74 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++\n 2 files changed, 79 insertions(+), 0 deletions(-)\n\ndiff --git a/diff.h b/diff.h\nindex d43da9d..13b4f10 100644\n--- a/diff.h\n+++ b/diff.h\n@@ -171,6 +171,11 @@ extern int diff_tree_sha1(const unsigned char *old, const unsigned char *new,\n extern int diff_root_tree_sha1(const unsigned char *new, const char *base,\n struct diff_options *opt);\n \n+extern int narrow_diff_tree_sha1(const unsigned char *old, const unsigned char *new,\n+\t\t\t\t const char *base, struct diff_options *opt);\n+extern int narrow_diff_root_tree_sha1(const unsigned char *new, const char *base,\n+\t\t\t\t struct diff_options *opt);\n+\n struct combine_diff_path {\n \tstruct combine_diff_path *next;\n \tint len;\ndiff --git a/tree-diff.c b/tree-diff.c\nindex cd659c6..31b0842 100644\n--- a/tree-diff.c\n+++ b/tree-diff.c\n@@ -433,6 +433,45 @@ int diff_tree_sha1(const unsigned char *old, const unsigned char *new, const cha\n \treturn retval;\n }\n \n+int narrow_diff_tree_sha1(const unsigned char *old, const unsigned char *new, const char *base, struct diff_options *opt)\n+{\n+\tunsigned char old2[20], new2[20];\n+\tchar *new_base = NULL;\n+\tint retval;\n+\n+\tif (get_narrow_prefix()) {\n+\t\tint len1 = strlen(get_narrow_prefix());\n+\t\tint len2 = base ? strlen(base) : 0;\n+\t\tunsigned long size;\n+\t\tunsigned char deref_sha1[20];\n+\n+\t\tif (len2 && !strncmp(base, get_narrow_prefix(), len1) &&\n+\t\t (base[len1] == '/' || base[len1] == '\\0'))\n+\t\t\t;\t/* good */\n+\t\telse {\n+\t\t\tnew_base = xmalloc(len1 + 1 + len2 + 1);\n+\t\t\tmemcpy(new_base, get_narrow_prefix, len1+1);\n+\t\t\tif (len2) {\n+\t\t\t\tnew_base[len1] = '/';\n+\t\t\t\tmemcpy(new_base+len1+1, base, len2+1);\n+\t\t\t}\n+\t\t\tbase = new_base;\n+\t\t}\n+\n+\t\tfree(read_object_with_reference(old, tree_type, &size, deref_sha1));\n+\t\tif (!find_tree(deref_sha1, old2, get_narrow_prefix()))\n+\t\t\thashcpy(old2, (const unsigned char *)EMPTY_TREE_SHA1_BIN);\n+\t\told = old2;\n+\t\tfree(read_object_with_reference(new, tree_type, &size, deref_sha1));\n+\t\tif (!find_tree(deref_sha1, new2, get_narrow_prefix()))\n+\t\t\thashcpy(new2, (const unsigned char *)EMPTY_TREE_SHA1_BIN);\n+\t\tnew = new2;\n+\t}\n+\tretval = diff_tree_sha1(old, new, base, opt);\n+\tfree(new_base);\n+\treturn retval;\n+}\n+\n int diff_root_tree_sha1(const unsigned char *new, const char *base, struct diff_options *opt)\n {\n \tint retval;\n@@ -451,6 +490,41 @@ int diff_root_tree_sha1(const unsigned char *new, const char *base, struct diff_\n \treturn retval;\n }\n \n+int narrow_diff_root_tree_sha1(const unsigned char *new, const char *base, struct diff_options *opt)\n+{\n+\tunsigned char new2[20];\n+\tchar *new_base = NULL;\n+\tint retval;\n+\n+\tif (get_narrow_prefix()) {\n+\t\tint len1 = strlen(get_narrow_prefix());\n+\t\tint len2 = base ? strlen(base) : 0;\n+\t\tunsigned long size;\n+\t\tunsigned char deref_sha1[20];\n+\n+\t\tif (len2 && !strncmp(base, get_narrow_prefix(), len1) &&\n+\t\t (base[len1] == '/' || base[len1] == '\\0'))\n+\t\t\t;\t/* good */\n+\t\telse {\n+\t\t\tnew_base = xmalloc(len1 + 1 + len2 + 1);\n+\t\t\tmemcpy(new_base, get_narrow_prefix, len1+1);\n+\t\t\tif (len2) {\n+\t\t\t\tnew_base[len1] = '/';\n+\t\t\t\tmemcpy(new_base+len1+1, base, len2+1);\n+\t\t\t}\n+\t\t\tbase = new_base;\n+\t\t}\n+\n+\t\tfree(read_object_with_reference(new, tree_type, &size, deref_sha1));\n+\t\tif (!find_tree(deref_sha1, new2, get_narrow_prefix()))\n+\t\t\thashcpy(new2, (const unsigned char *)EMPTY_TREE_SHA1_BIN);\n+\t\tnew = new2;\n+\t}\n+\tretval = diff_root_tree_sha1(new, base, opt);\n+\tfree(new_base);\n+\treturn retval;\n+}\n+\n static int count_paths(const char **paths)\n {\n \tint i = 0;\n-- \n1.7.1.rc1.69.g24c2f7\n"},{"id":"148886","messageId":"1282688422-7738-19-git-send-email-pclouds@gmail.com","threadId":"24851","inReplyTo":"1282688422-7738-1-git-send-email-pclouds@gmail.com","subject":"[PATCH 18/32] log-tree: use narrow version of diff_tree_sha1","fromName":"Nguyễn Thái Ngọc Duy","fromEmail":"pclouds@gmail.com","sentAt":"2010-08-24T22:20:08Z","receivedAt":"2010-08-24T22:20:08Z","isPatch":true,"sender":{"key":"pclouds@gmail.com","avatar":"https://avatars.githubusercontent.com/u/720?v=4"},"body":"This makes \"git log\" work in narrow repos.\n\nSigned-off-by: Nguyễn Thái Ngọc Duy \n---\n log-tree.c | 23 ++++++++++++++++++++---\n 1 files changed, 20 insertions(+), 3 deletions(-)\n\ndiff --git a/log-tree.c b/log-tree.c\nindex b46ed3b..693992a 100644\n--- a/log-tree.c\n+++ b/log-tree.c\n@@ -600,7 +600,10 @@ static int log_tree_diff(struct rev_info *opt, struct commit *commit, struct log\n \tparents = commit->parents;\n \tif (!parents) {\n \t\tif (opt->show_root_diff) {\n-\t\t\tdiff_root_tree_sha1(sha1, \"\", &opt->diffopt);\n+\t\t\tif (get_narrow_prefix())\n+\t\t\t\tnarrow_diff_root_tree_sha1(sha1, NULL, &opt->diffopt);\n+\t\t\telse\n+\t\t\t\tdiff_root_tree_sha1(sha1, \"\", &opt->diffopt);\n \t\t\tlog_tree_diff_flush(opt);\n \t\t}\n \t\treturn !opt->loginfo;\n@@ -618,7 +621,14 @@ static int log_tree_diff(struct rev_info *opt, struct commit *commit, struct log\n \t\t\t * parent, showing summary diff of the others\n \t\t\t * we merged _in_.\n \t\t\t */\n-\t\t\tdiff_tree_sha1(parents->item->object.sha1, sha1, \"\", &opt->diffopt);\n+\t\t\tif (get_narrow_prefix())\n+\t\t\t\tnarrow_diff_tree_sha1(parents->item->object.sha1,\n+\t\t\t\t\t\t sha1,\n+\t\t\t\t\t\t NULL,\n+\t\t\t\t\t\t &opt->diffopt);\n+\t\t\telse\n+\t\t\t\tdiff_tree_sha1(parents->item->object.sha1, sha1,\n+\t\t\t\t\t \"\", &opt->diffopt);\n \t\t\tlog_tree_diff_flush(opt);\n \t\t\treturn !opt->loginfo;\n \t\t}\n@@ -631,7 +641,14 @@ static int log_tree_diff(struct rev_info *opt, struct commit *commit, struct log\n \tfor (;;) {\n \t\tstruct commit *parent = parents->item;\n \n-\t\tdiff_tree_sha1(parent->object.sha1, sha1, \"\", &opt->diffopt);\n+\t\tif (get_narrow_prefix())\n+\t\t\tnarrow_diff_tree_sha1(parent->object.sha1,\n+\t\t\t\t\t sha1,\n+\t\t\t\t\t NULL,\n+\t\t\t\t\t &opt->diffopt);\n+\t\telse\n+\t\t\tdiff_tree_sha1(parent->object.sha1, sha1,\n+\t\t\t\t \"\", &opt->diffopt);\n \t\tlog_tree_diff_flush(opt);\n \n \t\tshowed_log |= !opt->loginfo;\n-- \n1.7.1.rc1.69.g24c2f7\n"},{"id":"148881","messageId":"1282688422-7738-20-git-send-email-pclouds@gmail.com","threadId":"24851","inReplyTo":"1282688422-7738-1-git-send-email-pclouds@gmail.com","subject":"[PATCH 19/32] clone: support --narrow option","fromName":"Nguyễn Thái Ngọc Duy","fromEmail":"pclouds@gmail.com","sentAt":"2010-08-24T22:20:09Z","receivedAt":"2010-08-24T22:20:09Z","isPatch":true,"sender":{"key":"pclouds@gmail.com","avatar":"https://avatars.githubusercontent.com/u/720?v=4"},"body":"\nSigned-off-by: Nguyễn Thái Ngọc Duy \n---\n builtin/clone.c | 15 ++++++++++++++-\n 1 files changed, 14 insertions(+), 1 deletions(-)\n\ndiff --git a/builtin/clone.c b/builtin/clone.c\nindex efb1e6f..65c4cee 100644\n--- a/builtin/clone.c\n+++ b/builtin/clone.c\n@@ -23,6 +23,7 @@\n #include \"branch.h\"\n #include \"remote.h\"\n #include \"run-command.h\"\n+#include \"narrow-tree.h\"\n \n /*\n * Overall FIXMEs:\n@@ -39,7 +40,7 @@ static const char * const builtin_clone_usage[] = {\n \n static int option_no_checkout, option_bare, option_mirror;\n static int option_local, option_no_hardlinks, option_shared, option_recursive;\n-static char *option_template, *option_reference, *option_depth;\n+static char *option_template, *option_reference, *option_depth, *option_narrow;\n static char *option_origin = NULL;\n static char *option_branch = NULL;\n static char *option_upload_pack = \"git-upload-pack\";\n@@ -78,6 +79,8 @@ static struct option builtin_clone_options[] = {\n \t\t \"path to git-upload-pack on the remote\"),\n \tOPT_STRING(0, \"depth\", &option_depth, \"depth\",\n \t\t \"create a shallow clone of that depth\"),\n+\tOPT_STRING(0, \"narrow\", &option_narrow, \"tree\",\n+\t\t \"narrow clone\"),\n \n \tOPT_END()\n };\n@@ -515,6 +518,8 @@ int cmd_clone(int argc, const char **argv, const char *prefix)\n \tstrbuf_reset(&value);\n \n \tif (path && !is_bundle) {\n+\t\tif (option_narrow)\n+\t\t\tdie(\"--narrow is not really for local clone. Please consider --shared\");\n \t\trefs = clone_local(path, git_dir);\n \t\tmapped_refs = wanted_peer_refs(refs, refspec);\n \t} else {\n@@ -530,6 +535,14 @@ int cmd_clone(int argc, const char **argv, const char *prefix)\n \t\t\ttransport_set_option(transport, TRANS_OPT_DEPTH,\n \t\t\t\t\t option_depth);\n \n+\t\t/* Do this early in order to set get_narrow_prefix() != NULL */\n+\t\tif (option_narrow) {\n+\t\t\tint fd = open(git_path(\"narrow\"), O_RDWR | O_CREAT, 0644);\n+\t\t\twrite(fd, option_narrow, strlen(option_narrow));\n+\t\t\tclose(fd);\n+\t\t\tcheck_narrow_prefix();\n+\t\t}\n+\n \t\ttransport_set_verbosity(transport, option_verbosity, option_progress);\n \n \t\tif (option_upload_pack)\n-- \n1.7.1.rc1.69.g24c2f7\n"},{"id":"148889","messageId":"1282688422-7738-21-git-send-email-pclouds@gmail.com","threadId":"24851","inReplyTo":"1282688422-7738-1-git-send-email-pclouds@gmail.com","subject":"[PATCH 20/32] narrow-tree: add join_narrow_tree to do tree fixup for commits","fromName":"Nguyễn Thái Ngọc Duy","fromEmail":"pclouds@gmail.com","sentAt":"2010-08-24T22:20:10Z","receivedAt":"2010-08-24T22:20:10Z","isPatch":true,"sender":{"key":"pclouds@gmail.com","avatar":"https://avatars.githubusercontent.com/u/720?v=4"},"body":"As stated somewhere, the index in narrow repo is also narrowed. When a\nusers are done with his changes and about to commit, the new narrow\ntree created from index will be grafted back to a base toplevel tree\n(usually from parent commit). The result is a new toplevel tree with\nuser's changes and suitable for commits.\n\nThis function does exactly that.\n\nSigned-off-by: Nguyễn Thái Ngọc Duy \n---\n narrow-tree.c | 66 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++\n narrow-tree.h | 2 +\n 2 files changed, 68 insertions(+), 0 deletions(-)\n\ndiff --git a/narrow-tree.c b/narrow-tree.c\nindex 46e913d..73d4f22 100644\n--- a/narrow-tree.c\n+++ b/narrow-tree.c\n@@ -1,4 +1,14 @@\n #include \"cache.h\"\n+#include \"commit.h\"\n+#include \"tree.h\"\n+#include \"diff.h\"\n+#include \"revision.h\"\n+#include \"refs.h\"\n+#include \"tag.h\"\n+#include \"progress.h\"\n+#include \"pack.h\"\n+#include \"sha1-lookup.h\"\n+#include \"csum-file.h\"\n #include \"narrow-tree.h\"\n \n static const char *narrow_prefix;\n@@ -29,3 +39,59 @@ const char *get_narrow_prefix()\n {\n \treturn narrow_prefix;\n }\n+\n+/*\n+ * The opposite of narrow_tree(). Put the subtree back to the original tree.\n+ */\n+int join_narrow_tree(const unsigned char *base,\n+\t\t unsigned char *newsha1,\n+\t\t const unsigned char *subtree_sha1,\n+\t\t const char *prefix)\n+{\n+\tstruct tree_desc desc;\n+\tstruct name_entry entry;\n+\tstruct strbuf buffer;\n+\tenum object_type type;\n+\tconst char *slash;\n+\tchar *buf;\n+\tint subtree_len;\n+\tunsigned long size;\n+\n+\tslash = strchr(prefix, '/');\n+\tsubtree_len = slash ? slash - prefix : strlen(prefix);\n+\n+\tbuf = read_sha1_file(base, &type, &size);\n+\tif (!buf || type != OBJ_TREE)\n+\t\tdie(\"Bad tree %s\", sha1_to_hex(base));\n+\n+\tinit_tree_desc(&desc, buf, size);\n+\tstrbuf_init(&buffer, 8192);\n+\twhile (tree_entry(&desc, &entry)) {\n+\t\tstrbuf_addf(&buffer, \"%o %.*s%c\", entry.mode, strlen(entry.path), entry.path, '\\0');\n+\n+\t\tif (S_ISDIR(entry.mode) &&\n+\t\t subtree_len == strlen(entry.path) &&\n+\t\t !strncmp(entry.path, prefix, subtree_len)) {\n+\t\t\tif (slash && slash[1]) /* trailing slash does not count */\n+\t\t\t\tjoin_narrow_tree(entry.sha1, newsha1,\n+\t\t\t\t\t\t subtree_sha1, slash+1);\n+\t\t\telse\n+\t\t\t\tmemcpy(newsha1, subtree_sha1, 20); /* replace the tree */\n+\n+\t\t\t/* FIXME, what if placeholder tree does not exist? */\n+\n+\t\t\tstrbuf_add(&buffer, newsha1, 20);\n+\t\t}\n+\t\telse\n+\t\t\tstrbuf_add(&buffer, entry.sha1, 20);\n+\t}\n+\n+\tfree(buf);\n+\tif (write_sha1_file(buffer.buf, buffer.len, tree_type, newsha1)) {\n+\t\terror(\"Could not write replaced tree for %s\", sha1_to_hex(subtree_sha1));\n+\t\tstrbuf_release(&buffer);\n+\t\treturn 1;\n+\t}\n+\tstrbuf_release(&buffer);\n+\treturn 0;\n+}\ndiff --git a/narrow-tree.h b/narrow-tree.h\nindex aa8c94f..ecb3ded 100644\n--- a/narrow-tree.h\n+++ b/narrow-tree.h\n@@ -1 +1,3 @@\n extern int check_narrow_prefix();\n+extern int join_narrow_tree(const unsigned char *base, unsigned char *newsha1,\n+\t\t\t const unsigned char *subtree_sha1, const char *prefix);\n-- \n1.7.1.rc1.69.g24c2f7\n"},{"id":"148888","messageId":"1282688422-7738-22-git-send-email-pclouds@gmail.com","threadId":"24851","inReplyTo":"1282688422-7738-1-git-send-email-pclouds@gmail.com","subject":"[PATCH 21/32] commit: add narrow's commit_tree version","fromName":"Nguyễn Thái Ngọc Duy","fromEmail":"pclouds@gmail.com","sentAt":"2010-08-24T22:20:11Z","receivedAt":"2010-08-24T22:20:11Z","isPatch":true,"sender":{"key":"pclouds@gmail.com","avatar":"https://avatars.githubusercontent.com/u/720?v=4"},"body":"The narrow version uses join_narrow_tree() to recreate a full tree\nfrom a base toplevel tree (typically commit parent's tree) and a tree\ncreated from index.\n\nSigned-off-by: Nguyễn Thái Ngọc Duy \n---\n commit.c | 21 +++++++++++++++++++++\n commit.h | 5 +++++\n 2 files changed, 26 insertions(+), 0 deletions(-)\n\ndiff --git a/commit.c b/commit.c\nindex 0094ec1..77ed9fd 100644\n--- a/commit.c\n+++ b/commit.c\n@@ -6,6 +6,7 @@\n #include \"diff.h\"\n #include \"revision.h\"\n #include \"notes.h\"\n+#include \"narrow-tree.h\"\n \n int save_commit_buffer = 1;\n \n@@ -864,3 +865,23 @@ int commit_tree(const char *msg, unsigned char *tree,\n \tstrbuf_release(&buffer);\n \treturn result;\n }\n+\n+int commit_narrow_tree(const char *msg, unsigned char *tree,\n+\t\t const unsigned char *narrow_base,\n+\t\t struct commit_list *parents, unsigned char *ret,\n+\t\t const char *author)\n+{\n+\tunsigned char result_tree[20];\n+\n+\tif (get_narrow_prefix()) {\n+\t\tunsigned char sha1[20];\n+\n+\t\tif (!find_tree(tree, sha1, get_narrow_prefix()))\n+\t\t\tdie(\"Narrow tree %s not found\", get_narrow_prefix());\n+\t\tif (join_narrow_tree(narrow_base, result_tree, sha1,\n+\t\t\t\t get_narrow_prefix()))\n+\t\t\tdie(\"Failed to join tree\");\n+\t\ttree = result_tree;\n+\t}\n+\treturn commit_tree(msg, tree, parents, ret, author);\n+}\ndiff --git a/commit.h b/commit.h\nindex 9113bbe..c718439 100644\n--- a/commit.h\n+++ b/commit.h\n@@ -170,5 +170,10 @@ struct commit_list *reduce_heads(struct commit_list *heads);\n extern int commit_tree(const char *msg, unsigned char *tree,\n \t\tstruct commit_list *parents, unsigned char *ret,\n \t\tconst char *author);\n+extern int commit_narrow_tree(const char *msg, unsigned char *tree,\n+\t\t\t const unsigned char *narrow_base,\n+\t\t\t struct commit_list *parents,\n+\t\t\t unsigned char *ret,\n+\t\t\t const char *author);\n \n #endif /* COMMIT_H */\n-- \n1.7.1.rc1.69.g24c2f7\n"},{"id":"148887","messageId":"1282688422-7738-23-git-send-email-pclouds@gmail.com","threadId":"24851","inReplyTo":"1282688422-7738-1-git-send-email-pclouds@gmail.com","subject":"[PATCH 22/32] commit: use commit_narrow_tree() to support narrow repo","fromName":"Nguyễn Thái Ngọc Duy","fromEmail":"pclouds@gmail.com","sentAt":"2010-08-24T22:20:12Z","receivedAt":"2010-08-24T22:20:12Z","isPatch":true,"sender":{"key":"pclouds@gmail.com","avatar":"https://avatars.githubusercontent.com/u/720?v=4"},"body":"\nSigned-off-by: Nguyễn Thái Ngọc Duy \n---\n builtin/commit.c | 10 +++++++++-\n 1 files changed, 9 insertions(+), 1 deletions(-)\n\ndiff --git a/builtin/commit.c b/builtin/commit.c\nindex 66fdd22..59da654 100644\n--- a/builtin/commit.c\n+++ b/builtin/commit.c\n@@ -1351,7 +1351,15 @@ int cmd_commit(int argc, const char **argv, const char *prefix)\n \t\texit(1);\n \t}\n \n-\tif (commit_tree(sb.buf, active_cache_tree->sha1, parents, commit_sha1,\n+\tif (get_narrow_prefix()) {\n+\t\tif (!parents)\n+\t\t\tdie(\"Narrow mode does not support initial commit (yet)\");\n+\t\tif (parse_commit(parents->item))\n+\t\t\tdie(\"Bad commit %s\", parents->item->object.sha1);\n+\t}\n+\tif (commit_narrow_tree(sb.buf, active_cache_tree->sha1,\n+\t\t\tparents ? parents->item->tree->object.sha1 : NULL,\n+\t\t\tparents, commit_sha1,\n \t\t\tfmt_ident(author_name, author_email, author_date,\n \t\t\t\tIDENT_ERROR_ON_NO_NAME))) {\n \t\trollback_index_files();\n-- \n1.7.1.rc1.69.g24c2f7\n"},{"id":"148894","messageId":"1282688422-7738-24-git-send-email-pclouds@gmail.com","threadId":"24851","inReplyTo":"1282688422-7738-1-git-send-email-pclouds@gmail.com","subject":"[PATCH 23/32] commit-tree: require --narrow-base in narrow repo","fromName":"Nguyễn Thái Ngọc Duy","fromEmail":"pclouds@gmail.com","sentAt":"2010-08-24T22:20:13Z","receivedAt":"2010-08-24T22:20:13Z","isPatch":true,"sender":{"key":"pclouds@gmail.com","avatar":"https://avatars.githubusercontent.com/u/720?v=4"},"body":"\nSigned-off-by: Nguyễn Thái Ngọc Duy \n---\n builtin/commit-tree.c | 20 +++++++++++++++++++-\n 1 files changed, 19 insertions(+), 1 deletions(-)\n\ndiff --git a/builtin/commit-tree.c b/builtin/commit-tree.c\nindex 87f0591..0c71925 100644\n--- a/builtin/commit-tree.c\n+++ b/builtin/commit-tree.c\n@@ -32,11 +32,29 @@ int cmd_commit_tree(int argc, const char **argv, const char *prefix)\n \tunsigned char tree_sha1[20];\n \tunsigned char commit_sha1[20];\n \tstruct strbuf buffer = STRBUF_INIT;\n+\tunsigned char narrow_base[20];\n \n \tgit_config(git_default_config, NULL);\n \n \tif (argc < 2 || !strcmp(argv[1], \"-h\"))\n \t\tusage(commit_tree_usage);\n+\n+\tif (get_narrow_prefix()) {\n+\t\tunsigned char sha1[20];\n+\t\tunsigned long size;\n+\t\tvoid *buf;\n+\n+\t\tif (prefixcmp(argv[1], \"--narrow-base=\"))\n+\t\t\tdie(\"--narrow-base required in narrow mode\");\n+\t\tif (get_sha1(argv[1]+4, sha1))\n+\t\t\tdie(\"Invalid SHA1\");\n+\t\tbuf = read_object_with_reference(sha1, tree_type, &size, narrow_base);\n+\t\tif (!buf)\n+\t\t\tdie(\"Bad treeish %s\", sha1_to_hex(sha1));\n+\t\tfree(buf);\n+\t\targv++;\n+\t}\n+\n \tif (get_sha1(argv[1], tree_sha1))\n \t\tdie(\"Not a valid object name %s\", argv[1]);\n \n@@ -56,7 +74,7 @@ int cmd_commit_tree(int argc, const char **argv, const char *prefix)\n \tif (strbuf_read(&buffer, 0, 0) < 0)\n \t\tdie_errno(\"git commit-tree: failed to read\");\n \n-\tif (!commit_tree(buffer.buf, tree_sha1, parents, commit_sha1, NULL)) {\n+\tif (!commit_narrow_tree(buffer.buf, tree_sha1, narrow_base, parents, commit_sha1, NULL)) {\n \t\tprintf(\"%s\\n\", sha1_to_hex(commit_sha1));\n \t\treturn 0;\n \t}\n-- \n1.7.1.rc1.69.g24c2f7\n"},{"id":"148893","messageId":"1282688422-7738-25-git-send-email-pclouds@gmail.com","threadId":"24851","inReplyTo":"1282688422-7738-1-git-send-email-pclouds@gmail.com","subject":"[PATCH 24/32] merge: refuse to merge if narrow bases are different","fromName":"Nguyễn Thái Ngọc Duy","fromEmail":"pclouds@gmail.com","sentAt":"2010-08-24T22:20:14Z","receivedAt":"2010-08-24T22:20:14Z","isPatch":true,"sender":{"key":"pclouds@gmail.com","avatar":"https://avatars.githubusercontent.com/u/720?v=4"},"body":"commit_narrow_tree() works with a single narrow base. Unfortunately a\nmerge may have more than one parent. If all parents have the same\ntrees outside $GIT_DIR/narrow tree, then it's actually \"a single\nnarrow base\".\n\nOther than that, refuse to merge. A merge in such case will need\nserver support because narrow repos do not have enough trees to\nperform merge locally.\n\nSigned-off-by: Nguyễn Thái Ngọc Duy \n---\n builtin/merge.c | 18 ++++++++++++++++++\n narrow-tree.c | 55 +++++++++++++++++++++++++++++++++++++++++++++++++++++++\n narrow-tree.h | 1 +\n 3 files changed, 74 insertions(+), 0 deletions(-)\n\ndiff --git a/builtin/merge.c b/builtin/merge.c\nindex 37ce4f5..c70d39d 100644\n--- a/builtin/merge.c\n+++ b/builtin/merge.c\n@@ -25,6 +25,7 @@\n #include \"help.h\"\n #include \"merge-recursive.h\"\n #include \"resolve-undo.h\"\n+#include \"narrow-tree.h\"\n \n #define DEFAULT_TWOHEAD (1<<0)\n #define DEFAULT_OCTOPUS (1<<1)\n@@ -1048,6 +1049,23 @@ int cmd_merge(int argc, const char **argv, const char *prefix)\n \t\t\tallow_trivial = 0;\n \t}\n \n+\tif (get_narrow_prefix()) {\n+\t\tstruct commit_list *item = remoteheads;\n+\t\tstruct commit *head_commit = lookup_commit(head);\n+\n+\t\tparse_commit(head_commit);\n+\t\twhile (item) {\n+\t\t\tparse_commit(item->item);\n+\t\t\tif (!same_narrow_base(head_commit->tree->object.sha1,\n+\t\t\t\t\t item->item->tree->object.sha1,\n+\t\t\t\t\t get_narrow_prefix()))\n+\t\t\t\tbreak;\n+\t\t\titem = item->next;\n+\t\t}\n+\t\tif (item)\n+\t\t\tdie(\"Different narrow base, couldn't do merge (yet)\");\n+\t}\n+\n \tif (!remoteheads->next)\n \t\tcommon = get_merge_bases(lookup_commit(head),\n \t\t\t\tremoteheads->item, 1);\ndiff --git a/narrow-tree.c b/narrow-tree.c\nindex 73d4f22..4a16647 100644\n--- a/narrow-tree.c\n+++ b/narrow-tree.c\n@@ -95,3 +95,58 @@ int join_narrow_tree(const unsigned char *base,\n \tstrbuf_release(&buffer);\n \treturn 0;\n }\n+\n+int same_narrow_base(const unsigned char *t1, const unsigned char *t2, const char *prefix)\n+{\n+\tstruct tree_desc desc1, desc2;\n+\tstruct name_entry entry1, entry2;\n+\tchar *buf1, *buf2;\n+\tenum object_type type;\n+\tunsigned long size;\n+\tconst char *slash;\n+\tint prefix_len;\n+\n+\tslash = strchr(prefix, '/');\n+\tprefix_len = slash ? slash - prefix : strlen(prefix);\n+\n+\tbuf1 = read_sha1_file(t1, &type, &size);\n+\tif (type != OBJ_TREE)\n+\t\tdie(\"Bad tree %s\", sha1_to_hex(t1));\n+\tinit_tree_desc(&desc1, buf1, size);\n+\n+\tbuf2 = read_sha1_file(t2, &type, &size);\n+\tif (type != OBJ_TREE)\n+\t\tdie(\"Bad tree %s\", sha1_to_hex(t2));\n+\tinit_tree_desc(&desc2, buf2, size);\n+\n+\twhile (tree_entry(&desc1, &entry1) && tree_entry(&desc2, &entry2)) {\n+\t\tif (strcmp(entry1.path, entry2.path) ||\n+\t\t entry1.mode != entry2.mode) {\n+\t\t\tfree(buf1);\n+\t\t\tfree(buf2);\n+\t\t\treturn 0;\n+\t\t}\n+\n+\t\tif (!hashcmp(entry1.sha1, entry2.sha1))\n+\t\t\tcontinue;\n+\n+\t\tif (S_ISDIR(entry1.mode) &&\n+\t\t !strncmp(entry1.path, prefix, prefix_len) &&\n+\t\t entry1.path[prefix_len] == '\\0') {\n+\n+\t\t\t/* This is subtree, SHA-1 does not matter */\n+\t\t\tif (!slash)\n+\t\t\t\tcontinue;\n+\n+\t\t\tif (same_narrow_base(entry1.sha1, entry2.sha1, slash+1))\n+\t\t\t\tcontinue;\n+\t\t}\n+\n+\t\tfree(buf1);\n+\t\tfree(buf2);\n+\t\treturn 0;\n+\t}\n+\tfree(buf1);\n+\tfree(buf2);\n+\treturn !desc1.size && !desc2.size;\n+}\ndiff --git a/narrow-tree.h b/narrow-tree.h\nindex ecb3ded..8756094 100644\n--- a/narrow-tree.h\n+++ b/narrow-tree.h\n@@ -1,3 +1,4 @@\n extern int check_narrow_prefix();\n extern int join_narrow_tree(const unsigned char *base, unsigned char *newsha1,\n \t\t\t const unsigned char *subtree_sha1, const char *prefix);\n+int same_narrow_base(const unsigned char *t1, const unsigned char *t2, const char *prefix);\n-- \n1.7.1.rc1.69.g24c2f7\n"},{"id":"148891","messageId":"1282688422-7738-26-git-send-email-pclouds@gmail.com","threadId":"24851","inReplyTo":"1282688422-7738-1-git-send-email-pclouds@gmail.com","subject":"[PATCH 25/32] merge: prepare commit properly in narrow mode","fromName":"Nguyễn Thái Ngọc Duy","fromEmail":"pclouds@gmail.com","sentAt":"2010-08-24T22:20:15Z","receivedAt":"2010-08-24T22:20:15Z","isPatch":true,"sender":{"key":"pclouds@gmail.com","avatar":"https://avatars.githubusercontent.com/u/720?v=4"},"body":"\nSigned-off-by: Nguyễn Thái Ngọc Duy \n---\n builtin/merge.c | 8 ++++++--\n 1 files changed, 6 insertions(+), 2 deletions(-)\n\ndiff --git a/builtin/merge.c b/builtin/merge.c\nindex c70d39d..d05c528 100644\n--- a/builtin/merge.c\n+++ b/builtin/merge.c\n@@ -57,6 +57,7 @@ static size_t xopts_nr, xopts_alloc;\n static const char *branch;\n static int verbosity;\n static int allow_rerere_auto;\n+static unsigned char narrow_base[20];\n \n static struct strategy all_strategy[] = {\n \t{ \"recursive\", DEFAULT_TWOHEAD | NO_TRIVIAL },\n@@ -780,7 +781,8 @@ static int merge_trivial(void)\n \tparent->next = xmalloc(sizeof(*parent->next));\n \tparent->next->item = remoteheads->item;\n \tparent->next->next = NULL;\n-\tcommit_tree(merge_msg.buf, result_tree, parent, result_commit, NULL);\n+\tcommit_narrow_tree(merge_msg.buf, result_tree, narrow_base,\n+\t\t\t parent, result_commit, NULL);\n \tfinish(result_commit, \"In-index merge\");\n \tdrop_save();\n \treturn 0;\n@@ -809,7 +811,8 @@ static int finish_automerge(struct commit_list *common,\n \t}\n \tfree_commit_list(remoteheads);\n \tstrbuf_addch(&merge_msg, '\\n');\n-\tcommit_tree(merge_msg.buf, result_tree, parents, result_commit, NULL);\n+\tcommit_narrow_tree(merge_msg.buf, result_tree, narrow_base,\n+\t\t\t parents, result_commit, NULL);\n \tstrbuf_addf(&buf, \"Merge made by %s.\", wt_strategy);\n \tfinish(result_commit, buf.buf);\n \tstrbuf_release(&buf);\n@@ -1064,6 +1067,7 @@ int cmd_merge(int argc, const char **argv, const char *prefix)\n \t\t}\n \t\tif (item)\n \t\t\tdie(\"Different narrow base, couldn't do merge (yet)\");\n+\t\thashcpy(narrow_base, lookup_commit(head)->tree->object.sha1);\n \t}\n \n \tif (!remoteheads->next)\n-- \n1.7.1.rc1.69.g24c2f7\n"},{"id":"148895","messageId":"1282688422-7738-27-git-send-email-pclouds@gmail.com","threadId":"24851","inReplyTo":"1282688422-7738-1-git-send-email-pclouds@gmail.com","subject":"[PATCH 26/32] Add upload-narrow-base command","fromName":"Nguyễn Thái Ngọc Duy","fromEmail":"pclouds@gmail.com","sentAt":"2010-08-24T22:20:16Z","receivedAt":"2010-08-24T22:20:16Z","isPatch":true,"sender":{"key":"pclouds@gmail.com","avatar":"https://avatars.githubusercontent.com/u/720?v=4"},"body":"A narrow repo only has enough trees within a given subtree. That would\nmean even a trivial three-way merge is impossible because it needs\nfull tree walk.\n\nOn the other hand, doing merge remotely is non-sense because conflicts\ncan happen, and there would be no index on remote side for people to\ninspect/resolve conflicts. Such a merge in narrow repo would be\nsplitted into two phases: 3-way trivial merge for everything outside\nnarrow tree, and real merge within narrow tree.\n\nThe first phase is done in server by this command. Given two commits,\nit will do 3-way merge and return result trees of the merge (no blob\nshould be created because this is trivial merge). No new objects will\nbe stored in server after this operation.\n\nFor local repo, parent trees of narrow tree should be enough for\njoin_narrow_tree() to do its job. But because remote side does not\nstore any result objects, in order to push a merge, we would need to\nsend all necessary trees that the remote side does not have.\n\nFor this reason, upload-narrow-base would return a pack of all new\ntrees (those that are not in either merge parents). It is expected\nthat git client would push all these trees back to server when it does\na push.\n\nSigned-off-by: Nguyễn Thái Ngọc Duy \n---\n Makefile | 1 +\n builtin.h | 1 +\n builtin/upload-narrow-base.c | 215 ++++++++++++++++++++++++++++++++++++++++++\n git.c | 1 +\n 4 files changed, 218 insertions(+), 0 deletions(-)\n create mode 100644 builtin/upload-narrow-base.c\n\ndiff --git a/Makefile b/Makefile\nindex 54c435e..7b33a0e 100644\n--- a/Makefile\n+++ b/Makefile\n@@ -737,6 +737,7 @@ BUILTIN_OBJS += builtin/update-index.o\n BUILTIN_OBJS += builtin/update-ref.o\n BUILTIN_OBJS += builtin/update-server-info.o\n BUILTIN_OBJS += builtin/upload-archive.o\n+BUILTIN_OBJS += builtin/upload-narrow-base.o\n BUILTIN_OBJS += builtin/var.o\n BUILTIN_OBJS += builtin/verify-pack.o\n BUILTIN_OBJS += builtin/verify-tag.o\ndiff --git a/builtin.h b/builtin.h\nindex ed6ee26..0383328 100644\n--- a/builtin.h\n+++ b/builtin.h\n@@ -131,6 +131,7 @@ extern int cmd_update_index(int argc, const char **argv, const char *prefix);\n extern int cmd_update_ref(int argc, const char **argv, const char *prefix);\n extern int cmd_update_server_info(int argc, const char **argv, const char *prefix);\n extern int cmd_upload_archive(int argc, const char **argv, const char *prefix);\n+extern int cmd_upload_narrow_base(int argc, const char **argv, const char *prefix);\n extern int cmd_upload_tar(int argc, const char **argv, const char *prefix);\n extern int cmd_var(int argc, const char **argv, const char *prefix);\n extern int cmd_verify_tag(int argc, const char **argv, const char *prefix);\ndiff --git a/builtin/upload-narrow-base.c b/builtin/upload-narrow-base.c\nnew file mode 100644\nindex 0000000..f31f03e\n--- /dev/null\n+++ b/builtin/upload-narrow-base.c\n@@ -0,0 +1,215 @@\n+#include \"cache.h\"\n+#include \"builtin.h\"\n+#include \"cache-tree.h\"\n+#include \"diff.h\"\n+#include \"diffcore.h\"\n+#include \"commit.h\"\n+#include \"revision.h\"\n+#include \"unpack-trees.h\"\n+#include \"pkt-line.h\"\n+#include \"pack.h\"\n+#include \"transport.h\"\n+#include \"sideband.h\"\n+\n+const char *narrow_prefix;\n+\n+#define PARENT1\t\t(1u<<16)\n+#define PARENT2\t\t(1u<<17)\n+\n+static void write_object(struct tree *t,\n+\t\t\t const char *base,\n+\t\t\t struct simple_pack *pack)\n+{\n+\tstruct name_entry entry;\n+\tstruct tree *subtree;\n+\tstruct tree_desc desc;\n+\tenum object_type type;\n+\tunsigned long size;\n+\tint len = 0;\n+\tvoid *buffer;\n+\tchar *path = xmalloc(PATH_MAX);\n+\n+\tif (base) {\n+\t\tlen = strlen(base);\n+\t\tmemcpy(path, base, len);\n+\t\tpath[len++] = '/';\n+\t}\n+\n+\tbuffer = read_sha1_file(t->object.sha1, &type, &size);\n+\tif (!buffer || type != OBJ_TREE)\n+\t\tdie(\"%s is not a tree\", sha1_to_hex(t->object.sha1));\n+\n+\tinit_tree_desc(&desc, buffer, size);\n+\twhile (tree_entry(&desc, &entry)) {\n+\t\tif (!S_ISDIR(entry.mode))\n+\t\t\tcontinue;\n+\n+\t\tsubtree = lookup_tree(entry.sha1);\n+\t\tstrcpy(path+len, entry.path);\n+\t\tif (!strcmp(path, narrow_prefix))\n+\t\t\t;\n+\t\telse if (!prefixcmp(narrow_prefix, path) && /* subtree predecessor */\n+\t\t\t narrow_prefix[strlen(path)] == '/')\n+\t\t\t;\n+\t\telse if (!(subtree->object.flags & (PARENT1 | PARENT2))) {\n+\t\t\tsubtree->object.flags |= TMP_MARK; /* non recursive */\n+\t\t\twrite_object_to_pack(pack, entry.sha1, subtree->buffer,\n+\t\t\t\t\t subtree->size, OBJ_TREE);\n+\t\t}\n+\n+\t\twrite_object(subtree, path, pack);\n+\t}\n+\tfree(path);\n+\tfree(buffer);\n+}\n+\n+static int read_tree_trivial(struct tree *trees[3])\n+{\n+\tint i;\n+\tstruct tree_desc t[3];\n+\tstruct unpack_trees_options opts;\n+\n+\tmemset(&opts, 0, sizeof(opts));\n+\topts.head_idx = 2;\n+\topts.src_index = &the_index;\n+\topts.dst_index = &the_index;\n+\topts.merge = 1;\n+\topts.fn = threeway_merge;\n+\tcache_tree_free(&active_cache_tree);\n+\tfor (i = 0; i < 3; i++) {\n+\t\tparse_tree(trees[i]);\n+\t\tinit_tree_desc(t+i, trees[i]->buffer, trees[i]->size);\n+\t}\n+\tif (unpack_trees(3, t, &opts))\n+\t\treturn -1;\n+\treturn 0;\n+}\n+\n+static int create_tree_object(const void *buf, unsigned long len,\n+\t\t\t const char *type, unsigned char *sha1)\n+{\n+\tstruct tree *t;\n+\tif (type != tree_type)\n+\t\tdie(\"Invalid type %s\", type);\n+\thash_sha1_file(buf, len, tree_type, sha1);\n+\tt = lookup_tree(sha1);\n+\tif (!t->object.parsed) {\n+\t\tt->buffer = xmalloc(len);\n+\t\tmemcpy(t->buffer, buf, len);\n+\t\tt->size = len;\n+\t\tt->object.parsed = 1;\n+\t}\n+\treturn 0;\n+}\n+\n+int cmd_upload_narrow_base(int argc, const char **argv, const char *prefix)\n+{\n+\tchar buf[LARGE_PACKET_MAX];\n+\tunsigned char sha1[2][20];\n+\tstruct tree *t[3];\n+\tstruct commit_list *common;\n+\tstruct simple_pack pack;\n+\tint i, len;\n+\n+\tstrcpy(buf, argv[1]); /* enter-repo smudges its argument */\n+\n+\tif (!enter_repo(buf, 0))\n+\t\tdie(\"'%s' does not appear to be a git repository\", buf);\n+\n+\tlen = packet_read_line(0, buf, sizeof(buf));\n+\tif (!len)\n+\t\tdie(\"narrow-merge: narrow-tree missing\");\n+\tif (!prefixcmp(buf, \"narrow-tree \")) {\n+\t\tnarrow_prefix = xstrdup(buf+12);\n+\t\tlen = strlen(narrow_prefix);\n+\t\tif (!len)\n+\t\t\tdie(\"narrow-merge: empty narrow-tree\");\n+\t\tif (narrow_prefix[len-1] == '/')\n+\t\t\tdie(\"narrow-merge: trailing slash not allowed %s\", narrow_prefix);\n+\t}\n+\telse\n+\t\tdie(\"narrow-merge: invalid narrow-tree %s\", buf);\n+\n+\tfor (i = 0; i < 2; i++) {\n+\t\tlen = packet_read_line(0, buf, sizeof(buf));\n+\t\tif (!len)\n+\t\t\tdie(\"narrow-merge: parent missing\");\n+\t\tif (!prefixcmp(buf, \"parent \")) {\n+\t\t\tif (get_sha1_hex(buf+7, sha1[i]))\n+\t\t\t\tdie(\"narrow-merge: invalid SHA1 %s\", buf);\n+\t\t\tif (!lookup_commit_reference(sha1[i]))\n+\t\t\t\treturn 1;\n+\t\t}\n+\t\telse\n+\t\t\tdie(\"narrow-merge: invalid parent %s\", buf);\n+\t}\n+\tlen = packet_read_line(0, buf, sizeof(buf));\n+\tif (len)\n+\t\tdie(\"narrow-merge: expected a flush\");\n+\n+\tcommon = get_merge_bases(lookup_commit(sha1[0]),\n+\t\t\t\t lookup_commit(sha1[1]),\n+\t\t\t\t 1);\n+\tif (!common || common->next)\n+\t\tdie(\"narrow-merge: no common or too many common found, can't merge\");\n+\n+\tt[0] = common->item->tree;\n+\tt[1] = lookup_commit(sha1[0])->tree;\n+\tt[2] = lookup_commit(sha1[1])->tree;\n+\n+\t/* Three way merge to index */\n+\tdiscard_index(&the_index);\n+\tif (read_tree_trivial(t))\n+\t\tdie(\"narrow-merge: non trivial merge\");\n+\n+\t/* Make sure merge is good (no conflicts outside subtree) */\n+\tlen = strlen(narrow_prefix);\n+\tfor (i = 0; i < the_index.cache_nr; i++) {\n+\t\tstruct cache_entry *ce = the_index.cache[i];\n+\n+\t\t/*\n+\t\t * No staged entries within subtree, prefer stage 1:\n+\t\t * (the goal is no staged entries, the content is\n+\t\t * not really important as subtree client must do\n+\t\t * a real merge within subtree)\n+\t\t * stage 0: move on\n+\t\t * stage 1: turn it to stage 0,\n+\t\t * stage 2: mark CE_REMOVE\n+\t\t */\n+\t\tif (!prefixcmp(ce->name, narrow_prefix) &&\n+\t\t ce->name[len] == '/') {\n+\t\t\tswitch (ce_stage(ce)) {\n+\t\t\tcase 1: ce->ce_flags &= ~CE_STAGEMASK; break;\n+\t\t\tcase 2: ce->ce_flags |= CE_REMOVE; break;\n+\t\t\t}\n+\t\t\tcontinue;\n+\t\t}\n+\t\tif (ce_stage(ce))\n+\t\t\tdie(\"git upload-pack: unmerged entry %s\", ce->name);\n+\t}\n+\n+\t/* Generate trees in memory */\n+\tthe_index.cache_tree = cache_tree();\n+\tif (cache_tree_update_fn(the_index.cache_tree,\n+\t\t\t\t the_index.cache,\n+\t\t\t\t the_index.cache_nr,\n+\t\t\t\t create_tree_object,\n+\t\t\t\t 0, 0))\n+\t\tdie(\"narrow-merge: error generating trees\");\n+\n+\t/* Everything is OK. Send root SHA-1 */\n+\tpacket_write(1, \"ACK %s\\n\", sha1_to_hex(the_index.cache_tree->sha1));\n+\n+\t/* Traverse and send only missing trees */\n+\tmemset(&pack, 0, sizeof(pack));\n+\tpack.fd = 1;\n+\tcreate_pack(&pack);\n+\n+\tt[0] = lookup_tree(the_index.cache_tree->sha1);\n+\tset_tree_marks(t[1], PARENT1);\n+\tset_tree_marks(t[2], PARENT2);\n+\twrite_object(t[0], NULL, &pack);\n+\n+\tclose_pack(&pack);\n+\treturn 0;\n+}\ndiff --git a/git.c b/git.c\nindex f37028b..edb379f 100644\n--- a/git.c\n+++ b/git.c\n@@ -392,6 +392,7 @@ static void handle_internal_command(int argc, const char **argv)\n \t\t{ \"update-ref\", cmd_update_ref, RUN_SETUP },\n \t\t{ \"update-server-info\", cmd_update_server_info, RUN_SETUP },\n \t\t{ \"upload-archive\", cmd_upload_archive },\n+\t\t{ \"upload-narrow-base\", cmd_upload_narrow_base },\n \t\t{ \"var\", cmd_var },\n \t\t{ \"verify-tag\", cmd_verify_tag, RUN_SETUP },\n \t\t{ \"version\", cmd_version },\n-- \n1.7.1.rc1.69.g24c2f7\n"},{"id":"148890","messageId":"1282688422-7738-28-git-send-email-pclouds@gmail.com","threadId":"24851","inReplyTo":"1282688422-7738-1-git-send-email-pclouds@gmail.com","subject":"[PATCH 27/32] rev-list: traverse some more trees to make upload-narrow-base happy","fromName":"Nguyễn Thái Ngọc Duy","fromEmail":"pclouds@gmail.com","sentAt":"2010-08-24T22:20:17Z","receivedAt":"2010-08-24T22:20:17Z","isPatch":true,"sender":{"key":"pclouds@gmail.com","avatar":"https://avatars.githubusercontent.com/u/720?v=4"},"body":"When in narrow repo and a merge is met, just traverse every valid\ntrees that could be found, regardless whether they are in narrow tree.\n\nThis is a weak point because fsck won't be able to find out missing\ntrees. Hopefully the remote end won't accept incomplete push.\n\nSigned-off-by: Nguyễn Thái Ngọc Duy \n---\n list-objects.c | 61 +++++++++++++++++++++++++++++++++++++++++++++++++++++--\n 1 files changed, 58 insertions(+), 3 deletions(-)\n\ndiff --git a/list-objects.c b/list-objects.c\nindex 7014926..ccdd64b 100644\n--- a/list-objects.c\n+++ b/list-objects.c\n@@ -62,7 +62,8 @@ static void process_tree(struct rev_info *revs,\n \t\t\t show_object_fn show,\n \t\t\t struct name_path *path,\n \t\t\t const char *name,\n-\t\t\t const char *subtree)\n+\t\t\t const char *subtree,\n+\t\t\t int recursive)\n {\n \tstruct object *obj = &tree->object;\n \tstruct tree_desc desc;\n@@ -81,6 +82,10 @@ static void process_tree(struct rev_info *revs,\n \t\tdie(\"bad tree object %s\", sha1_to_hex(obj->sha1));\n \tobj->flags |= SEEN;\n \tshow(obj, path, name);\n+\n+\tif (!recursive)\n+\t\treturn;\n+\n \tme.up = path;\n \tme.elem = name;\n \tme.elem_len = strlen(name);\n@@ -112,7 +117,8 @@ static void process_tree(struct rev_info *revs,\n \t\t\tprocess_tree(revs,\n \t\t\t\t lookup_tree(entry.sha1),\n \t\t\t\t show, &me, entry.path,\n-\t\t\t\t slash ? slash+1 : NULL);\n+\t\t\t\t slash ? slash+1 : NULL,\n+\t\t\t\t 1);\n \t\telse if (S_ISGITLINK(entry.mode))\n \t\t\tprocess_gitlink(revs, entry.sha1,\n \t\t\t\t\tshow, &me, entry.path);\n@@ -163,6 +169,48 @@ static void add_pending_tree(struct rev_info *revs, struct tree *tree)\n \tadd_pending_object(revs, &tree->object, \"\");\n }\n \n+static void add_pending_reachable_tree(struct rev_info *revs,\n+\t\t\t\t const unsigned char *sha1,\n+\t\t\t\t const char *base)\n+{\n+\tstruct name_entry entry;\n+\tstruct object *subtree;\n+\tstruct tree_desc desc;\n+\tenum object_type type;\n+\tunsigned long size;\n+\tvoid *buffer;\n+\tchar *path = xmalloc(PATH_MAX);\n+\tint len;\n+\n+\tbuffer = read_sha1_file(sha1, &type, &size);\n+\tif (!buffer) {\n+\t\tfree(path);\n+\t\treturn;\n+\t}\n+\tif (type != OBJ_TREE)\n+\t\tdie(\"%s is not a tree\", sha1_to_hex(sha1));\n+\n+\tlen = strlen(base);\n+\tmemcpy(path, base, len);\n+\tpath[len++] = '/';\n+\n+\tinit_tree_desc(&desc, buffer, size);\n+\twhile (tree_entry(&desc, &entry)) {\n+\t\tif (!S_ISDIR(entry.mode))\n+\t\t\tcontinue;\n+\n+\t\tsubtree = lookup_object(entry.sha1);\n+\t\tstrcpy(path+len, entry.path);\n+\t\tif (subtree) {\n+\t\t\tsubtree->flags |= TMP_MARK; /* non recursive */\n+\t\t\tadd_pending_object(revs, subtree, xstrdup(path));\n+\t\t}\n+\t\tadd_pending_reachable_tree(revs, entry.sha1, path);\n+\t}\n+\tfree(path);\n+\tfree(buffer);\n+}\n+\n void traverse_commit_list(struct rev_info *revs,\n \t\t\t show_commit_fn show_commit,\n \t\t\t show_object_fn show_object,\n@@ -177,6 +225,8 @@ void traverse_commit_list(struct rev_info *revs,\n \t}\n \n \twhile ((commit = get_revision(revs)) != NULL) {\n+\t\tif (get_narrow_prefix())\n+\t\t\tadd_pending_reachable_tree(revs, commit->tree->object.sha1, \"\");\n \t\tadd_pending_tree(revs, commit->tree);\n \t\tshow_commit(commit, data);\n \t}\n@@ -194,7 +244,12 @@ void traverse_commit_list(struct rev_info *revs,\n \t\tif (obj->type == OBJ_TREE) {\n \t\t\tprocess_tree(revs, (struct tree *)obj, show_object,\n \t\t\t\t NULL, name,\n-\t\t\t\t revs->narrow_tree ? revs->narrow_prefix : NULL);\n+\t\t\t\t revs->narrow_tree ? revs->narrow_prefix : NULL,\n+\t\t\t\t (obj->flags & TMP_MARK) == 0);\n+\t\t\tif (obj->flags & TMP_MARK) {\n+\t\t\t\tobj->flags &= ~TMP_MARK;\n+\t\t\t\t//free((char*)name);\n+\t\t\t}\n \t\t\tcontinue;\n \t\t}\n \t\tif (obj->type == OBJ_BLOB) {\n-- \n1.7.1.rc1.69.g24c2f7\n"},{"id":"148899","messageId":"1282688422-7738-29-git-send-email-pclouds@gmail.com","threadId":"24851","inReplyTo":"1282688422-7738-1-git-send-email-pclouds@gmail.com","subject":"[PATCH 28/32] narrow-tree: add oldest_narrow_base()","fromName":"Nguyễn Thái Ngọc Duy","fromEmail":"pclouds@gmail.com","sentAt":"2010-08-24T22:20:18Z","receivedAt":"2010-08-24T22:20:18Z","isPatch":true,"sender":{"key":"pclouds@gmail.com","avatar":"https://avatars.githubusercontent.com/u/720?v=4"},"body":"This function helps solve another problem at client. Because\nupload-narrow-base receives commits. What if they don't have those\ncommits because they are created locally?\n\nThe solution is to find the oldest commits that still have the same\nnarrow base (trees outside narrow tree) and use those commits with\nhope that the server side also have them.\n\nSigned-off-by: Nguyễn Thái Ngọc Duy \n---\n narrow-tree.c | 48 ++++++++++++++++++++++++++++++++++++++++++++++++\n narrow-tree.h | 1 +\n 2 files changed, 49 insertions(+), 0 deletions(-)\n\ndiff --git a/narrow-tree.c b/narrow-tree.c\nindex 4a16647..f1b8902 100644\n--- a/narrow-tree.c\n+++ b/narrow-tree.c\n@@ -150,3 +150,51 @@ int same_narrow_base(const unsigned char *t1, const unsigned char *t2, const cha\n \tfree(buf2);\n \treturn !desc1.size && !desc2.size;\n }\n+\n+int oldest_narrow_base(const unsigned char *sha1, unsigned char *newsha1)\n+{\n+\tenum object_type type;\n+\tunsigned long size;\n+\tchar *buf, *bufptr;\n+\tunsigned char base[20], newbase[20], parent[20];\n+\n+\tbuf = read_sha1_file(sha1, &type, &size);\n+\tif (!buf || type != OBJ_COMMIT) {\n+\t\tif (buf)\n+\t\t\tfree(buf);\n+\t\terror(\"Not a commit %s\", sha1_to_hex(sha1));\n+\t\treturn 1;\n+\t}\n+\tget_sha1_hex(buf+5, base);\n+\thashcpy(newsha1, sha1);\n+\n+\twhile (1) {\n+\t\tbufptr = buf + 46;\t/* \"tree \" + \"hex sha1\" + \"\\n\" */\n+\t\tif (!memcmp(bufptr, \"parent \", 7)) {\n+\n+\t\t\t/* more than one parent, a merge */\n+\t\t\tif (!memcmp(bufptr + 48, \"parent \", 7))\n+\t\t\t\tbreak;\n+\n+\t\t\tfree(buf);\n+\t\t\tget_sha1_hex(bufptr + 7, parent);\n+\t\t\tbuf = read_sha1_file(parent, &type, &size);\n+\t\t\tif (!buf ||type != OBJ_COMMIT) {\n+\t\t\t\tif (buf)\n+\t\t\t\t\tfree(buf);\n+\t\t\t\terror(\"Not a commit %s\", sha1_to_hex(sha1));\n+\t\t\t\treturn 1;\n+\t\t\t}\n+\t\t\tget_sha1_hex(buf+5, newbase);\n+\t\t\tif (!same_narrow_base(base, newbase, get_narrow_prefix()))\n+\t\t\t\tbreak;\n+\n+\t\t\thashcpy(newsha1, parent);\n+\t\t\t/* keep searching */\n+\t\t}\n+\t\telse\t\t/* root commit */\n+\t\t\tbreak;\n+\t}\n+\tfree(buf);\n+\treturn 0;\n+}\ndiff --git a/narrow-tree.h b/narrow-tree.h\nindex 8756094..78d6f39 100644\n--- a/narrow-tree.h\n+++ b/narrow-tree.h\n@@ -2,3 +2,4 @@ extern int check_narrow_prefix();\n extern int join_narrow_tree(const unsigned char *base, unsigned char *newsha1,\n \t\t\t const unsigned char *subtree_sha1, const char *prefix);\n int same_narrow_base(const unsigned char *t1, const unsigned char *t2, const char *prefix);\n+int oldest_narrow_base(const unsigned char *sha1, unsigned char *newsha1);\n-- \n1.7.1.rc1.69.g24c2f7\n"},{"id":"148896","messageId":"1282688422-7738-30-git-send-email-pclouds@gmail.com","threadId":"24851","inReplyTo":"1282688422-7738-1-git-send-email-pclouds@gmail.com","subject":"[PATCH 29/32] Add command fetch-narrow-base","fromName":"Nguyễn Thái Ngọc Duy","fromEmail":"pclouds@gmail.com","sentAt":"2010-08-24T22:20:19Z","receivedAt":"2010-08-24T22:20:19Z","isPatch":true,"sender":{"key":"pclouds@gmail.com","avatar":"https://avatars.githubusercontent.com/u/720?v=4"},"body":"This is the client side of upload-narrow-base. It sends two commits, a\nnarrow prefix, receives a pack and unpacks it.\n\nSigned-off-by: Nguyễn Thái Ngọc Duy \n---\n Makefile | 1 +\n builtin.h | 1 +\n builtin/fetch-narrow-base.c | 89 +++++++++++++++++++++++++++++++++++++++++++\n git.c | 1 +\n narrow-tree.h | 4 ++\n 5 files changed, 96 insertions(+), 0 deletions(-)\n create mode 100644 builtin/fetch-narrow-base.c\n\ndiff --git a/Makefile b/Makefile\nindex 7b33a0e..a10eb00 100644\n--- a/Makefile\n+++ b/Makefile\n@@ -676,6 +676,7 @@ BUILTIN_OBJS += builtin/diff-index.o\n BUILTIN_OBJS += builtin/diff-tree.o\n BUILTIN_OBJS += builtin/diff.o\n BUILTIN_OBJS += builtin/fast-export.o\n+BUILTIN_OBJS += builtin/fetch-narrow-base.o\n BUILTIN_OBJS += builtin/fetch-pack.o\n BUILTIN_OBJS += builtin/fetch.o\n BUILTIN_OBJS += builtin/fmt-merge-msg.o\ndiff --git a/builtin.h b/builtin.h\nindex 0383328..9eefa0f 100644\n--- a/builtin.h\n+++ b/builtin.h\n@@ -66,6 +66,7 @@ extern int cmd_diff(int argc, const char **argv, const char *prefix);\n extern int cmd_diff_tree(int argc, const char **argv, const char *prefix);\n extern int cmd_fast_export(int argc, const char **argv, const char *prefix);\n extern int cmd_fetch(int argc, const char **argv, const char *prefix);\n+extern int cmd_fetch_narrow_base(int argc, const char **argv, const char *prefix);\n extern int cmd_fetch_pack(int argc, const char **argv, const char *prefix);\n extern int cmd_fmt_merge_msg(int argc, const char **argv, const char *prefix);\n extern int cmd_for_each_ref(int argc, const char **argv, const char *prefix);\ndiff --git a/builtin/fetch-narrow-base.c b/builtin/fetch-narrow-base.c\nnew file mode 100644\nindex 0000000..0162ad0\n--- /dev/null\n+++ b/builtin/fetch-narrow-base.c\n@@ -0,0 +1,89 @@\n+#include \"cache.h\"\n+#include \"builtin.h\"\n+#include \"commit.h\"\n+#include \"pkt-line.h\"\n+#include \"transport.h\"\n+#include \"sideband.h\"\n+#include \"run-command.h\"\n+#include \"parse-options.h\"\n+\n+int fetch_narrow_base(const unsigned char *c1,\n+\t\t const unsigned char *c2,\n+\t\t unsigned char *newsha1,\n+\t\t const char *narrow_prefix,\n+\t\t const char *remote,\n+\t\t const char *exec)\n+{\n+\tchar buf[LARGE_PACKET_MAX];\n+\tint fd[2], len;\n+\tstruct transport *transport;\n+\tstruct remote *_remote;\n+\tconst char *argv[] = { \"unpack-objects\", \"-q\", NULL };\n+\tstruct child_process cmd;\n+\n+\t_remote = remote_get(remote);\n+\tif (!_remote->url[0])\n+\t\tdie(\"narrow-base: Remote with no URL\");\n+\ttransport = transport_get(_remote, _remote->url[0]);\n+\tif (!exec)\n+\t\texec = \"git-upload-narrow-base\";\n+\ttransport_connect(transport, \"git-upload-narrow-base\", exec, fd);\n+\n+\tpacket_write(fd[1], \"narrow-tree %s\\n\", narrow_prefix);\n+\tpacket_write(fd[1], \"parent %s\\n\", sha1_to_hex(c1));\n+\tpacket_write(fd[1], \"parent %s\\n\", sha1_to_hex(c2));\n+\tpacket_flush(fd[1]);\n+\n+\tlen = packet_read_line(fd[0], buf, sizeof(buf));\n+\tif (!len)\n+\t\tdie(\"narrow-base: expected ACK/NAK, got EOF\");\n+\tif (buf[len-1] == '\\n')\n+\t\tbuf[--len] = 0;\n+\tif (prefixcmp(buf, \"ACK \"))\n+\t\tdie(\"narrow-base: protocol error\");\n+\n+\tget_sha1_hex(buf+4, newsha1);\n+\n+\tmemset(&cmd, 0, sizeof(cmd));\n+\tcmd.argv = argv;\n+\tcmd.in = fd[0];\n+\tcmd.git_cmd = 1;\n+\n+\tif (run_command(&cmd))\n+\t\tdie(\"unable to run %s\", argv[0]);\n+\n+\treturn transport_disconnect(transport);\n+}\n+\n+int cmd_fetch_narrow_base(int argc, const char **argv, const char *prefix)\n+{\n+\tunsigned char c1[20], c2[20], sha1[20];\n+\tconst char *remote;\n+\tconst char *exec;\n+\tconst char *narrow_prefix = get_narrow_prefix();\n+\tstruct option options[] = {\n+\t\tOPT_STRING(0, \"remote\", &remote, \"remote\", \"remote host\"),\n+\t\tOPT_STRING(0, \"upload-narrow-base\", &exec, \"path\", \"path to upload-narrow-base\"),\n+\t\tOPT_STRING(0, \"narrow-tree\", &narrow_prefix, \"narrow prefix\", \"narrow prefix\"),\n+\t\tOPT_END()\n+\t};\n+\tconst char *usage[] = { \"git narrow-base [options] parent1 parent2\", NULL };\n+\n+\targc = parse_options(argc, argv, prefix, options, usage, 0);\n+\n+\tif (argc != 2)\n+\t\tdie(\"Insufficient arguments\");\n+\n+\tif (get_sha1(argv[0], c1) ||\n+\t get_sha1(argv[1], c2) ||\n+\t !lookup_commit_reference(c1) ||\n+\t !lookup_commit_reference(c2))\n+\t\treturn 1;\n+\n+\tif (!fetch_narrow_base(c1, c2, sha1, narrow_prefix, remote, exec)) {\n+\t\tprintf(\"%s\\n\", sha1_to_hex(sha1));\n+\t\treturn 0;\n+\t}\n+\n+\treturn 1;\n+}\ndiff --git a/git.c b/git.c\nindex edb379f..f4370b1 100644\n--- a/git.c\n+++ b/git.c\n@@ -322,6 +322,7 @@ static void handle_internal_command(int argc, const char **argv)\n \t\t{ \"diff-tree\", cmd_diff_tree, RUN_SETUP },\n \t\t{ \"fast-export\", cmd_fast_export, RUN_SETUP },\n \t\t{ \"fetch\", cmd_fetch, RUN_SETUP },\n+\t\t{ \"fetch-narrow-base\", cmd_fetch_narrow_base, RUN_SETUP },\n \t\t{ \"fetch-pack\", cmd_fetch_pack, RUN_SETUP },\n \t\t{ \"fmt-merge-msg\", cmd_fmt_merge_msg, RUN_SETUP },\n \t\t{ \"for-each-ref\", cmd_for_each_ref, RUN_SETUP },\ndiff --git a/narrow-tree.h b/narrow-tree.h\nindex 78d6f39..614d677 100644\n--- a/narrow-tree.h\n+++ b/narrow-tree.h\n@@ -3,3 +3,7 @@ extern int join_narrow_tree(const unsigned char *base, unsigned char *newsha1,\n \t\t\t const unsigned char *subtree_sha1, const char *prefix);\n int same_narrow_base(const unsigned char *t1, const unsigned char *t2, const char *prefix);\n int oldest_narrow_base(const unsigned char *sha1, unsigned char *newsha1);\n+int fetch_narrow_base(const unsigned char *c1, const unsigned char *c2,\n+\t\t unsigned char *newsha1,\n+\t\t const char *narrow_prefix,\n+\t\t const char *remote, const char *exec);\n-- \n1.7.1.rc1.69.g24c2f7\n"},{"id":"148898","messageId":"1282688422-7738-31-git-send-email-pclouds@gmail.com","threadId":"24851","inReplyTo":"1282688422-7738-1-git-send-email-pclouds@gmail.com","subject":"[PATCH 30/32] merge: support merging when narrow bases are different","fromName":"Nguyễn Thái Ngọc Duy","fromEmail":"pclouds@gmail.com","sentAt":"2010-08-24T22:20:20Z","receivedAt":"2010-08-24T22:20:20Z","isPatch":true,"sender":{"key":"pclouds@gmail.com","avatar":"https://avatars.githubusercontent.com/u/720?v=4"},"body":"\nSigned-off-by: Nguyễn Thái Ngọc Duy \n---\n builtin/merge.c | 25 ++++++++++++++++++++++---\n 1 files changed, 22 insertions(+), 3 deletions(-)\n\ndiff --git a/builtin/merge.c b/builtin/merge.c\nindex d05c528..1c70193 100644\n--- a/builtin/merge.c\n+++ b/builtin/merge.c\n@@ -58,6 +58,7 @@ static const char *branch;\n static int verbosity;\n static int allow_rerere_auto;\n static unsigned char narrow_base[20];\n+static const char *narrow_remote, *narrow_exec;\n \n static struct strategy all_strategy[] = {\n \t{ \"recursive\", DEFAULT_TWOHEAD | NO_TRIVIAL },\n@@ -195,6 +196,10 @@ static struct option builtin_merge_options[] = {\n \tOPT_CALLBACK('m', \"message\", &merge_msg, \"message\",\n \t\t\"message to be used for the merge commit (if any)\",\n \t\toption_parse_message),\n+\tOPT_STRING(0, \"remote\", &narrow_remote, \"repo\",\n+\t\t \"remote repo to perform narrow base calculation\"),\n+\tOPT_STRING(0, \"upload-narrow-base\", &narrow_exec, \"path\",\n+\t\t \"path to git-upload-narrow-base\"),\n \tOPT__VERBOSITY(&verbosity),\n \tOPT_END()\n };\n@@ -1065,9 +1070,23 @@ int cmd_merge(int argc, const char **argv, const char *prefix)\n \t\t\t\tbreak;\n \t\t\titem = item->next;\n \t\t}\n-\t\tif (item)\n-\t\t\tdie(\"Different narrow base, couldn't do merge (yet)\");\n-\t\thashcpy(narrow_base, lookup_commit(head)->tree->object.sha1);\n+\t\tif (item) {\n+\t\t\tunsigned char sha1[2][20];\n+\n+\t\t\tif (remoteheads->next)\n+\t\t\t\tdie(\"Octopus merge is not supported in narrow repository\");\n+\t\t\tif (!narrow_remote)\n+\t\t\t\tdie(\"This merge cannot be fully performed locally.\\n\"\n+\t\t\t\t \"Please specify remote repo with --remote.\");\n+\t\t\toldest_narrow_base(head, sha1[0]);\n+\t\t\toldest_narrow_base(remoteheads->item->object.sha1, sha1[1]);\n+\t\t\tif (fetch_narrow_base(sha1[0], sha1[1], narrow_base,\n+\t\t\t\t\t get_narrow_prefix(),\n+\t\t\t\t\t narrow_remote, narrow_exec))\n+\t\t\t\tdie(\"Failed to request narrow base from remote repo\");\n+\t\t}\n+\t\telse\n+\t\t\thashcpy(narrow_base, lookup_commit(head)->tree->object.sha1);\n \t}\n \n \tif (!remoteheads->next)\n-- \n1.7.1.rc1.69.g24c2f7\n"},{"id":"148897","messageId":"1282688422-7738-32-git-send-email-pclouds@gmail.com","threadId":"24851","inReplyTo":"1282688422-7738-1-git-send-email-pclouds@gmail.com","subject":"[PATCH 31/32] send-pack: do not use thin pack in narrow mode","fromName":"Nguyễn Thái Ngọc Duy","fromEmail":"pclouds@gmail.com","sentAt":"2010-08-24T22:20:21Z","receivedAt":"2010-08-24T22:20:21Z","isPatch":true,"sender":{"key":"pclouds@gmail.com","avatar":"https://avatars.githubusercontent.com/u/720?v=4"},"body":"\nSigned-off-by: Nguyễn Thái Ngọc Duy \n---\n builtin/send-pack.c | 4 ++++\n 1 files changed, 4 insertions(+), 0 deletions(-)\n\ndiff --git a/builtin/send-pack.c b/builtin/send-pack.c\nindex 481602d..c2138df 100644\n--- a/builtin/send-pack.c\n+++ b/builtin/send-pack.c\n@@ -457,6 +457,10 @@ int cmd_send_pack(int argc, const char **argv, const char *prefix)\n \t}\n \tif (!dest)\n \t\tusage(send_pack_usage);\n+\n+\tif (get_narrow_prefix())\n+\t\targs.use_thin_pack = 0;\n+\n \t/*\n \t * --all and --mirror are incompatible; neither makes sense\n \t * with any refspecs.\n-- \n1.7.1.rc1.69.g24c2f7\n"},{"id":"148900","messageId":"1282688422-7738-33-git-send-email-pclouds@gmail.com","threadId":"24851","inReplyTo":"1282688422-7738-1-git-send-email-pclouds@gmail.com","subject":"[PATCH 32/32] daemon: support upload-narrow-base","fromName":"Nguyễn Thái Ngọc Duy","fromEmail":"pclouds@gmail.com","sentAt":"2010-08-24T22:20:22Z","receivedAt":"2010-08-24T22:20:22Z","isPatch":true,"sender":{"key":"pclouds@gmail.com","avatar":"https://avatars.githubusercontent.com/u/720?v=4"},"body":"\nSigned-off-by: Nguyễn Thái Ngọc Duy \n---\n daemon.c | 7 +++++++\n 1 files changed, 7 insertions(+), 0 deletions(-)\n\ndiff --git a/daemon.c b/daemon.c\nindex e22a2b7..58794b7 100644\n--- a/daemon.c\n+++ b/daemon.c\n@@ -356,6 +356,12 @@ static int upload_archive(void)\n \treturn run_service_command(argv);\n }\n \n+static int upload_narrow_base(void)\n+{\n+\tstatic const char *argv[] = { \"upload-narrow-base\", \".\", NULL };\n+\treturn run_service_command(argv);\n+}\n+\n static int receive_pack(void)\n {\n \tstatic const char *argv[] = { \"receive-pack\", \".\", NULL };\n@@ -364,6 +370,7 @@ static int receive_pack(void)\n \n static struct daemon_service daemon_service[] = {\n \t{ \"upload-archive\", \"uploadarch\", upload_archive, 0, 1 },\n+\t{ \"upload-narrow-base\", \"narrowbase\", upload_narrow_base, 0, 1 },\n \t{ \"upload-pack\", \"uploadpack\", upload_pack, 1, 1 },\n \t{ \"receive-pack\", \"receivepack\", receive_pack, 0, 1 },\n };\n-- \n1.7.1.rc1.69.g24c2f7\n"},{"id":"148901","messageId":"20100824223741.GB2376@burratino","threadId":"24851","inReplyTo":"1282688422-7738-1-git-send-email-pclouds@gmail.com","subject":"Re: [RFD PATCH 00/32] subtree clone v2","fromName":"Jonathan Nieder","fromEmail":"jrnieder@gmail.com","sentAt":"2010-08-24T22:37:42Z","receivedAt":"2010-08-24T22:37:42Z","isPatch":true,"sender":{"key":"jrnieder@gmail.com","avatar":"https://avatars.githubusercontent.com/u/281595?v=4"},"body":"Nguyễn Thái Ngọc Duy wrote:\n\n> Patches 26..30\n> Add upload-narrow-base command\n> rev-list: traverse some more trees to make upload-narrow-base happy\n> narrow-tree: add oldest_narrow_base()\n> Add command fetch-narrow-base\n> merge: support merging when narrow bases are different\n> \n> Remote merge part.\n> \n> Split a merge operation into two parts, the real merge will be done\n> within narrow tree. Conflicts can happen and be resolved in the narrow\n> index, locally.\n> \n> Everything outside narrow tree will be merged (trivially) by\n> server. Then server sends the base tree back, so join_narrow_tree() in\n> patch 20 can be used to create proper commit.\n> \n> Server can disable this remote merge feature, which means users are\n> forced to do rebase/fast-forward. Not too bad.\n\nYikes. Naïve question (please forgive my laziness): is it possible to\nmerge without remote contact in the boring case, when no changes have\noccured outside the narrow tree?\n"},{"id":"148902","messageId":"20100824224156.GC2376@burratino","threadId":"24851","inReplyTo":"1282688422-7738-3-git-send-email-pclouds@gmail.com","subject":"Re: [PATCH 02/32] cache-tree: abstract out write_sha1_file from cache_tree_update()","fromName":"Jonathan Nieder","fromEmail":"jrnieder@gmail.com","sentAt":"2010-08-24T22:41:56Z","receivedAt":"2010-08-24T22:41:56Z","isPatch":true,"sender":{"key":"jrnieder@gmail.com","avatar":"https://avatars.githubusercontent.com/u/281595?v=4"},"body":"Nguyễn Thái Ngọc Duy wrote:\n\n> This allows cache-tree to produce trees directly in a pack, for instance.\n\nNeat.\n"},{"id":"148903","messageId":"AANLkTikipryFVf_XvvbHopWSo5Ey_mvgTevY2NAvhygQ@mail.gmail.com","threadId":"24851","inReplyTo":"20100824223741.GB2376@burratino","subject":"Re: [RFD PATCH 00/32] subtree clone v2","fromName":"Nguyen Thai Ngoc Duy","fromEmail":"pclouds@gmail.com","sentAt":"2010-08-24T22:47:31Z","receivedAt":"2010-08-24T22:47:31Z","isPatch":true,"sender":{"key":"pclouds@gmail.com","avatar":"https://avatars.githubusercontent.com/u/720?v=4"},"body":"On Wed, Aug 25, 2010 at 8:37 AM, Jonathan Nieder wrote:\n> Nguyễn Thái Ngọc Duy wrote:\n>\n>> Patches 26..30\n>>   Add upload-narrow-base command\n>>   rev-list: traverse some more trees to make upload-narrow-base happy\n>>   narrow-tree: add oldest_narrow_base()\n>>   Add command fetch-narrow-base\n>>   merge: support merging when narrow bases are different\n>>\n>> Remote merge part.\n>>\n>> Split a merge operation into two parts, the real merge will be done\n>> within narrow tree. Conflicts can happen and be resolved in the narrow\n>> index, locally.\n>>\n>> Everything outside narrow tree will be merged (trivially) by\n>> server. Then server sends the base tree back, so join_narrow_tree() in\n>> patch 20 can be used to create proper commit.\n>>\n>> Server can disable this remote merge feature, which means users are\n>> forced to do rebase/fast-forward. Not too bad.\n>\n> Yikes.  Naïve question (please forgive my laziness): is it possible to\n> merge without remote contact in the boring case, when no changes have\n> occured outside the narrow tree?\n\nThat's possible (and is implemented in my series). But I guess as soon\nas you do \"git pull\", the boring case is likely not applicable\nanymore.\n-- \nDuy\n"},{"id":"148904","messageId":"20100824230907.GD2376@burratino","threadId":"24851","inReplyTo":"AANLkTikipryFVf_XvvbHopWSo5Ey_mvgTevY2NAvhygQ@mail.gmail.com","subject":"Re: [RFD PATCH 00/32] subtree clone v2","fromName":"Jonathan Nieder","fromEmail":"jrnieder@gmail.com","sentAt":"2010-08-24T23:09:07Z","receivedAt":"2010-08-24T23:09:07Z","isPatch":true,"sender":{"key":"jrnieder@gmail.com","avatar":"https://avatars.githubusercontent.com/u/281595?v=4"},"body":"Nguyen Thai Ngoc Duy wrote:\n> On Wed, Aug 25, 2010 at 8:37 AM, Jonathan Nieder wrote:\n\n>> is it possible to\n>> merge without remote contact in the boring case, when no changes have\n>> occured outside the narrow tree?\n>\n> That's possible (and is implemented in my series). But I guess as soon\n> as you do \"git pull\", the boring case is likely not applicable\n> anymore.\n\nMakes sense.\n\nOne slightly less boring case. Is it possible to merge when the\nsimplified history, looking only at changes outside the narrow tree,\nwould have permitted a fast-forward?\n\nIf so, a git contributor who is only interested in Documentation/\ncould work on \"next\" between releases and keep up with \"maint\" and\n\"master\" longer term, without the help of a full-tree-merging machine.\n\nMore realistically, a linux-2.6 contributor only interested in one\nsubsystem could always keep up with \"master\". Which would be nice.\n"},{"id":"148905","messageId":"20100824231536.GE2376@burratino","threadId":"24851","inReplyTo":"1282688422-7738-4-git-send-email-pclouds@gmail.com","subject":"Re: [PATCH 03/32] cache-tree: ignore CE_REMOVE entries in verify_cache()","fromName":"Jonathan Nieder","fromEmail":"jrnieder@gmail.com","sentAt":"2010-08-24T23:15:36Z","receivedAt":"2010-08-24T23:15:36Z","isPatch":true,"sender":{"key":"jrnieder@gmail.com","avatar":"https://avatars.githubusercontent.com/u/281595?v=4"},"body":"Nguyễn Thái Ngọc Duy wrote:\n\n> --- a/cache-tree.c\n> +++ b/cache-tree.c\n> @@ -156,6 +156,8 @@ static int verify_cache(struct cache_entry **cache,\n> \tfunny = 0;\n> \tfor (i = 0; i < entries; i++) {\n> \t\tstruct cache_entry *ce = cache[i];\n> +\t\tif (ce->ce_flags & CE_REMOVE)\n> +\t\t\tcontinue;\n> \t\tif (ce_stage(ce) || (ce->ce_flags & CE_INTENT_TO_ADD)) {\n> \t\t\tif (10 < ++funny) {\n> \t\t\t\tfprintf(stderr, \"...\\n\");\n\nIn other words, this teaches internal write-tree callers to ignore\nunmerged and intent-to-add entries marked with CE_REMOVE.\n\nWhy? When does this come up?\n"},{"id":"148906","messageId":"20100824232554.GF2376@burratino","threadId":"24851","inReplyTo":"1282688422-7738-5-git-send-email-pclouds@gmail.com","subject":"Re: [PATCH 04/32] move do_compress() from pack-objects.c to pack-write.c","fromName":"Jonathan Nieder","fromEmail":"jrnieder@gmail.com","sentAt":"2010-08-24T23:25:54Z","receivedAt":"2010-08-24T23:25:54Z","isPatch":true,"sender":{"key":"jrnieder@gmail.com","avatar":"https://avatars.githubusercontent.com/u/281595?v=4"},"body":"Nguyễn Thái Ngọc Duy wrote:\n\n> Subject: [PATCH 04/32] move do_compress() from pack-objects.c to pack-write.c\n\nexposing it as internal API. Makes sense.\n\n> +unsigned long compress_object(void **pptr, unsigned long size)\n> +{\n> +\tz_stream stream;\n> +\tvoid *in, *out;\n> +\tunsigned long maxsize;\n> +\n> +\tmemset(&stream, 0, sizeof(stream));\n> +\tdeflateInit(&stream, Z_DEFAULT_COMPRESSION);\n> +\tmaxsize = deflateBound(&stream, size);\n> +\n> +\tin = *pptr;\n> +\tout = xmalloc(maxsize);\n> +\t*pptr = out;\n> +\n> +\tstream.next_in = in;\n> +\tstream.avail_in = size;\n> +\tstream.next_out = out;\n> +\tstream.avail_out = maxsize;\n> +\twhile (deflate(&stream, Z_FINISH) == Z_OK)\n> +\t\t; /* nothing */\n> +\tdeflateEnd(&stream);\n> +\n> +\treturn stream.total_out;\n> +}\n\nHowever, there is nothing particularly specialized to git objects\nabout this. e.g. deflate_it() from diff.c could use this.\n"},{"id":"148908","messageId":"20100824234301.GG2376@burratino","threadId":"24851","inReplyTo":"1282688422-7738-10-git-send-email-pclouds@gmail.com","subject":"Re: [PATCH 09/32] index: make narrow index incompatible with older git","fromName":"Jonathan Nieder","fromEmail":"jrnieder@gmail.com","sentAt":"2010-08-24T23:43:01Z","receivedAt":"2010-08-24T23:43:01Z","isPatch":true,"sender":{"key":"jrnieder@gmail.com","avatar":"https://avatars.githubusercontent.com/u/281595?v=4"},"body":"Nguyễn Thái Ngọc Duy wrote:\n\n> Index in narrow repos is not a full index and should not be used\n> to create commits without modification (to be explained later on).\n[...]\n> +++ b/read-cache.c\n> @@ -25,8 +25,9 @@ static struct cache_entry *refresh_cache_entry(struct cache_entry *ce, int reall\n> */\n> \n> #define CACHE_EXT(s) ( (s[0]<<24)|(s[1]<<16)|(s[2]<<8)|(s[3]) )\n> -#define CACHE_EXT_TREE 0x54524545\t/* \"TREE\" */\n> +#define CACHE_EXT_TREE 0x54524545\t /* \"TREE\" */\n> #define CACHE_EXT_RESOLVE_UNDO 0x52455543 /* \"REUC\" */\n> +#define CACHE_EXT_NARROW 0x4e415257\t /* \"NARW\" */\n\nJust curious: why aren't we using\n\n #define CACHE_EXT_TREE CACHE_EXT(\"TREE\")\n #define CACHE_EXT_RESOLVE_UNDO CACHE_EXT(\"REUC\")\n #define CACHE_EXT_NARROW CACHE_EXT(\"NARW\")\n\nAre they invalid case labels?\n\nI think non-optional extensions might need to be lowercase (\"narw\")\nthough I am not sure.\n\n> @@ -1168,7 +1169,9 @@ static int verify_hdr(struct cache_header *hdr, unsigned long size)\n> \n> \tif (hdr->hdr_signature != htonl(CACHE_SIGNATURE))\n> \t\treturn error(\"bad signature\");\n> -\tif (hdr->hdr_version != htonl(2) && hdr->hdr_version != htonl(3))\n> +\tif (hdr->hdr_version != htonl(2) &&\n> +\t hdr->hdr_version != htonl(3) &&\n> +\t hdr->hdr_version != htonl(4))\n> \t\treturn error(\"bad index version\");\n[...]\n> @@ -1568,7 +1584,11 @@ int write_index(struct index_state *istate, int newfd)\n[...]\n> +\tif (get_narrow_prefix() && ver < 4)\n> +\t\tver = 4;\t/* narrow-unaware git should to touch this index */\n> +\n> +\thdr.hdr_version = htonl(ver);\n> \thdr.hdr_entries = htonl(entries - removed);\n> \n> \tgit_SHA1_Init(&c);\n\nOh, or you can do this. :)\n\nThanks for a pleasant read. Looks good so far, though I didn't read\nthe tests.\n"},{"id":"148909","messageId":"AANLkTimKEtkjM-sXrqa=LHTKzUb78TZBXA5siDPyfB_h@mail.gmail.com","threadId":"24851","inReplyTo":"20100824230907.GD2376@burratino","subject":"Re: [RFD PATCH 00/32] subtree clone v2","fromName":"Nguyen Thai Ngoc Duy","fromEmail":"pclouds@gmail.com","sentAt":"2010-08-25T00:20:57Z","receivedAt":"2010-08-25T00:20:57Z","isPatch":true,"sender":{"key":"pclouds@gmail.com","avatar":"https://avatars.githubusercontent.com/u/720?v=4"},"body":"On Wed, Aug 25, 2010 at 9:09 AM, Jonathan Nieder wrote:\n> Nguyen Thai Ngoc Duy wrote:\n>> On Wed, Aug 25, 2010 at 8:37 AM, Jonathan Nieder \n>> wrote:\n>\n>>> is it possible to\n>>> merge without remote contact in the boring case, when no changes have\n>>> occured outside the narrow tree?\n>>\n>> That's possible (and is implemented in my series). But I guess as soon\n>> as you do \"git pull\", the boring case is likely not applicable\n>> anymore.\n>\n> Makes sense.\n>\n> One slightly less boring case.  Is it possible to merge when the\n> simplified history, looking only at changes outside the narrow tree,\n> would have permitted a fast-forward?\n\nYes, as long as there is no changes outside narrow tree.\n\n>\n> If so, a git contributor who is only interested in Documentation/\n> could work on \"next\" between releases and keep up with \"maint\" and\n> \"master\" longer term, without the help of a full-tree-merging machine.\n>\n> More realistically, a linux-2.6 contributor only interested in one\n> subsystem could always keep up with \"master\".  Which would be nice.\n>\n\n\n\n-- \nDuy\n"},{"id":"148910","messageId":"AANLkTi=P-PoBWwDbfGs7v1BTwvJ7uwfe7hd7A+Y7M361@mail.gmail.com","threadId":"24851","inReplyTo":"20100824231536.GE2376@burratino","subject":"Re: [PATCH 03/32] cache-tree: ignore CE_REMOVE entries in verify_cache()","fromName":"Nguyen Thai Ngoc Duy","fromEmail":"pclouds@gmail.com","sentAt":"2010-08-25T00:23:44Z","receivedAt":"2010-08-25T00:23:44Z","isPatch":true,"sender":{"key":"pclouds@gmail.com","avatar":"https://avatars.githubusercontent.com/u/720?v=4"},"body":"On Wed, Aug 25, 2010 at 9:15 AM, Jonathan Nieder wrote:\n> Nguyễn Thái Ngọc Duy wrote:\n>\n>> --- a/cache-tree.c\n>> +++ b/cache-tree.c\n>> @@ -156,6 +156,8 @@ static int verify_cache(struct cache_entry **cache,\n>>       funny = 0;\n>>       for (i = 0; i < entries; i++) {\n>>               struct cache_entry *ce = cache[i];\n>> +             if (ce->ce_flags & CE_REMOVE)\n>> +                     continue;\n>>               if (ce_stage(ce) || (ce->ce_flags & CE_INTENT_TO_ADD)) {\n>>                       if (10 < ++funny) {\n>>                               fprintf(stderr, \"...\\n\");\n>\n> In other words, this teaches internal write-tree callers to ignore\n> unmerged and intent-to-add entries marked with CE_REMOVE.\n>\n> Why?  When does this come up?\n\nI was lazy. In patch 26, upload-narrow-merge do three way merge and\ndrop staged entries within narrow tree (this is at server side,\nconflicts within narrow tree will be handled at client side later).\nInstead of removing staged entries, I mark them CE_REMOVE.\n-- \nDuy\n"},{"id":"148911","messageId":"AANLkTinMu_jdT2k=NCX3gwVKeMo63jte_K68PjkFGtQi@mail.gmail.com","threadId":"24851","inReplyTo":"20100824234301.GG2376@burratino","subject":"Re: [PATCH 09/32] index: make narrow index incompatible with older git","fromName":"Nguyen Thai Ngoc Duy","fromEmail":"pclouds@gmail.com","sentAt":"2010-08-25T00:25:37Z","receivedAt":"2010-08-25T00:25:37Z","isPatch":true,"sender":{"key":"pclouds@gmail.com","avatar":"https://avatars.githubusercontent.com/u/720?v=4"},"body":"On Wed, Aug 25, 2010 at 9:43 AM, Jonathan Nieder wrote:\n> Nguyễn Thái Ngọc Duy wrote:\n>\n>> Index in narrow repos is not a full index and should not be used\n>> to create commits without modification (to be explained later on).\n> [...]\n>> +++ b/read-cache.c\n>> @@ -25,8 +25,9 @@ static struct cache_entry *refresh_cache_entry(struct\n>> cache_entry *ce, int reall\n>>   */\n>>\n>>  #define CACHE_EXT(s) ( (s[0]<<24)|(s[1]<<16)|(s[2]<<8)|(s[3]) )\n>> -#define CACHE_EXT_TREE 0x54524545    /* \"TREE\" */\n>> +#define CACHE_EXT_TREE 0x54524545      /* \"TREE\" */\n>>  #define CACHE_EXT_RESOLVE_UNDO 0x52455543 /* \"REUC\" */\n>> +#define CACHE_EXT_NARROW 0x4e415257    /* \"NARW\" */\n>\n> Just curious: why aren't we using\n>\n>  #define CACHE_EXT_TREE CACHE_EXT(\"TREE\")\n>  #define CACHE_EXT_RESOLVE_UNDO CACHE_EXT(\"REUC\")\n>  #define CACHE_EXT_NARROW CACHE_EXT(\"NARW\")\n>\n> Are they invalid case labels?\n>\n> I think non-optional extensions might need to be lowercase (\"narw\")\n> though I am not sure.\n\nNo idea. I need to dig into history to figure it out later.\n-- \nDuy\n"},{"id":"148913","messageId":"20100825004821.GA10423@burratino","threadId":"24851","inReplyTo":"AANLkTi=P-PoBWwDbfGs7v1BTwvJ7uwfe7hd7A+Y7M361@mail.gmail.com","subject":"Re: [PATCH 03/32] cache-tree: ignore CE_REMOVE entries in verify_cache()","fromName":"Jonathan Nieder","fromEmail":"jrnieder@gmail.com","sentAt":"2010-08-25T00:48:21Z","receivedAt":"2010-08-25T00:48:21Z","isPatch":true,"sender":{"key":"jrnieder@gmail.com","avatar":"https://avatars.githubusercontent.com/u/281595?v=4"},"body":"Nguyen Thai Ngoc Duy wrote:\n>> Nguyễn Thái Ngọc Duy wrote:\n\n>>> --- a/cache-tree.c\n>>> +++ b/cache-tree.c\n>>> @@ -156,6 +156,8 @@ static int verify_cache(struct cache_entry **cache,\n>>>       funny = 0;\n>>>       for (i = 0; i < entries; i++) {\n>>>               struct cache_entry *ce = cache[i];\n>>> +             if (ce->ce_flags & CE_REMOVE)\n>>> +                     continue;\n[...]\n> I was lazy. In patch 26, upload-narrow-merge do three way merge and\n> drop staged entries within narrow tree (this is at server side,\n> conflicts within narrow tree will be handled at client side later).\n> Instead of removing staged entries, I mark them CE_REMOVE.\n\nAh, ok; so it's to avoid spending time in remove_marked_cache_entries().\n\nI think (though it wouldn't come up in your application) that to keep\nthis and avoid caller confusion one would need something like\n\n\t\tif ((cache[i]->ce_flags & CE_REMOVE) ||\n\t\t (cache[i+1]->ce_flags & CE_REMOVE))\n\t\t\tcontinue;\n\nin the other loop. I say \"something like\" because a person could have\nmarked some some but not all stages with CE_REMOVE, causing\nfile/directory conflicts to be missed.\n\nThanks for the explanation.\n"},{"id":"148928","messageId":"AANLkTi=-90e8c5Wi2TYWnAxsWvRMx_PjJZHhhRuSuRwb@mail.gmail.com","threadId":"24851","inReplyTo":"20100824232554.GF2376@burratino","subject":"Re: [PATCH 04/32] move do_compress() from pack-objects.c to pack-write.c","fromName":"Nguyen Thai Ngoc Duy","fromEmail":"pclouds@gmail.com","sentAt":"2010-08-25T03:19:26Z","receivedAt":"2010-08-25T03:19:26Z","isPatch":true,"sender":{"key":"pclouds@gmail.com","avatar":"https://avatars.githubusercontent.com/u/720?v=4"},"body":"On Wed, Aug 25, 2010 at 9:25 AM, Jonathan Nieder wrote:\n>> +unsigned long compress_object(void **pptr, unsigned long size)\n>> +{\n>> +     z_stream stream;\n>> +     void *in, *out;\n>> +     unsigned long maxsize;\n>> +\n>> +     memset(&stream, 0, sizeof(stream));\n>> +     deflateInit(&stream, Z_DEFAULT_COMPRESSION);\n>> +     maxsize = deflateBound(&stream, size);\n>> +\n>> +     in = *pptr;\n>> +     out = xmalloc(maxsize);\n>> +     *pptr = out;\n>> +\n>> +     stream.next_in = in;\n>> +     stream.avail_in = size;\n>> +     stream.next_out = out;\n>> +     stream.avail_out = maxsize;\n>> +     while (deflate(&stream, Z_FINISH) == Z_OK)\n>> +             ; /* nothing */\n>> +     deflateEnd(&stream);\n>> +\n>> +     return stream.total_out;\n>> +}\n>\n> However, there is nothing particularly specialized to git objects\n> about this.  e.g. deflate_it() from diff.c could use this.\n\nAlso archive-zip.c, fast-import.c, http-push.c and remote-curl.c have\nvery similar code. I'll see if I can reduce some code duplication.\n-- \nDuy\n"},{"id":"148929","messageId":"AANLkTi=5OoRWXND4x7YJKTU2kPaBAiUwYMFRocB8RC4A@mail.gmail.com","threadId":"24851","inReplyTo":"1282688422-7738-8-git-send-email-pclouds@gmail.com","subject":"Re: [PATCH 07/32] tree.c: find_subtree() to search for a tree","fromName":"Elijah Newren","fromEmail":"newren@gmail.com","sentAt":"2010-08-25T03:35:05Z","receivedAt":"2010-08-25T03:35:05Z","isPatch":true,"sender":{"key":"newren@gmail.com","avatar":"https://avatars.githubusercontent.com/u/5455730?v=4"},"body":"2010/8/24 Nguyễn Thái Ngọc Duy :\n> Signed-off-by: Nguyễn Thái Ngọc Duy \n\n> +int find_tree(const unsigned char *sha1, unsigned char *newsha1, const char *path)\n\nTrivial nit: Subject doesn't match the name of the function in the code. ;-)\n\nAlso, when trying to apply your series locally on master, it seems to\ndie on this patch. What commit is your series based on?\n"},{"id":"148930","messageId":"AANLkTinAzJHt3Bc9R0t50dUWqYqP5PGKG6-5Vj3sM=pA@mail.gmail.com","threadId":"24851","inReplyTo":"AANLkTi=5OoRWXND4x7YJKTU2kPaBAiUwYMFRocB8RC4A@mail.gmail.com","subject":"Re: [PATCH 07/32] tree.c: find_subtree() to search for a tree","fromName":"Nguyen Thai Ngoc Duy","fromEmail":"pclouds@gmail.com","sentAt":"2010-08-25T03:43:58Z","receivedAt":"2010-08-25T03:43:58Z","isPatch":true,"sender":{"key":"pclouds@gmail.com","avatar":"https://avatars.githubusercontent.com/u/720?v=4"},"body":"On Wed, Aug 25, 2010 at 1:35 PM, Elijah Newren wrote:\n> 2010/8/24 Nguyễn Thái Ngọc Duy :\n>> Signed-off-by: Nguyễn Thái Ngọc Duy \n> \n>> +int find_tree(const unsigned char *sha1, unsigned char *newsha1, const\n>> char *path)\n>\n> Trivial nit: Subject doesn't match the name of the function in the code.\n>  ;-)\n\nYeah. Been busy fixing code than commit messages..\n\n> Also, when trying to apply your series locally on master, it seems to\n> die on this patch.  What commit is your series based on?\n\nc11969de, master at Aug 19.\n-- \nDuy\n"},{"id":"148934","messageId":"AANLkTi=KBThiZ_FBUh0RfzLtP=YAtUPcx5PJW1q5OCd4@mail.gmail.com","threadId":"24851","inReplyTo":"1282688422-7738-12-git-send-email-pclouds@gmail.com","subject":"Re: [PATCH 11/32] rev-list: support --narrow-tree","fromName":"Elijah Newren","fromEmail":"newren@gmail.com","sentAt":"2010-08-25T03:59:58Z","receivedAt":"2010-08-25T03:59:58Z","isPatch":true,"sender":{"key":"newren@gmail.com","avatar":"https://avatars.githubusercontent.com/u/5455730?v=4"},"body":"Hi,\n\n2010/8/24 Nguyễn Thái Ngọc Duy :\n> These options allow plumbing to access narrow tree traverse modes.\n> As a consequence, tests can now be written to test these modes.\n>\n\n> +                       if (!prefixcmp(arg, \"--narrow-tree=\")) {\n> +                               revs->narrow_prefix = arg + 14;\n> +                               revs->narrow_tree = 1;\n> +                               continue;\n> +                       }\n\nSeems somewhat non-gittish. Why not 'git rev-list --objects HEAD --\n' rather than 'git rev-list --objects\n--narrow-tree= HEAD'? (Was it due to the current bug in\nrev-list when specifying both --objects and paths? If so, that's\naddressed in the patch series I just sent.)\n\n\n> +cat <merge.expected\n> +0da28f8308e336bd4b2c26b61c7fc44e41a30e49\n> +7475cb8a389b36ce238b9ee6cbfdfa26a1b67e35\n> +af9c31c0e217154296d93d66b9a5a41892c7b321\n> +ba7a30916c792624a8372cb26da50f5594098222\n> +74a398027f0b59183db54ca8c67dc30b5aeed0ff t2\n\nIs this right? Do you expect to see a t2 entry when filtering for t1/t12?\n\n> +3b52e9990a52d9ea46b25199b5810566324759f4\n> +fa0bba1767985729582729a12a5459cd041d6cf6 t1\n> +d81015b2a1c40fc6cbc5bf5a8b16949018c3aede t1/t12\n> +52bd8e43afb01d0c9747f1fedf2fc94684ee4cc4 t1/t12/f120\n> +aa0602ee56ea4cb5e5bfff8713bb8a9a6ab4303e\n> +d77d258d806b2aa7cdb5a301e6a54f023d9bdcfe t1\n> +ab246f7eb05e94f695d2a0e30093d94fde7b837e t1/t12\n> +04156ae83e615fa5b6019170928bc131539f9996 t1/t12/f120\n> +72e05e456b9ff4c01ccf6178ce60d9b52b41aae4\n> +4fc1656b01ce8b21987c55a2870b8c9a431ec772 t1\n> +EOF\n> +\n> +test_expect_success 'rev-list --narrow-tree=t1/t12 with merges' '\n> +       git rev-list --objects --narrow-tree=t1/t12 $MERGE >result &&\n> +       test_cmp merge.expected result\n"},{"id":"148935","messageId":"AANLkTimmd-6CWEA9qjHGnAn8svjU+iEFDxwbck6KAQtX@mail.gmail.com","threadId":"24851","inReplyTo":"1282688422-7738-11-git-send-email-pclouds@gmail.com","subject":"Re: [PATCH 10/32] rev-list: support traversing in narrow repository mode","fromName":"Elijah Newren","fromEmail":"newren@gmail.com","sentAt":"2010-08-25T04:11:43Z","receivedAt":"2010-08-25T04:11:43Z","isPatch":true,"sender":{"key":"newren@gmail.com","avatar":"https://avatars.githubusercontent.com/u/5455730?v=4"},"body":"2010/8/24 Nguyễn Thái Ngọc Duy :\n> In this mode, only trees within revs->narrow_prefix is traversed. This\n> narrows down the whole repository to the given subtree. This mode will\n> be used by upload-pack/pack-objects to create a narrow pack, and by\n> all git operations in narrow repository (i.e. $GIT_DIR/narrow exists).\n\nMy fix-rev-list series I posted is going to cause a few conflicts with\nthis patch...\n\n\n> +               if (!subtree)\n> +                       ;       /* no subtree restriction, go on */\n> +               else if (S_ISDIR(entry.mode) &&\n> +                        !strncmp(entry.path, subtree, subtree_len) &&\n> +                        entry.path[subtree_len] == '\\0')\n> +                       ;       /* inside subtree, go on */\n\nInstead of writing your own comparison code, you may want to consider\nusing tree_entry_interesting() here. That would also provide you with\na natural extension to multiple subtrees, rather than your currently\nhardcoded single tree. You may want to take a look at\ntree-diff.c:show_tree() for inspiration here.\n"},{"id":"148938","messageId":"AANLkTinsNvVup43B6nQtU6dvJy789n8kQm6N6na0J9oa@mail.gmail.com","threadId":"24851","inReplyTo":"AANLkTikipryFVf_XvvbHopWSo5Ey_mvgTevY2NAvhygQ@mail.gmail.com","subject":"Re: [RFD PATCH 00/32] subtree clone v2","fromName":"Elijah Newren","fromEmail":"newren@gmail.com","sentAt":"2010-08-25T04:37:07Z","receivedAt":"2010-08-25T04:37:07Z","isPatch":true,"sender":{"key":"newren@gmail.com","avatar":"https://avatars.githubusercontent.com/u/5455730?v=4"},"body":"On Tue, Aug 24, 2010 at 4:47 PM, Nguyen Thai Ngoc Duy wrote:\n> On Wed, Aug 25, 2010 at 8:37 AM, Jonathan Nieder wrote:\n>> Nguyễn Thái Ngọc Duy wrote:\n>>\n>>> Patches 26..30\n>>>   Add upload-narrow-base command\n>>>   rev-list: traverse some more trees to make upload-narrow-base happy\n>>>   narrow-tree: add oldest_narrow_base()\n>>>   Add command fetch-narrow-base\n>>>   merge: support merging when narrow bases are different\n>>>\n>>> Remote merge part.\n>>>\n>>> Split a merge operation into two parts, the real merge will be done\n>>> within narrow tree. Conflicts can happen and be resolved in the narrow\n>>> index, locally.\n>>>\n>>> Everything outside narrow tree will be merged (trivially) by\n>>> server. Then server sends the base tree back, so join_narrow_tree() in\n>>> patch 20 can be used to create proper commit.\n>>>\n>>> Server can disable this remote merge feature, which means users are\n>>> forced to do rebase/fast-forward. Not too bad.\n>>\n>> Yikes.  Naïve question (please forgive my laziness): is it possible to\n>> merge without remote contact in the boring case, when no changes have\n>> occured outside the narrow tree?\n>\n> That's possible (and is implemented in my series). But I guess as soon\n> as you do \"git pull\", the boring case is likely not applicable\n> anymore.\n\nI'm not sure I follow. Are you allowing changes outside the narrow\ntree to occur? If you're not, I would have assumed that repeated\npulls just work, without any need to talk to the server, using a\nresolve-like strategy (with no special rename detection).\n\nHere's my understanding, though it might have holes:\n\nIf you have a narrow/subtree clone, it means that you only have the\ndata for a certain paths. I'm assuming that also meant you would only\nallow modifying those paths. In other words, you have no changes\noutside the narrow tree. Because of that, I think you can handle\npaths outside the narrow region using trivial-merge logic: From\nDocumentation/technical/trivial-merge.txt, I think the relevant cases\nare 2, 3, 8, 10, 13, or 14. 13 & 14 already have a specified\nresolution. There's already a comment in the file that cases 8 & 10\ncould validly be resolved as (empty), it just hasn't been done in the\ncode as it tends to happen with the follow-up automatic merge anyway.\nThat only leaves cases 2 & 3 as being slightly tricky -- if a path on\none side of the merge started empty and ended empty, it would seem to\nmake sense that the non-empty path on the other side would be the\nresolution. We can't do that in the non-narrow clone case because the\nnon-empty path may have been created due to a rename and we'd like to\nhave changes follow the rename appropriately. However, in the narrow\nclone case, one can't rename from a path you don't have to a path you\ndo, so this possibility is eliminated.\n\n\nSo, if my understanding is correct, then if you have no changes from\nupstream outside the sparse/narrow/whatever paths (where \"paths\"\ncurrently means a single tree in your current patches), I think you\nshould be able to do a merge locally in that sparse clone. The end\nresult will also have no changes from upstream outside those paths.\nThus, you should be able to merge again.\n\n\nDoes that seem reasonable? Am I missing anything?\n"},{"id":"148941","messageId":"AANLkTinaF6e+bfVqRRThuZ=2PjpOs4P88RKFsxOTVSJR@mail.gmail.com","threadId":"24851","inReplyTo":"1282688422-7738-16-git-send-email-pclouds@gmail.com","subject":"Re: [PATCH 15/32] unpack_trees: only unpack $GIT_DIR/narrow subtree in narrow repository","fromName":"Elijah Newren","fromEmail":"newren@gmail.com","sentAt":"2010-08-25T05:04:30Z","receivedAt":"2010-08-25T05:04:30Z","isPatch":true,"sender":{"key":"newren@gmail.com","avatar":"https://avatars.githubusercontent.com/u/5455730?v=4"},"body":"Hi,\n\n2010/8/24 Nguyễn Thái Ngọc Duy :\n> By definition, narrow repository is incomplete. It does not even have\n> enough tree for a single commit. So populating a full index is\n> impossible.\n>\n> Because of this, unpack_trees() is modified to only unpack trees\n> within $GIT_DIR/narrow, which narrow repo has all needed trees. This\n> makes the resulting index unsuitable for creating commits later on.\n> This is the reason index version is increased to 4, to avoid older\n> git from using it.\n>\n> The resulting tree objects created from the index is only part of the\n> full tree. Manipulation will be needed at commit time to create proper\n> tree for commits.\n\nI spent a while thinking about this a couple weeks ago and never came\nto a strong conclusion about which of two alternatives should be\npreferred; I'm curious why you decided to go for this solution. An\nalternative I thought of was having the index have entries for missing\nfiles (whose contents did not exist in the repository or the working\ncopy; rather all we know is the filename and its sha1sum) and also\ngain the ability to have entries for missing trees (which behave\nsimilarly; all we know is their name and their sha1sum, but the\ncontents of that sha1sum are not in the repository or the working\ndirectory) Is there a reason to prefer one alternative over the\nother? Does the alternative I thought of even make sense?\n"},{"id":"148943","messageId":"AANLkTikkybj784PHoVCkVZyxo40q6BhdkOzFho-xBS6G@mail.gmail.com","threadId":"24851","inReplyTo":"AANLkTinsNvVup43B6nQtU6dvJy789n8kQm6N6na0J9oa@mail.gmail.com","subject":"Re: [RFD PATCH 00/32] subtree clone v2","fromName":"Nguyen Thai Ngoc Duy","fromEmail":"pclouds@gmail.com","sentAt":"2010-08-25T05:21:00Z","receivedAt":"2010-08-25T05:21:00Z","isPatch":true,"sender":{"key":"pclouds@gmail.com","avatar":"https://avatars.githubusercontent.com/u/720?v=4"},"body":"On Wed, Aug 25, 2010 at 2:37 PM, Elijah Newren wrote:\n>>>> Remote merge part.\n>>>>\n>>>> Split a merge operation into two parts, the real merge will be done\n>>>> within narrow tree. Conflicts can happen and be resolved in the narrow\n>>>> index, locally.\n>>>>\n>>>> Everything outside narrow tree will be merged (trivially) by\n>>>> server. Then server sends the base tree back, so join_narrow_tree() in\n>>>> patch 20 can be used to create proper commit.\n>>>>\n>>>> Server can disable this remote merge feature, which means users are\n>>>> forced to do rebase/fast-forward. Not too bad.\n>>>\n>>> Yikes.  Naïve question (please forgive my laziness): is it possible to\n>>> merge without remote contact in the boring case, when no changes have\n>>> occured outside the narrow tree?\n>>\n>> That's possible (and is implemented in my series). But I guess as soon\n>> as you do \"git pull\", the boring case is likely not applicable\n>> anymore.\n>\n> I'm not sure I follow.  Are you allowing changes outside the narrow\n> tree to occur?  If you're not, I would have assumed that repeated\n> pulls just work, without any need to talk to the server, using a\n> resolve-like strategy (with no special rename detection).\n\nI don't. But I can't stop all other users (who use full repos) change\noutside the narrow tree and push their changes back upstream. When\nnarrow user pulls from upstream again, the tree outside narrow tree\nmight be not the same as before.\n\n> Here's my understanding, though it might have holes:\n>\n> If you have a narrow/subtree clone, it means that you only have the\n> data for a certain paths.  I'm assuming that also meant you would only\n> allow modifying those paths.  In other words, you have no changes\n> outside the narrow tree.  Because of that, I think you can handle\n> paths outside the narrow region using trivial-merge logic:  From\n> Documentation/technical/trivial-merge.txt, I think the relevant cases\n> are 2, 3, 8, 10, 13, or 14.  13 & 14 already have a specified\n> resolution.  There's already a comment in the file that cases 8 & 10\n> could validly be resolved as (empty), it just hasn't been done in the\n> code as it tends to happen with the follow-up automatic merge anyway.\n> That only leaves cases 2 & 3 as being slightly tricky -- if a path on\n> one side of the merge started empty and ended empty, it would seem to\n> make sense that the non-empty path on the other side would be the\n> resolution.  We can't do that in the non-narrow clone case because the\n> non-empty path may have been created due to a rename and we'd like to\n> have changes follow the rename appropriately.  However, in the narrow\n> clone case, one can't rename from a path you don't have to a path you\n> do, so this possibility is eliminated.\n\nTrivial merge did not work outside narrow tree for me because I did\nnot have all trees of a commit. So as soon as it steps on the narrow\nborder, it has to stop (without retrieving full path to files outside\nnarrow tree). trivial-merge.txt takes input as files, not directories.\nGiven two directories (because we don't have further details what\ninside those directories), I don't know how to do trivial merge.\n\n> So, if my understanding is correct, then if you have no changes from\n> upstream outside the sparse/narrow/whatever paths (where \"paths\"\n> currently means a single tree in your current patches), I think you\n> should be able to do a merge locally in that sparse clone.  The end\n> result will also have no changes from upstream outside those paths.\n> Thus, you should be able to merge again.\n\nYes. But other people will change outside narrow tree, unfortunately.\n-- \nDuy\n"},{"id":"148944","messageId":"AANLkTikv9zHGRdJRoFV6Pr0HMUbKPPz_1kUXC6DnoSXL@mail.gmail.com","threadId":"24851","inReplyTo":"AANLkTinsNvVup43B6nQtU6dvJy789n8kQm6N6na0J9oa@mail.gmail.com","subject":"Re: [RFD PATCH 00/32] subtree clone v2","fromName":"Elijah Newren","fromEmail":"newren@gmail.com","sentAt":"2010-08-25T05:21:22Z","receivedAt":"2010-08-25T05:21:22Z","isPatch":true,"sender":{"key":"newren@gmail.com","avatar":"https://avatars.githubusercontent.com/u/5455730?v=4"},"body":"Replying to myself...\n\nOn Tue, Aug 24, 2010 at 10:37 PM, Elijah Newren wrote:\n\n> outside the narrow tree.  Because of that, I think you can handle\n> paths outside the narrow region using trivial-merge logic:  From\n> Documentation/technical/trivial-merge.txt, I think the relevant cases\n> are 2, 3, 8, 10, 13, or 14.  13 & 14 already have a specified\n\n> That only leaves cases 2 & 3 as being slightly tricky -- if a path on\n> one side of the merge started empty and ended empty, it would seem to\n> make sense that the non-empty path on the other side would be the\n> resolution.  We can't do that in the non-narrow clone case because the\n> non-empty path may have been created due to a rename and we'd like to\n> have changes follow the rename appropriately.  However, in the narrow\n> clone case, one can't rename from a path you don't have to a path you\n> do, so this possibility is eliminated.\n\nI just realized I was assuming the path was empty in the merge base\nand in upstream, but non-empty on the side of the merge corresponding\nto the local sparse/narrow repository. My wording certainly reflected\nthat assumption. If so, I still think what I said was correct. If\nnot, it's slightly more subtle. If the non-empty side is upstream,\nthen it may be a rename case that needs special handling. If it's a\nrename from a path outside our subtree (and we can somehow detect this\ncase), then although we don't have the data to do any special\nhandling, it wouldn't be necessary since we did not modify either the\nsource or destination of the rename. The tricky case is if it's a\nrename from a path within our subtree to a file outside; we'd only be\nable to handle such situations if there was no modification other than\nthe rename. Unfortunately, we can't distinguish between (1) renames\nfrom paths outside our subtree to outside our subtree and (2) renames\nplus changes from paths inside our subtree to outside our subtree.\n\nThat puts us in a bit of a pickle for the empty->empty vs. non-empty\ncase. We can only resolve it if the non-empty side is in our subtree,\nor if we decide to just ignore renames in our merges.\n"},{"id":"148945","messageId":"AANLkTikOtinR1EjzrzMqzR8Pn2hpDRVJrWDmx5b5tGQ_@mail.gmail.com","threadId":"24851","inReplyTo":"AANLkTikkybj784PHoVCkVZyxo40q6BhdkOzFho-xBS6G@mail.gmail.com","subject":"Re: [RFD PATCH 00/32] subtree clone v2","fromName":"Elijah Newren","fromEmail":"newren@gmail.com","sentAt":"2010-08-25T05:31:31Z","receivedAt":"2010-08-25T05:31:31Z","isPatch":true,"sender":{"key":"newren@gmail.com","avatar":"https://avatars.githubusercontent.com/u/5455730?v=4"},"body":"Hi,\n\nOn Tue, Aug 24, 2010 at 11:21 PM, Nguyen Thai Ngoc Duy\n wrote:\n> On Wed, Aug 25, 2010 at 2:37 PM, Elijah Newren wrote:\n\n>> I'm not sure I follow.  Are you allowing changes outside the narrow\n>> tree to occur?  If you're not, I would have assumed that repeated\n>> pulls just work, without any need to talk to the server, using a\n>> resolve-like strategy (with no special rename detection).\n>\n> I don't. But I can't stop all other users (who use full repos) change\n> outside the narrow tree and push their changes back upstream. When\n> narrow user pulls from upstream again, the tree outside narrow tree\n> might be not the same as before.\n\nSure, they will modify paths outside your subtree, but you know that\nyou didn't do so. So if you neglect renames then the situation is\npretty simple:\n\nMerge-side: ancestor you upstream\n -------- -------- --------------\nSha1sum: ancestor ancestor new-stuff\n\nWhich could be true for either a file or a tree. Either way, if you\nignore renames, then the trivial merge is 'new-stuff' for any such\nblob/tree. (Yeah, you'd have to create a new merge algorithm that\ndoes 'trivial' merges at tree levels in addition to file levels, but\nthe concept is relatively simple at least.)\n\nAfter your merge, a subsequent pull would look like this at the same\nfile/tree path:\n\nMerge-side: ancestor you upstream\n -------- -------- --------------\nSha1sum: new-stuff new-stuff newer-stuff\n\nFor which the resolution would be 'newer-stuff' (again, ignoring the\nproblems that renames could cause, which I discussed elsewhere).\n\nUnless, of course, I'm missing something still...\n"},{"id":"148946","messageId":"AANLkTi=cuT9HiZyp67rMBM63KjuOpADjAVMYbQ-cfDWm@mail.gmail.com","threadId":"24851","inReplyTo":"AANLkTinAzJHt3Bc9R0t50dUWqYqP5PGKG6-5Vj3sM=pA@mail.gmail.com","subject":"Re: [PATCH 07/32] tree.c: find_subtree() to search for a tree","fromName":"Elijah Newren","fromEmail":"newren@gmail.com","sentAt":"2010-08-25T05:35:42Z","receivedAt":"2010-08-25T05:35:42Z","isPatch":true,"sender":{"key":"newren@gmail.com","avatar":"https://avatars.githubusercontent.com/u/5455730?v=4"},"body":"On Tue, Aug 24, 2010 at 9:43 PM, Nguyen Thai Ngoc Duy wrote:\n>> Also, when trying to apply your series locally on master, it seems to\n>> die on this patch.  What commit is your series based on?\n>\n> c11969de, master at Aug 19.\n\nHmm...seems they don't apply there either, but the reason is because\ngmane changed the order of your patches. That's unfortunate. :-/\n"},{"id":"148947","messageId":"AANLkTikiazizLaXtotxL2UcuJ64KgWwGCwy+E6rv42_G@mail.gmail.com","threadId":"24851","inReplyTo":"AANLkTinaF6e+bfVqRRThuZ=2PjpOs4P88RKFsxOTVSJR@mail.gmail.com","subject":"Re: [PATCH 15/32] unpack_trees: only unpack $GIT_DIR/narrow subtree in narrow repository","fromName":"Nguyen Thai Ngoc Duy","fromEmail":"pclouds@gmail.com","sentAt":"2010-08-25T05:38:54Z","receivedAt":"2010-08-25T05:38:54Z","isPatch":true,"sender":{"key":"pclouds@gmail.com","avatar":"https://avatars.githubusercontent.com/u/720?v=4"},"body":"On Wed, Aug 25, 2010 at 3:04 PM, Elijah Newren wrote:\n> Hi,\n>\n> 2010/8/24 Nguyễn Thái Ngọc Duy :\n>> By definition, narrow repository is incomplete. It does not even have\n>> enough tree for a single commit. So populating a full index is\n>> impossible.\n>>\n>> Because of this, unpack_trees() is modified to only unpack trees\n>> within $GIT_DIR/narrow, which narrow repo has all needed trees. This\n>> makes the resulting index unsuitable for creating commits later on.\n>> This is the reason index version is increased to 4, to avoid older\n>> git from using it.\n>>\n>> The resulting tree objects created from the index is only part of the\n>> full tree. Manipulation will be needed at commit time to create proper\n>> tree for commits.\n>\n> I spent a while thinking about this a couple weeks ago and never came\n> to a strong conclusion about which of two alternatives should be\n> preferred; I'm curious why you decided to go for this solution.  An\n> alternative I thought of was having the index have entries for missing\n> files (whose contents did not exist in the repository or the working\n> copy; rather all we know is the filename and its sha1sum) and also\n> gain the ability to have entries for missing trees (which behave\n> similarly; all we know is their name and their sha1sum, but the\n> contents of that sha1sum are not in the repository or the working\n> directory)  Is there a reason to prefer one alternative over the\n> other?  Does the alternative I thought of even make sense?\n\nThat was nightmare. I had to deal with unpack_callback() and skip\nuninterested paths. And that function is, I think, quite optimized. I\ntried another approach, putting directories in index with hope that it\nwould cut down the number of code path I'd have to touch. It did, but\nthen index sorting order is just weird and I couldn't get it right\n(felt to risky).\n\nunpack_trees() is used by diff (all diffs except diff_tree_sha1).\ntraverse_trees()/unpack_trees() is also used by merge strategy. All\nthose code is really complicated that I'd rather stay away from them.\n\nThat was before I went with tree rewrites. Soon after I realized the\nnice effect of tree rewrites is that the index is narrowed down. So I\ncan get rid of tree rewrites as long as the index is still narrow.\nThat's how I come to this approach.\n\nBack to your questions. I think the alternative makes sense, it just\nlooks a lot of work and quite intrusive. On the other hand, my current\napproach is quite simple (but it probably won't work for more than a\nsingle narrow tree)\n-- \nDuy\n"},{"id":"148949","messageId":"AANLkTikCa_Yfw4t1=JAOkZPE3_txof67C4CfgrELoPsh@mail.gmail.com","threadId":"24851","inReplyTo":"AANLkTikOtinR1EjzrzMqzR8Pn2hpDRVJrWDmx5b5tGQ_@mail.gmail.com","subject":"Re: [RFD PATCH 00/32] subtree clone v2","fromName":"Nguyen Thai Ngoc Duy","fromEmail":"pclouds@gmail.com","sentAt":"2010-08-25T06:21:25Z","receivedAt":"2010-08-25T06:21:25Z","isPatch":true,"sender":{"key":"pclouds@gmail.com","avatar":"https://avatars.githubusercontent.com/u/720?v=4"},"body":"On Wed, Aug 25, 2010 at 3:31 PM, Elijah Newren wrote:\n> Sure, they will modify paths outside your subtree, but you know that\n> you didn't do so.  So if you neglect renames then the situation is\n> pretty simple:\n>\n> Merge-side: ancestor    you         upstream\n>            --------    --------    --------------\n> Sha1sum:    ancestor    ancestor    new-stuff\n>\n> Which could be true for either a file or a tree.  Either way, if you\n> ignore renames, then the trivial merge is 'new-stuff' for any such\n> blob/tree.  (Yeah, you'd have to create a new merge algorithm that\n> does 'trivial' merges at tree levels in addition to file levels, but\n> the concept is relatively simple at least.)\n>\n> After your merge, a subsequent pull would look like this at the same\n> file/tree path:\n>\n> Merge-side: ancestor    you         upstream\n>            --------    --------    --------------\n> Sha1sum:    new-stuff   new-stuff   newer-stuff\n>\n> For which the resolution would be 'newer-stuff' (again, ignoring the\n> problems that renames could cause, which I discussed elsewhere).\n>\n> Unless, of course, I'm missing something still...\n\nOK here is the story. And I could be wrong on this merge stuff.\n\nI was deep in merge-tree.c code (which was the base for my narrow\nmerge) and was stuck at\n\n\"If either \"you\" or \"upstream\" is as same as ancestor (all as dirs),\nthen it's easy. What if neither of them is the same as ancestor? If we\nhave more trees, we could just descend and figure out. But we don't\nbecause it's outside narrow area. It could be that \"you\" added a new\nfile and \"upstream\" added another file, which is trivial at file\nlevel.\"\n\nI was deep in code and forgot that in narrow repo, I did not allow to\nchange outside narrow area, so it \"you\" must be as same as ancestor.\nBut the question was, what if user decides to branch from some commit\nin history? If he/she does, \"you\" might not be as same as ancestor. If\nit's non trivial merge, merge should fail (expected).\n\nBut the \"it could be that \"you\" added ... at file level\" above\nbothered me. I had a feeling that I could miss a trivial merge (at\nfile level) because I could not make a judgement at (shallow) tree\nlevel. And I did not have all trees to do merge at file level.\n\nStill puzzling thinking about this.. Did I make wrong assumptions?\n-- \nDuy\n"},{"id":"148968","messageId":"AANLkTi=sPL4mAH=_Fba8Er6BQvq=NDO+-GF+MaGc8yAm@mail.gmail.com","threadId":"24851","inReplyTo":"AANLkTikCa_Yfw4t1=JAOkZPE3_txof67C4CfgrELoPsh@mail.gmail.com","subject":"Re: [RFD PATCH 00/32] subtree clone v2","fromName":"Elijah Newren","fromEmail":"newren@gmail.com","sentAt":"2010-08-25T13:06:36Z","receivedAt":"2010-08-25T13:06:36Z","isPatch":true,"sender":{"key":"newren@gmail.com","avatar":"https://avatars.githubusercontent.com/u/5455730?v=4"},"body":"Hi,\n\nOn Wed, Aug 25, 2010 at 12:21 AM, Nguyen Thai Ngoc Duy\n wrote:\n> OK here is the story. And I could be wrong on this merge stuff.\n>\n> I was deep in merge-tree.c code (which was the base for my narrow\n> merge) and was stuck at\n>\n> \"If either \"you\" or \"upstream\" is as same as ancestor (all as dirs),\n> then it's easy. What if neither of them is the same as ancestor? If we\n> have more trees, we could just descend and figure out. But we don't\n> because it's outside narrow area. It could be that \"you\" added a new\n> file and \"upstream\" added another file, which is trivial at file\n> level.\"\n>\n> I was deep in code and forgot that in narrow repo, I did not allow to\n> change outside narrow area, so it \"you\" must be as same as ancestor.\n> But the question was, what if user decides to branch from some commit\n> in history? If he/she does, \"you\" might not be as same as ancestor. If\n> it's non trivial merge, merge should fail (expected).\n\nIf you branch from some older commit, then naturally the common\nancestor between you and upstream changes as well (namely, to that\nolder commit). Since you don't/can't change anything outside your\nsubtree, that puts you in the following situation for paths outside\nyour subtree when you try to merge:\n\nMerge-side: ancestor you upstream\n -------- -------- --------------\nSha1sum: old-stuff old-stuff newest-stuff\n\n> But the \"it could be that \"you\" added ... at file level\" above\n> bothered me. I had a feeling that I could miss a trivial merge (at\n> file level) because I could not make a judgement at (shallow) tree\n> level. And I did not have all trees to do merge at file level.\n>\n> Still puzzling thinking about this.. Did I make wrong assumptions?\n\nI'm not sure I understand why there would be any issue with you adding\none file in your subtree, and upstream adding (another/the same?) file\noutside your subtree. The only problematic case I see is when files\nare renamed upstream from within the subtree to outside the subtree\nand subsequently changed. If on your side of history you modify one\nof these files that were renamed from upstream, there's no way to\napply your changes to the new file. It'd look something like this:\n\nMerge-side: ancestor you upstream\n -------- -------- --------------\nold-file: original modify1 (missing)\nnew-file: (missing) (missing) modify2\n\nThe correct resolution is a three-way merge between original, modify1,\nand modify2, stored in new-file with old-file being deleted. However,\nyou don't have access to new-file/modify2 since it's outside your\nsubtree. All you'd see is a modify/delete conflict in old-file and\nthe 'trivial merge' logic I suggested above (which ignores renames)\nwould result in new-file still having contents of modify2.\n\nHopefully the modify/delete conflict would be enough to trigger the\nuser to ask rather than just randomly pick some resolution and\nproceed, which makes me think it should be okay to have a renaming\nignoring 'trivial' merge algorithm for subtrees.\n\n\nElijah\n"},{"id":"148988","messageId":"7vpqx6qypx.fsf@alter.siamese.dyndns.org","threadId":"24851","inReplyTo":"AANLkTinsNvVup43B6nQtU6dvJy789n8kQm6N6na0J9oa@mail.gmail.com","subject":"Re: [RFD PATCH 00/32] subtree clone v2","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2010-08-25T19:27:54Z","receivedAt":"2010-08-25T19:27:54Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Elijah Newren writes:\n\n> I'm not sure I follow. Are you allowing changes outside the narrow\n> tree to occur? If you're not, I would have assumed that repeated\n> pulls just work, without any need to talk to the server, using a\n> resolve-like strategy (with no special rename detection).\n>\n> Here's my understanding, though it might have holes:\n>\n> If you have a narrow/subtree clone, it means that you only have the\n> data for a certain paths. I'm assuming that also meant you would only\n> allow modifying those paths. In other words, you have no changes\n> outside the narrow tree. Because of that, I think you can handle\n> paths outside the narrow region using trivial-merge logic ...\n\nI think you can do a merge only once, but after that you would be lost.\n\nSuppose you forked from commit A, narrowed to one subdirectory and created\nB (iow, diff(A, B) contains only changes to that subdirectory). Somebody\nelse worked on the whole tree and queued commit C on top of A.\n\n B---?\n / /\n ---A---C\n\nYour merge between B and C that uses A as the common ancestor can take all\nchanges outside the subdirectory from C (but you need to know all the tree\nobject names in C near the top outside your area). So far, so good.\n\nWhile this was going on, somebody else also forked from A and made changes\nthat touch both inside and outside your area, and its tip commit is E.\n\n B---D---?\n / / /\n ---A---C /\n \\ /\n o---E\n\nNow you want to merge changes made on the branch that leads to E to your\ncopy. How would we do that?\n\nYou cannot merge D and E without having enough information necessary to\nmake a merge between C and E with full tree, can you?\n"},{"id":"148993","messageId":"AANLkTi=rt5ho9EXg7_NF+JqPW=cs8yqAHqm5+mPooYcC@mail.gmail.com","threadId":"24851","inReplyTo":"7vpqx6qypx.fsf@alter.siamese.dyndns.org","subject":"Re: [RFD PATCH 00/32] subtree clone v2","fromName":"Elijah Newren","fromEmail":"newren@gmail.com","sentAt":"2010-08-25T20:43:10Z","receivedAt":"2010-08-25T20:43:10Z","isPatch":true,"sender":{"key":"newren@gmail.com","avatar":"https://avatars.githubusercontent.com/u/5455730?v=4"},"body":"On Wed, Aug 25, 2010 at 1:27 PM, Junio C Hamano wrote:\n\n>> If you have a narrow/subtree clone, it means that you only have the\n>> data for a certain paths.  I'm assuming that also meant you would only\n>> allow modifying those paths.  In other words, you have no changes\n>> outside the narrow tree.  Because of that, I think you can handle\n>> paths outside the narrow region using trivial-merge logic ...\n>\n> I think you can do a merge only once, but after that you would be lost.\n>\n> Suppose you forked from commit A, narrowed to one subdirectory and created\n> B (iow, diff(A, B) contains only changes to that subdirectory).  Somebody\n> else worked on the whole tree and queued commit C on top of A.\n>\n>      B---?\n>     /   /\n>  ---A---C\n>\n> Your merge between B and C that uses A as the common ancestor can take all\n> changes outside the subdirectory from C (but you need to know all the tree\n> object names in C near the top outside your area).  So far, so good.\n>\n> While this was going on, somebody else also forked from A and made changes\n> that touch both inside and outside your area, and its tip commit is E.\n>\n>      B---D---?\n>     /   /   /\n>  ---A---C   /\n>     \\     /\n>      o---E\n>\n> Now you want to merge changes made on the branch that leads to E to your\n> copy.  How would we do that?\n>\n> You cannot merge D and E without having enough information necessary to\n> make a merge between C and E with full tree, can you?\n\nAh, now I think I see what Duy was getting at; thanks for making that clear.\n\nYou are correct, of course[1]. Failing merges with\n\"not-enough-information-available\" will sometimes be a fact of life\nwith sparse/narrow/subtree/partial/whatever clones[2] -- either that\nor \"server-side merging\" as Duy suggests (though I personally am not a\nfan of that). I was just pointing out that I think we're not limited\nto a single merge ever; for example, if I understand correctly, if\nsomeone else merges C & E then you'll be able to merge from them[3]\n(and thus, you'll be able to merge \"more than once\").\n\nElijah\n\n\n[1] Outside your subtree, D does not match the merge-base A, thus so\nyou can't employ 'trivial merge' logic and you don't have the data for\nany other kind of merge.\n\n[2] As noted as item 7 at\nhttp://article.gmane.org/gmane.comp.version-control.git/152020.\n\n[3] This is because outside your subtree, D would match the new\nmerge-base (C), allowing you to use trivial merge logic again.\n"},{"id":"149009","messageId":"AANLkTimMGn9M9im2LdjYJkbg9QhUKq2a_4tRQEUoQM0c@mail.gmail.com","threadId":"24851","inReplyTo":"AANLkTi=KBThiZ_FBUh0RfzLtP=YAtUPcx5PJW1q5OCd4@mail.gmail.com","subject":"Re: [PATCH 11/32] rev-list: support --narrow-tree","fromName":"Nguyen Thai Ngoc Duy","fromEmail":"pclouds@gmail.com","sentAt":"2010-08-25T22:11:11Z","receivedAt":"2010-08-25T22:11:11Z","isPatch":true,"sender":{"key":"pclouds@gmail.com","avatar":"https://avatars.githubusercontent.com/u/720?v=4"},"body":"On Wed, Aug 25, 2010 at 1:59 PM, Elijah Newren wrote:\n> Hi,\n>\n> 2010/8/24 Nguyễn Thái Ngọc Duy :\n>> These options allow plumbing to access narrow tree traverse modes.\n>> As a consequence, tests can now be written to test these modes.\n>>\n> \n>> +                       if (!prefixcmp(arg, \"--narrow-tree=\")) {\n>> +                               revs->narrow_prefix = arg + 14;\n>> +                               revs->narrow_tree = 1;\n>> +                               continue;\n>> +                       }\n>\n> Seems somewhat non-gittish.  Why not 'git rev-list --objects HEAD --\n> ' rather than 'git rev-list --objects\n> --narrow-tree= HEAD'?  (Was it due to the current bug in\n> rev-list when specifying both --objects and paths?  If so, that's\n> addressed in the patch series I just sent.)\n\nYeah probably because of the --object bug that I did not know. Anyway\nI still need \"--narrow\" signal because in my series, I would send all\ntrees that are not found in parents' trees for merge commits. A\nrequirement from remote merge.\n\n>> +cat <merge.expected\n>> +0da28f8308e336bd4b2c26b61c7fc44e41a30e49\n>> +7475cb8a389b36ce238b9ee6cbfdfa26a1b67e35\n>> +af9c31c0e217154296d93d66b9a5a41892c7b321\n>> +ba7a30916c792624a8372cb26da50f5594098222\n>> +74a398027f0b59183db54ca8c67dc30b5aeed0ff t2\n>\n> Is this right?  Do you expect to see a t2 entry when filtering for t1/t12?\n\nAs I said above, if this \"t2\" is not seen in either parents, then it\nmust be sent/traversed, unfortunately.\n-- \nDuy\n"},{"id":"149011","messageId":"AANLkTi=02T+60AjJON8MFt7DSbtA=bx39YKigZHq_b_u@mail.gmail.com","threadId":"24851","inReplyTo":"AANLkTikOtinR1EjzrzMqzR8Pn2hpDRVJrWDmx5b5tGQ_@mail.gmail.com","subject":"Re: [RFD PATCH 00/32] subtree clone v2","fromName":"Nguyen Thai Ngoc Duy","fromEmail":"pclouds@gmail.com","sentAt":"2010-08-25T22:13:21Z","receivedAt":"2010-08-25T22:13:21Z","isPatch":true,"sender":{"key":"pclouds@gmail.com","avatar":"https://avatars.githubusercontent.com/u/720?v=4"},"body":"On Wed, Aug 25, 2010 at 3:31 PM, Elijah Newren wrote:\n> Sure, they will modify paths outside your subtree, but you know that\n> you didn't do so.  So if you neglect renames then the situation is\n> pretty simple:\n> ....\n>\n> Unless, of course, I'm missing something still...\n\nI remember it!\n\nSo there are two points: the \"no changes outside narrow tree from\n\"you\"\" assumption and whether it is trivial to do a merge outside\nnarrow tree without whole trees.\n\nThe first point. That assumption holds if user clones, starts working,\nthen does \"pull origin\". But if user merges another branch, say \"next\"\ninto \"master\", the common commit may be somewhere down in history and\nthere may be changes outside narrow tree from both \"you\" and\n\"upstream\" (though user does not make those changes).\n\nThe second point (probably not needed if the above assumption is no\nlonger true, but I post anyway in case I forget it again). Without\nwhole trees, it's impossible to determine a trivial merge reliably. If\n\"you\" adds a file and \"upstream\" adds another file, predecessor trees\nwill look different but merging them is trivial (at file level). If we\nmiss some trees that lead to those new files, the best thing we can do\nis to claim it non-trivial.\n-- \nDuy\n"},{"id":"149027","messageId":"AANLkTinQYfpWYJ8Lr3=VX+x2Xj+eQfYztT1GHh2AQKff@mail.gmail.com","threadId":"24851","inReplyTo":"AANLkTi=02T+60AjJON8MFt7DSbtA=bx39YKigZHq_b_u@mail.gmail.com","subject":"Re: [RFD PATCH 00/32] subtree clone v2","fromName":"Elijah Newren","fromEmail":"newren@gmail.com","sentAt":"2010-08-26T02:50:53Z","receivedAt":"2010-08-26T02:50:53Z","isPatch":true,"sender":{"key":"newren@gmail.com","avatar":"https://avatars.githubusercontent.com/u/5455730?v=4"},"body":"On Wed, Aug 25, 2010 at 4:13 PM, Nguyen Thai Ngoc Duy wrote:\n> On Wed, Aug 25, 2010 at 3:31 PM, Elijah Newren wrote:\n> I remember it!\n>\n> So there are two points: the \"no changes outside narrow tree from\n> \"you\"\" assumption and whether it is trivial to do a merge outside\n> narrow tree without whole trees.\n>\n> The first point. That assumption holds if user clones, starts working,\n> then does \"pull origin\". But if user merges another branch, say \"next\"\n> into \"master\", the common commit may be somewhere down in history and\n> there may be changes outside narrow tree from both \"you\" and\n> \"upstream\" (though user does not make those changes).\n\nYes, I agree; you can't merge with the data you have in such a case.\n\n> The second point (probably not needed if the above assumption is no\n> longer true, but I post anyway in case I forget it again). Without\n> whole trees, it's impossible to determine a trivial merge reliably. If\n> \"you\" adds a file and \"upstream\" adds another file, predecessor trees\n> will look different but merging them is trivial (at file level). If we\n> miss some trees that lead to those new files, the best thing we can do\n> is to claim it non-trivial.\n\nI'm not following this one. Could you provide more detail?\n"},{"id":"149029","messageId":"AANLkTi=3tGf=Gw6SnRLOSaURw7JX=qEPtLCzLhZN1D+y@mail.gmail.com","threadId":"24851","inReplyTo":"AANLkTinQYfpWYJ8Lr3=VX+x2Xj+eQfYztT1GHh2AQKff@mail.gmail.com","subject":"Re: [RFD PATCH 00/32] subtree clone v2","fromName":"Nguyen Thai Ngoc Duy","fromEmail":"pclouds@gmail.com","sentAt":"2010-08-26T03:52:05Z","receivedAt":"2010-08-26T03:52:05Z","isPatch":true,"sender":{"key":"pclouds@gmail.com","avatar":"https://avatars.githubusercontent.com/u/720?v=4"},"body":"On Thu, Aug 26, 2010 at 12:50 PM, Elijah Newren wrote:\n>> The second point (probably not needed if the above assumption is no\n>> longer true, but I post anyway in case I forget it again). Without\n>> whole trees, it's impossible to determine a trivial merge reliably. If\n>> \"you\" adds a file and \"upstream\" adds another file, predecessor trees\n>> will look different but merging them is trivial (at file level). If we\n>> miss some trees that lead to those new files, the best thing we can do\n>> is to claim it non-trivial.\n>\n> I'm not following this one.  Could you provide more detail?\n>\n\nLet's say in \"ancestor\" tree, we have\n\nt1/f0\nt2/t3/f0\n\nIn \"you\" tree, we have\n\nt1/f0\nt2/t3/f0\nt2/t3/f1\n\nIn \"upstream\" tree, we have\n\nt1/f0\nt2/t3/f0\nt2/t3/f2\n\nThe narrow tree is t1, so we the trees we have are toplevel tree and\nt1. If we have all trees, that should be a trivial merge, which\nresults in f0, f1 and f2 inside t2/t3. But we don't have t2 and t3\ntrees in narrow repo. When we traverse toplevel tree of \"ancestor\",\n\"you\" and \"upstream\", we can only see that t2 sha-1 is different. If\n\"t2\" is a file, not a tree, then we can conclude non-trivial here. And\nbecause we don't have t2, we can't descend to make better conclusion.\n-- \nDuy\n"},{"id":"149030","messageId":"AANLkTinH8UWbkCUBFa1yr1fOQJgs9rLOj_kcp2UR3iua@mail.gmail.com","threadId":"24851","inReplyTo":"AANLkTi=3tGf=Gw6SnRLOSaURw7JX=qEPtLCzLhZN1D+y@mail.gmail.com","subject":"Re: [RFD PATCH 00/32] subtree clone v2","fromName":"Elijah Newren","fromEmail":"newren@gmail.com","sentAt":"2010-08-26T04:39:42Z","receivedAt":"2010-08-26T04:39:42Z","isPatch":true,"sender":{"key":"newren@gmail.com","avatar":"https://avatars.githubusercontent.com/u/5455730?v=4"},"body":"On Wed, Aug 25, 2010 at 9:52 PM, Nguyen Thai Ngoc Duy wrote:\n> Let's say in \"ancestor\" tree, we have\n>\n> t1/f0\n> t2/t3/f0\n>\n> In \"you\" tree, we have\n>\n> t1/f0\n> t2/t3/f0\n> t2/t3/f1\n>\n> In \"upstream\" tree, we have\n>\n> t1/f0\n> t2/t3/f0\n> t2/t3/f2\n>\n> The narrow tree is t1, so we the trees we have are toplevel tree and\n\nHow would you get the \"you\" tree? Since you don't have the data for\nt2 or t2/t3, there's no way you can create and commit this tree.\n"},{"id":"149031","messageId":"AANLkTinZk-cF6ksN2UhAyHNwJAS=fm6eLKJEP+evdJGU@mail.gmail.com","threadId":"24851","inReplyTo":"AANLkTinH8UWbkCUBFa1yr1fOQJgs9rLOj_kcp2UR3iua@mail.gmail.com","subject":"Re: [RFD PATCH 00/32] subtree clone v2","fromName":"Nguyen Thai Ngoc Duy","fromEmail":"pclouds@gmail.com","sentAt":"2010-08-26T04:45:35Z","receivedAt":"2010-08-26T04:45:35Z","isPatch":true,"sender":{"key":"pclouds@gmail.com","avatar":"https://avatars.githubusercontent.com/u/720?v=4"},"body":"On Thu, Aug 26, 2010 at 2:39 PM, Elijah Newren wrote:\n> On Wed, Aug 25, 2010 at 9:52 PM, Nguyen Thai Ngoc Duy wrote:\n>> Let's say in \"ancestor\" tree, we have\n>>\n>> t1/f0\n>> t2/t3/f0\n>>\n>> In \"you\" tree, we have\n>>\n>> t1/f0\n>> t2/t3/f0\n>> t2/t3/f1\n>>\n>> In \"upstream\" tree, we have\n>>\n>> t1/f0\n>> t2/t3/f0\n>> t2/t3/f2\n>>\n>> The narrow tree is t1, so we the trees we have are toplevel tree and\n>\n> How would you get the \"you\" tree?  Since you don't have the data for\n> t2 or t2/t3, there's no way you can create and commit this tree.\n\n\"you\" tree does not create that file. It's been there. It's the\n\"ancestor\" tree that did not have that file. So if you happen to\nchoose an \"upstream\" tree that git-merge-base would give you that\nparticular \"ancestor\" tree, we're in this situation.\n-- \nDuy\n"}]}