{"thread":{"id":"803","subject":"[PATCH] diff: mode bits fixes","startedAt":"2005-06-01T18:38:07Z","lastAt":"2005-06-04T00:09:56Z","messageCount":20,"participants":["Junio C Hamano","Linus Torvalds","McMullan, Jason","Nicolas Pitre","Petr Baudis"],"isPatch":true,"patchVersion":1,"patchTotal":null},"messages":[{"id":"4383","messageId":"7vis0xkjn4.fsf@assigned-by-dhcp.cox.net","threadId":"803","inReplyTo":"7vy89ums2l.fsf@assigned-by-dhcp.cox.net","subject":"[PATCH] diff: mode bits fixes","fromName":"Junio C Hamano","fromEmail":"junkio@cox.net","sentAt":"2005-06-01T18:38:07Z","receivedAt":"2005-06-01T18:38:07Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"The core GIT repository has trees that record regular file mode\nin 0664 instead of normalized 0644 pattern.  Comparing such a\ntree with another tree that records the same file in 0644\npattern without content changes with git-diff-tree causes it to\nfeed otherwise unmodified pairs to the diff_change() routine,\nwhich triggers a sanity check routine and barfs.  This patch\nfixes the problem, along with the fix to another caller that\nuses unnormalized mode bits to call diff_change() routine in a\nsimilar way.\n\nWithout this patch, you will see \"fatal error\" from diff-tree\nwhen you run git-deltafy-script on the core GIT repository\nitself.\n\nSigned-off-by: Junio C Hamano <junkio@cox.net>\n---\n\n*** Linus, I decided to bite the bullet and audited the callers.\n*** There was only one that was not quite right, other than the\n*** diff-tree one.  Please disregard the one I sent last night\n*** which incorrectly said \"S_ISDIR() || S_ISREG()\".  Please\n*** also disregard the other one that silently ignores\n*** unmodified filepairs in the output routine.  Warning about\n*** callers should be the most appropriate action as this\n*** version does.\n\n diff.h       |    4 ++++\n diffcore.h   |    4 ----\n diff-files.c |    8 +++-----\n diff-tree.c  |    4 +++-\n diff.c       |   12 +++++++-----\n 5 files changed, 17 insertions(+), 15 deletions(-)\n\ndiff --git a/diff.h b/diff.h\n--- a/diff.h\n+++ b/diff.h\n@@ -4,6 +4,10 @@\n #ifndef DIFF_H\n #define DIFF_H\n \n+#define DIFF_FILE_CANON_MODE(mode) \\\n+\t(S_ISREG(mode) ? (S_IFREG | ce_permissions(mode)) : \\\n+\tS_ISLNK(mode) ? S_IFLNK : S_IFDIR)\n+\n extern void diff_addremove(int addremove,\n \t\t\t   unsigned mode,\n \t\t\t   const unsigned char *sha1,\ndiff --git a/diffcore.h b/diffcore.h\n--- a/diffcore.h\n+++ b/diffcore.h\n@@ -59,10 +59,6 @@ struct diff_filepair {\n \n #define DIFF_PAIR_MODE_CHANGED(p) ((p)->one->mode != (p)->two->mode)\n \n-#define DIFF_FILE_CANON_MODE(mode) \\\n-\t(S_ISREG(mode) ? (S_IFREG | ce_permissions(mode)) : \\\n-\tS_ISLNK(mode) ? S_IFLNK : S_IFDIR)\n-\n extern void diff_free_filepair(struct diff_filepair *);\n \n extern int diff_unmodified_pair(struct diff_filepair *);\ndiff --git a/diff-files.c b/diff-files.c\n--- a/diff-files.c\n+++ b/diff-files.c\n@@ -88,7 +88,7 @@ int main(int argc, const char **argv)\n \n \tfor (i = 0; i < entries; i++) {\n \t\tstruct stat st;\n-\t\tunsigned int oldmode, mode;\n+\t\tunsigned int oldmode;\n \t\tstruct cache_entry *ce = active_cache[i];\n \t\tint changed;\n \n@@ -116,10 +116,8 @@ int main(int argc, const char **argv)\n \t\t\tcontinue;\n \n \t\toldmode = ntohl(ce->ce_mode);\n-\t\tmode = (S_ISLNK(st.st_mode) ? S_IFLNK :\n-\t\t\tS_IFREG | ce_permissions(st.st_mode));\n-\n-\t\tshow_modified(oldmode, mode, ce->sha1, null_sha1,\n+\t\tshow_modified(oldmode, DIFF_FILE_CANON_MODE(st.st_mode),\n+\t\t\t      ce->sha1, null_sha1,\n \t\t\t      ce->name);\n \t}\n \tdiffcore_std((1 < argc) ? argv + 1 : NULL,\ndiff --git a/diff-tree.c b/diff-tree.c\n--- a/diff-tree.c\n+++ b/diff-tree.c\n@@ -44,10 +44,12 @@ static const unsigned char *extract(void\n \tint len = strlen(tree)+1;\n \tconst unsigned char *sha1 = tree + len;\n \tconst char *path = strchr(tree, ' ');\n+\tunsigned int mode;\n \n-\tif (!path || size < len + 20 || sscanf(tree, \"%o\", modep) != 1)\n+\tif (!path || size < len + 20 || sscanf(tree, \"%o\", &mode) != 1)\n \t\tdie(\"corrupt tree file\");\n \t*pathp = path+1;\n+\t*modep = DIFF_FILE_CANON_MODE(mode);\n \treturn sha1;\n }\n \ndiff --git a/diff.c b/diff.c\n--- a/diff.c\n+++ b/diff.c\n@@ -854,12 +854,14 @@ static void diff_resolve_rename_copy(voi\n \t\telse if (memcmp(p->one->sha1, p->two->sha1, 20) ||\n \t\t\t p->one->mode != p->two->mode)\n \t\t\tp->status = 'M';\n-\t\telse\n-\t\t\t/* this is a \"no-change\" entry.\n-\t\t\t * should not happen anymore.\n-\t\t\t * p->status = 'X';\n+\t\telse {\n+\t\t\t/* This is a \"no-change\" entry and should not\n+\t\t\t * happen anymore, but prepare for broken callers.\n \t\t\t */\n-\t\t\tdie(\"internal error in diffcore: unmodified entry remains\");\n+\t\t\terror(\"feeding unmodified %s to diffcore\",\n+\t\t\t      p->one->path);\n+\t\t\tp->status = 'X';\n+\t\t}\n \t}\n \tdiff_debug_queue(\"resolve-rename-copy done\", q);\n }\n------------\n\n"},{"id":"4424","messageId":"7v4qcg906f.fsf_-_@assigned-by-dhcp.cox.net","threadId":"803","inReplyTo":"7vis0xkjn4.fsf@assigned-by-dhcp.cox.net","subject":"[PATCH] Handle deltified object correctly in git-*-pull family.","fromName":"Junio C Hamano","fromEmail":"junkio@cox.net","sentAt":"2005-06-02T16:46:16Z","receivedAt":"2005-06-02T16:46:16Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"When a remote repository is deltified, we need to get the\nobjects that a deltified object we want to obtain is based upon.\nThe initial parts of each retrieved SHA1 file is inflated and\ninspected to see if it is deltified, and its base object is\nasked from the remote side when it is.  Since this partial\ninflation and inspection has a small performance hit, it can\noptionally be skipped by giving -d flag to git-*-pull commands.\nThis flag should be used only when the remote repository is\nknown to have no deltified objects.\n\nRsync transport does not have this problem since it fetches\neverything the remote side has.\n\nSigned-off-by: Junio C Hamano <junkio@cox.net>\n---\n\n*** Linus, this uses the new helper you wrote.  The interface is\n*** much more pleasant to use.\n\n Documentation/git-http-pull.txt  |    6 +++++-\n Documentation/git-local-pull.txt |    6 +++++-\n Documentation/git-rpull.txt      |    6 +++++-\n cache.h                          |    3 +++\n pull.h                           |    3 +++\n http-pull.c                      |    4 +++-\n local-pull.c                     |    4 +++-\n pull.c                           |    6 ++++++\n rpull.c                          |    4 +++-\n sha1_file.c                      |   40 ++++++++++++++++++++++++++++++++++++++\n 10 files changed, 76 insertions(+), 6 deletions(-)\n\ndiff --git a/Documentation/git-http-pull.txt b/Documentation/git-http-pull.txt\n--- a/Documentation/git-http-pull.txt\n+++ b/Documentation/git-http-pull.txt\n@@ -9,7 +9,7 @@ git-http-pull - Downloads a remote GIT r\n \n SYNOPSIS\n --------\n-'git-http-pull' [-c] [-t] [-a] [-v] commit-id url\n+'git-http-pull' [-c] [-t] [-a] [-v] [-d] commit-id url\n \n DESCRIPTION\n -----------\n@@ -21,6 +21,10 @@ Downloads a remote GIT repository via HT\n \tGet trees associated with the commit objects.\n -a::\n \tGet all the objects.\n+-d::\n+\tDo not check for delta base objects (use this option\n+\tonly when you know the remote repository is not\n+\tdeltified).\n -v::\n \tReport what is downloaded.\n \ndiff --git a/Documentation/git-local-pull.txt b/Documentation/git-local-pull.txt\n--- a/Documentation/git-local-pull.txt\n+++ b/Documentation/git-local-pull.txt\n@@ -9,7 +9,7 @@ git-local-pull - Duplicates another GIT \n \n SYNOPSIS\n --------\n-'git-local-pull' [-c] [-t] [-a] [-l] [-s] [-n] [-v] commit-id path\n+'git-local-pull' [-c] [-t] [-a] [-l] [-s] [-n] [-v] [-d] commit-id path\n \n DESCRIPTION\n -----------\n@@ -23,6 +23,10 @@ OPTIONS\n \tGet trees associated with the commit objects.\n -a::\n \tGet all the objects.\n+-d::\n+\tDo not check for delta base objects (use this option\n+\tonly when you know the remote repository is not\n+\tdeltified).\n -v::\n \tReport what is downloaded.\n \ndiff --git a/Documentation/git-rpull.txt b/Documentation/git-rpull.txt\n--- a/Documentation/git-rpull.txt\n+++ b/Documentation/git-rpull.txt\n@@ -10,7 +10,7 @@ git-rpull - Pulls from a remote reposito\n \n SYNOPSIS\n --------\n-'git-rpull' [-c] [-t] [-a] [-v] commit-id url\n+'git-rpull' [-c] [-t] [-a] [-d] [-v] commit-id url\n \n DESCRIPTION\n -----------\n@@ -25,6 +25,10 @@ OPTIONS\n \tGet trees associated with the commit objects.\n -a::\n \tGet all the objects.\n+-d::\n+\tDo not check for delta base objects (use this option\n+\tonly when you know the remote repository is not\n+\tdeltified).\n -v::\n \tReport what is downloaded.\n \ndiff --git a/cache.h b/cache.h\n--- a/cache.h\n+++ b/cache.h\n@@ -158,6 +158,9 @@ extern int write_sha1_file(void *buf, un\n \n extern int check_sha1_signature(unsigned char *sha1, void *buf, unsigned long size, const char *type);\n \n+extern int sha1_delta_base(const unsigned char *, unsigned char *);\n+\n+\n /* Read a tree into the cache */\n extern int read_tree(void *buffer, unsigned long size, int stage);\n \ndiff --git a/pull.h b/pull.h\n--- a/pull.h\n+++ b/pull.h\n@@ -13,6 +13,9 @@ extern int get_history;\n /** Set to fetch the trees in the commit history. **/\n extern int get_all;\n \n+/* Set to zero to skip the check for delta object base. */\n+extern int get_delta;\n+\n /* Set to be verbose */\n extern int get_verbosely;\n \ndiff --git a/http-pull.c b/http-pull.c\n--- a/http-pull.c\n+++ b/http-pull.c\n@@ -103,6 +103,8 @@ int main(int argc, char **argv)\n \t\t\tget_tree = 1;\n \t\t} else if (argv[arg][1] == 'c') {\n \t\t\tget_history = 1;\n+\t\t} else if (argv[arg][1] == 'd') {\n+\t\t\tget_delta = 0;\n \t\t} else if (argv[arg][1] == 'a') {\n \t\t\tget_all = 1;\n \t\t\tget_tree = 1;\n@@ -113,7 +115,7 @@ int main(int argc, char **argv)\n \t\targ++;\n \t}\n \tif (argc < arg + 2) {\n-\t\tusage(\"git-http-pull [-c] [-t] [-a] [-v] commit-id url\");\n+\t\tusage(\"git-http-pull [-c] [-t] [-a] [-d] [-v] commit-id url\");\n \t\treturn 1;\n \t}\n \tcommit_id = argv[arg];\ndiff --git a/local-pull.c b/local-pull.c\n--- a/local-pull.c\n+++ b/local-pull.c\n@@ -74,7 +74,7 @@ int fetch(unsigned char *sha1)\n }\n \n static const char *local_pull_usage = \n-\"git-local-pull [-c] [-t] [-a] [-l] [-s] [-n] [-v] commit-id path\";\n+\"git-local-pull [-c] [-t] [-a] [-l] [-s] [-n] [-v] [-d] commit-id path\";\n \n /* \n  * By default we only use file copy.\n@@ -92,6 +92,8 @@ int main(int argc, char **argv)\n \t\t\tget_tree = 1;\n \t\telse if (argv[arg][1] == 'c')\n \t\t\tget_history = 1;\n+\t\telse if (argv[arg][1] == 'd')\n+\t\t\tget_delta = 0;\n \t\telse if (argv[arg][1] == 'a') {\n \t\t\tget_all = 1;\n \t\t\tget_tree = 1;\ndiff --git a/pull.c b/pull.c\n--- a/pull.c\n+++ b/pull.c\n@@ -6,6 +6,7 @@\n \n int get_tree = 0;\n int get_history = 0;\n+int get_delta = 1;\n int get_all = 0;\n int get_verbosely = 0;\n static unsigned char current_commit_sha1[20];\n@@ -37,6 +38,11 @@ static int make_sure_we_have_it(const ch\n \tstatus = fetch(sha1);\n \tif (status && what)\n \t\treport_missing(what, sha1);\n+\tif (get_delta) {\n+\t\tchar delta_sha1[20];\n+\t\tif (sha1_delta_base(sha1, delta_sha1))\n+\t\t\tstatus = make_sure_we_have_it(what, delta_sha1);\n+\t}\n \treturn status;\n }\n \ndiff --git a/rpull.c b/rpull.c\n--- a/rpull.c\n+++ b/rpull.c\n@@ -27,6 +27,8 @@ int main(int argc, char **argv)\n \t\t\tget_tree = 1;\n \t\t} else if (argv[arg][1] == 'c') {\n \t\t\tget_history = 1;\n+\t\t} else if (argv[arg][1] == 'd') {\n+\t\t\tget_delta = 0;\n \t\t} else if (argv[arg][1] == 'a') {\n \t\t\tget_all = 1;\n \t\t\tget_tree = 1;\n@@ -37,7 +39,7 @@ int main(int argc, char **argv)\n \t\targ++;\n \t}\n \tif (argc < arg + 2) {\n-\t\tusage(\"git-rpull [-c] [-t] [-a] [-v] commit-id url\");\n+\t\tusage(\"git-rpull [-c] [-t] [-a] [-v] [-d] commit-id url\");\n \t\treturn 1;\n \t}\n \tcommit_id = argv[arg];\ndiff --git a/sha1_file.c b/sha1_file.c\n--- a/sha1_file.c\n+++ b/sha1_file.c\n@@ -347,6 +347,46 @@ void * unpack_sha1_file(void *map, unsig\n \treturn buf;\n }\n \n+int sha1_delta_base(const unsigned char *sha1, unsigned char *delta_sha1)\n+{\n+\tunsigned long mapsize, size;\n+\tvoid *map;\n+\tchar type[20];\n+\tchar buffer[200];\n+\tz_stream stream;\n+\tint ret, bytes, status;\n+\n+\tmap = map_sha1_file(sha1, &mapsize);\n+\tif (!map)\n+\t\treturn 0;\n+\tret = unpack_sha1_header(&stream, map, mapsize, buffer,\n+\t\t\t\t sizeof(buffer));\n+\tstatus = 0;\n+\n+\tif (ret < Z_OK ||\n+\t    sscanf(buffer, \"%10s %lu\", type, &size) != 2 ||\n+\t    strcmp(type, \"delta\"))\n+\t\tgoto out;\n+\tbytes = strlen(buffer) + 1;\n+\tif (size - bytes < 20)\n+\t\tgoto out;\n+\n+\tmemmove(buffer, buffer + bytes, stream.total_out - bytes);\n+\tbytes = stream.total_out - bytes;\n+\tif (bytes < 20 && ret == Z_OK) {\n+\t\tstream.next_out = buffer + bytes;\n+\t\tstream.avail_out = sizeof(buffer) - bytes;\n+\t\twhile (inflate(&stream, Z_FINISH) == Z_OK)\n+\t\t\t; /* nothing */\n+\t}\n+\tstatus = 1;\n+\tmemcpy(delta_sha1, buffer, 20);\n+ out:\n+\tinflateEnd(&stream);\n+\tmunmap(map, mapsize);\n+\treturn status;\n+}\n+\n void * read_sha1_file(const unsigned char *sha1, char *type, unsigned long *size)\n {\n \tunsigned long mapsize;\n------------\n\n"},{"id":"4425","messageId":"7vwtpc7lju.fsf_-_@assigned-by-dhcp.cox.net","threadId":"803","inReplyTo":"7vis0xkjn4.fsf@assigned-by-dhcp.cox.net","subject":"[PATCH] Use correct U*MAX.","fromName":"Junio C Hamano","fromEmail":"junkio@cox.net","sentAt":"2005-06-02T16:47:33Z","receivedAt":"2005-06-02T16:47:33Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"The largest \"unsigned long\" value is ULONG_MAX, not UINT_MAX.\n\nSigned-off-by: Junio C Hamano <junkio@cox.net>\n---\n\n*** Linus, you may be unable to spell, but I cannot type ;-).\n\n count-delta.c     |    4 ++--\n diff.c            |    4 ++--\n diffcore-break.c  |    2 +-\n diffcore-rename.c |    2 +-\n 4 files changed, 6 insertions(+), 6 deletions(-)\n\ndiff --git a/count-delta.c b/count-delta.c\n--- a/count-delta.c\n+++ b/count-delta.c\n@@ -46,7 +46,7 @@ unsigned long count_delta(void *delta_bu\n \n \t/* the smallest delta size possible is 6 bytes */\n \tif (delta_size < 6)\n-\t\treturn UINT_MAX;\n+\t\treturn ULONG_MAX;\n \n \tdata = delta_buf;\n \ttop = delta_buf + delta_size;\n@@ -83,7 +83,7 @@ unsigned long count_delta(void *delta_bu\n \n \t/* sanity check */\n \tif (data != top || out != dst_size)\n-\t\treturn UINT_MAX;\n+\t\treturn ULONG_MAX;\n \n \t/* delete size is what was _not_ copied from source.\n \t * edit size is that and literal additions.\ndiff --git a/diff.c b/diff.c\n--- a/diff.c\n+++ b/diff.c\n@@ -256,7 +256,7 @@ static struct sha1_size_cache *locate_si\n \t\tfirst = next+1;\n \t}\n \t/* not found */\n-\tif (size == UINT_MAX)\n+\tif (size == ULONG_MAX)\n \t\treturn NULL;\n \t/* insert to make it at \"first\" */\n \tif (sha1_size_cache_alloc <= sha1_size_cache_nr) {\n@@ -338,7 +338,7 @@ int diff_populate_filespec(struct diff_f\n \t\tstruct sha1_size_cache *e;\n \n \t\tif (size_only) {\n-\t\t\te = locate_size_cache(s->sha1, UINT_MAX);\n+\t\t\te = locate_size_cache(s->sha1, ULONG_MAX);\n \t\t\tif (e) {\n \t\t\t\ts->size = e->size;\n \t\t\t\treturn 0;\ndiff --git a/diffcore-break.c b/diffcore-break.c\n--- a/diffcore-break.c\n+++ b/diffcore-break.c\n@@ -63,7 +63,7 @@ static int very_different(struct diff_fi\n \t/* Estimate the edit size by interpreting delta. */\n \tdelta_size = count_delta(delta, delta_size);\n \tfree(delta);\n-\tif (delta_size == UINT_MAX)\n+\tif (delta_size == ULONG_MAX)\n \t\treturn 0; /* error in delta computation */\n \n \tif (base_size < delta_size)\ndiff --git a/diffcore-rename.c b/diffcore-rename.c\n--- a/diffcore-rename.c\n+++ b/diffcore-rename.c\n@@ -176,7 +176,7 @@ static int estimate_similarity(struct di\n \t/* Estimate the edit size by interpreting delta. */\n \tdelta_size = count_delta(delta, delta_size);\n \tfree(delta);\n-\tif (delta_size == UINT_MAX)\n+\tif (delta_size == ULONG_MAX)\n \t\treturn 0;\n \n \t/*\n------------\n\n"},{"id":"4426","messageId":"7vpsv47lh7.fsf_-_@assigned-by-dhcp.cox.net","threadId":"803","inReplyTo":"7vis0xkjn4.fsf@assigned-by-dhcp.cox.net","subject":"[PATCH] Find size of SHA1 object without inflating everything.","fromName":"Junio C Hamano","fromEmail":"junkio@cox.net","sentAt":"2005-06-02T16:49:08Z","receivedAt":"2005-06-02T16:49:08Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"This adds sha1_file_size() helper function and uses it in the\nrename/copy similarity estimator.  The helper function handles\ndeltified object as well.\n\nSigned-off-by: Junio C Hamano <junkio@cox.net>\n---\n\n*** Linus, this also uses the new helper you wrote.\n\n cache.h     |    2 +-\n diff.c      |   11 +++++---\n sha1_file.c |   78 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++\n 3 files changed, 85 insertions(+), 6 deletions(-)\n\ndiff --git a/cache.h b/cache.h\n--- a/cache.h\n+++ b/cache.h\n@@ -159,7 +159,7 @@ extern int write_sha1_file(void *buf, un\n extern int check_sha1_signature(unsigned char *sha1, void *buf, unsigned long size, const char *type);\n \n extern int sha1_delta_base(const unsigned char *, unsigned char *);\n-\n+extern int sha1_file_size(const unsigned char *, unsigned long *);\n \n /* Read a tree into the cache */\n extern int read_tree(void *buffer, unsigned long size, int stage);\ndiff --git a/diff.c b/diff.c\n--- a/diff.c\n+++ b/diff.c\n@@ -333,7 +333,6 @@ int diff_populate_filespec(struct diff_f\n \t\tclose(fd);\n \t}\n \telse {\n-\t\t/* We cannot do size only for SHA1 blobs */\n \t\tchar type[20];\n \t\tstruct sha1_size_cache *e;\n \n@@ -343,11 +342,13 @@ int diff_populate_filespec(struct diff_f\n \t\t\t\ts->size = e->size;\n \t\t\t\treturn 0;\n \t\t\t}\n+\t\t\tif (!sha1_file_size(s->sha1, &s->size))\n+\t\t\t\tlocate_size_cache(s->sha1, s->size);\n+\t\t}\n+\t\telse {\n+\t\t\ts->data = read_sha1_file(s->sha1, type, &s->size);\n+\t\t\ts->should_free = 1;\n \t\t}\n-\t\ts->data = read_sha1_file(s->sha1, type, &s->size);\n-\t\ts->should_free = 1;\n-\t\tif (s->data && size_only)\n-\t\t\tlocate_size_cache(s->sha1, s->size);\n \t}\n \treturn 0;\n }\ndiff --git a/sha1_file.c b/sha1_file.c\n--- a/sha1_file.c\n+++ b/sha1_file.c\n@@ -387,6 +387,84 @@ int sha1_delta_base(const unsigned char \n \treturn status;\n }\n \n+int sha1_file_size(const unsigned char *sha1, unsigned long *sizep)\n+{\n+\tunsigned long mapsize, size;\n+\tvoid *map;\n+\tchar type[20];\n+\tchar buffer[200];\n+\tz_stream stream;\n+\tint ret, bytes, status;\n+\n+\tmap = map_sha1_file(sha1, &mapsize);\n+\tif (!map)\n+\t\treturn 0;\n+\tret = unpack_sha1_header(&stream, map, mapsize, buffer,\n+\t\t\t\t sizeof(buffer));\n+\tstatus = -1;\n+\n+\tif (ret < Z_OK || sscanf(buffer, \"%10s %lu\", type, &size) != 2)\n+\t\tgoto out;\n+\tif (strcmp(type, \"delta\")) {\n+\t\t*sizep = size;\n+\t\tstatus = 0;\n+\t\tgoto out;\n+\t}\n+\n+\t/* We are dealing with delta object.  Inflated, the first 20\n+\t * bytes hold the base object SHA1, and delta data follows\n+\t * immediately after it.\n+\t */\n+\tbytes = strlen(buffer) + 1;\n+\tif (size < bytes + 6 + 20)\n+\t\tgoto out; /* the smallest delta size is 6 bytes */\n+\n+\tmemmove(buffer, buffer + bytes, stream.total_out - bytes);\n+\tbytes = stream.total_out - bytes;\n+\tif (bytes < sizeof(buffer) && ret == Z_OK) {\n+\t\tstream.next_out = buffer + bytes;\n+\t\tstream.avail_out = sizeof(buffer) - bytes;\n+\t\twhile (inflate(&stream, Z_FINISH) == Z_OK)\n+\t\t\t; /* nothing */\n+\t}\n+\n+\t/* We have read initial part of the delta, which starts at\n+\t * buffer+20.  Borrow code from patch-delta to read the\n+\t * result size.\n+\t */\n+\t{\n+\t\tconst unsigned char *data = buffer + 20;\n+\t\tunsigned char cmd;\n+\t\tint i;\n+\n+\t\t/* Skip over the source size; we are not interested in\n+\t\t * it and we cannot verify it because we do not want\n+\t\t * to read the base object.\n+\t\t */\n+\t\tcmd = *data++;\n+\t\twhile (cmd) {\n+\t\t\tif (cmd & 1)\n+\t\t\t\tdata++;\n+\t\t\tcmd >>= 1;\n+\t\t}\n+\t\t/* Read the result size */\n+\t\tsize = i = 0;\n+\t\tcmd = *data++;\n+\t\twhile (cmd) {\n+\t\t\tif (cmd & 1)\n+\t\t\t\tsize |= *data++ << i;\n+\t\t\ti += 8;\n+\t\t\tcmd >>= 1;\n+\t\t}\n+\t\t*sizep = size;\n+\t}\n+\tstatus = 0;\n+ out:\n+\tinflateEnd(&stream);\n+\tmunmap(map, mapsize);\n+\treturn status;\n+}\n+\n void * read_sha1_file(const unsigned char *sha1, char *type, unsigned long *size)\n {\n \tunsigned long mapsize;\n------------\n\n"},{"id":"4427","messageId":"Pine.LNX.4.58.0506020959250.1876@ppc970.osdl.org","threadId":"803","inReplyTo":"7v4qcg906f.fsf_-_@assigned-by-dhcp.cox.net","subject":"Re: [PATCH] Handle deltified object correctly in git-*-pull family.","fromName":"Linus Torvalds","fromEmail":"torvalds@osdl.org","sentAt":"2005-06-02T17:03:29Z","receivedAt":"2005-06-02T17:03:29Z","isPatch":true,"sender":{"key":"torvalds@linux-foundation.org","avatar":"https://avatars.githubusercontent.com/u/1024025?v=4"},"body":"\n\nOn Thu, 2 Jun 2005, Junio C Hamano wrote:\n> \n> *** Linus, this uses the new helper you wrote.  The interface is\n> *** much more pleasant to use.\n\nCan you update it for the fact that I split things up even more?\n\nIn particular, see commit 5180cacc202bb20b15981469487eb8d6b0509997: \"Split \nup unpack_sha1_file() some more\".\n\nYou should be able to unpack a delta with just something like this:\n\n\tint ret;\n\tz_stream stream;\n\tchar hdr[100], type[10];\n\tunsigned long size;\n\n\tret = unpack_sha1_header(&stream, map, mapsize, hdr, sizeof(hdr));\n\tif (ret < Z_OK)\n\t\treturn NULL;\n\tif (parse_sha1_header(hdr, type, &size) < 0)\n\t\treturn NULL;\n\tif (strcmp(\"delta\", type))\n\t\treturn NULL;\n\tbuffer = unpack_sha1_rest(&stream, hdr, size);\n\tif (!buffer)\n\t\treturn NULL;\n\t.. we now have the delta of size \"size\" in \"buffer\" ..\n\nie it now has proper helper functions for every single stage in unpacking\nan object.\n\n\t\tLinus\n"},{"id":"4428","messageId":"1117731830.32257.52.camel@jmcmullan.timesys","threadId":"803","inReplyTo":"7v4qcg906f.fsf_-_@assigned-by-dhcp.cox.net","subject":"Re: [PATCH] Handle deltified object correctly in git-*-pull family.","fromName":"McMullan, Jason","fromEmail":"jason.mcmullan@timesys.com","sentAt":"2005-06-02T17:03:49Z","receivedAt":"2005-06-02T17:03:49Z","isPatch":true,"sender":{"key":"jason.mcmullan@timesys.com","avatar":null},"body":"On Thu, 2005-06-02 at 09:46 -0700, Junio C Hamano wrote:\n> When a remote repository is deltified, we need to get the\n> objects that a deltified object we want to obtain is based upon.\n> The initial parts of each retrieved SHA1 file is inflated and\n> inspected to see if it is deltified, and its base object is\n> asked from the remote side when it is.  Since this partial\n> inflation and inspection has a small performance hit, it can\n> optionally be skipped by giving -d flag to git-*-pull commands.\n> This flag should be used only when the remote repository is\n> known to have no deltified objects.\n\nEww. Don't you want to attempt to get the referenced sha1 *before*\nyou stick the delta blob into the repository?\n\nOtherwise, if a transfer fails, there's no good way to recover your\ndatabase through the git-*pull interfaces, ie:\n\nTry to pull session 1:\n  pull tree a\n    pull delta b (references blob c)\n    pull blob c -> fails!\n\nTry to pull session 2:\n  pull tree a\n    delta b - Found in database!\n    ...\n\nOr do I not understand your code properly?\n\n(I'm working on a similar piece of code for my git-daemon protocol,\n which is why I'm worried about this issue)\n   \n-- \nJason McMullan <jason.mcmullan@timesys.com>\nTimeSys Corporation\n\n"},{"id":"4430","messageId":"7vhdgg7i2a.fsf@assigned-by-dhcp.cox.net","threadId":"803","inReplyTo":"1117731830.32257.52.camel@jmcmullan.timesys","subject":"Re: [PATCH] Handle deltified object correctly in git-*-pull family.","fromName":"Junio C Hamano","fromEmail":"junkio@cox.net","sentAt":"2005-06-02T18:02:53Z","receivedAt":"2005-06-02T18:02:53Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":">>>>> \"JM\" == McMullan, Jason <jason.mcmullan@timesys.com> writes:\n\nJM> Eww. Don't you want to attempt to get the referenced sha1 *before*\nJM> you stick the delta blob into the repository?\n\nThat issue crossed my mind, and I admit I haven't looked at the\nissues closely enough, but I suspect that it may not worth it\nwith the current pull.c structure.\n\nThe current pull code fetches and stores a commit object before\nit retrieves the tree object associate with it, and similarly a\ntree object before its subtree and blobs, which has the same\nissue.  Adding -r (recover) option to the pull family to not\ncheck for the existence of required object but its dependents\nwould be necessary if my suspition turns out to be correct, and\ndelta dependency should be handled the same way commit and tree\ndependencies are handled there.\n\n"},{"id":"4434","messageId":"7v3bs07fmu.fsf@assigned-by-dhcp.cox.net","threadId":"803","inReplyTo":"Pine.LNX.4.58.0506020959250.1876@ppc970.osdl.org","subject":"[PATCH] Handle deltified object correctly in git-*-pull family.","fromName":"Junio C Hamano","fromEmail":"junkio@cox.net","sentAt":"2005-06-02T18:55:21Z","receivedAt":"2005-06-02T18:55:21Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"When a remote repository is deltified, we need to get the\nobjects that a deltified object we want to obtain is based upon.\nThe initial parts of each retrieved SHA1 file is inflated and\ninspected to see if it is deltified, and its base object is\nasked from the remote side when it is.  Since this partial\ninflation and inspection has a small performance hit, it can\noptionally be skipped by giving -d flag to git-*-pull commands.\nThis flag should be used only when the remote repository is\nknown to have no deltified objects.\n\nRsync transport does not have this problem since it fetches\neverything the remote side has.\n\nSigned-off-by: Junio C Hamano <junkio@cox.net>\n---\n\n*** Now uses parse_sha1_header() and unpack_sha1_rest().  I\n*** decided not to make it the callers responsibility to check\n*** what we have already got and fixed unpack_sha1_rest() to\n*** avoid copying more than size bytes.\n\n Documentation/git-http-pull.txt  |    6 ++++-\n Documentation/git-local-pull.txt |    6 ++++-\n Documentation/git-rpull.txt      |    6 ++++-\n cache.h                          |    1 +\n pull.h                           |    3 +++\n http-pull.c                      |    4 +++-\n local-pull.c                     |    4 +++-\n pull.c                           |    7 ++++++\n rpull.c                          |    4 +++-\n sha1_file.c                      |   43 +++++++++++++++++++++++++++++++++++++-\n 10 files changed, 77 insertions(+), 7 deletions(-)\n\ndiff --git a/Documentation/git-http-pull.txt b/Documentation/git-http-pull.txt\n--- a/Documentation/git-http-pull.txt\n+++ b/Documentation/git-http-pull.txt\n@@ -9,7 +9,7 @@ git-http-pull - Downloads a remote GIT r\n \n SYNOPSIS\n --------\n-'git-http-pull' [-c] [-t] [-a] [-v] commit-id url\n+'git-http-pull' [-c] [-t] [-a] [-v] [-d] commit-id url\n \n DESCRIPTION\n -----------\n@@ -21,6 +21,10 @@ Downloads a remote GIT repository via HT\n \tGet trees associated with the commit objects.\n -a::\n \tGet all the objects.\n+-d::\n+\tDo not check for delta base objects (use this option\n+\tonly when you know the remote repository is not\n+\tdeltified).\n -v::\n \tReport what is downloaded.\n \ndiff --git a/Documentation/git-local-pull.txt b/Documentation/git-local-pull.txt\n--- a/Documentation/git-local-pull.txt\n+++ b/Documentation/git-local-pull.txt\n@@ -9,7 +9,7 @@ git-local-pull - Duplicates another GIT \n \n SYNOPSIS\n --------\n-'git-local-pull' [-c] [-t] [-a] [-l] [-s] [-n] [-v] commit-id path\n+'git-local-pull' [-c] [-t] [-a] [-l] [-s] [-n] [-v] [-d] commit-id path\n \n DESCRIPTION\n -----------\n@@ -23,6 +23,10 @@ OPTIONS\n \tGet trees associated with the commit objects.\n -a::\n \tGet all the objects.\n+-d::\n+\tDo not check for delta base objects (use this option\n+\tonly when you know the remote repository is not\n+\tdeltified).\n -v::\n \tReport what is downloaded.\n \ndiff --git a/Documentation/git-rpull.txt b/Documentation/git-rpull.txt\n--- a/Documentation/git-rpull.txt\n+++ b/Documentation/git-rpull.txt\n@@ -10,7 +10,7 @@ git-rpull - Pulls from a remote reposito\n \n SYNOPSIS\n --------\n-'git-rpull' [-c] [-t] [-a] [-v] commit-id url\n+'git-rpull' [-c] [-t] [-a] [-d] [-v] commit-id url\n \n DESCRIPTION\n -----------\n@@ -25,6 +25,10 @@ OPTIONS\n \tGet trees associated with the commit objects.\n -a::\n \tGet all the objects.\n+-d::\n+\tDo not check for delta base objects (use this option\n+\tonly when you know the remote repository is not\n+\tdeltified).\n -v::\n \tReport what is downloaded.\n \ndiff --git a/cache.h b/cache.h\n--- a/cache.h\n+++ b/cache.h\n@@ -153,6 +153,7 @@ extern char *sha1_file_name(const unsign\n extern void * map_sha1_file(const unsigned char *sha1, unsigned long *size);\n extern int unpack_sha1_header(z_stream *stream, void *map, unsigned long mapsize, void *buffer, unsigned long size);\n extern int parse_sha1_header(char *hdr, char *type, unsigned long *sizep);\n+extern int sha1_delta_base(const unsigned char *, unsigned char *);\n extern void * unpack_sha1_file(void *map, unsigned long mapsize, char *type, unsigned long *size);\n extern void * read_sha1_file(const unsigned char *sha1, char *type, unsigned long *size);\n extern int write_sha1_file(void *buf, unsigned long len, const char *type, unsigned char *return_sha1);\ndiff --git a/pull.h b/pull.h\n--- a/pull.h\n+++ b/pull.h\n@@ -13,6 +13,9 @@ extern int get_history;\n /** Set to fetch the trees in the commit history. **/\n extern int get_all;\n \n+/* Set to zero to skip the check for delta object base. */\n+extern int get_delta;\n+\n /* Set to be verbose */\n extern int get_verbosely;\n \ndiff --git a/http-pull.c b/http-pull.c\n--- a/http-pull.c\n+++ b/http-pull.c\n@@ -103,6 +103,8 @@ int main(int argc, char **argv)\n \t\t\tget_tree = 1;\n \t\t} else if (argv[arg][1] == 'c') {\n \t\t\tget_history = 1;\n+\t\t} else if (argv[arg][1] == 'd') {\n+\t\t\tget_delta = 0;\n \t\t} else if (argv[arg][1] == 'a') {\n \t\t\tget_all = 1;\n \t\t\tget_tree = 1;\n@@ -113,7 +115,7 @@ int main(int argc, char **argv)\n \t\targ++;\n \t}\n \tif (argc < arg + 2) {\n-\t\tusage(\"git-http-pull [-c] [-t] [-a] [-v] commit-id url\");\n+\t\tusage(\"git-http-pull [-c] [-t] [-a] [-d] [-v] commit-id url\");\n \t\treturn 1;\n \t}\n \tcommit_id = argv[arg];\ndiff --git a/local-pull.c b/local-pull.c\n--- a/local-pull.c\n+++ b/local-pull.c\n@@ -74,7 +74,7 @@ int fetch(unsigned char *sha1)\n }\n \n static const char *local_pull_usage = \n-\"git-local-pull [-c] [-t] [-a] [-l] [-s] [-n] [-v] commit-id path\";\n+\"git-local-pull [-c] [-t] [-a] [-l] [-s] [-n] [-v] [-d] commit-id path\";\n \n /* \n  * By default we only use file copy.\n@@ -92,6 +92,8 @@ int main(int argc, char **argv)\n \t\t\tget_tree = 1;\n \t\telse if (argv[arg][1] == 'c')\n \t\t\tget_history = 1;\n+\t\telse if (argv[arg][1] == 'd')\n+\t\t\tget_delta = 0;\n \t\telse if (argv[arg][1] == 'a') {\n \t\t\tget_all = 1;\n \t\t\tget_tree = 1;\ndiff --git a/pull.c b/pull.c\n--- a/pull.c\n+++ b/pull.c\n@@ -6,6 +6,7 @@\n \n int get_tree = 0;\n int get_history = 0;\n+int get_delta = 1;\n int get_all = 0;\n int get_verbosely = 0;\n static unsigned char current_commit_sha1[20];\n@@ -37,6 +38,12 @@ static int make_sure_we_have_it(const ch\n \tstatus = fetch(sha1);\n \tif (status && what)\n \t\treport_missing(what, sha1);\n+\tif (get_delta) {\n+\t\tchar delta_sha1[20];\n+\t\tstatus = sha1_delta_base(sha1, delta_sha1);\n+\t\tif (0 < status)\n+\t\t\tstatus = make_sure_we_have_it(what, delta_sha1);\n+\t}\n \treturn status;\n }\n \ndiff --git a/rpull.c b/rpull.c\n--- a/rpull.c\n+++ b/rpull.c\n@@ -27,6 +27,8 @@ int main(int argc, char **argv)\n \t\t\tget_tree = 1;\n \t\t} else if (argv[arg][1] == 'c') {\n \t\t\tget_history = 1;\n+\t\t} else if (argv[arg][1] == 'd') {\n+\t\t\tget_delta = 0;\n \t\t} else if (argv[arg][1] == 'a') {\n \t\t\tget_all = 1;\n \t\t\tget_tree = 1;\n@@ -37,7 +39,7 @@ int main(int argc, char **argv)\n \t\targ++;\n \t}\n \tif (argc < arg + 2) {\n-\t\tusage(\"git-rpull [-c] [-t] [-a] [-v] commit-id url\");\n+\t\tusage(\"git-rpull [-c] [-t] [-a] [-v] [-d] commit-id url\");\n \t\treturn 1;\n \t}\n \tcommit_id = argv[arg];\ndiff --git a/sha1_file.c b/sha1_file.c\n--- a/sha1_file.c\n+++ b/sha1_file.c\n@@ -325,7 +325,13 @@ void *unpack_sha1_rest(z_stream *stream,\n \tint bytes = strlen(buffer) + 1;\n \tchar *buf = xmalloc(1+size);\n \n-\tmemcpy(buf, buffer + bytes, stream->total_out - bytes);\n+\t/* (stream->total_out - bytes) is what we already have.  The\n+\t * caller could be asking for something smaller than that.\n+\t */\n+\tif (size < stream->total_out - bytes)\n+\t\tmemcpy(buf, buffer + bytes, size);\n+\telse\n+\t\tmemcpy(buf, buffer + bytes, stream->total_out - bytes);\n \tbytes = stream->total_out - bytes;\n \tif (bytes < size) {\n \t\tstream->next_out = buf + bytes;\n@@ -401,6 +407,41 @@ void * unpack_sha1_file(void *map, unsig\n \treturn unpack_sha1_rest(&stream, hdr, *size);\n }\n \n+int sha1_delta_base(const unsigned char *sha1, unsigned char *base_sha1)\n+{\n+\tint ret;\n+\tunsigned long mapsize, size;\n+\tvoid *map;\n+\tz_stream stream;\n+\tchar hdr[1024], type[20];\n+\tvoid *delta_data_head;\n+\n+\tmap = map_sha1_file(sha1, &mapsize);\n+\tif (!map)\n+\t\treturn -1;\n+\tret = unpack_sha1_header(&stream, map, mapsize, hdr, sizeof(hdr));\n+\tif (ret < Z_OK || parse_sha1_header(hdr, type, &size) < 0) {\n+\t\tret = -1;\n+\t\tgoto out;\n+\t}\n+\tif (strcmp(type, \"delta\")) {\n+\t\tret = 0;\n+\t\tgoto out;\n+\t}\n+\tdelta_data_head = unpack_sha1_rest(&stream, hdr, 20);\n+\tif (!delta_data_head) {\n+\t\tret = -1;\n+\t\tgoto out;\n+\t}\n+\tret = 1;\n+\tmemcpy(base_sha1, delta_data_head, 20);\n+\tfree(delta_data_head);\n+ out:\n+\tinflateEnd(&stream);\n+\tmunmap(map, mapsize);\n+\treturn ret;\n+}\n+\n void * read_sha1_file(const unsigned char *sha1, char *type, unsigned long *size)\n {\n \tunsigned long mapsize;\n------------\n\n"},{"id":"4435","messageId":"7vwtpc60z3.fsf_-_@assigned-by-dhcp.cox.net","threadId":"803","inReplyTo":"Pine.LNX.4.58.0506020959250.1876@ppc970.osdl.org","subject":"[PATCH] Find size of SHA1 object without inflating everything.","fromName":"Junio C Hamano","fromEmail":"junkio@cox.net","sentAt":"2005-06-02T18:57:20Z","receivedAt":"2005-06-02T18:57:20Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"This adds sha1_file_size() helper function and uses it in the\nrename/copy similarity estimator.  The helper function handles\ndeltified object as well.\n\nSigned-off-by: Junio C Hamano <junkio@cox.net>\n---\n\n*** The U*MAX fix patch from previous round still applies, so I am\n*** resending it to you.  This probably would depend on it, so\n*** please apply U*MAX fix patch before this one.\n\n cache.h     |    1 +\n diff.c      |   11 ++++++----\n sha1_file.c |   64 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++\n 3 files changed, 71 insertions(+), 5 deletions(-)\n\ndiff --git a/cache.h b/cache.h\n--- a/cache.h\n+++ b/cache.h\n@@ -154,6 +154,7 @@ extern void * map_sha1_file(const unsign\n extern int unpack_sha1_header(z_stream *stream, void *map, unsigned long mapsize, void *buffer, unsigned long size);\n extern int parse_sha1_header(char *hdr, char *type, unsigned long *sizep);\n extern int sha1_delta_base(const unsigned char *, unsigned char *);\n+extern int sha1_file_size(const unsigned char *, unsigned long *);\n extern void * unpack_sha1_file(void *map, unsigned long mapsize, char *type, unsigned long *size);\n extern void * read_sha1_file(const unsigned char *sha1, char *type, unsigned long *size);\n extern int write_sha1_file(void *buf, unsigned long len, const char *type, unsigned char *return_sha1);\ndiff --git a/diff.c b/diff.c\n--- a/diff.c\n+++ b/diff.c\n@@ -333,7 +333,6 @@ int diff_populate_filespec(struct diff_f\n \t\tclose(fd);\n \t}\n \telse {\n-\t\t/* We cannot do size only for SHA1 blobs */\n \t\tchar type[20];\n \t\tstruct sha1_size_cache *e;\n \n@@ -343,11 +342,13 @@ int diff_populate_filespec(struct diff_f\n \t\t\t\ts->size = e->size;\n \t\t\t\treturn 0;\n \t\t\t}\n+\t\t\tif (!sha1_file_size(s->sha1, &s->size))\n+\t\t\t\tlocate_size_cache(s->sha1, s->size);\n+\t\t}\n+\t\telse {\n+\t\t\ts->data = read_sha1_file(s->sha1, type, &s->size);\n+\t\t\ts->should_free = 1;\n \t\t}\n-\t\ts->data = read_sha1_file(s->sha1, type, &s->size);\n-\t\ts->should_free = 1;\n-\t\tif (s->data && size_only)\n-\t\t\tlocate_size_cache(s->sha1, s->size);\n \t}\n \treturn 0;\n }\ndiff --git a/sha1_file.c b/sha1_file.c\n--- a/sha1_file.c\n+++ b/sha1_file.c\n@@ -442,6 +442,70 @@ int sha1_delta_base(const unsigned char \n \treturn ret;\n }\n \n+int sha1_file_size(const unsigned char *sha1, unsigned long *sizep)\n+{\n+\tint ret, status;\n+\tunsigned long mapsize, size;\n+\tvoid *map;\n+\tz_stream stream;\n+\tchar hdr[1024], type[20];\n+\tvoid *delta_data_head;\n+\tconst unsigned char *data;\n+\tunsigned char cmd;\n+\tint i;\n+\n+\tmap = map_sha1_file(sha1, &mapsize);\n+\tif (!map)\n+\t\treturn -1;\n+\tret = unpack_sha1_header(&stream, map, mapsize, hdr, sizeof(hdr));\n+\tstatus = -1;\n+\tif (ret < Z_OK || parse_sha1_header(hdr, type, &size) < 0)\n+\t\tgoto out;\n+\tif (strcmp(type, \"delta\")) {\n+\t\t*sizep = size;\n+\t\tstatus = 0;\n+\t\tgoto out;\n+\t}\n+\n+\t/* We are dealing with delta object.  Inflated, the first 20\n+\t * bytes hold the base object SHA1, and delta data follows\n+\t * immediately after it.\n+\t */\n+\tdelta_data_head = unpack_sha1_rest(&stream, hdr, 200);\n+\n+\t/* The initial part of the delta starts at delta_data_head +\n+\t * 20.  Borrow code from patch-delta to read the result size.\n+\t */\n+\n+\tdata = delta_data_head + 20;\n+\n+\t/* Skip over the source size; we are not interested in\n+\t * it and we cannot verify it because we do not want\n+\t * to read the base object.\n+\t */\n+\tcmd = *data++;\n+\twhile (cmd) {\n+\t\tif (cmd & 1)\n+\t\t\tdata++;\n+\t\tcmd >>= 1;\n+\t}\n+\t/* Read the result size */\n+\tsize = i = 0;\n+\tcmd = *data++;\n+\twhile (cmd) {\n+\t\tif (cmd & 1)\n+\t\t\tsize |= *data++ << i;\n+\t\ti += 8;\n+\t\tcmd >>= 1;\n+\t}\n+\t*sizep = size;\n+\tstatus = 0;\n+ out:\n+\tinflateEnd(&stream);\n+\tmunmap(map, mapsize);\n+\treturn status;\n+}\n+\n void * read_sha1_file(const unsigned char *sha1, char *type, unsigned long *size)\n {\n \tunsigned long mapsize;\n------------\n\n"},{"id":"4454","messageId":"Pine.LNX.4.63.0506021713330.17354@localhost.localdomain","threadId":"803","inReplyTo":"7v3bs07fmu.fsf@assigned-by-dhcp.cox.net","subject":"Re: [PATCH] Handle deltified object correctly in git-*-pull family.","fromName":"Nicolas Pitre","fromEmail":"nico@cam.org","sentAt":"2005-06-02T21:31:20Z","receivedAt":"2005-06-02T21:31:20Z","isPatch":true,"sender":{"key":"nico@fluxnic.net","avatar":"https://avatars.githubusercontent.com/u/702790?v=4"},"body":"On Thu, 2 Jun 2005, Junio C Hamano wrote:\n\n> The initial parts of each retrieved SHA1 file is inflated and\n> inspected to see if it is deltified, and its base object is\n> asked from the remote side when it is.  Since this partial\n> inflation and inspection has a small performance hit, it can\n> optionally be skipped by giving -d flag to git-*-pull commands.\n\nIt is still way more expensive than it could.\n\n> diff --git a/sha1_file.c b/sha1_file.c\n> --- a/sha1_file.c\n> +++ b/sha1_file.c\n> @@ -325,7 +325,13 @@ void *unpack_sha1_rest(z_stream *stream,\n>  \tint bytes = strlen(buffer) + 1;\n>  \tchar *buf = xmalloc(1+size);\n>  \n> -\tmemcpy(buf, buffer + bytes, stream->total_out - bytes);\n> +\t/* (stream->total_out - bytes) is what we already have.  The\n> +\t * caller could be asking for something smaller than that.\n> +\t */\n> +\tif (size < stream->total_out - bytes)\n> +\t\tmemcpy(buf, buffer + bytes, size);\n> +\telse\n> +\t\tmemcpy(buf, buffer + bytes, stream->total_out - bytes);\n>  \tbytes = stream->total_out - bytes;\n>  \tif (bytes < size) {\n>  \t\tstream->next_out = buf + bytes;\n\nThis hunk is completely unneeded.\n\n> @@ -401,6 +407,41 @@ void * unpack_sha1_file(void *map, unsig\n>  \treturn unpack_sha1_rest(&stream, hdr, *size);\n>  }\n>  \n> +int sha1_delta_base(const unsigned char *sha1, unsigned char *base_sha1)\n> +{\n> +\tint ret;\n> +\tunsigned long mapsize, size;\n> +\tvoid *map;\n> +\tz_stream stream;\n> +\tchar hdr[1024], type[20];\n\nDon't make hdr 1024 bytes long.  If you do so unpack_sha1_header() will \nuncompress up to 1024 bytes which is way above required.  Instead, \nconsider a value of say 64 which is plenty sufficient (10 for the type \nstring, another 10 for the size, 20 for the reference sha1 and another \n10 for the beginning of the delta data that must include the size, and \nthe rest for good measure).\n\n> +\tvoid *delta_data_head;\n> +\n> +\tmap = map_sha1_file(sha1, &mapsize);\n> +\tif (!map)\n> +\t\treturn -1;\n> +\tret = unpack_sha1_header(&stream, map, mapsize, hdr, sizeof(hdr));\n> +\tif (ret < Z_OK || parse_sha1_header(hdr, type, &size) < 0) {\n> +\t\tret = -1;\n> +\t\tgoto out;\n> +\t}\n> +\tif (strcmp(type, \"delta\")) {\n> +\t\tret = 0;\n> +\t\tgoto out;\n> +\t}\n> +\tdelta_data_head = unpack_sha1_rest(&stream, hdr, 20);\n\nHere you don't need to call unpack_sha1_rest() at all which would call \nxmalloc and another memcpy needlessly.  Instead, just use:\n\n\tmemcpy(base_sha1, hdr + strlen(hdr) + 1, 20);\n\nand you're done.  No need to call an extra free() either.\n\nAnd maybe this function should live in delta.c instead?\n\n\nNicolas\n"},{"id":"4455","messageId":"Pine.LNX.4.63.0506021733520.17354@localhost.localdomain","threadId":"803","inReplyTo":"Pine.LNX.4.63.0506021713330.17354@localhost.localdomain","subject":"Re: [PATCH] Handle deltified object correctly in git-*-pull family.","fromName":"Nicolas Pitre","fromEmail":"nico@cam.org","sentAt":"2005-06-02T21:36:37Z","receivedAt":"2005-06-02T21:36:37Z","isPatch":true,"sender":{"key":"nico@fluxnic.net","avatar":"https://avatars.githubusercontent.com/u/702790?v=4"},"body":"On Thu, 2 Jun 2005, Nicolas Pitre wrote:\n\n> Here you don't need to call unpack_sha1_rest() at all which would call \n> xmalloc and another memcpy needlessly.  Instead, just use:\n> \n> \tmemcpy(base_sha1, hdr + strlen(hdr) + 1, 20);\n> \n> and you're done.  No need to call an extra free() either.\n> \n> And maybe this function should live in delta.c instead?\n\nForget about my suggestion of moving it to delta.c.  Since it assumes \nknowledge of the object header format it better stay close to the other \nfunctions in sha1_file.c.\n\n\nNicolas\n"},{"id":"4456","messageId":"Pine.LNX.4.58.0506021508020.1876@ppc970.osdl.org","threadId":"803","inReplyTo":"7vwtpc60z3.fsf_-_@assigned-by-dhcp.cox.net","subject":"Re: [PATCH] Find size of SHA1 object without inflating everything.","fromName":"Linus Torvalds","fromEmail":"torvalds@osdl.org","sentAt":"2005-06-02T22:10:11Z","receivedAt":"2005-06-02T22:10:11Z","isPatch":true,"sender":{"key":"torvalds@linux-foundation.org","avatar":"https://avatars.githubusercontent.com/u/1024025?v=4"},"body":"\n\nOn Thu, 2 Jun 2005, Junio C Hamano wrote:\n>  \n> +int sha1_file_size(const unsigned char *sha1, unsigned long *sizep)\n...\n> +\tret = unpack_sha1_header(&stream, map, mapsize, hdr, sizeof(hdr));\n...\n> +\tdelta_data_head = unpack_sha1_rest(&stream, hdr, 200);\n\nWhy do you do this? You've already unpacked 1024 bytes (including the\nheader), now you want to unpack at least 200 bytes past the header (which\nis less than what you already did.\n\nSo here \"unpack_sha1_rest()\" just ends up being a \"xmalloc + memcpy\", but \nsince you don't actually want the malloc (indeed, you're leaking it, as \nfar as I can tell), it seems to be all bad..\n\n\t\tLinus\n"},{"id":"4457","messageId":"7vu0kg1jxn.fsf_-_@assigned-by-dhcp.cox.net","threadId":"803","inReplyTo":"Pine.LNX.4.63.0506021733520.17354@localhost.localdomain","subject":"[PATCH 1/2] Handle deltified object correctly in git-*-pull family.","fromName":"Junio C Hamano","fromEmail":"junkio@cox.net","sentAt":"2005-06-02T22:19:00Z","receivedAt":"2005-06-02T22:19:00Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":">>>>> \"NP\" == Nicolas Pitre <nico@cam.org> writes:\n\n>> Here you don't need to call unpack_sha1_rest() at all which would call \n>> xmalloc and another memcpy needlessly.  Instead,...\n\nLike this...\n\n------------\nWhen a remote repository is deltified, we need to get the\nobjects that a deltified object we want to obtain is based upon.\nThe initial parts of each retrieved SHA1 file is inflated and\ninspected to see if it is deltified, and its base object is\nasked from the remote side when it is.  Since this partial\ninflation and inspection has a small performance hit, it can\noptionally be skipped by giving -d flag to git-*-pull commands.\nThis flag should be used only when the remote repository is\nknown to have no deltified objects.\n\nRsync transport does not have this problem since it fetches\neverything the remote side has.\n\nSigned-off-by: Junio C Hamano <junkio@cox.net>\n---\n\n*** Thanks and credits goes to Nico for suggesting not to\n*** use unpack_sha1_rest().\n\n Documentation/git-http-pull.txt  |    6 +++++-\n Documentation/git-local-pull.txt |    6 +++++-\n Documentation/git-rpull.txt      |    6 +++++-\n cache.h                          |    1 +\n pull.h                           |    3 +++\n http-pull.c                      |    4 +++-\n local-pull.c                     |    4 +++-\n pull.c                           |    7 +++++++\n rpull.c                          |    4 +++-\n sha1_file.c                      |   31 +++++++++++++++++++++++++++++++\n 10 files changed, 66 insertions(+), 6 deletions(-)\n\ndiff --git a/Documentation/git-http-pull.txt b/Documentation/git-http-pull.txt\n--- a/Documentation/git-http-pull.txt\n+++ b/Documentation/git-http-pull.txt\n@@ -9,7 +9,7 @@ git-http-pull - Downloads a remote GIT r\n \n SYNOPSIS\n --------\n-'git-http-pull' [-c] [-t] [-a] [-v] commit-id url\n+'git-http-pull' [-c] [-t] [-a] [-v] [-d] commit-id url\n \n DESCRIPTION\n -----------\n@@ -21,6 +21,10 @@ Downloads a remote GIT repository via HT\n \tGet trees associated with the commit objects.\n -a::\n \tGet all the objects.\n+-d::\n+\tDo not check for delta base objects (use this option\n+\tonly when you know the remote repository is not\n+\tdeltified).\n -v::\n \tReport what is downloaded.\n \ndiff --git a/Documentation/git-local-pull.txt b/Documentation/git-local-pull.txt\n--- a/Documentation/git-local-pull.txt\n+++ b/Documentation/git-local-pull.txt\n@@ -9,7 +9,7 @@ git-local-pull - Duplicates another GIT \n \n SYNOPSIS\n --------\n-'git-local-pull' [-c] [-t] [-a] [-l] [-s] [-n] [-v] commit-id path\n+'git-local-pull' [-c] [-t] [-a] [-l] [-s] [-n] [-v] [-d] commit-id path\n \n DESCRIPTION\n -----------\n@@ -23,6 +23,10 @@ OPTIONS\n \tGet trees associated with the commit objects.\n -a::\n \tGet all the objects.\n+-d::\n+\tDo not check for delta base objects (use this option\n+\tonly when you know the remote repository is not\n+\tdeltified).\n -v::\n \tReport what is downloaded.\n \ndiff --git a/Documentation/git-rpull.txt b/Documentation/git-rpull.txt\n--- a/Documentation/git-rpull.txt\n+++ b/Documentation/git-rpull.txt\n@@ -10,7 +10,7 @@ git-rpull - Pulls from a remote reposito\n \n SYNOPSIS\n --------\n-'git-rpull' [-c] [-t] [-a] [-v] commit-id url\n+'git-rpull' [-c] [-t] [-a] [-d] [-v] commit-id url\n \n DESCRIPTION\n -----------\n@@ -25,6 +25,10 @@ OPTIONS\n \tGet trees associated with the commit objects.\n -a::\n \tGet all the objects.\n+-d::\n+\tDo not check for delta base objects (use this option\n+\tonly when you know the remote repository is not\n+\tdeltified).\n -v::\n \tReport what is downloaded.\n \ndiff --git a/cache.h b/cache.h\n--- a/cache.h\n+++ b/cache.h\n@@ -153,6 +153,7 @@ extern char *sha1_file_name(const unsign\n extern void * map_sha1_file(const unsigned char *sha1, unsigned long *size);\n extern int unpack_sha1_header(z_stream *stream, void *map, unsigned long mapsize, void *buffer, unsigned long size);\n extern int parse_sha1_header(char *hdr, char *type, unsigned long *sizep);\n+extern int sha1_delta_base(const unsigned char *, unsigned char *);\n extern void * unpack_sha1_file(void *map, unsigned long mapsize, char *type, unsigned long *size);\n extern void * read_sha1_file(const unsigned char *sha1, char *type, unsigned long *size);\n extern int write_sha1_file(void *buf, unsigned long len, const char *type, unsigned char *return_sha1);\ndiff --git a/pull.h b/pull.h\n--- a/pull.h\n+++ b/pull.h\n@@ -13,6 +13,9 @@ extern int get_history;\n /** Set to fetch the trees in the commit history. **/\n extern int get_all;\n \n+/* Set to zero to skip the check for delta object base. */\n+extern int get_delta;\n+\n /* Set to be verbose */\n extern int get_verbosely;\n \ndiff --git a/http-pull.c b/http-pull.c\n--- a/http-pull.c\n+++ b/http-pull.c\n@@ -103,6 +103,8 @@ int main(int argc, char **argv)\n \t\t\tget_tree = 1;\n \t\t} else if (argv[arg][1] == 'c') {\n \t\t\tget_history = 1;\n+\t\t} else if (argv[arg][1] == 'd') {\n+\t\t\tget_delta = 0;\n \t\t} else if (argv[arg][1] == 'a') {\n \t\t\tget_all = 1;\n \t\t\tget_tree = 1;\n@@ -113,7 +115,7 @@ int main(int argc, char **argv)\n \t\targ++;\n \t}\n \tif (argc < arg + 2) {\n-\t\tusage(\"git-http-pull [-c] [-t] [-a] [-v] commit-id url\");\n+\t\tusage(\"git-http-pull [-c] [-t] [-a] [-d] [-v] commit-id url\");\n \t\treturn 1;\n \t}\n \tcommit_id = argv[arg];\ndiff --git a/local-pull.c b/local-pull.c\n--- a/local-pull.c\n+++ b/local-pull.c\n@@ -74,7 +74,7 @@ int fetch(unsigned char *sha1)\n }\n \n static const char *local_pull_usage = \n-\"git-local-pull [-c] [-t] [-a] [-l] [-s] [-n] [-v] commit-id path\";\n+\"git-local-pull [-c] [-t] [-a] [-l] [-s] [-n] [-v] [-d] commit-id path\";\n \n /* \n  * By default we only use file copy.\n@@ -92,6 +92,8 @@ int main(int argc, char **argv)\n \t\t\tget_tree = 1;\n \t\telse if (argv[arg][1] == 'c')\n \t\t\tget_history = 1;\n+\t\telse if (argv[arg][1] == 'd')\n+\t\t\tget_delta = 0;\n \t\telse if (argv[arg][1] == 'a') {\n \t\t\tget_all = 1;\n \t\t\tget_tree = 1;\ndiff --git a/pull.c b/pull.c\n--- a/pull.c\n+++ b/pull.c\n@@ -6,6 +6,7 @@\n \n int get_tree = 0;\n int get_history = 0;\n+int get_delta = 1;\n int get_all = 0;\n int get_verbosely = 0;\n static unsigned char current_commit_sha1[20];\n@@ -37,6 +38,12 @@ static int make_sure_we_have_it(const ch\n \tstatus = fetch(sha1);\n \tif (status && what)\n \t\treport_missing(what, sha1);\n+\tif (get_delta) {\n+\t\tchar delta_sha1[20];\n+\t\tstatus = sha1_delta_base(sha1, delta_sha1);\n+\t\tif (0 < status)\n+\t\t\tstatus = make_sure_we_have_it(what, delta_sha1);\n+\t}\n \treturn status;\n }\n \ndiff --git a/rpull.c b/rpull.c\n--- a/rpull.c\n+++ b/rpull.c\n@@ -27,6 +27,8 @@ int main(int argc, char **argv)\n \t\t\tget_tree = 1;\n \t\t} else if (argv[arg][1] == 'c') {\n \t\t\tget_history = 1;\n+\t\t} else if (argv[arg][1] == 'd') {\n+\t\t\tget_delta = 0;\n \t\t} else if (argv[arg][1] == 'a') {\n \t\t\tget_all = 1;\n \t\t\tget_tree = 1;\n@@ -37,7 +39,7 @@ int main(int argc, char **argv)\n \t\targ++;\n \t}\n \tif (argc < arg + 2) {\n-\t\tusage(\"git-rpull [-c] [-t] [-a] [-v] commit-id url\");\n+\t\tusage(\"git-rpull [-c] [-t] [-a] [-v] [-d] commit-id url\");\n \t\treturn 1;\n \t}\n \tcommit_id = argv[arg];\ndiff --git a/sha1_file.c b/sha1_file.c\n--- a/sha1_file.c\n+++ b/sha1_file.c\n@@ -401,6 +401,37 @@ void * unpack_sha1_file(void *map, unsig\n \treturn unpack_sha1_rest(&stream, hdr, *size);\n }\n \n+int sha1_delta_base(const unsigned char *sha1, unsigned char *base_sha1)\n+{\n+\tint ret;\n+\tunsigned long mapsize, size;\n+\tvoid *map;\n+\tz_stream stream;\n+\tchar hdr[64], type[20];\n+\tvoid *delta_data_head;\n+\n+\tmap = map_sha1_file(sha1, &mapsize);\n+\tif (!map)\n+\t\treturn -1;\n+\tret = unpack_sha1_header(&stream, map, mapsize, hdr, sizeof(hdr));\n+\tif (ret < Z_OK || parse_sha1_header(hdr, type, &size) < 0) {\n+\t\tret = -1;\n+\t\tgoto out;\n+\t}\n+\tif (strcmp(type, \"delta\")) {\n+\t\tret = 0;\n+\t\tgoto out;\n+\t}\n+\n+\tdelta_data_head = hdr + strlen(hdr) + 1;\n+\tret = 1;\n+\tmemcpy(base_sha1, delta_data_head, 20);\n+ out:\n+\tinflateEnd(&stream);\n+\tmunmap(map, mapsize);\n+\treturn ret;\n+}\n+\n void * read_sha1_file(const unsigned char *sha1, char *type, unsigned long *size)\n {\n \tunsigned long mapsize;\n------------\n\n"},{"id":"4458","messageId":"7voeao1juh.fsf_-_@assigned-by-dhcp.cox.net","threadId":"803","inReplyTo":"Pine.LNX.4.63.0506021733520.17354@localhost.localdomain","subject":"[PATCH 2/2] Find size of SHA1 object without inflating everything.","fromName":"Junio C Hamano","fromEmail":"junkio@cox.net","sentAt":"2005-06-02T22:20:54Z","receivedAt":"2005-06-02T22:20:54Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"This adds sha1_file_size() helper function and uses it in the\nrename/copy similarity estimator.  The helper function handles\ndeltified object as well.\n\nSigned-off-by: Junio C Hamano <junkio@cox.net>\n---\n\n*** Thanks and credits goes to Nico for suggesting not to\n*** use unpack_sha1_rest().\n\n cache.h     |    1 +\n diff.c      |   11 ++++++-----\n sha1_file.c |   60 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++\n 3 files changed, 67 insertions(+), 5 deletions(-)\n\ndiff --git a/cache.h b/cache.h\n--- a/cache.h\n+++ b/cache.h\n@@ -154,6 +154,7 @@ extern void * map_sha1_file(const unsign\n extern int unpack_sha1_header(z_stream *stream, void *map, unsigned long mapsize, void *buffer, unsigned long size);\n extern int parse_sha1_header(char *hdr, char *type, unsigned long *sizep);\n extern int sha1_delta_base(const unsigned char *, unsigned char *);\n+extern int sha1_file_size(const unsigned char *, unsigned long *);\n extern void * unpack_sha1_file(void *map, unsigned long mapsize, char *type, unsigned long *size);\n extern void * read_sha1_file(const unsigned char *sha1, char *type, unsigned long *size);\n extern int write_sha1_file(void *buf, unsigned long len, const char *type, unsigned char *return_sha1);\ndiff --git a/diff.c b/diff.c\n--- a/diff.c\n+++ b/diff.c\n@@ -333,7 +333,6 @@ int diff_populate_filespec(struct diff_f\n \t\tclose(fd);\n \t}\n \telse {\n-\t\t/* We cannot do size only for SHA1 blobs */\n \t\tchar type[20];\n \t\tstruct sha1_size_cache *e;\n \n@@ -343,11 +342,13 @@ int diff_populate_filespec(struct diff_f\n \t\t\t\ts->size = e->size;\n \t\t\t\treturn 0;\n \t\t\t}\n+\t\t\tif (!sha1_file_size(s->sha1, &s->size))\n+\t\t\t\tlocate_size_cache(s->sha1, s->size);\n+\t\t}\n+\t\telse {\n+\t\t\ts->data = read_sha1_file(s->sha1, type, &s->size);\n+\t\t\ts->should_free = 1;\n \t\t}\n-\t\ts->data = read_sha1_file(s->sha1, type, &s->size);\n-\t\ts->should_free = 1;\n-\t\tif (s->data && size_only)\n-\t\t\tlocate_size_cache(s->sha1, s->size);\n \t}\n \treturn 0;\n }\ndiff --git a/sha1_file.c b/sha1_file.c\n--- a/sha1_file.c\n+++ b/sha1_file.c\n@@ -432,6 +432,66 @@ int sha1_delta_base(const unsigned char \n \treturn ret;\n }\n \n+int sha1_file_size(const unsigned char *sha1, unsigned long *sizep)\n+{\n+\tint ret, status;\n+\tunsigned long mapsize, size;\n+\tvoid *map;\n+\tz_stream stream;\n+\tchar hdr[64], type[20];\n+\tconst unsigned char *data;\n+\tunsigned char cmd;\n+\tint i;\n+\n+\tmap = map_sha1_file(sha1, &mapsize);\n+\tif (!map)\n+\t\treturn -1;\n+\tret = unpack_sha1_header(&stream, map, mapsize, hdr, sizeof(hdr));\n+\tstatus = -1;\n+\tif (ret < Z_OK || parse_sha1_header(hdr, type, &size) < 0)\n+\t\tgoto out;\n+\tif (strcmp(type, \"delta\")) {\n+\t\t*sizep = size;\n+\t\tstatus = 0;\n+\t\tgoto out;\n+\t}\n+\n+\t/* We are dealing with a delta object.  Inflated, the first\n+\t * 20 bytes hold the base object SHA1, and delta data follows\n+\t * immediately after it.\n+\t *\n+\t * The initial part of the delta starts at delta_data_head +\n+\t * 20.  Borrow code from patch-delta to read the result size.\n+\t */\n+\tdata = hdr + strlen(hdr) + 1 + 20;\n+\n+\t/* Skip over the source size; we are not interested in\n+\t * it and we cannot verify it because we do not want\n+\t * to read the base object.\n+\t */\n+\tcmd = *data++;\n+\twhile (cmd) {\n+\t\tif (cmd & 1)\n+\t\t\tdata++;\n+\t\tcmd >>= 1;\n+\t}\n+\t/* Read the result size */\n+\tsize = i = 0;\n+\tcmd = *data++;\n+\twhile (cmd) {\n+\t\tif (cmd & 1)\n+\t\t\tsize |= *data++ << i;\n+\t\ti += 8;\n+\t\tcmd >>= 1;\n+\t}\n+\t*sizep = size;\n+\tstatus = 0;\n+ out:\n+\tinflateEnd(&stream);\n+\tmunmap(map, mapsize);\n+\treturn status;\n+}\n+\n void * read_sha1_file(const unsigned char *sha1, char *type, unsigned long *size)\n {\n \tunsigned long mapsize;\n------------\n\n"},{"id":"4461","messageId":"7vis0w1ime.fsf@assigned-by-dhcp.cox.net","threadId":"803","inReplyTo":"Pine.LNX.4.58.0506021508020.1876@ppc970.osdl.org","subject":"Re: [PATCH] Find size of SHA1 object without inflating everything.","fromName":"Junio C Hamano","fromEmail":"junkio@cox.net","sentAt":"2005-06-02T22:47:21Z","receivedAt":"2005-06-02T22:47:21Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":">>>>> \"LT\" == Linus Torvalds <torvalds@osdl.org> writes:\n\nLT> On Thu, 2 Jun 2005, Junio C Hamano wrote:\n>> \n>> +int sha1_file_size(const unsigned char *sha1, unsigned long *sizep)\nLT> ...\n>> +\tret = unpack_sha1_header(&stream, map, mapsize, hdr, sizeof(hdr));\nLT> ...\n>> +\tdelta_data_head = unpack_sha1_rest(&stream, hdr, 200);\n\nLT> Why do you do this?\n\nBecause I was not thinking when I wrote \"char hdr[1024]\".  \n\nNico pointed out the same problem and you have a fixed version\nof both in your mailbox.\n\n\n\n\n"},{"id":"4462","messageId":"Pine.LNX.4.58.0506021546550.1876@ppc970.osdl.org","threadId":"803","inReplyTo":"7vu0kg1jxn.fsf_-_@assigned-by-dhcp.cox.net","subject":"Re: [PATCH 1/2] Handle deltified object correctly in git-*-pull family.","fromName":"Linus Torvalds","fromEmail":"torvalds@osdl.org","sentAt":"2005-06-02T22:48:52Z","receivedAt":"2005-06-02T22:48:52Z","isPatch":true,"sender":{"key":"torvalds@linux-foundation.org","avatar":"https://avatars.githubusercontent.com/u/1024025?v=4"},"body":"\n\nOn Thu, 2 Jun 2005, Junio C Hamano wrote:\n> \n> Like this...\n\nYup. Applied, thanks,\n\n\t\tLinus\n"},{"id":"4518","messageId":"20050603230234.GC13093@pasky.ji.cz","threadId":"803","inReplyTo":"7vwtpc7lju.fsf_-_@assigned-by-dhcp.cox.net","subject":"Re: [PATCH] Use correct U*MAX.","fromName":"Petr Baudis","fromEmail":"pasky@ucw.cz","sentAt":"2005-06-03T23:02:37Z","receivedAt":"2005-06-03T23:02:37Z","isPatch":true,"sender":{"key":"pasky@ucw.cz","avatar":"https://avatars.githubusercontent.com/u/18439?v=4"},"body":"Dear diary, on Thu, Jun 02, 2005 at 06:47:33PM CEST, I got a letter\nwhere Junio C Hamano <junkio@cox.net> told me that...\n> The largest \"unsigned long\" value is ULONG_MAX, not UINT_MAX.\n> \n> Signed-off-by: Junio C Hamano <junkio@cox.net>\n> diff --git a/diff.c b/diff.c\n> --- a/diff.c\n> +++ b/diff.c\n> @@ -256,7 +256,7 @@ static struct sha1_size_cache *locate_si\n>  \t\tfirst = next+1;\n>  \t}\n>  \t/* not found */\n> -\tif (size == UINT_MAX)\n> +\tif (size == ULONG_MAX)\n>  \t\treturn NULL;\n>  \t/* insert to make it at \"first\" */\n>  \tif (sha1_size_cache_alloc <= sha1_size_cache_nr) {\n> @@ -338,7 +338,7 @@ int diff_populate_filespec(struct diff_f\n>  \t\tstruct sha1_size_cache *e;\n>  \n>  \t\tif (size_only) {\n> -\t\t\te = locate_size_cache(s->sha1, UINT_MAX);\n> +\t\t\te = locate_size_cache(s->sha1, ULONG_MAX);\n>  \t\t\tif (e) {\n>  \t\t\t\ts->size = e->size;\n>  \t\t\t\treturn 0;\n\nThis one still applies, but it might be better to get rid of it\naltogether, like...\n\n[PATCH] Kill UINT_MAX usage in locate_size_cache()\n\nUse -1 instead of UINT_MAX to indicate that locate_size_cache() should\ndo only the lookup and not create new stuff in case the looked up hash\nwas not found.\n\nSigned-off-by: Petr Baudis <pasky@ucw.cz>\n\ndiff --git a/diff.c b/diff.c\n--- a/diff.c\n+++ b/diff.c\n@@ -256,7 +256,7 @@ static struct sha1_size_cache *locate_si\n \t\tfirst = next+1;\n \t}\n \t/* not found */\n-\tif (size == UINT_MAX)\n+\tif (size == -1)\n \t\treturn NULL;\n \t/* insert to make it at \"first\" */\n \tif (sha1_size_cache_alloc <= sha1_size_cache_nr) {\n@@ -337,7 +337,7 @@ int diff_populate_filespec(struct diff_f\n \t\tstruct sha1_size_cache *e;\n \n \t\tif (size_only) {\n-\t\t\te = locate_size_cache(s->sha1, UINT_MAX);\n+\t\t\te = locate_size_cache(s->sha1, -1);\n \t\t\tif (e) {\n \t\t\t\ts->size = e->size;\n \t\t\t\treturn 0;\n\n|\n-- \n\t\t\t\tPetr \"Pasky\" Baudis\nStuff: http://pasky.or.cz/\nC++: an octopus made by nailing extra legs onto a dog. -- Steve Taylor\n"},{"id":"4519","messageId":"7v4qcfko0q.fsf@assigned-by-dhcp.cox.net","threadId":"803","inReplyTo":"20050603230234.GC13093@pasky.ji.cz","subject":"Re: [PATCH] Use correct U*MAX.","fromName":"Junio C Hamano","fromEmail":"junkio@cox.net","sentAt":"2005-06-03T23:40:21Z","receivedAt":"2005-06-03T23:40:21Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"I'd rather see it use the correct U*MAX.\n\n"},{"id":"4520","messageId":"20050604000042.GG13093@pasky.ji.cz","threadId":"803","inReplyTo":"7v4qcfko0q.fsf@assigned-by-dhcp.cox.net","subject":"Re: [PATCH] Use correct U*MAX.","fromName":"Petr Baudis","fromEmail":"pasky@ucw.cz","sentAt":"2005-06-04T00:00:42Z","receivedAt":"2005-06-04T00:00:42Z","isPatch":true,"sender":{"key":"pasky@ucw.cz","avatar":"https://avatars.githubusercontent.com/u/18439?v=4"},"body":"Dear diary, on Sat, Jun 04, 2005 at 01:40:21AM CEST, I got a letter\nwhere Junio C Hamano <junkio@cox.net> told me that...\n> I'd rather see it use the correct U*MAX.\n\nCare to elaborate? It doesn't make sense to me to use any U*MAX stuff\nthere whatsoever. (And how do you define the \"correct\" U*MAX anyway?)\n\nP.S.: Could you please always keep at least some context in the mail?\nThanks.\n\n-- \n\t\t\t\tPetr \"Pasky\" Baudis\nStuff: http://pasky.or.cz/\nC++: an octopus made by nailing extra legs onto a dog. -- Steve Taylor\n"},{"id":"4521","messageId":"7vslzzj82z.fsf@assigned-by-dhcp.cox.net","threadId":"803","inReplyTo":"20050604000042.GG13093@pasky.ji.cz","subject":"Re: [PATCH] Use correct U*MAX.","fromName":"Junio C Hamano","fromEmail":"junkio@cox.net","sentAt":"2005-06-04T00:09:56Z","receivedAt":"2005-06-04T00:09:56Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":">>>>> \"PB\" == Petr Baudis <pasky@ucw.cz> writes:\n\n>> I'd rather see it use the correct U*MAX.\n\nPB> Care to elaborate? It doesn't make sense to me to use any U*MAX stuff\nPB> there whatsoever. (And how do you define the \"correct\" U*MAX anyway?)\n\nIt just feels wrong to spell a parameter to the function\nlocate_size_cache() \"-1\" when I know the argument it expects is\nof type unsigned long.  And correct U*MAX for that case is\nobviously ULONG_MAX, _assuming_ that you agree to the function's\n(unwritten) calling convention of \"passing the largest possible\nvalue to me means 'do not create', not 'you are telling me that\nsha1 is such a large file'\".\n\nIf you feel strongly about that calling convention, you could\nrewrite it to take the third argument \"int do_not_create\" and\npass that information separately, which is conceptually cleaner.\nI just did not think that was worth it for such an internal\nhelper when I wrote it.\n\n"}]}