{"thread":{"id":"8771","subject":"Applying patches in a directory that isn't a repository","startedAt":"2007-06-29T06:07:31Z","lastAt":"2007-06-30T04:18:11Z","messageCount":9,"participants":["Geoff Russell","Junio C Hamano","しらいしななこ"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"46021","messageId":"93c3eada0706282307i7a22bd27w6ca10839d36ea4eb@mail.gmail.com","threadId":"8771","inReplyTo":"93c3eada0706280153w1898be80u7785ef2b2b1dd188@mail.gmail.com","subject":"Applying patches in a directory that isn't a repository","fromName":"Geoff Russell","fromEmail":"geoffrey.russell@gmail.com","sentAt":"2007-06-29T06:07:31Z","receivedAt":"2007-06-29T06:07:31Z","isPatch":false,"sender":{"key":"geoffrey.russell@gmail.com","avatar":"https://gravatar.com/avatar/c30f497ccfa6bf06d86f30bd2ba092a2dd124c61c6bc902f7cb5c3f6486947de?d=mp&s=160"},"body":"Hi,\n\nI'm trying to use git for software distribution.\n\nThe system is a directory of about 450mb comprising binaries,\ndatafiles and scripts. The\nfull repository is about 1.1gb.   The users don't really need .git\nexcept when we\nlogin to pull upgrades for them. It would be nicer not to have the\n.git directory and\njust distribute patches which we would apply for our users.\n\nBut \"git am\" needs (AFAIK) a full repository. Is there a way to apply\na patch without\n.git being present?\n\nI can see the advantages of our users having the full repository, so maybe\nwe should just distribute by DVD (currently we use CDs) and forget\nabout it, but for\ndownload purposes, 450mb is nicer than  1.1gb.\n\nCheers,\nGeoff Russell\n"},{"id":"46022","messageId":"7v3b0bi88r.fsf@assigned-by-dhcp.pobox.com","threadId":"8771","inReplyTo":"93c3eada0706282307i7a22bd27w6ca10839d36ea4eb@mail.gmail.com","subject":"Re: Applying patches in a directory that isn't a repository","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2007-06-29T06:29:24Z","receivedAt":"2007-06-29T06:29:24Z","isPatch":false,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"\"Geoff Russell\" <geoffrey.russell@gmail.com> writes:\n\n> But \"git am\" needs (AFAIK) a full repository. Is there a way to apply\n> a patch without\n> .git being present?\n\nIf the recipient does not have a git repository, there is no\npoint using \"git am\", as it is about making commits out of\ne-mails.\n\ngit-apply acts as a plain \"patch applicator\".\n\nAnother possibility is to use \"GNU patch\", with -p1.  That\nhistorically has been how the patches were applied, and the\noutput from git-diff is designed to be very compatible with it,\nunless you choose to use more advanced features that \"GNU patch\"\ndoes not support yet (notably, renames, mode changes, and binary\ndiffs).\n"},{"id":"46024","messageId":"11830989612019-git-send-email-gitster@pobox.com","threadId":"8771","inReplyTo":"7v3b0bi88r.fsf@assigned-by-dhcp.pobox.com","subject":"[PATCH 1/4] diffcore_count_changes: pass diffcore_filespec","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2007-06-29T06:35:58Z","receivedAt":"2007-06-29T06:35:58Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"We may want to use richer information on the data we are dealing\nwith in this function, so instead of passing a buffer address\nand length, just pass the diffcore_filespec structure.  Existing\ncallers always call this function with parameters taken from a\nfilespec anyway, so there is no functionality changes.\n\nSigned-off-by: Junio C Hamano <gitster@pobox.com>\n---\n diffcore-break.c  |    3 +--\n diffcore-delta.c  |    8 ++++----\n diffcore-rename.c |    3 +--\n diffcore.h        |    4 ++--\n 4 files changed, 8 insertions(+), 10 deletions(-)\n\ndiff --git a/diffcore-break.c b/diffcore-break.c\nindex 9c19b8c..ae8a7d0 100644\n--- a/diffcore-break.c\n+++ b/diffcore-break.c\n@@ -66,8 +66,7 @@ static int should_break(struct diff_filespec *src,\n \tif (base_size < MINIMUM_BREAK_SIZE)\n \t\treturn 0; /* we do not break too small filepair */\n \n-\tif (diffcore_count_changes(src->data, src->size,\n-\t\t\t\t   dst->data, dst->size,\n+\tif (diffcore_count_changes(src, dst,\n \t\t\t\t   NULL, NULL,\n \t\t\t\t   0,\n \t\t\t\t   &src_copied, &literal_added))\ndiff --git a/diffcore-delta.c b/diffcore-delta.c\nindex 7338a40..0e1fae7 100644\n--- a/diffcore-delta.c\n+++ b/diffcore-delta.c\n@@ -156,8 +156,8 @@ static struct spanhash_top *hash_chars(unsigned char *buf, unsigned int sz)\n \treturn hash;\n }\n \n-int diffcore_count_changes(void *src, unsigned long src_size,\n-\t\t\t   void *dst, unsigned long dst_size,\n+int diffcore_count_changes(struct diff_filespec *src,\n+\t\t\t   struct diff_filespec *dst,\n \t\t\t   void **src_count_p,\n \t\t\t   void **dst_count_p,\n \t\t\t   unsigned long delta_limit,\n@@ -172,14 +172,14 @@ int diffcore_count_changes(void *src, unsigned long src_size,\n \tif (src_count_p)\n \t\tsrc_count = *src_count_p;\n \tif (!src_count) {\n-\t\tsrc_count = hash_chars(src, src_size);\n+\t\tsrc_count = hash_chars(src->data, src->size);\n \t\tif (src_count_p)\n \t\t\t*src_count_p = src_count;\n \t}\n \tif (dst_count_p)\n \t\tdst_count = *dst_count_p;\n \tif (!dst_count) {\n-\t\tdst_count = hash_chars(dst, dst_size);\n+\t\tdst_count = hash_chars(dst->data, dst->size);\n \t\tif (dst_count_p)\n \t\t\t*dst_count_p = dst_count;\n \t}\ndiff --git a/diffcore-rename.c b/diffcore-rename.c\nindex 79c984c..cb22736 100644\n--- a/diffcore-rename.c\n+++ b/diffcore-rename.c\n@@ -189,8 +189,7 @@ static int estimate_similarity(struct diff_filespec *src,\n \n \tdelta_limit = (unsigned long)\n \t\t(base_size * (MAX_SCORE-minimum_score) / MAX_SCORE);\n-\tif (diffcore_count_changes(src->data, src->size,\n-\t\t\t\t   dst->data, dst->size,\n+\tif (diffcore_count_changes(src, dst,\n \t\t\t\t   &src->cnt_data, &dst->cnt_data,\n \t\t\t\t   delta_limit,\n \t\t\t\t   &src_copied, &literal_added))\ndiff --git a/diffcore.h b/diffcore.h\nindex 7b9294e..990dec5 100644\n--- a/diffcore.h\n+++ b/diffcore.h\n@@ -103,8 +103,8 @@ void diff_debug_queue(const char *, struct diff_queue_struct *);\n #define diff_debug_queue(a,b) do {} while(0)\n #endif\n \n-extern int diffcore_count_changes(void *src, unsigned long src_size,\n-\t\t\t\t  void *dst, unsigned long dst_size,\n+extern int diffcore_count_changes(struct diff_filespec *src,\n+\t\t\t\t  struct diff_filespec *dst,\n \t\t\t\t  void **src_count_p,\n \t\t\t\t  void **dst_count_p,\n \t\t\t\t  unsigned long delta_limit,\n-- \n1.5.2.2.1414.g1e7d9\n"},{"id":"46025","messageId":"11830989621033-git-send-email-gitster@pobox.com","threadId":"8771","inReplyTo":"7v3b0bi88r.fsf@assigned-by-dhcp.pobox.com","subject":"[PATCH 2/4] diffcore_filespec: add is_binary","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2007-06-29T06:35:59Z","receivedAt":"2007-06-29T06:35:59Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"diffcore-break and diffcore-rename would want to behave slightly\ndifferently depending on the binary-ness of the data, so add one\nbit to the filespec, as the structure is now passed down to\ndiffcore_count_changes() function.\n\nSigned-off-by: Junio C Hamano <gitster@pobox.com>\n---\n diff.c     |   16 ++++++++++++++++\n diffcore.h |    1 +\n 2 files changed, 17 insertions(+), 0 deletions(-)\n\ndiff --git a/diff.c b/diff.c\nindex 9938969..74c1198 100644\n--- a/diff.c\n+++ b/diff.c\n@@ -3005,6 +3005,22 @@ void diffcore_std(struct diff_options *options)\n {\n \tif (options->quiet)\n \t\treturn;\n+\n+\t/*\n+\t * break/rename count similarity differently depending on\n+\t * the binary-ness.\n+\t */\n+\tif ((options->break_opt != -1) || (options->detect_rename)) {\n+\t\tstruct diff_queue_struct *q = &diff_queued_diff;\n+\t\tint i;\n+\n+\t\tfor (i = 0; i < q->nr; i++) {\n+\t\t\tstruct diff_filepair *p = q->queue[i];\n+\t\t\tp->one->is_binary = file_is_binary(p->one);\n+\t\t\tp->two->is_binary = file_is_binary(p->two);\n+\t\t}\n+\t}\n+\n \tif (options->break_opt != -1)\n \t\tdiffcore_break(options->break_opt);\n \tif (options->detect_rename)\ndiff --git a/diffcore.h b/diffcore.h\nindex 990dec5..0c8abb5 100644\n--- a/diffcore.h\n+++ b/diffcore.h\n@@ -37,6 +37,7 @@ struct diff_filespec {\n #define DIFF_FILE_VALID(spec) (((spec)->mode) != 0)\n \tunsigned should_free : 1; /* data should be free()'ed */\n \tunsigned should_munmap : 1; /* data should be munmap()'ed */\n+\tunsigned is_binary : 1; /* data should be considered \"binary\" */\n };\n \n extern struct diff_filespec *alloc_filespec(const char *);\n-- \n1.5.2.2.1414.g1e7d9\n"},{"id":"46026","messageId":"11830989623983-git-send-email-gitster@pobox.com","threadId":"8771","inReplyTo":"7v3b0bi88r.fsf@assigned-by-dhcp.pobox.com","subject":"[PATCH 3/4] diffcore-delta.c: update the comment on the algorithm.","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2007-06-29T06:36:00Z","receivedAt":"2007-06-29T06:36:00Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"The comment at the top of the file described an old algorithm\nthat was neutral to text/binary differences (it hashed sliding\nwindow of N-byte sequences and counted overlaps), but long time\nago we switched to a new heuristics that are more suitable for\nline oriented (read: text) files that are much faster.\n\nSigned-off-by: Junio C Hamano <gitster@pobox.com>\n---\n diffcore-delta.c |   21 +++++++++------------\n 1 files changed, 9 insertions(+), 12 deletions(-)\n\ndiff --git a/diffcore-delta.c b/diffcore-delta.c\nindex 0e1fae7..7116df0 100644\n--- a/diffcore-delta.c\n+++ b/diffcore-delta.c\n@@ -5,23 +5,20 @@\n /*\n  * Idea here is very simple.\n  *\n- * We have total of (sz-N+1) N-byte overlapping sequences in buf whose\n- * size is sz.  If the same N-byte sequence appears in both source and\n- * destination, we say the byte that starts that sequence is shared\n- * between them (i.e. copied from source to destination).\n+ * Almost all data we are interested in are text, but sometimes we have\n+ * to deal with binary data.  So we cut them into chunks delimited by\n+ * LF byte, or 64-byte sequence, whichever comes first, and hash them.\n  *\n- * For each possible N-byte sequence, if the source buffer has more\n- * instances of it than the destination buffer, that means the\n- * difference are the number of bytes not copied from source to\n- * destination.  If the counts are the same, everything was copied\n- * from source to destination.  If the destination has more,\n- * everything was copied, and destination added more.\n+ * For those chunks, if the source buffer has more instances of it\n+ * than the destination buffer, that means the difference are the\n+ * number of bytes not copied from source to destination.  If the\n+ * counts are the same, everything was copied from source to\n+ * destination.  If the destination has more, everything was copied,\n+ * and destination added more.\n  *\n  * We are doing an approximation so we do not really have to waste\n  * memory by actually storing the sequence.  We just hash them into\n  * somewhere around 2^16 hashbuckets and count the occurrences.\n- *\n- * The length of the sequence is arbitrarily set to 8 for now.\n  */\n \n /* Wild guess at the initial hash size */\n-- \n1.5.2.2.1414.g1e7d9\n"},{"id":"46027","messageId":"1183098962312-git-send-email-gitster@pobox.com","threadId":"8771","inReplyTo":"7v3b0bi88r.fsf@assigned-by-dhcp.pobox.com","subject":"[PATCH 4/4] diffcore-delta.c: Ignore CR in CRLF for text files","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2007-06-29T06:36:01Z","receivedAt":"2007-06-29T06:36:01Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"This ignores CR byte in CRLF sequence in text file when\ncomputing similarity of two blobs.\n\nUsually this should not matter as nobody sane would be checking\nin a file with CRLF line endings to the repository (they would\nuse autocrlf so that the repository copy would have LF line\nendings).\n\nSigned-off-by: Junio C Hamano <gitster@pobox.com>\n---\n diffcore-delta.c       |   14 +++++++++++---\n t/t0022-crlf-rename.sh |   33 +++++++++++++++++++++++++++++++++\n 2 files changed, 44 insertions(+), 3 deletions(-)\n create mode 100755 t/t0022-crlf-rename.sh\n\ndiff --git a/diffcore-delta.c b/diffcore-delta.c\nindex 7116df0..a038b16 100644\n--- a/diffcore-delta.c\n+++ b/diffcore-delta.c\n@@ -122,11 +122,14 @@ static struct spanhash_top *add_spanhash(struct spanhash_top *top,\n \t}\n }\n \n-static struct spanhash_top *hash_chars(unsigned char *buf, unsigned int sz)\n+static struct spanhash_top *hash_chars(struct diff_filespec *one)\n {\n \tint i, n;\n \tunsigned int accum1, accum2, hashval;\n \tstruct spanhash_top *hash;\n+\tunsigned char *buf = one->data;\n+\tunsigned int sz = one->size;\n+\tint is_text = !one->is_binary;\n \n \ti = INITIAL_HASH_SIZE;\n \thash = xmalloc(sizeof(*hash) + sizeof(struct spanhash) * (1<<i));\n@@ -140,6 +143,11 @@ static struct spanhash_top *hash_chars(unsigned char *buf, unsigned int sz)\n \t\tunsigned int c = *buf++;\n \t\tunsigned int old_1 = accum1;\n \t\tsz--;\n+\n+\t\t/* Ignore CR in CRLF sequence if text */\n+\t\tif (is_text && c == '\\r' && sz && *buf == '\\n')\n+\t\t\tcontinue;\n+\n \t\taccum1 = (accum1 << 7) ^ (accum2 >> 25);\n \t\taccum2 = (accum2 << 7) ^ (old_1 >> 25);\n \t\taccum1 += c;\n@@ -169,14 +177,14 @@ int diffcore_count_changes(struct diff_filespec *src,\n \tif (src_count_p)\n \t\tsrc_count = *src_count_p;\n \tif (!src_count) {\n-\t\tsrc_count = hash_chars(src->data, src->size);\n+\t\tsrc_count = hash_chars(src);\n \t\tif (src_count_p)\n \t\t\t*src_count_p = src_count;\n \t}\n \tif (dst_count_p)\n \t\tdst_count = *dst_count_p;\n \tif (!dst_count) {\n-\t\tdst_count = hash_chars(dst->data, dst->size);\n+\t\tdst_count = hash_chars(dst);\n \t\tif (dst_count_p)\n \t\t\t*dst_count_p = dst_count;\n \t}\ndiff --git a/t/t0022-crlf-rename.sh b/t/t0022-crlf-rename.sh\nnew file mode 100755\nindex 0000000..430a1d1\n--- /dev/null\n+++ b/t/t0022-crlf-rename.sh\n@@ -0,0 +1,33 @@\n+#!/bin/sh\n+\n+test_description='ignore CR in CRLF sequence while computing similiarity'\n+\n+. ./test-lib.sh\n+\n+test_expect_success setup '\n+\n+\tcat ../t0022-crlf-rename.sh >sample &&\n+\tgit add sample &&\n+\n+\ttest_tick &&\n+\tgit commit -m Initial &&\n+\n+\tsed -e \"s/\\$/\n/\" ../t0022-crlf-rename.sh >elpmas &&\n+\tgit add elpmas &&\n+\trm -f sample &&\n+\n+\ttest_tick &&\n+\tgit commit -a -m Second\n+\n+'\n+\n+test_expect_success 'diff -M' '\n+\n+\tgit diff-tree -M -r --name-status HEAD^ HEAD |\n+\tsed -e \"s/R[0-9]*/RNUM/\" >actual &&\n+\techo \"RNUM\tsample\telpmas\" >expect &&\n+\tdiff -u expect actual\n+\n+'\n+\n+test_done\n-- \n1.5.2.2.1414.g1e7d9\n"},{"id":"46034","messageId":"200706290813.l5T8DJ6w024507@mi1.bluebottle.com","threadId":"8771","inReplyTo":"1183098962312-git-send-email-gitster@pobox.com","subject":"Re: [PATCH 4/4] diffcore-delta.c: Ignore CR in CRLF for text files","fromName":"しらいしななこ","fromEmail":"nanako3@bluebottle.com","sentAt":"2007-06-29T08:14:19Z","receivedAt":"2007-06-29T08:14:19Z","isPatch":true,"sender":{"key":"nanako3@lavabit.com","avatar":"https://gravatar.com/avatar/3777b9e201c5883a62b1a6fdf7c53f2d712d1d80989146063ea861e33aad72a8?d=mp&s=160"},"body":"Quoting Junio C Hamano <gitster@pobox.com>:\n\n> This ignores CR byte in CRLF sequence in text file when\n> computing similarity of two blobs.\n> ...\n> +test_expect_success 'diff -M' '\n> +\n> +\tgit diff-tree -M -r --name-status HEAD^ HEAD |\n> +\tsed -e \"s/R[0-9]*/RNUM/\" >actual &&\n> +\techo \"RNUM\tsample\telpmas\" >expect &&\n> +\tdiff -u expect actual\n> +\n> +'\n\nI tried this test but it does not give R100.  The new file is unchanged except for\nLF -> CRLF.  Could you explain why?\n\n-- \nNanako Shiraishi\nhttp://ivory.ap.teacup.com/nanako3/\n\n----------------------------------------------------------------------\nGet a free email address with REAL anti-spam protection.\nhttp://www.bluebottle.com\n"},{"id":"46038","messageId":"7vbqezf8is.fsf@assigned-by-dhcp.pobox.com","threadId":"8771","inReplyTo":"200706290813.l5T8DJ6w024507@mi1.bluebottle.com","subject":"Re: [PATCH 4/4] diffcore-delta.c: Ignore CR in CRLF for text files","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2007-06-29T08:51:39Z","receivedAt":"2007-06-29T08:51:39Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"しらいしななこ  <nanako3@bluebottle.com> writes:\n\n> Quoting Junio C Hamano <gitster@pobox.com>:\n>\n>> This ignores CR byte in CRLF sequence in text file when\n>> computing similarity of two blobs.\n>> ...\n>> +test_expect_success 'diff -M' '\n>> +\n>> +\tgit diff-tree -M -r --name-status HEAD^ HEAD |\n>> +\tsed -e \"s/R[0-9]*/RNUM/\" >actual &&\n>> +\techo \"RNUM\tsample\telpmas\" >expect &&\n>> +\tdiff -u expect actual\n>> +\n>> +'\n>\n> I tried this test but it does not give R100.  The new file is unchanged except for\n> LF -> CRLF.  Could you explain why?\n\nHeh, I hate when people nitpick me ;-)\n\nThe similarity code counts \"bytes copied from the source\nmaterial to the destination\" and \"bytes added to the source to\ncreate the destination\".\n\nRename detection uses only the former value.  The amount of data\ncopied from the source is divided by the larger of the size of\nsource or destination.\n\nIn the case of our test script, the source material is about 560\nbytes, while the destination material is about 590 bytes, after \nadding CR to the end of every line.  We find that 560 bytes have\nbeen copied from the source material, and floor(560 * 100 / 590)\nis 94%, which is what you would see as the result.\n\nWe could adjust max_size variable inside diffcore_count_changes,\nbut I am not sure if it is worth the trouble.\n"},{"id":"46062","messageId":"93c3eada0706292118u2058553bj103ba707aa144d16@mail.gmail.com","threadId":"8771","inReplyTo":"7v3b0bi88r.fsf@assigned-by-dhcp.pobox.com","subject":"Re: Applying patches in a directory that isn't a repository","fromName":"Geoff Russell","fromEmail":"geoffrey.russell@gmail.com","sentAt":"2007-06-30T04:18:11Z","receivedAt":"2007-06-30T04:18:11Z","isPatch":false,"sender":{"key":"geoffrey.russell@gmail.com","avatar":"https://gravatar.com/avatar/c30f497ccfa6bf06d86f30bd2ba092a2dd124c61c6bc902f7cb5c3f6486947de?d=mp&s=160"},"body":"On 6/29/07, Junio C Hamano <gitster@pobox.com> wrote:\n> \"Geoff Russell\" <geoffrey.russell@gmail.com> writes:\n>\n> > But \"git am\" needs (AFAIK) a full repository. Is there a way to apply\n> > a patch without\n> > .git being present?\n>\n> If the recipient does not have a git repository, there is no\n> point using \"git am\", as it is about making commits out of\n> e-mails.\n>\n> git-apply acts as a plain \"patch applicator\".\n\nFor some reason I thought git-apply also needed a repository --- but it doesn't\nand I've just tested it and, bingo, it is perfect for my needs. Many thanks.\n\nCheers,\nGeoff\n"}]}