{"thread":{"id":"20022","subject":"[PATCHv2 0/2] git apply: cope with whitespace differences","startedAt":"2009-07-04T11:53:47Z","lastAt":"2009-07-05T23:21:04Z","messageCount":4,"participants":["Giuseppe Bilotta","Ramsay Jones"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"117435","messageId":"1246708429-21823-1-git-send-email-giuseppe.bilotta@gmail.com","threadId":"20022","inReplyTo":null,"subject":"[PATCHv2 0/2] git apply: cope with whitespace differences","fromName":"Giuseppe Bilotta","fromEmail":"giuseppe.bilotta@gmail.com","sentAt":"2009-07-04T11:53:47Z","receivedAt":"2009-07-04T11:53:47Z","isPatch":false,"sender":{"key":"giuseppe.bilotta@gmail.com","avatar":"https://avatars.githubusercontent.com/u/1464?v=4"},"body":"This second version includes significant cleanups and improvements to my\nprevious attempt at a --ignore-whitespace option for git-apply.\n\nFollowing Junio's review, the patch now includes proper handling for\nleading whitespace (specifically the case when one part has it and the\nother one doesn't), and more test cases.\n\nThe second patch fixes the updating of the postimage when whitespace is\nadjusted to match the target.\n\nGiuseppe Bilotta (2):\n  git apply: option to ignore whitespace differences\n  git apply: preserve original whitespace with --ignore-whitespace\n\n builtin-apply.c                        |  108 +++++++++++++++++++++++++++++---\n contrib/completion/git-completion.bash |    2 +\n git-am.sh                              |    3 +-\n git-rebase.sh                          |    3 +\n t/t4107-apply-ignore-whitespace.sh     |  107 +++++++++++++++++++++++++++++++\n 5 files changed, 214 insertions(+), 9 deletions(-)\n create mode 100755 t/t4107-apply-ignore-whitespace.sh\n"},{"id":"117436","messageId":"1246708429-21823-2-git-send-email-giuseppe.bilotta@gmail.com","threadId":"20022","inReplyTo":"1246708429-21823-1-git-send-email-giuseppe.bilotta@gmail.com","subject":"[PATCHv2 1/2] git apply: option to ignore whitespace differences","fromName":"Giuseppe Bilotta","fromEmail":"giuseppe.bilotta@gmail.com","sentAt":"2009-07-04T11:53:48Z","receivedAt":"2009-07-04T11:53:48Z","isPatch":false,"sender":{"key":"giuseppe.bilotta@gmail.com","avatar":"https://avatars.githubusercontent.com/u/1464?v=4"},"body":"Introduce --ignore-whitespace option to ignore whitespace differences\nwhile applying the patch. The feature is based on Robert Fitzsimons'\nwork from\nhttp://permalink.gmane.org/gmane.comp.version-control.git/7876\n\nA test is included, and 'git am' and 'git rebase' are made aware of this\noption and pass it through to 'git apply', and so is the bash git\ncompletion.\n---\n builtin-apply.c                        |   55 ++++++++++++++++-\n contrib/completion/git-completion.bash |    2 +\n git-am.sh                              |    3 +-\n git-rebase.sh                          |    3 +\n t/t4107-apply-ignore-whitespace.sh     |  107 ++++++++++++++++++++++++++++++++\n 5 files changed, 167 insertions(+), 3 deletions(-)\n create mode 100755 t/t4107-apply-ignore-whitespace.sh\n\ndiff --git a/builtin-apply.c b/builtin-apply.c\nindex dc0ff5e..01230f1 100644\n--- a/builtin-apply.c\n+++ b/builtin-apply.c\n@@ -39,6 +39,7 @@ static int diffstat;\n static int numstat;\n static int summary;\n static int check;\n+static int ignore_whitespace;\n static int apply = 1;\n static int apply_in_reverse;\n static int apply_with_reject;\n@@ -214,6 +215,52 @@ static uint32_t hash_line(const char *cp, size_t len)\n \treturn h;\n }\n \n+/*\n+ * Compare s1 to s2 up to length n, ignoring whitespace differences.\n+ * It is acceptable if s2 is a substring of s1.\n+ */\n+static int memcmp_ignore_whitespace(const char *s1, const char *s2, size_t n)\n+{\n+\tconst char *stop = s2 + n;\n+\tint result;\n+\n+\tif (!n)\n+\t\treturn 0;\n+\n+\t/* skip leading whitespace */\n+\twhile (isspace(*s1))\n+\t\ts1++;\n+\twhile (isspace(*s2))\n+\t\ts2++;\n+\twhile (!result && s2 < stop) {\n+\t\tresult = *s1++ - *s2++;\n+\t\t/*\n+\t\t * skip whitespace inside if we have whitespace\n+\t\t * on both buffers\n+\t\t */\n+\t\tif (isspace(*s1) && isspace(*s2)) {\n+\t\t\twhile (isspace(*s1))\n+\t\t\t\ts1++;\n+\t\t\twhile (isspace(*s2))\n+\t\t\t\ts2++;\n+\t\t}\n+\t}\n+\n+\treturn result;\n+}\n+\n+/*\n+ * Returns true if the lines in s2 match the buffer in s1 up to length n,\n+ * according to the ignore_whitespace setting\n+ */\n+static int lines_match(const char *s1, const char *s2, size_t n)\n+{\n+\tif (ignore_whitespace)\n+\t\treturn !memcmp_ignore_whitespace(s1, s2, n);\n+\telse\n+\t\treturn !memcmp(s1, s2, n);\n+}\n+\n static void add_line_info(struct image *img, const char *bol, size_t len, unsigned flag)\n {\n \tALLOC_GROW(img->line_allocated, img->nr + 1, img->alloc);\n@@ -1690,7 +1737,7 @@ static int match_fragment(struct image *img,\n \tif ((match_end\n \t     ? (try + preimage->len == img->len)\n \t     : (try + preimage->len <= img->len)) &&\n-\t    !memcmp(img->buf + try, preimage->buf, preimage->len))\n+\t    lines_match(img->buf + try, preimage->buf, preimage->len))\n \t\treturn 1;\n \n \tif (ws_error_action != correct_ws_error)\n@@ -1698,7 +1745,9 @@ static int match_fragment(struct image *img,\n \n \t/*\n \t * The hunk does not apply byte-by-byte, but the hash says\n-\t * it might with whitespace fuzz.\n+\t * it might with whitespace fuzz. We haven't been asked to\n+\t * ignore whitespace, we were asked to correct whitespace\n+\t * errors, so let's try matching after whitespace correction.\n \t */\n \tfixed_buf = xmalloc(preimage->len + 1);\n \tbuf = fixed_buf;\n@@ -3304,6 +3353,8 @@ int cmd_apply(int argc, const char **argv, const char *unused_prefix)\n \t\t{ OPTION_CALLBACK, 0, \"whitespace\", &whitespace_option, \"action\",\n \t\t\t\"detect new or modified lines that have whitespace errors\",\n \t\t\t0, option_parse_whitespace },\n+\t\tOPT_BOOLEAN(0, \"ignore-whitespace\", &ignore_whitespace,\n+\t\t\t\"ignore whitespace differences when finding context\"),\n \t\tOPT_BOOLEAN('R', \"reverse\", &apply_in_reverse,\n \t\t\t\"apply the patch in reverse\"),\n \t\tOPT_BOOLEAN(0, \"unidiff-zero\", &unidiff_zero,\ndiff --git a/contrib/completion/git-completion.bash b/contrib/completion/git-completion.bash\nindex ddb71e2..d3415b5 100755\n--- a/contrib/completion/git-completion.bash\n+++ b/contrib/completion/git-completion.bash\n@@ -663,6 +663,7 @@ _git_am ()\n \t--*)\n \t\t__gitcomp \"\n \t\t\t--3way --committer-date-is-author-date --ignore-date\n+\t\t\t--ignore-whitespace\n \t\t\t--interactive --keep --no-utf8 --signoff --utf8\n \t\t\t--whitespace=\n \t\t\t\"\n@@ -684,6 +685,7 @@ _git_apply ()\n \t\t\t--stat --numstat --summary --check --index\n \t\t\t--cached --index-info --reverse --reject --unidiff-zero\n \t\t\t--apply --no-add --exclude=\n+\t\t\t--ignore-whitespace\n \t\t\t--whitespace= --inaccurate-eof --verbose\n \t\t\t\"\n \t\treturn\ndiff --git a/git-am.sh b/git-am.sh\nindex d64d997..fe024b1 100755\n--- a/git-am.sh\n+++ b/git-am.sh\n@@ -16,6 +16,7 @@ s,signoff       add a Signed-off-by line to the commit message\n u,utf8          recode into utf8 (default)\n k,keep          pass -k flag to git-mailinfo\n whitespace=     pass it through git-apply\n+ignore-whitespace pass it through git-apply\n directory=      pass it through git-apply\n C=              pass it through git-apply\n p=              pass it through git-apply\n@@ -303,7 +304,7 @@ do\n \t\tgit_apply_opt=\"$git_apply_opt $(sq \"$1$2\")\"; shift ;;\n \t--patch-format)\n \t\tshift ; patch_format=\"$1\" ;;\n-\t--reject)\n+\t--reject|--ignore-whitespace)\n \t\tgit_apply_opt=\"$git_apply_opt $1\" ;;\n \t--committer-date-is-author-date)\n \t\tcommitter_date_is_author_date=t ;;\ndiff --git a/git-rebase.sh b/git-rebase.sh\nindex 18bc694..d741752 100755\n--- a/git-rebase.sh\n+++ b/git-rebase.sh\n@@ -333,6 +333,9 @@ do\n \t\t\t;;\n \t\tesac\n \t\t;;\n+\t--ignore-whitespace)\n+\t\tgit_am_opt=\"$git_am_opt $1\"\n+\t\t;;\n \t--committer-date-is-author-date|--ignore-date)\n \t\tgit_am_opt=\"$git_am_opt $1\"\n \t\tforce_rebase=t\ndiff --git a/t/t4107-apply-ignore-whitespace.sh b/t/t4107-apply-ignore-whitespace.sh\nnew file mode 100755\nindex 0000000..b9e6f14\n--- /dev/null\n+++ b/t/t4107-apply-ignore-whitespace.sh\n@@ -0,0 +1,107 @@\n+#!/bin/sh\n+#\n+# Copyright (c) 2005 Junio C Hamano\n+# Copyright (c) 2005 Robert Fitzsimons\n+#\n+\n+test_description='git-apply --ignore-whitespace.\n+\n+'\n+. ./test-lib.sh\n+\n+# setup\n+\n+cat > patch1.patch <<\\EOF\n+diff --git a/main.c b/main.c\n+new file mode 100644\n+--- /dev/null\n++++ b/main.c\n+@@ -0,0 +1,22 @@\n++#include <stdio.h>\n++\n++void print_int(int num);\n++int func(int num);\n++\n++int main() {\n++       int i;\n++\n++       for (i = 0; i < 10; i++) {\n++               print_int(func(i)); /* stuff */\n++       }\n++\n++       return 0;\n++}\n++\n++int func(int num) {\n++       return num * num;\n++}\n++\n++void print_int(int num) {\n++       printf(\"%d\", num);\n++}\n+EOF\n+cat > patch2.patch <<\\EOF\n+diff --git a/main.c b/main.c\n+--- a/main.c\n++++ b/main.c\n+@@ -10,6 +10,8 @@\n+ \t\tprint_int(func(i)); /* stuff */\n+ \t}\n+ \n++\tprintf(\"\\n\");\n++\n+ \treturn 0;\n+ }\n+ \n+EOF\n+\n+cat > patch3.patch <<\\EOF\n+diff --git a/main.c b/main.c\n+--- a/main.c\n++++ b/main.c\n+@@ -10,1 +10,1 @@\n+ \t\tprint_int(func(i)); \n+EOF\n+\n+cat > patch4.patch <<\\EOF\n+diff --git a/main.c b/main.c\n+--- a/main.c\n++++ b/main.c\n+@@ -21,1 +21,1 @@\n+\t}; \n+\\ No newline at end of file\n+EOF\n+\n+cat > patch5.patch <<\\EOF\n+diff --git a/main.c b/main.c\n+--- a/main.c\n++++ b/main.c\n+@@ -2,1 +2,1 @@\n+ \tvoid print_int(int num);\n+EOF\n+\n+test_expect_success \"S = patch1\" \\\n+    'git-apply patch1.patch'\n+\n+test_expect_failure \"F = patch2\" \\\n+    'git-apply patch2.patch'\n+\n+test_expect_success \"S = patch2 (--ignore-whitespace)\" \\\n+    'git-apply --ignore-whitespace patch2.patch'\n+\n+test_expect_failure \"S = patch3 (missing string at EOL)\" \\\n+    'git-apply --ignore-whitespace patch3.patch'\n+\n+test_expect_failure \"S = patch4 (missing EOL at EOF)\" \\\n+    'git-apply --ignore-whitespace patch4.patch'\n+\n+test_expect_success \"S = patch5 (leading whitespace)\" \\\n+    'git-apply --ignore-whitespace patch5.patch'\n+\n+rm -f main.c\n+test_expect_success \"S = patch1 (--ignore-whitespace)\" \\\n+    'git-apply --ignore-whitespace patch1.patch'\n+\n+test_done\n+\n+\n-- \n1.6.3.3.512.g4fff\n"},{"id":"117437","messageId":"1246708429-21823-3-git-send-email-giuseppe.bilotta@gmail.com","threadId":"20022","inReplyTo":"1246708429-21823-1-git-send-email-giuseppe.bilotta@gmail.com","subject":"[PATCHv2 2/2] git apply: preserve original whitespace with --ignore-whitespace","fromName":"Giuseppe Bilotta","fromEmail":"giuseppe.bilotta@gmail.com","sentAt":"2009-07-04T11:53:49Z","receivedAt":"2009-07-04T11:53:49Z","isPatch":false,"sender":{"key":"giuseppe.bilotta@gmail.com","avatar":"https://avatars.githubusercontent.com/u/1464?v=4"},"body":"---\n builtin-apply.c |   55 ++++++++++++++++++++++++++++++++++++++++++++++++-------\n 1 files changed, 48 insertions(+), 7 deletions(-)\n\ndiff --git a/builtin-apply.c b/builtin-apply.c\nindex 01230f1..6c6ccbd 100644\n--- a/builtin-apply.c\n+++ b/builtin-apply.c\n@@ -1639,10 +1639,17 @@ static int read_old_data(struct stat *st, const char *path, struct strbuf *buf)\n \t}\n }\n \n+/*\n+ * Update the preimage, and the common lines in postimage,\n+ * from buffer buf of length len. If postlen is 0 the postimage\n+ * is updated in place, otherwise it's updated on a new buffer\n+ * of length postlen\n+ */\n+\n static void update_pre_post_images(struct image *preimage,\n \t\t\t\t   struct image *postimage,\n \t\t\t\t   char *buf,\n-\t\t\t\t   size_t len)\n+\t\t\t\t   size_t len, size_t postlen)\n {\n \tint i, ctx;\n \tchar *new, *old, *fixed;\n@@ -1661,11 +1668,19 @@ static void update_pre_post_images(struct image *preimage,\n \t*preimage = fixed_preimage;\n \n \t/*\n-\t * Adjust the common context lines in postimage, in place.\n-\t * This is possible because whitespace fixing does not make\n-\t * the string grow.\n+\t * Adjust the common context lines in postimage. This can be\n+\t * done in-place when we are just doing whitespace fixing,\n+\t * which does not make the string grow, but needs a new buffer\n+\t * when ignore_whitespace causes the update, since in this case\n+\t * we could have e.g. tabs converted to multiple spaces.\n+\t * We trust the caller to tell us if the update can be done\n+\t * in place (postlen==0) or not.\n \t */\n-\tnew = old = postimage->buf;\n+\told = postimage->buf;\n+\tif (postlen)\n+\t\tnew = postimage->buf = xmalloc(postlen);\n+\telse\n+\t\tnew = old;\n \tfixed = preimage->buf;\n \tfor (i = ctx = 0; i < postimage->nr; i++) {\n \t\tsize_t len = postimage->line[i].len;\n@@ -1737,8 +1752,34 @@ static int match_fragment(struct image *img,\n \tif ((match_end\n \t     ? (try + preimage->len == img->len)\n \t     : (try + preimage->len <= img->len)) &&\n-\t    lines_match(img->buf + try, preimage->buf, preimage->len))\n+\t    lines_match(img->buf + try, preimage->buf, preimage->len)) {\n+\t\t/*\n+\t\t * When ignoring whitespace, we want to preserve the\n+\t\t * target image whitespace in lines that are not modified,\n+\t\t * so we update the preimage and the common lines in\n+\t\t * the postimage to the target whitespace.\n+\t\t */\n+\t\tif (ignore_whitespace) {\n+\t\t\t/*\n+\t\t\t * New length for the updated pre and postimages\n+\t\t\t */\n+\t\t\tsize_t prelen = 0;\n+\t\t\tsize_t postlen = postimage->len;\n+\t\t\tfor (i = 0; i < preimage->nr; i++) {\n+\t\t\t\tsize_t len = img->line[try_lno + i].len;\n+\t\t\t\tif (preimage->line[i].flag & LINE_COMMON)\n+\t\t\t\t\tpostlen += len - preimage->line[i].len;\n+\t\t\t\tprelen += preimage->line[i].len = len;\n+\t\t\t}\n+\t\t\tfixed_buf = xmalloc(prelen);\n+\t\t\tmemcpy(fixed_buf, img->buf + try, prelen);\n+\n+\t\t\tupdate_pre_post_images(\n+\t\t\t\tpreimage, postimage,\n+\t\t\t\tfixed_buf, prelen, postlen);\n+\t\t}\n \t\treturn 1;\n+\t}\n \n \tif (ws_error_action != correct_ws_error)\n \t\treturn 0;\n@@ -1799,7 +1840,7 @@ static int match_fragment(struct image *img,\n \t * hunk match.  Update the context lines in the postimage.\n \t */\n \tupdate_pre_post_images(preimage, postimage,\n-\t\t\t       fixed_buf, buf - fixed_buf);\n+\t\t\t       fixed_buf, buf - fixed_buf, 0);\n \treturn 1;\n \n  unmatch_exit:\n-- \n1.6.3.3.512.g4fff\n"},{"id":"117484","messageId":"4A513560.5040001@ramsay1.demon.co.uk","threadId":"20022","inReplyTo":"1246708429-21823-2-git-send-email-giuseppe.bilotta@gmail.com","subject":"Re: [PATCHv2 1/2] git apply: option to ignore whitespace differences","fromName":"Ramsay Jones","fromEmail":"ramsay@ramsay1.demon.co.uk","sentAt":"2009-07-05T23:21:04Z","receivedAt":"2009-07-05T23:21:04Z","isPatch":false,"sender":{"key":"ramsay@ramsayjones.plus.com","avatar":"https://avatars.githubusercontent.com/u/33702710?v=4"},"body":"[Sorry for the late reply to this; I've been catching up on\nemail after several days away - so if this has already been\naddressed, just ignore and sorry for the noise!]\n\nGiuseppe Bilotta wrote:\n[snip]\n> + * Compare s1 to s2 up to length n, ignoring whitespace differences.\n> + * It is acceptable if s2 is a substring of s1.\n> + */\n> +static int memcmp_ignore_whitespace(const char *s1, const char *s2, size_t n)\n> +{\n> +\tconst char *stop = s2 + n;\n> +\tint result;\n\nresult is uninitialized here...\n\n> +\n> +\tif (!n)\n> +\t\treturn 0;\n> +\n> +\t/* skip leading whitespace */\n> +\twhile (isspace(*s1))\n> +\t\ts1++;\n> +\twhile (isspace(*s2))\n> +\t\ts2++;\n> +\twhile (!result && s2 < stop) {\n\n... and is still uninitialized the first time it hits\nthe loop condition...\n\n> +\t\tresult = *s1++ - *s2++;\n\n... so this may or may not be set...\n\n> +\t\t/*\n> +\t\t * skip whitespace inside if we have whitespace\n> +\t\t * on both buffers\n> +\t\t */\n> +\t\tif (isspace(*s1) && isspace(*s2)) {\n> +\t\t\twhile (isspace(*s1))\n> +\t\t\t\ts1++;\n> +\t\t\twhile (isspace(*s2))\n> +\t\t\t\ts2++;\n> +\t\t}\n> +\t}\n> +\n> +\treturn result;\n\n... so this may still be uninitialized.\n> +}\n> +\n\nI haven't been following this thread; I just noticed this problem\nas the patch floated past...\n\nATB,\nRamsay Jones\n"}]}