{"thread":{"id":"26552","subject":"Merging limitations after directory renames -- interesting test repo","startedAt":"2011-02-18T18:58:48Z","lastAt":"2011-02-23T08:02:33Z","messageCount":34,"participants":["Martin Langhoff","Jeff King","Linus Torvalds","Junio C Hamano","Piotr Krukowiecki","Ævar Arnfjörð Bjarmason"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"161559","messageId":"AANLkTimsQmOLDENX27YqpicBeFFZrfgEAsLvFiJqoV7w@mail.gmail.com","threadId":"26552","inReplyTo":null,"subject":"Merging limitations after directory renames -- interesting test repo","fromName":"Martin Langhoff","fromEmail":"martin.langhoff@gmail.com","sentAt":"2011-02-18T18:58:48Z","receivedAt":"2011-02-18T18:58:48Z","isPatch":false,"sender":{"key":"martin.langhoff@gmail.com","avatar":"https://gravatar.com/avatar/1e3f311b6c4c15836501901ca58f8c0b0667246488084ba524d8bc9867e22fd9?d=mp&s=160"},"body":"Briefly,\n\nFetch git://dev.laptop.org/git/users/martin/etherpad and try to merge\npg and master. A ton of work is left to do by hand, and you don't even\nget useful conflicts.\n\n - Many paths that shoud succeed don't seem to succeed. After using\ngit merge, tell me if this file doesn't look like at least git should\nhave _attempted_ a diff3 on it:\n   gitk --all  -- etherpad/bin/rebuildjar.sh\ntrunk/etherpad/bin/rebuildjar.sh\ntrunk/trunk/etherpad/bin/rebuildjar.sh\n\n - Git diff seems confused, says \"path needs merge\" instead of giving\nme a diff. May be related to the first problem\n    git diff -- etherpad/bin/rebuildjar.sh\n\n   However I can\n   git diff master -- etherpad/bin/rebuildjar.sh\n   git diff pg -- etherpad/bin/rebuildjar.sh\n\nNote note! The \"pg\" branch you see has been artificially rebuilt so I\nmay have made a mistake. However, I've reviewed all the commits and it\nseems correct. Original repo comes from SVN via hg -- see\ngit://github.com/ether/pad.git\n\nMost importantly, even if I misfired here or there, the resulting\nhistory of the paths reads like something git _can_ make sense of if\nit traces the history of the paths carefully.\n\ncheers,\n\n\n\nm\n-- \n martin.langhoff@gmail.com\n martin@laptop.org -- Software Architect - OLPC\n - ask interesting questions\n - don't get distracted with shiny stuff  - working code first\n - http://wiki.laptop.org/go/User:Martinlanghoff\n"},{"id":"161566","messageId":"20110218222151.GB4258@sigill.intra.peff.net","threadId":"26552","inReplyTo":"AANLkTimsQmOLDENX27YqpicBeFFZrfgEAsLvFiJqoV7w@mail.gmail.com","subject":"Re: Merging limitations after directory renames -- interesting test repo","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2011-02-18T22:21:51Z","receivedAt":"2011-02-18T22:21:51Z","isPatch":false,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"[cc'ing Linus, who is usually interested in deep rename issues]\n\nOn Fri, Feb 18, 2011 at 01:58:48PM -0500, Martin Langhoff wrote:\n\n> Fetch git://dev.laptop.org/git/users/martin/etherpad and try to merge\n> pg and master. A ton of work is left to do by hand, and you don't even\n\ns{git/}{} in your url.\n\n>  - Many paths that shoud succeed don't seem to succeed.\n\nHmm. Two questions:\n\n  1. Did you bump up your merge.renamelimit? It's hard to see because it\n     scrolls off the screen amidst the many conflicts, but the first\n     message is:\n\n       warning: too many files (created: 425 deleted: 1093), skipping\n       inexact rename detection\n\n     which you want to use. Try \"git config merge.renamelimit\n     10000\". Which runs pretty snappily on my machine; I wonder if we\n     should up the default limit.\n\n  2. Which version of git are you using? Commit 83c9031 produces some\n     funky results on merges. It's not in any released version of git,\n     but it is in master.\n\n> After using git merge, tell me if this file doesn't look like at least\n> git should have _attempted_ a diff3 on it:\n>\n>    gitk --all  -- etherpad/bin/rebuildjar.sh\n>    trunk/etherpad/bin/rebuildjar.sh\n>    trunk/trunk/etherpad/bin/rebuildjar.sh\n\nSo assuming the two issues above are not a problem, merge says:\n\n  CONFLICT (rename/delete): Rename\n  trunk/etherpad/bin/rebuildjar.sh->etherpad/bin/rebuildjar.sh in\n  origin/pg and deleted in HEAD\n\nabout this and many other files. But that doesn't seem right. Both sides\ndid the same rename. But we don't seem to detect it on \"master\":\n\n  $ base=`git merge-base master origin/pg`\n  $ git diff -M --summary $base master | grep rebuildjar\n   create mode 100755 etherpad/bin/rebuildjar.sh\n   delete mode 100755 trunk/etherpad/bin/rebuildjar.sh\n\nHmm. That is probably because it was substantially rewritten:\n\n  $ wc -l etherpad/bin/rebuildjar.sh\n  161\n\n  $ git diff --stat \\\n      $base:trunk/etherpad/bin/rebuildjar.sh \\\n      master:etherpad/bin/rebuildjar.sh\n   .../etherpad => master:etherpad}/bin/rebuildjar.sh |   92 +++++++++++++-\n   1 files changed, 89 insertions(+), 3 deletions(-)\n\nYou can ignore the rename-looking bit in the diffstat; it's an artifact\nof the way single-blob diffs are done. But the important thing to note\nis that it was mostly rewritten on the master branch, and that's why the\nrename isn't found.\n\nThis seems like an interesting case with respect to git's philosophy of\nautomatic rename detection. Usually the notion of \"if it is so far apart\nthat we can't do rename detection, a 3-way merge is not likely to be\ninteresting\" works.\n\nBut in this case, that may not hold. We have identical renames on either\nside, but one side has been modified a lot. The 3-way merge may very\nwell be useful, but we never get there, because we don't even consider\nthat the two endpoints may be related.\n\nYou can simulate the 3-way merge that would happen with:\n\n  $ git show origin/pg:etherpad/bin/rebuildjar.sh >pg\n  $ git show master:etherpad/bin/rebuildjar.sh >master\n  $ git show $base:trunk/etherpad/bin/rebuildjar.sh >base\n  $ git merge-file master base pg\n  $ $EDITOR master\n\nIt's an ugly merge, but way more useful than dumping with a\nrename/delete conflict.\n\nI'm not sure of the right solution. I guess if there was some way to say\n\"no really, these are renames\" it would make your merge easier. But I\nwonder if we can somehow be more clever.\n\n>  - Git diff seems confused, says \"path needs merge\" instead of giving\n> me a diff. May be related to the first problem\n>     git diff -- etherpad/bin/rebuildjar.sh\n\nI think this is a side effect of the rename/delete conflict. There is\nnothing to diff against on one side.\n\n-Peff\n"},{"id":"161568","messageId":"AANLkTimKp+Z==QXJg2Bagot+Df4REeANuxwVi7bpPCXr@mail.gmail.com","threadId":"26552","inReplyTo":"20110218222151.GB4258@sigill.intra.peff.net","subject":"Re: Merging limitations after directory renames -- interesting test repo","fromName":"Linus Torvalds","fromEmail":"torvalds@linux-foundation.org","sentAt":"2011-02-18T23:27:36Z","receivedAt":"2011-02-18T23:27:36Z","isPatch":false,"sender":{"key":"torvalds@linux-foundation.org","avatar":"https://avatars.githubusercontent.com/u/1024025?v=4"},"body":"On Fri, Feb 18, 2011 at 2:21 PM, Jeff King <peff@peff.net> wrote:\n>\n>  1. Did you bump up your merge.renamelimit? It's hard to see because it\n>     scrolls off the screen amidst the many conflicts, but the first\n>     message is:\n>\n>       warning: too many files (created: 425 deleted: 1093), skipping\n>       inexact rename detection\n>\n>     which you want to use. Try \"git config merge.renamelimit\n>     10000\". Which runs pretty snappily on my machine; I wonder if we\n>     should up the default limit.\n\nYeah, for the kernel, I have\n\n\t[diff]\n\t\trenamelimit=0\n\nto disable the limit entirely, because the default limit is very low\nindeed. Git is quite good at the rename detection.\n\nHowever, the reason for the low default is not because it's not snappy\nenough - it's because it can end up using a lot of memory (and if\nyou're low on memory, the swapping will mean that it goes from \"quite\nsnappy\" to \"slow as molasses\" - but it still will not be CPU limited,\nit's just paging like crazy).\n\nThat said, having the error message scroll by and not even be noticed\nby somebody doing a merge is really not good. It's an important hint,\nand if you miss that hint you can easily have a totally trivial merge\nturn into a totally unacceptable one.\n\nSo I do think we could try to lift the default a bit, but it might be\neven more important to just make the message much more noticeable and\navoid scrolling past it. For example, setting a flag, and not printing\nout the message immediately, but instead print it out only if it turns\ninto trouble at the end.\n\nI dunno.\n\n>  2. Which version of git are you using? Commit 83c9031 produces some\n>     funky results on merges. It's not in any released version of git,\n>     but it is in master.\n\nOuch. I thought that commit wasn't _supposed_ to make any difference.\nIf it does, then that sounds like a bad bug. Junio?\n\n> Hmm. That is probably because it was substantially rewritten:\n\nDamn. We suck.\n\nLookie here, here's the problem (well, at least part of it):\n\n  [torvalds@i5 etherpad]$ git diff --summary -M10 master...origin/pg |\ngrep rebuildjar\n   rename {trunk/etherpad => etherpad}/bin/rebuildjar.sh (89%)\n\nThat looks fine. But how about doing it the other way around:\n\n  [torvalds@i5 etherpad]$ git diff --summary -M10 origin/pg...master |\ngrep rebuildjar\n   copy trunk/infrastructure/bin/comp.sh => etherpad/bin/rebuildjar.sh (13%)\n   delete mode 100755 trunk/etherpad/bin/rebuildjar.sh\n\nYeah, not such a good match. And it's total crap too: that rename is\nmuch worse than the real one. So we've actually done something\nactively wrong. Because we have:\n\n   .../etherpad => master:etherpad}/bin/rebuildjar.sh |   92\n+++++++++++++++++++-\n   1 files changed, 89 insertions(+), 3 deletions(-)\n\nand\n\n   .../comp.sh => master:etherpad/bin/rebuildjar.sh   |  296\n+++++++++-----------\n   1 files changed, 129 insertions(+), 167 deletions(-)\n\nso that choice of comp.sh was total and utter crap, and the 13%\nsimilarity comes from that.\n\nSo we did something wrong in picking that comp.sh file. And I bet it's\nbecause on a very superficial level it looks better: despite the diff\nbeing *obviously* bigger for the crazy comp.sh choice, I think it\ndecided that they had more lines in common.\n\nWe've had stupid bugs in the \"diffcore_count_changes()\" logic before.\nIt's just that they're _usually_ hidden by overwhelming common code.\n\nIn fact, the attached patch improves things a bit. At a huge added CPU\ncost in rename detection. Then I get\n\n  [torvalds@i5 etherpad]$ git diff --summary -M10 origin/pg...master |\ngrep rebuildjar\n   rename {trunk/etherpad => etherpad}/bin/rebuildjar.sh (32%)\n\nwhich is at least the sane choice for rename detection. It won't fix\nreally the merge-from-hell issue, though, since we won't consider \"32%\nsimilar\" to be a rename.\n\nWe might want to have a way to force rename detection for certain\npatterns for cases like this.\n\n                                    Linus\n\n\n diffcore-rename.c |    2 +-\n 1 files changed, 1 insertions(+), 1 deletions(-)\n\ndiff --git a/diffcore-rename.c b/diffcore-rename.c\nindex df41be5..19957cd 100644\n--- a/diffcore-rename.c\n+++ b/diffcore-rename.c\n@@ -170,7 +170,7 @@ static int estimate_similarity(struct diff_filespec *src,\n \t * and the final score computation below would not have a\n \t * divide-by-zero issue.\n \t */\n-\tif (base_size * (MAX_SCORE-minimum_score) < delta_size * MAX_SCORE)\n+\tif (max_size * (MAX_SCORE-minimum_score) < delta_size * MAX_SCORE)\n \t\treturn 0;\n \n \tif (!src->cnt_data && diff_populate_filespec(src, 0))\n"},{"id":"161570","messageId":"AANLkTimuU4A7sUqo-dpW3ch4H_WJg+G2ynNmagx=C9t8@mail.gmail.com","threadId":"26552","inReplyTo":"AANLkTimKp+Z==QXJg2Bagot+Df4REeANuxwVi7bpPCXr@mail.gmail.com","subject":"Re: Merging limitations after directory renames -- interesting test repo","fromName":"Linus Torvalds","fromEmail":"torvalds@linux-foundation.org","sentAt":"2011-02-19T00:26:14Z","receivedAt":"2011-02-19T00:26:14Z","isPatch":false,"sender":{"key":"torvalds@linux-foundation.org","avatar":"https://avatars.githubusercontent.com/u/1024025?v=4"},"body":"On Fri, Feb 18, 2011 at 3:27 PM, Linus Torvalds\n<torvalds@linux-foundation.org> wrote:\n>\n> We've had stupid bugs in the \"diffcore_count_changes()\" logic before.\n> It's just that they're _usually_ hidden by overwhelming common code.\n>\n> In fact, the attached patch improves things a bit.\n\nThere's a secondary problem too, which is illustrated by this:\n\n  [torvalds@i5 etherpad]$ git diff --summary -M30 $BASE.. | grep\nm0024_statistics_table.js\n   copy {trunk/etherpad =>\netherpad}/src/etherpad/db_migrations/m0024_statistics_table.js (100%)\n   rename trunk/etherpad/src/etherpad/db_migrations/m0024_statistics_table.js\n=> etherpad/src/etherpad/db_migrations/m0040_create_plugin_tables.js\n(52%)\n\nwhich also ends up terminally confusing the merge. It sees that dual\nsource of m0024_statistics_table.js and just gets really really\nconfused.\n\nAnd the bug is that we didn't even ask for copy detection! This just\nconfuses merging more.\n\nAttached is the ugliest patch ever. I'm in no way implying this should\never be accepted, with\n\n - that crazy-ugly static variable to pass in the copy state to the\n'for_each_hash()' callback\n\n - that really ugly \"if (detect_rename == DIFF_DETECT_COPY)\" with\nbroken indentation just to get rid of the copy-checking phase.\n\nSo please consider the attached patch just a \"look, guys, this is\nwrong, and here's the ugliest hack you've ever seen to fix it\".\n\nAnyway, with this, I can at least do\n\n   git merge -Xrename-threshold=30 origin/pg\n\nand while it fails miserably, the failures are now no longer \"totally\nobviously a git bug\". Now it has real rename-rename conflicts like\n\nCONFLICT (rename/rename): Rename\n\"trunk/etherpad/src/static/crossdomain.xml\"->\"etherpad/src/static/crossdomain.xml\"\nin branch \"HEAD\"\n                          rename\n\"trunk/etherpad/src/static/crossdomain.xml\"->\"etherpad/src/static/crossdomain.xml.in\"\nin \"origin/pg\"\n\nwhich really _is_ a conflict that needs user input.\n\nNow, I didn't check that they are *all* of this valid kind, but most\nof them really are. The directories are:\n  (HEAD): etherpad/src/themes/default/templates\n (origin/pg): etherpad/src/templates\n\nso you'd need to fix that up.\n\nWhatever. It's still a nasty merge, but at least git seems to do a\nmuch better job. That \"-Xrename-threshold=30\" thing is a total hack,\nbut it's a valid way to say \"ok, git didn't find some renames I want\nit to, so let's see if I can force it to do a less critical search\".\n\n                          Linus\n\n\n diffcore-rename.c |   12 +++++++++---\n 1 files changed, 9 insertions(+), 3 deletions(-)\n\ndiff --git a/diffcore-rename.c b/diffcore-rename.c\nindex df41be5..daa85e4 100644\n--- a/diffcore-rename.c\n+++ b/diffcore-rename.c\n@@ -170,7 +170,7 @@ static int estimate_similarity(struct diff_filespec *src,\n \t * and the final score computation below would not have a\n \t * divide-by-zero issue.\n \t */\n-\tif (base_size * (MAX_SCORE-minimum_score) < delta_size * MAX_SCORE)\n+\tif (max_size * (MAX_SCORE-minimum_score) < delta_size * MAX_SCORE)\n \t\treturn 0;\n \n \tif (!src->cnt_data && diff_populate_filespec(src, 0))\n@@ -246,6 +246,8 @@ struct file_similarity {\n \tstruct file_similarity *next;\n };\n \n+static int find_copies_too;\n+\n static int find_identical_files(struct file_similarity *src,\n \t\t\t\tstruct file_similarity *dst)\n {\n@@ -277,6 +279,8 @@ static int find_identical_files(struct file_similarity *src,\n \t\t\t}\n \t\t\t/* Give higher scores to sources that haven't been used already */\n \t\t\tscore = !source->rename_used;\n+\t\t\tif (source->rename_used && !find_copies_too)\n+\t\t\t\tcontinue;\n \t\t\tscore += basename_same(source, target);\n \t\t\tif (score > best_score) {\n \t\t\t\tbest = p;\n@@ -377,11 +381,12 @@ static void insert_file_table(struct hash_table *table, int src_dst, int index,\n  * and then during the second round we try to match\n  * cache-dirty entries as well.\n  */\n-static int find_exact_renames(void)\n+static int find_exact_renames(int copies)\n {\n \tint i;\n \tstruct hash_table file_table;\n \n+\tfind_copies_too = copies;\n \tinit_hash(&file_table);\n \tfor (i = 0; i < rename_src_nr; i++)\n \t\tinsert_file_table(&file_table, -1, i, rename_src[i].one);\n@@ -467,7 +472,7 @@ void diffcore_rename(struct diff_options *options)\n \t * We really want to cull the candidates list early\n \t * with cheap tests in order to avoid doing deltas.\n \t */\n-\trename_count = find_exact_renames();\n+\trename_count = find_exact_renames(detect_rename == DIFF_DETECT_COPY);\n \n \t/* Did we only want exact renames? */\n \tif (minimum_score == MAX_SCORE)\n@@ -551,6 +556,7 @@ void diffcore_rename(struct diff_options *options)\n \t\trename_count++;\n \t}\n \n+\tif (detect_rename == DIFF_DETECT_COPY)\n \tfor (i = 0; i < dst_cnt * NUM_CANDIDATE_PER_DST; i++) {\n \t\tstruct diff_rename_dst *dst;\n \n"},{"id":"161572","messageId":"7vwrkwn993.fsf@alter.siamese.dyndns.org","threadId":"26552","inReplyTo":"AANLkTimKp+Z==QXJg2Bagot+Df4REeANuxwVi7bpPCXr@mail.gmail.com","subject":"Re: Merging limitations after directory renames -- interesting test repo","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2011-02-19T00:44:24Z","receivedAt":"2011-02-19T00:44:24Z","isPatch":false,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Linus Torvalds <torvalds@linux-foundation.org> writes:\n\n> Ouch. I thought that commit wasn't _supposed_ to make any difference.\n> If it does, then that sounds like a bad bug. Junio?\n\nIt already has been reverted.\n\nI haven't really looked at the issue yet, but for one thing it seemed to\nbreak synthesis of the virtual common tree inside merge-recursive for some\nreason, and ends up producing a lot more diffs than necessary between the\ncommon and the tips being merged.\n"},{"id":"161575","messageId":"7vsjvkn8vh.fsf@alter.siamese.dyndns.org","threadId":"26552","inReplyTo":"AANLkTimuU4A7sUqo-dpW3ch4H_WJg+G2ynNmagx=C9t8@mail.gmail.com","subject":"Re: Merging limitations after directory renames -- interesting test repo","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2011-02-19T00:52:34Z","receivedAt":"2011-02-19T00:52:34Z","isPatch":false,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Linus Torvalds <torvalds@linux-foundation.org> writes:\n\n> And the bug is that we didn't even ask for copy detection! This just\n> confuses merging more.\n\nYeah, I think it is probably a good idea to really limit the rename\ndetection only to renames at least inside merge-recursive.  The static\nvariable does look ugly but it shouldn't be a rocket surgery to pass it\ndown if we want to.\n"},{"id":"161576","messageId":"AANLkTikORqg_BUUbBtUQTYTVO3nttKpMDtCbxm4ikJCU@mail.gmail.com","threadId":"26552","inReplyTo":"AANLkTimuU4A7sUqo-dpW3ch4H_WJg+G2ynNmagx=C9t8@mail.gmail.com","subject":"Re: Merging limitations after directory renames -- interesting test repo","fromName":"Linus Torvalds","fromEmail":"torvalds@linux-foundation.org","sentAt":"2011-02-19T00:55:35Z","receivedAt":"2011-02-19T00:55:35Z","isPatch":false,"sender":{"key":"torvalds@linux-foundation.org","avatar":"https://avatars.githubusercontent.com/u/1024025?v=4"},"body":"On Fri, Feb 18, 2011 at 4:26 PM, Linus Torvalds\n<torvalds@linux-foundation.org> wrote:\n>\n> So please consider the attached patch just a \"look, guys, this is\n> wrong, and here's the ugliest hack you've ever seen to fix it\".\n\nBtw, the more I think about it, the more I suspect that the\n\"estimate_similarity()\" part of the patch is correct, or at least\nbetter than what we used to have.\n\nIf we have a file that expanded from 100 lines to 200 lines, and all\nof the old contents are there, then I think that logically people\nwould expect it to be a \"50% similarity\".\n\nBut the thing is, with the old code, we would look at the old smaller\nsize (100 lines), and take 50% of that. And then when the delta (also\n100 lines) is bigger than that 50%, then we'd totally dismiss that\nthing from similarity analysis, because it obviously isn't similar\nenough.\n\nSo using the bigger size as the basis (and taking 50% of _that_ and\ncomparing it to the delta) is probably the sane thing to do.\n\nThe rest of the patch I still think is total crap. The _intention_ is\ngood, but the patch was written to be small rather than the right way\nof doing things.\n\n                          Linus\n"},{"id":"161577","messageId":"AANLkTimK0K=khbPk22Fpo99i02y1gdsSgMqWcRgXWe4w@mail.gmail.com","threadId":"26552","inReplyTo":"7vsjvkn8vh.fsf@alter.siamese.dyndns.org","subject":"Re: Merging limitations after directory renames -- interesting test repo","fromName":"Linus Torvalds","fromEmail":"torvalds@linux-foundation.org","sentAt":"2011-02-19T01:03:33Z","receivedAt":"2011-02-19T01:03:33Z","isPatch":false,"sender":{"key":"torvalds@linux-foundation.org","avatar":"https://avatars.githubusercontent.com/u/1024025?v=4"},"body":"On Fri, Feb 18, 2011 at 4:52 PM, Junio C Hamano <gitster@pobox.com> wrote:\n>\n> Yeah, I think it is probably a good idea to really limit the rename\n> detection only to renames at least inside merge-recursive.  The static\n> variable does look ugly but it shouldn't be a rocket surgery to pass it\n> down if we want to.\n\nHere's a slight update. It still has the \"too ugly to live\"\nif-statment without proper indentation, but it now changes\n\"for_each_hash()\" to pass in a flag value and uses that.\n\nIt also fixes one of the tests that depended on the \"find copies\"\nbehavior for its result to actually ask for copies now that we don't\ngive them unless you do.\n\n                                 Linus\n\n\n builtin/describe.c            |    4 ++--\n diffcore-rename.c             |   17 ++++++++++-------\n hash.c                        |    4 ++--\n hash.h                        |    2 +-\n t/t4008-diff-break-rewrite.sh |    4 ++--\n 5 files changed, 17 insertions(+), 14 deletions(-)\n\ndiff --git a/builtin/describe.c b/builtin/describe.c\nindex 342129f..4f4fe3e 100644\n--- a/builtin/describe.c\n+++ b/builtin/describe.c\n@@ -63,7 +63,7 @@ static inline struct commit_name *find_commit_name(const unsigned char *peeled)\n \treturn n;\n }\n \n-static int set_util(void *chain)\n+static int set_util(void *chain, int flag)\n {\n \tstruct commit_name *n;\n \tfor (n = chain; n; n = n->next) {\n@@ -289,7 +289,7 @@ static void describe(const char *arg, int last_one)\n \t\tfprintf(stderr, \"searching to describe %s\\n\", arg);\n \n \tif (!have_util) {\n-\t\tfor_each_hash(&names, set_util);\n+\t\tfor_each_hash(&names, set_util, 0);\n \t\thave_util = 1;\n \t}\n \ndiff --git a/diffcore-rename.c b/diffcore-rename.c\nindex df41be5..3a937ac 100644\n--- a/diffcore-rename.c\n+++ b/diffcore-rename.c\n@@ -170,7 +170,7 @@ static int estimate_similarity(struct diff_filespec *src,\n \t * and the final score computation below would not have a\n \t * divide-by-zero issue.\n \t */\n-\tif (base_size * (MAX_SCORE-minimum_score) < delta_size * MAX_SCORE)\n+\tif (max_size * (MAX_SCORE-minimum_score) < delta_size * MAX_SCORE)\n \t\treturn 0;\n \n \tif (!src->cnt_data && diff_populate_filespec(src, 0))\n@@ -247,7 +247,7 @@ struct file_similarity {\n };\n \n static int find_identical_files(struct file_similarity *src,\n-\t\t\t\tstruct file_similarity *dst)\n+\t\t\t\tstruct file_similarity *dst, int copies)\n {\n \tint renames = 0;\n \n@@ -277,6 +277,8 @@ static int find_identical_files(struct file_similarity *src,\n \t\t\t}\n \t\t\t/* Give higher scores to sources that haven't been used already */\n \t\t\tscore = !source->rename_used;\n+\t\t\tif (source->rename_used && !copies)\n+\t\t\t\tcontinue;\n \t\t\tscore += basename_same(source, target);\n \t\t\tif (score > best_score) {\n \t\t\t\tbest = p;\n@@ -306,7 +308,7 @@ static void free_similarity_list(struct file_similarity *p)\n \t}\n }\n \n-static int find_same_files(void *ptr)\n+static int find_same_files(void *ptr, int copies)\n {\n \tint ret;\n \tstruct file_similarity *p = ptr;\n@@ -329,7 +331,7 @@ static int find_same_files(void *ptr)\n \t * If we have both sources *and* destinations, see if\n \t * we can match them up\n \t */\n-\tret = (src && dst) ? find_identical_files(src, dst) : 0;\n+\tret = (src && dst) ? find_identical_files(src, dst, copies) : 0;\n \n \t/* Free the hashes and return the number of renames found */\n \tfree_similarity_list(src);\n@@ -377,7 +379,7 @@ static void insert_file_table(struct hash_table *table, int src_dst, int index,\n  * and then during the second round we try to match\n  * cache-dirty entries as well.\n  */\n-static int find_exact_renames(void)\n+static int find_exact_renames(int copies)\n {\n \tint i;\n \tstruct hash_table file_table;\n@@ -390,7 +392,7 @@ static int find_exact_renames(void)\n \t\tinsert_file_table(&file_table, 1, i, rename_dst[i].two);\n \n \t/* Find the renames */\n-\ti = for_each_hash(&file_table, find_same_files);\n+\ti = for_each_hash(&file_table, find_same_files, copies);\n \n \t/* .. and free the hash data structure */\n \tfree_hash(&file_table);\n@@ -467,7 +469,7 @@ void diffcore_rename(struct diff_options *options)\n \t * We really want to cull the candidates list early\n \t * with cheap tests in order to avoid doing deltas.\n \t */\n-\trename_count = find_exact_renames();\n+\trename_count = find_exact_renames(detect_rename == DIFF_DETECT_COPY);\n \n \t/* Did we only want exact renames? */\n \tif (minimum_score == MAX_SCORE)\n@@ -551,6 +553,7 @@ void diffcore_rename(struct diff_options *options)\n \t\trename_count++;\n \t}\n \n+\tif (detect_rename == DIFF_DETECT_COPY)\n \tfor (i = 0; i < dst_cnt * NUM_CANDIDATE_PER_DST; i++) {\n \t\tstruct diff_rename_dst *dst;\n \ndiff --git a/hash.c b/hash.c\nindex 1cd4c9d..c5dd002 100644\n--- a/hash.c\n+++ b/hash.c\n@@ -81,7 +81,7 @@ void **insert_hash(unsigned int hash, void *ptr, struct hash_table *table)\n \treturn insert_hash_entry(hash, ptr, table);\n }\n \n-int for_each_hash(const struct hash_table *table, int (*fn)(void *))\n+int for_each_hash(const struct hash_table *table, int (*fn)(void *, int), int flag)\n {\n \tint sum = 0;\n \tunsigned int i;\n@@ -92,7 +92,7 @@ int for_each_hash(const struct hash_table *table, int (*fn)(void *))\n \t\tvoid *ptr = array->ptr;\n \t\tarray++;\n \t\tif (ptr) {\n-\t\t\tint val = fn(ptr);\n+\t\t\tint val = fn(ptr, flag);\n \t\t\tif (val < 0)\n \t\t\t\treturn val;\n \t\t\tsum += val;\ndiff --git a/hash.h b/hash.h\nindex 69e33a4..1e2de55 100644\n--- a/hash.h\n+++ b/hash.h\n@@ -30,7 +30,7 @@ struct hash_table {\n \n extern void *lookup_hash(unsigned int hash, const struct hash_table *table);\n extern void **insert_hash(unsigned int hash, void *ptr, struct hash_table *table);\n-extern int for_each_hash(const struct hash_table *table, int (*fn)(void *));\n+extern int for_each_hash(const struct hash_table *table, int (*fn)(void *, int), int flag);\n extern void free_hash(struct hash_table *table);\n \n static inline void init_hash(struct hash_table *table)\ndiff --git a/t/t4008-diff-break-rewrite.sh b/t/t4008-diff-break-rewrite.sh\nindex d79d9e1e..73b4a24 100755\n--- a/t/t4008-diff-break-rewrite.sh\n+++ b/t/t4008-diff-break-rewrite.sh\n@@ -173,8 +173,8 @@ test_expect_success \\\n     'compare_diff_raw expected current'\n \n test_expect_success \\\n-    'run diff with -B -M' \\\n-    'git diff-index -B -M \"$tree\" >current'\n+    'run diff with -B -C' \\\n+    'git diff-index -B -C \"$tree\" >current'\n \n cat >expected <<\\EOF\n :100644 100644 f5deac7be59e7eeab8657fd9ae706fd6a57daed2 08bb2fb671deff4c03a4d4a0a1315dff98d5732c C095\tfile0\tfile1\n"},{"id":"161580","messageId":"AANLkTi=b1Ez9-jQbxCR1KwqVwn=hw4iKRwmRBkK3bULf@mail.gmail.com","threadId":"26552","inReplyTo":"AANLkTimK0K=khbPk22Fpo99i02y1gdsSgMqWcRgXWe4w@mail.gmail.com","subject":"Re: Merging limitations after directory renames -- interesting test repo","fromName":"Linus Torvalds","fromEmail":"torvalds@linux-foundation.org","sentAt":"2011-02-19T01:20:24Z","receivedAt":"2011-02-19T01:20:24Z","isPatch":false,"sender":{"key":"torvalds@linux-foundation.org","avatar":"https://avatars.githubusercontent.com/u/1024025?v=4"},"body":"On Fri, Feb 18, 2011 at 5:03 PM, Linus Torvalds\n<torvalds@linux-foundation.org> wrote:\n>\n> It also fixes one of the tests that depended on the \"find copies\"\n> behavior for its result to actually ask for copies now that we don't\n> give them unless you do.\n\nGahh. Here's all of them - there were six cases.\n\n                     Linus\n\n\n t/t4003-diff-rename-1.sh       |    2 +-\n t/t4004-diff-rename-symlink.sh |    2 +-\n t/t4005-diff-rename-2.sh       |    2 +-\n t/t4008-diff-break-rewrite.sh  |    4 ++--\n t/t4009-diff-rename-4.sh       |    2 +-\n 5 files changed, 6 insertions(+), 6 deletions(-)\n\ndiff --git a/t/t4003-diff-rename-1.sh b/t/t4003-diff-rename-1.sh\nindex c6130c4..bfa8835 100755\n--- a/t/t4003-diff-rename-1.sh\n+++ b/t/t4003-diff-rename-1.sh\n@@ -29,7 +29,7 @@ test_expect_success \\\n # copy-and-edit one, and rename-and-edit the other.  We do not say\n # anything about rezrov.\n \n-GIT_DIFF_OPTS=--unified=0 git diff-index -M -p $tree >current\n+GIT_DIFF_OPTS=--unified=0 git diff-index -C -p $tree >current\n cat >expected <<\\EOF\n diff --git a/COPYING b/COPYING.1\n copy from COPYING\ndiff --git a/t/t4004-diff-rename-symlink.sh b/t/t4004-diff-rename-symlink.sh\nindex 92a65f4..6e562c8 100755\n--- a/t/t4004-diff-rename-symlink.sh\n+++ b/t/t4004-diff-rename-symlink.sh\n@@ -35,7 +35,7 @@ test_expect_success SYMLINKS \\\n # a new creation.\n \n test_expect_success SYMLINKS 'setup diff output' \"\n-    GIT_DIFF_OPTS=--unified=0 git diff-index -M -p $tree >current &&\n+    GIT_DIFF_OPTS=--unified=0 git diff-index -C -p $tree >current &&\n     cat >expected <<\\EOF\n diff --git a/bozbar b/bozbar\n new file mode 120000\ndiff --git a/t/t4005-diff-rename-2.sh b/t/t4005-diff-rename-2.sh\nindex 1ba359d..77d7f49 100755\n--- a/t/t4005-diff-rename-2.sh\n+++ b/t/t4005-diff-rename-2.sh\n@@ -29,7 +29,7 @@ test_expect_success \\\n # and COPYING.2 are based on COPYING, and do not say anything about\n # rezrov.\n \n-git diff-index -M $tree >current\n+git diff-index -C $tree >current\n \n cat >expected <<\\EOF\n :100644 100644 6ff87c4664981e4397625791c8ea3bbb5f2279a3 0603b3238a076dc6c8022aedc6648fa523a17178 C1234\tCOPYING\tCOPYING.1\ndiff --git a/t/t4008-diff-break-rewrite.sh b/t/t4008-diff-break-rewrite.sh\nindex d79d9e1e..73b4a24 100755\n--- a/t/t4008-diff-break-rewrite.sh\n+++ b/t/t4008-diff-break-rewrite.sh\n@@ -173,8 +173,8 @@ test_expect_success \\\n     'compare_diff_raw expected current'\n \n test_expect_success \\\n-    'run diff with -B -M' \\\n-    'git diff-index -B -M \"$tree\" >current'\n+    'run diff with -B -C' \\\n+    'git diff-index -B -C \"$tree\" >current'\n \n cat >expected <<\\EOF\n :100644 100644 f5deac7be59e7eeab8657fd9ae706fd6a57daed2 08bb2fb671deff4c03a4d4a0a1315dff98d5732c C095\tfile0\tfile1\ndiff --git a/t/t4009-diff-rename-4.sh b/t/t4009-diff-rename-4.sh\nindex de3f174..f22c8e3 100755\n--- a/t/t4009-diff-rename-4.sh\n+++ b/t/t4009-diff-rename-4.sh\n@@ -29,7 +29,7 @@ test_expect_success \\\n # and COPYING.2 are based on COPYING, and do not say anything about\n # rezrov.\n \n-git diff-index -z -M $tree >current\n+git diff-index -z -C $tree >current\n \n cat >expected <<\\EOF\n :100644 100644 6ff87c4664981e4397625791c8ea3bbb5f2279a3 0603b3238a076dc6c8022aedc6648fa523a17178 C1234\n"},{"id":"161594","messageId":"20110219090812.GA20577@sigill.intra.peff.net","threadId":"26552","inReplyTo":"AANLkTimKp+Z==QXJg2Bagot+Df4REeANuxwVi7bpPCXr@mail.gmail.com","subject":"Re: Merging limitations after directory renames -- interesting test repo","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2011-02-19T09:08:12Z","receivedAt":"2011-02-19T09:08:12Z","isPatch":false,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Fri, Feb 18, 2011 at 03:27:36PM -0800, Linus Torvalds wrote:\n\n> >  1. Did you bump up your merge.renamelimit? It's hard to see because it\n> >     scrolls off the screen amidst the many conflicts, but the first\n> >     message is:\n> >\n> >       warning: too many files (created: 425 deleted: 1093), skipping\n> >       inexact rename detection\n> >\n> >     which you want to use. Try \"git config merge.renamelimit\n> >     10000\". Which runs pretty snappily on my machine; I wonder if we\n> >     should up the default limit.\n> \n> Yeah, for the kernel, I have\n> \n> \t[diff]\n> \t\trenamelimit=0\n> \n> to disable the limit entirely, because the default limit is very low\n> indeed. Git is quite good at the rename detection.\n> \n> However, the reason for the low default is not because it's not snappy\n> enough - it's because it can end up using a lot of memory (and if\n> you're low on memory, the swapping will mean that it goes from \"quite\n> snappy\" to \"slow as molasses\" - but it still will not be CPU limited,\n> it's just paging like crazy).\n\nI think it can be both. There is an O(n^2) part to the algorithm. I did\nsome timings a few years ago that showed an n^2 increase in time as you\nbumped the limit:\n\n  http://article.gmane.org/gmane.comp.version-control.git/73519\n\nThat's staying within a reasonable memory size. I would not be surprised\nif you can get much worse behavior by going into swap, but I didn't\nmeasure peak memory use there.\n\nThose tests led to:\n\n  commit 50705915eae89eae490dff30fa370ed02e4d6e72\n  Author: Jeff King <peff@peff.net>\n  Date:   Wed Apr 30 13:24:43 2008 -0400\n\n    bump rename limit defaults\n\n    The current rename limit default of 100 was arbitrarily\n    chosen. Testing[1] has shown that on modern hardware, a\n    limit of 200 adds about a second of computation time, and a\n    limit of 500 adds about 5 seconds of computation time.\n\n    This patch bumps the default limit to 200 for viewing diffs,\n    and to 500 for performing a merge. The limit for generating\n    git-status templates is set independently; we bump it up to\n    200 here, as well, to match the diff limit.\n\nBut perhaps it's time to revisit the test; it's been 2 years, and my\nhardware at the time was probably 2 years out of date. :)\n\nHere are the old and new times for various sizes of rename. Details\nabout the test are in the message referenced above.\n\n   N   Old CPU Seconds   New CPU Seconds\n  10              0.43              0.02\n 100              0.44              0.20\n 200              1.40              0.55\n 400              4.87              1.90\n 800             18.08              7.01\n1000             27.82             10.83\n\nSo maybe bump the diff limit to 400 and the merge limit to 1000,\ndoubling both? That leaves us at around 2 seconds per-commit for a log,\nand 10 seconds tacked onto a merge. We could maybe even go higher with\nthe merge limit. If it's such a big merge, the conflict resolution is\nprobably going to take forever anyway, so 30 extra seconds if it makes\nrename detection work is probably a good thing.\n\nAccording to top, git only hit around 17M resident on the 1000-sized\none, so I don't think memory is a problem, at least for average repos\n(and yes, I know top is an awful way to measure, but it's quick and it\nwould need to be orders of magnitude off for it to be a problem).\n\nSo I'm in favor of bumping the limits, or possibly even removing the\nhard number limit and putting in a \"try to do renames for this many\nseconds\" option. If we're going to have something like 30 second delays\non merge, though, we should perhaps write some eye candy to stderr after\n2 seconds or so (like we do with \"git checkout\").\n\n> So I do think we could try to lift the default a bit, but it might be\n> even more important to just make the message much more noticeable and\n> avoid scrolling past it. For example, setting a flag, and not printing\n> out the message immediately, but instead print it out only if it turns\n> into trouble at the end.\n\nYeah, I also think that would be useful. And if that information filters\nup to the merge command, it can even give better advice (like how to\ntweak the limit).\n\n-Peff\n"},{"id":"161598","messageId":"20110219101936.GB20577@sigill.intra.peff.net","threadId":"26552","inReplyTo":"AANLkTimKp+Z==QXJg2Bagot+Df4REeANuxwVi7bpPCXr@mail.gmail.com","subject":"Re: Merging limitations after directory renames -- interesting test repo","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2011-02-19T10:19:36Z","receivedAt":"2011-02-19T10:19:36Z","isPatch":false,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Fri, Feb 18, 2011 at 03:27:36PM -0800, Linus Torvalds wrote:\n\n> So I do think we could try to lift the default a bit, but it might be\n> even more important to just make the message much more noticeable and\n> avoid scrolling past it. For example, setting a flag, and not printing\n> out the message immediately, but instead print it out only if it turns\n> into trouble at the end.\n\nSo here's a patch series. It's built on your other 3 patches (all of\nwhich I thought looked good, btw).\n\n  [1/4]: merge: improve inexact rename limit warning\n  [2/4]: bump rename limit defaults (again)\n  [3/4]: commit: stop setting rename limit\n  [4/4]: inexact rename detection eye candy\n\n-Peff\n"},{"id":"161599","messageId":"20110219102051.GA22508@sigill.intra.peff.net","threadId":"26552","inReplyTo":"20110219101936.GB20577@sigill.intra.peff.net","subject":"[PATCH 1/4] merge: improve inexact rename limit warning","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2011-02-19T10:20:51Z","receivedAt":"2011-02-19T10:20:51Z","isPatch":true,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"The warning is generated deep in the diffcore code, which\nmeans that it will come first, followed possibly by a spew\nof conflicts, making it hard to see.\n\nInstead, let's have diffcore pass back the information about\nhow big the rename limit would needed to have been, and then\nthe caller can provide a more appropriate message (and at a\nmore appropriate time).\n\nNo refactoring of other non-merge callers is necessary,\nbecause nobody else was even using the warn_on_rename_limit\nfeature.\n\nSigned-off-by: Jeff King <peff@peff.net>\n---\n diff.h            |    2 +-\n diffcore-rename.c |    5 +++--\n merge-recursive.c |   10 +++++++++-\n merge-recursive.h |    1 +\n 4 files changed, 14 insertions(+), 4 deletions(-)\n\ndiff --git a/diff.h b/diff.h\nindex 0083d92..f774c9a 100644\n--- a/diff.h\n+++ b/diff.h\n@@ -110,7 +110,7 @@ struct diff_options {\n \tint pickaxe_opts;\n \tint rename_score;\n \tint rename_limit;\n-\tint warn_on_too_large_rename;\n+\tint needed_rename_limit;\n \tint dirstat_percent;\n \tint setup;\n \tint abbrev;\ndiff --git a/diffcore-rename.c b/diffcore-rename.c\nindex 0cd4c13..1d3d5cd 100644\n--- a/diffcore-rename.c\n+++ b/diffcore-rename.c\n@@ -518,12 +518,13 @@ void diffcore_rename(struct diff_options *options)\n \t * but handles the potential overflow case specially (and we\n \t * assume at least 32-bit integers)\n \t */\n+\toptions->needed_rename_limit = 0;\n \tif (rename_limit <= 0 || rename_limit > 32767)\n \t\trename_limit = 32767;\n \tif ((num_create > rename_limit && num_src > rename_limit) ||\n \t    (num_create * num_src > rename_limit * rename_limit)) {\n-\t\tif (options->warn_on_too_large_rename)\n-\t\t\twarning(\"too many files (created: %d deleted: %d), skipping inexact rename detection\", num_create, num_src);\n+\t\toptions->needed_rename_limit =\n+\t\t\tnum_src > num_create ? num_src : num_create;\n \t\tgoto cleanup;\n \t}\n \ndiff --git a/merge-recursive.c b/merge-recursive.c\nindex 16c2dbe..2ecd456 100644\n--- a/merge-recursive.c\n+++ b/merge-recursive.c\n@@ -22,6 +22,11 @@\n #include \"dir.h\"\n #include \"submodule.h\"\n \n+static const char rename_limit_advice[] =\n+\"inexact rename detection was skipped because there were too many\\n\"\n+\"  files. You may want to set your merge.renamelimit variable to at least\\n\"\n+\"  %d and retry this merge.\";\n+\n static struct tree *shift_tree_object(struct tree *one, struct tree *two,\n \t\t\t\t      const char *subtree_shift)\n {\n@@ -436,12 +441,13 @@ static struct string_list *get_renames(struct merge_options *o,\n \t\t\t    o->diff_rename_limit >= 0 ? o->diff_rename_limit :\n \t\t\t    500;\n \topts.rename_score = o->rename_score;\n-\topts.warn_on_too_large_rename = 1;\n \topts.output_format = DIFF_FORMAT_NO_OUTPUT;\n \tif (diff_setup_done(&opts) < 0)\n \t\tdie(\"diff setup failed\");\n \tdiff_tree_sha1(o_tree->object.sha1, tree->object.sha1, \"\", &opts);\n \tdiffcore_std(&opts);\n+\tif (opts.needed_rename_limit > o->needed_rename_limit)\n+\t\to->needed_rename_limit = opts.needed_rename_limit;\n \tfor (i = 0; i < diff_queued_diff.nr; ++i) {\n \t\tstruct string_list_item *item;\n \t\tstruct rename *re;\n@@ -1666,6 +1672,8 @@ int merge_recursive(struct merge_options *o,\n \t\tcommit_list_insert(h2, &(*result)->parents->next);\n \t}\n \tflush_output(o);\n+\tif (o->needed_rename_limit)\n+\t\twarning(rename_limit_advice, o->needed_rename_limit);\n \treturn clean;\n }\n \ndiff --git a/merge-recursive.h b/merge-recursive.h\nindex 981ed6a..1a113e2 100644\n--- a/merge-recursive.h\n+++ b/merge-recursive.h\n@@ -20,6 +20,7 @@ struct merge_options {\n \tint diff_rename_limit;\n \tint merge_rename_limit;\n \tint rename_score;\n+\tint needed_rename_limit;\n \tint call_depth;\n \tstruct strbuf obuf;\n \tstruct string_list current_file_set;\n-- \n1.7.4.1.26.g5e991\n"},{"id":"161600","messageId":"20110219102128.GB22508@sigill.intra.peff.net","threadId":"26552","inReplyTo":"20110219101936.GB20577@sigill.intra.peff.net","subject":"[PATCH 2/4] bump rename limit defaults (again)","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2011-02-19T10:21:28Z","receivedAt":"2011-02-19T10:21:28Z","isPatch":true,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"We did this once before in 5070591 (bump rename limit\ndefaults, 2008-04-30). Back then, we were shooting for about\n1 second for a diff/log calculation, and 5 seconds for a\nmerge.\n\nThere are a few new things to consider, though:\n\n  1. Average processors are faster now.\n\n  2. We've seen on the mailing list some ugly merges where\n     not using inexact rename detection leads to many more\n     conflicts. Merges of this size take a long time\n     anyway, so users are probably happy to spend a little\n     bit of time computing the renames.\n\nLet's bump the diff/merge default limits from 200/500 to\n400/1000. Those are 2 seconds and 10 seconds respectively on\nmy modern hardware.\n\nSigned-off-by: Jeff King <peff@peff.net>\n---\n diff.c            |    2 +-\n merge-recursive.c |    2 +-\n 2 files changed, 2 insertions(+), 2 deletions(-)\n\ndiff --git a/diff.c b/diff.c\nindex 5422c43..869cca7 100644\n--- a/diff.c\n+++ b/diff.c\n@@ -23,7 +23,7 @@\n #endif\n \n static int diff_detect_rename_default;\n-static int diff_rename_limit_default = 200;\n+static int diff_rename_limit_default = 400;\n static int diff_suppress_blank_empty;\n int diff_use_color_default = -1;\n static const char *diff_word_regex_cfg;\ndiff --git a/merge-recursive.c b/merge-recursive.c\nindex 2ecd456..089aa10 100644\n--- a/merge-recursive.c\n+++ b/merge-recursive.c\n@@ -439,7 +439,7 @@ static struct string_list *get_renames(struct merge_options *o,\n \topts.detect_rename = DIFF_DETECT_RENAME;\n \topts.rename_limit = o->merge_rename_limit >= 0 ? o->merge_rename_limit :\n \t\t\t    o->diff_rename_limit >= 0 ? o->diff_rename_limit :\n-\t\t\t    500;\n+\t\t\t    1000;\n \topts.rename_score = o->rename_score;\n \topts.output_format = DIFF_FORMAT_NO_OUTPUT;\n \tif (diff_setup_done(&opts) < 0)\n-- \n1.7.4.1.26.g5e991\n"},{"id":"161601","messageId":"20110219102157.GC22508@sigill.intra.peff.net","threadId":"26552","inReplyTo":"20110219101936.GB20577@sigill.intra.peff.net","subject":"[PATCH 3/4] commit: stop setting rename limit","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2011-02-19T10:21:57Z","receivedAt":"2011-02-19T10:21:57Z","isPatch":true,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"For its post-commit summary, commit was explicitly setting\nthe default rename limit to 100. Presumably when the code\nwas added, it was necessary to do so. These days, however,\nit will fall back properly to the diff default, and that\ndefault has long since changed from 100.\n\nSigned-off-by: Jeff King <peff@peff.net>\n---\n builtin/commit.c |    1 -\n 1 files changed, 0 insertions(+), 1 deletions(-)\n\ndiff --git a/builtin/commit.c b/builtin/commit.c\nindex d7f55e3..aa1510e 100644\n--- a/builtin/commit.c\n+++ b/builtin/commit.c\n@@ -1215,7 +1215,6 @@ static void print_summary(const char *prefix, const unsigned char *sha1)\n \tget_commit_format(format.buf, &rev);\n \trev.always_show_header = 0;\n \trev.diffopt.detect_rename = 1;\n-\trev.diffopt.rename_limit = 100;\n \trev.diffopt.break_opt = 0;\n \tdiff_setup_done(&rev.diffopt);\n \n-- \n1.7.4.1.26.g5e991\n"},{"id":"161602","messageId":"20110219102533.GD22508@sigill.intra.peff.net","threadId":"26552","inReplyTo":"20110219101936.GB20577@sigill.intra.peff.net","subject":"[RFC/PATCH 4/4] inexact rename detection eye candy","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2011-02-19T10:25:33Z","receivedAt":"2011-02-19T10:25:33Z","isPatch":true,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"During a merge, we might spend many seconds doing inexact\nrename detection. It's nice to let the user know that\nsomething is actually happening.\n\nSigned-off-by: Jeff King <peff@peff.net>\n---\nThis feels wrong because it's in such a deep library function. At the\nvery least we probably need some way to turn it off, so callers can pass\nalong any --quiet or --no-progress indicators. Though I think it should\nperhaps be off entirely for any revision traversals (since the progress\nwill be per-commit, so it will be in the middle of log output). And then\non for diff (unless --quiet is given), and on for merge.\n\nI made it update progress for each of the rename_src * rename_dst\nsimilarity estimates. We could just as easily count rename_dst items we\nlook at, but hey, it's eye candy, and obviously bigger numbers are\nbetter.\n\n diffcore-rename.c |    8 ++++++++\n 1 files changed, 8 insertions(+), 0 deletions(-)\n\ndiff --git a/diffcore-rename.c b/diffcore-rename.c\nindex 1d3d5cd..136a86f 100644\n--- a/diffcore-rename.c\n+++ b/diffcore-rename.c\n@@ -5,6 +5,7 @@\n #include \"diff.h\"\n #include \"diffcore.h\"\n #include \"hash.h\"\n+#include \"progress.h\"\n \n /* Table of rename/copy destinations */\n \n@@ -449,6 +450,7 @@ void diffcore_rename(struct diff_options *options)\n \tstruct diff_score *mx;\n \tint i, j, rename_count;\n \tint num_create, num_src, dst_cnt;\n+\tstruct progress *progress;\n \n \tif (!minimum_score)\n \t\tminimum_score = DEFAULT_RENAME_SCORE;\n@@ -528,6 +530,10 @@ void diffcore_rename(struct diff_options *options)\n \t\tgoto cleanup;\n \t}\n \n+\tprogress = start_progress_delay(\n+\t\t\t\"Performing inexact rename detection\",\n+\t\t\trename_dst_nr * rename_src_nr, 50, 1);\n+\n \tmx = xcalloc(num_create * NUM_CANDIDATE_PER_DST, sizeof(*mx));\n \tfor (dst_cnt = i = 0; i < rename_dst_nr; i++) {\n \t\tstruct diff_filespec *two = rename_dst[i].two;\n@@ -555,9 +561,11 @@ void diffcore_rename(struct diff_options *options)\n \t\t\t */\n \t\t\tdiff_free_filespec_blob(one);\n \t\t\tdiff_free_filespec_blob(two);\n+\t\t\tdisplay_progress(progress, i*rename_src_nr + j);\n \t\t}\n \t\tdst_cnt++;\n \t}\n+\tstop_progress(&progress);\n \n \t/* cost matrix sorted by most to least similar pair */\n \tqsort(mx, dst_cnt * NUM_CANDIDATE_PER_DST, sizeof(*mx), score_compare);\n-- \n1.7.4.1.26.g5e991\n"},{"id":"161619","messageId":"AANLkTimiZo8yiwnqzbOTg1FmM4rZn7ypSdZ3-zukLees@mail.gmail.com","threadId":"26552","inReplyTo":"20110219101936.GB20577@sigill.intra.peff.net","subject":"Re: Merging limitations after directory renames -- interesting test repo","fromName":"Martin Langhoff","fromEmail":"martin.langhoff@gmail.com","sentAt":"2011-02-19T15:22:29Z","receivedAt":"2011-02-19T15:22:29Z","isPatch":false,"sender":{"key":"martin.langhoff@gmail.com","avatar":"https://gravatar.com/avatar/1e3f311b6c4c15836501901ca58f8c0b0667246488084ba524d8bc9867e22fd9?d=mp&s=160"},"body":"On Sat, Feb 19, 2011 at 5:19 AM, Jeff King <peff@peff.net> wrote:\n> So here's a patch series. It's built on your other 3 patches (all of\n> which I thought looked good, btw).\n\nRight, this is looking better! Applied Linus and Jeff's patches, and\nthings are much more civilized.\n\nNow, I still have a ton of cases where the 2 branches renamed it just\nslightly different. In those cases, what I want to do is have a merge\nhelper script that does...\n\n git merge-helper --prefer-path foo/orig/README foo/preferred/README\nfoo/other/README\n\nif git could keep the rename trios that it recognizes during the merge\noperation in a \"merge notes\" file, we could just name\n\n git merge-helper --prefer-path foo/preferred/README\n\nand have it infer the \"non-preferred\" paths.\n\nThe result of the operation should be the same as if git had magically\nknown what I wanted to do during merge: git attempts a git-smart diff3\ntype merge, and stores the results accordingly in the index.\n\nEven for a large messy merge like this, it means I can say\n\n git merge\n git merge-helper --prefer-path src/templates/themes/default/* other/dir/foo/*\n\nand that will clear out 99% of the stupid work with minimal fuss;\nleaving the interesting merge work to be cleared up by hand.\n\nI suspect that teaching git merge to accept about 'preferred paths',\nand perhaps \"cache/learn\" about them, is also desirable. Harder\nthough.\n\nThoughts?\n\ncheers,\n\n\n\nm\n-- \n martin.langhoff@gmail.com\n martin@laptop.org -- Software Architect - OLPC\n - ask interesting questions\n - don't get distracted with shiny stuff  - working code first\n - http://wiki.laptop.org/go/User:Martinlanghoff\n"},{"id":"161625","messageId":"AANLkTi=WCqUE-wAnhuG8Osrm3UcA5OXXRScqL-0COrVj@mail.gmail.com","threadId":"26552","inReplyTo":"AANLkTimiZo8yiwnqzbOTg1FmM4rZn7ypSdZ3-zukLees@mail.gmail.com","subject":"Re: Merging limitations after directory renames -- interesting test repo","fromName":"Martin Langhoff","fromEmail":"martin.langhoff@gmail.com","sentAt":"2011-02-19T15:31:37Z","receivedAt":"2011-02-19T15:31:37Z","isPatch":false,"sender":{"key":"martin.langhoff@gmail.com","avatar":"https://gravatar.com/avatar/1e3f311b6c4c15836501901ca58f8c0b0667246488084ba524d8bc9867e22fd9?d=mp&s=160"},"body":"On Sat, Feb 19, 2011 at 10:22 AM, Martin Langhoff\n<martin.langhoff@gmail.com> wrote:\n>  git merge-helper --prefer-path foo/preferred/README\n\nThe concrete example in this repo could be\n\n gitk pg master --  trunk/etherpad/src/templates/store/eepnet_eval_signup.ejs \\\n    etherpad/src/themes/default/templates/store/eepnet_eval_signup.ejs \\\n   etherpad/src/templates/store/eepnet_eval_signup.ejs\n\nThe ideal thing would be to have a helper that knows what to do if I say\n\n  git merge-helper --prefer\netherpad/src/themes/default/templates/store/eepnet_eval_signup.ejs\n\ncheers,\n\n\nm\n-- \n martin.langhoff@gmail.com\n martin@laptop.org -- Software Architect - OLPC\n - ask interesting questions\n - don't get distracted with shiny stuff  - working code first\n - http://wiki.laptop.org/go/User:Martinlanghoff\n"},{"id":"161628","messageId":"AANLkTik=pd3vVMERz=H3sp835Ft8OvrOzBE4PUS7vrO7@mail.gmail.com","threadId":"26552","inReplyTo":"20110219102533.GD22508@sigill.intra.peff.net","subject":"Re: [RFC/PATCH 4/4] inexact rename detection eye candy","fromName":"Linus Torvalds","fromEmail":"torvalds@linux-foundation.org","sentAt":"2011-02-19T15:57:57Z","receivedAt":"2011-02-19T15:57:57Z","isPatch":true,"sender":{"key":"torvalds@linux-foundation.org","avatar":"https://avatars.githubusercontent.com/u/1024025?v=4"},"body":"On Sat, Feb 19, 2011 at 2:25 AM, Jeff King <peff@peff.net> wrote:\n>\n> This feels wrong because it's in such a deep library function. At the\n> very least we probably need some way to turn it off, so callers can pass\n> along any --quiet or --no-progress indicators.\n\nYeah.\n\n> I made it update progress for each of the rename_src * rename_dst\n> similarity estimates. We could just as easily count rename_dst items we\n> look at, but hey, it's eye candy, and obviously bigger numbers are\n> better.\n\nUhh. My only big reaction to your patch was literally \"why don't you\njust do it on the 'dst' items\". I really don't think bigger numbers\nare better, and if you have _so_ many sources that each dst takes so\nlong that you'd want updates at that granularity, you're too screwed\nanyway.\n\nDon't make the \"update progress\" be part of the O(n^2) problem.\n\n                  Linus\n"},{"id":"161634","messageId":"AANLkTi=s6FxnYVv3UinV4Q+4C_9dX9rCLQQoCFD7kG9S@mail.gmail.com","threadId":"26552","inReplyTo":"20110219102533.GD22508@sigill.intra.peff.net","subject":"Re: [RFC/PATCH 4/4] inexact rename detection eye candy","fromName":"Martin Langhoff","fromEmail":"martin.langhoff@gmail.com","sentAt":"2011-02-19T16:29:33Z","receivedAt":"2011-02-19T16:29:33Z","isPatch":true,"sender":{"key":"martin.langhoff@gmail.com","avatar":"https://gravatar.com/avatar/1e3f311b6c4c15836501901ca58f8c0b0667246488084ba524d8bc9867e22fd9?d=mp&s=160"},"body":"On Sat, Feb 19, 2011 at 5:25 AM, Jeff King <peff@peff.net> wrote:\n> During a merge, we might spend many seconds doing inexact\n> rename detection. It's nice to let the user know that\n> something is actually happening.\n\nGiven that we're doing costly work, could the results of rename\nmatching be recorded in a .git/MERGE_RENAMES file?\n\nIf that file's available, the simple merge-helper script I've\ndescribed is possible...\n\ngit commit would have to unlink it on successful commit...\n\ncheers,\n\n\n\nm\n-- \n martin.langhoff@gmail.com\n martin@laptop.org -- Software Architect - OLPC\n - ask interesting questions\n - don't get distracted with shiny stuff  - working code first\n - http://wiki.laptop.org/go/User:Martinlanghoff\n"},{"id":"161637","messageId":"AANLkTim2ETC0T8=8D5pWstZRZ95eo5V0+Pr62mNXRBH1@mail.gmail.com","threadId":"26552","inReplyTo":"20110219102128.GB22508@sigill.intra.peff.net","subject":"Re: [PATCH 2/4] bump rename limit defaults (again)","fromName":"Piotr Krukowiecki","fromEmail":"piotr.krukowiecki@gmail.com","sentAt":"2011-02-19T17:54:47Z","receivedAt":"2011-02-19T17:54:47Z","isPatch":true,"sender":{"key":"piotr.krukowiecki@gmail.com","avatar":"https://avatars.githubusercontent.com/u/3259959?v=4"},"body":"On Sat, Feb 19, 2011 at 11:21 AM, Jeff King <peff@peff.net> wrote:\n> Let's bump the diff/merge default limits from 200/500 to\n> 400/1000. Those are 2 seconds and 10 seconds respectively on\n> my modern hardware.\n\nJust curious - what is your modern hardware?\n\n-- \nPiotrek\n"},{"id":"161714","messageId":"AANLkTi=aWC_sXK1VBspsiVAwhkC4pP8Z+uq7QceLMopS@mail.gmail.com","threadId":"26552","inReplyTo":"20110219102128.GB22508@sigill.intra.peff.net","subject":"Re: [PATCH 2/4] bump rename limit defaults (again)","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2011-02-19T20:12:49Z","receivedAt":"2011-02-19T20:12:49Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"On Sat, Feb 19, 2011 at 11:21, Jeff King <peff@peff.net> wrote:\n\n> Let's bump the diff/merge default limits from 200/500 to\n> 400/1000. Those are 2 seconds and 10 seconds respectively on\n> my modern hardware.\n\nThis is somewhat outside the scope of your patch, but rather than\nmaking these decisions for the user and compiling them into Git\nwouldn't it be better to expose it somehow?\n\nE.g. when you do a git-merge show a progress bar (similar to\ngit-clone) showing that we're trying to do rename detection, show that\nwe stopped, and when we fail point out that the user could run\ngit-merge with some switch that would make Git try harder.\n"},{"id":"161741","messageId":"20110220094803.GA988@sigill.intra.peff.net","threadId":"26552","inReplyTo":"AANLkTik=pd3vVMERz=H3sp835Ft8OvrOzBE4PUS7vrO7@mail.gmail.com","subject":"Re: [RFC/PATCH 4/4] inexact rename detection eye candy","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2011-02-20T09:48:04Z","receivedAt":"2011-02-20T09:48:04Z","isPatch":true,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Sat, Feb 19, 2011 at 07:57:57AM -0800, Linus Torvalds wrote:\n\n> > I made it update progress for each of the rename_src * rename_dst\n> > similarity estimates. We could just as easily count rename_dst items we\n> > look at, but hey, it's eye candy, and obviously bigger numbers are\n> > better.\n> \n> Uhh. My only big reaction to your patch was literally \"why don't you\n> just do it on the 'dst' items\". I really don't think bigger numbers\n> are better, and if you have _so_ many sources that each dst takes so\n> long that you'd want updates at that granularity, you're too screwed\n> anyway.\n> \n> Don't make the \"update progress\" be part of the O(n^2) problem.\n\nI timed it and it's not. The progress code is smart enough not to\nactually print anything more than once per second. So it's just an extra\nfunction call per loop, which is dwarfed by the massive\nestimate_similarity(). It's really not that tight a loop.\n\nThat being said, the output seems a little smoother to me hoisting it\nout, so I've put that in my re-roll. The bigger numbers are worth\nkeeping, IMHO, as they are a more accurate reflection of how much work\nis being done. You do the same amount of work with 2 dests and 1000\nsources as you do with 1000 dests and 2 sources. But one will count to\n1000, and the other will count to 2. They should probably both count to\n2000, the number of estimate_similarity() calls we must make, which is\nthe expensive part. (Actually that is not quite accurate, as we may skip\nsome calls for destinations already found, but it's not worth the effort\nto figure out how many calls we'll actually make ahead of time).\n\nSo here is the re-roll, which I think is probably OK for inclusion. It\nreplaces 4/4 from my last series.\n\n  [1/3]: add inexact rename detection progress infrastructure\n  [2/3]: merge: enable progress reporting for rename detection\n  [3/3]: pull: propagate --progress to merge\n\n-Peff\n"},{"id":"161742","messageId":"20110220095115.GA1082@sigill.intra.peff.net","threadId":"26552","inReplyTo":"20110220094803.GA988@sigill.intra.peff.net","subject":"[PATCH 1/3] add inexact rename detection progress infrastructure","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2011-02-20T09:51:16Z","receivedAt":"2011-02-20T09:51:16Z","isPatch":true,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"We might spend many seconds doing inexact rename detection\nwith no output.  It's nice to let the user know that\nsomething is actually happening.\n\nThis patch adds the infrastructure, but no callers actually\nturn on progress reporting.\n\nSigned-off-by: Jeff King <peff@peff.net>\n---\nReplaces the old 4/4. Now not on by default, progress reporting is\nhoisted to the outer loop, and actually counts all the way to 100%.\n\n diff.h            |    1 +\n diffcore-rename.c |   10 ++++++++++\n 2 files changed, 11 insertions(+), 0 deletions(-)\n\ndiff --git a/diff.h b/diff.h\nindex f774c9a..9585e41 100644\n--- a/diff.h\n+++ b/diff.h\n@@ -111,6 +111,7 @@ struct diff_options {\n \tint rename_score;\n \tint rename_limit;\n \tint needed_rename_limit;\n+\tint show_rename_progress;\n \tint dirstat_percent;\n \tint setup;\n \tint abbrev;\ndiff --git a/diffcore-rename.c b/diffcore-rename.c\nindex 1d3d5cd..d40e40a 100644\n--- a/diffcore-rename.c\n+++ b/diffcore-rename.c\n@@ -5,6 +5,7 @@\n #include \"diff.h\"\n #include \"diffcore.h\"\n #include \"hash.h\"\n+#include \"progress.h\"\n \n /* Table of rename/copy destinations */\n \n@@ -449,6 +450,7 @@ void diffcore_rename(struct diff_options *options)\n \tstruct diff_score *mx;\n \tint i, j, rename_count;\n \tint num_create, num_src, dst_cnt;\n+\tstruct progress *progress = NULL;\n \n \tif (!minimum_score)\n \t\tminimum_score = DEFAULT_RENAME_SCORE;\n@@ -528,6 +530,12 @@ void diffcore_rename(struct diff_options *options)\n \t\tgoto cleanup;\n \t}\n \n+\tif (options->show_rename_progress) {\n+\t\tprogress = start_progress_delay(\n+\t\t\t\t\"Performing inexact rename detection\",\n+\t\t\t\trename_dst_nr * rename_src_nr, 50, 1);\n+\t}\n+\n \tmx = xcalloc(num_create * NUM_CANDIDATE_PER_DST, sizeof(*mx));\n \tfor (dst_cnt = i = 0; i < rename_dst_nr; i++) {\n \t\tstruct diff_filespec *two = rename_dst[i].two;\n@@ -557,7 +565,9 @@ void diffcore_rename(struct diff_options *options)\n \t\t\tdiff_free_filespec_blob(two);\n \t\t}\n \t\tdst_cnt++;\n+\t\tdisplay_progress(progress, (i+1)*rename_src_nr);\n \t}\n+\tstop_progress(&progress);\n \n \t/* cost matrix sorted by most to least similar pair */\n \tqsort(mx, dst_cnt * NUM_CANDIDATE_PER_DST, sizeof(*mx), score_compare);\n-- \n1.7.4.1.26.g5e991\n"},{"id":"161743","messageId":"20110220095320.GB1082@sigill.intra.peff.net","threadId":"26552","inReplyTo":"20110220094803.GA988@sigill.intra.peff.net","subject":"[PATCH 2/3] merge: enable progress reporting for rename detection","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2011-02-20T09:53:21Z","receivedAt":"2011-02-20T09:53:21Z","isPatch":true,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"The user can enable or disable it explicitly with the new\n--progress, but it defaults to checking isatty(2).\n\nThis works only with merge-recursive and subtree. In theory\nwe could pass a progress flag to other strategies, but none\nof them support progress at this point, so let's wait until\nthey grow such a feature before worrying about propagating\nit.\n\nSigned-off-by: Jeff King <peff@peff.net>\n---\nIn theory somebody could have a third-party strategy that cares about\nprogress. Passing the information would probably have to go through the\nenvironment, as strategies will barf on command lines they don't\nunderstand.\n\nI punted on it until somebody shows up who actually cares.\n\n Documentation/merge-options.txt |   10 +++++++++-\n builtin/merge.c                 |    7 +++++++\n merge-recursive.c               |    1 +\n merge-recursive.h               |    1 +\n 4 files changed, 18 insertions(+), 1 deletions(-)\n\ndiff --git a/Documentation/merge-options.txt b/Documentation/merge-options.txt\nindex e33e0f8..b613d4e 100644\n--- a/Documentation/merge-options.txt\n+++ b/Documentation/merge-options.txt\n@@ -75,9 +75,17 @@ option can be used to override --squash.\n ifndef::git-pull[]\n -q::\n --quiet::\n-\tOperate quietly.\n+\tOperate quietly. Implies --no-progress.\n \n -v::\n --verbose::\n \tBe verbose.\n+\n+--progress::\n+--no-progress::\n+\tTurn progress on/off explicitly. If neither is specified,\n+\tprogress is shown if standard error is connected to a terminal.\n+\tNote that not all merge strategies may support progress\n+\treporting.\n+\n endif::git-pull[]\ndiff --git a/builtin/merge.c b/builtin/merge.c\nindex 8c58c3c..d81bee3 100644\n--- a/builtin/merge.c\n+++ b/builtin/merge.c\n@@ -58,6 +58,7 @@ static int option_renormalize;\n static int verbosity;\n static int allow_rerere_auto;\n static int abort_current_merge;\n+static int show_progress = -1;\n \n static struct strategy all_strategy[] = {\n \t{ \"recursive\",  DEFAULT_TWOHEAD | NO_TRIVIAL },\n@@ -200,6 +201,7 @@ static struct option builtin_merge_options[] = {\n \tOPT__VERBOSITY(&verbosity),\n \tOPT_BOOLEAN(0, \"abort\", &abort_current_merge,\n \t\t\"abort the current in-progress merge\"),\n+\tOPT_SET_INT(0, \"progress\", &show_progress, \"force progress reporting\", 1),\n \tOPT_END()\n };\n \n@@ -660,6 +662,8 @@ static int try_merge_strategy(const char *strategy, struct commit_list *common,\n \t\t\to.subtree_shift = \"\";\n \n \t\to.renormalize = option_renormalize;\n+\t\to.show_rename_progress =\n+\t\t\tshow_progress == -1 ? isatty(2) : show_progress;\n \n \t\tfor (x = 0; x < xopts_nr; x++)\n \t\t\tif (parse_merge_opt(&o, xopts[x]))\n@@ -946,6 +950,9 @@ int cmd_merge(int argc, const char **argv, const char *prefix)\n \targc = parse_options(argc, argv, prefix, builtin_merge_options,\n \t\t\tbuiltin_merge_usage, 0);\n \n+\tif (verbosity < 0 && show_progress == -1)\n+\t\tshow_progress = 0;\n+\n \tif (abort_current_merge) {\n \t\tint nargc = 2;\n \t\tconst char *nargv[] = {\"reset\", \"--merge\", NULL};\ndiff --git a/merge-recursive.c b/merge-recursive.c\nindex 089aa10..6c8f957 100644\n--- a/merge-recursive.c\n+++ b/merge-recursive.c\n@@ -441,6 +441,7 @@ static struct string_list *get_renames(struct merge_options *o,\n \t\t\t    o->diff_rename_limit >= 0 ? o->diff_rename_limit :\n \t\t\t    1000;\n \topts.rename_score = o->rename_score;\n+\topts.show_rename_progress = o->show_rename_progress;\n \topts.output_format = DIFF_FORMAT_NO_OUTPUT;\n \tif (diff_setup_done(&opts) < 0)\n \t\tdie(\"diff setup failed\");\ndiff --git a/merge-recursive.h b/merge-recursive.h\nindex 1a113e2..7e1e972 100644\n--- a/merge-recursive.h\n+++ b/merge-recursive.h\n@@ -21,6 +21,7 @@ struct merge_options {\n \tint merge_rename_limit;\n \tint rename_score;\n \tint needed_rename_limit;\n+\tint show_rename_progress;\n \tint call_depth;\n \tstruct strbuf obuf;\n \tstruct string_list current_file_set;\n-- \n1.7.4.1.26.g5e991\n"},{"id":"161744","messageId":"20110220095655.GC1082@sigill.intra.peff.net","threadId":"26552","inReplyTo":"20110220094803.GA988@sigill.intra.peff.net","subject":"[PATCH 3/3] pull: propagate --progress to merge","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2011-02-20T09:56:56Z","receivedAt":"2011-02-20T09:56:56Z","isPatch":true,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"Now that merge understands progress, we should pass it\nalong. While we're at it, pass along --no-progress, too.\n\nSigned-off-by: Jeff King <peff@peff.net>\n---\nProbably not a big deal, but good for people who do \"git pull\n--no-progress\" from a cronjob (which should be !isatty(2), but ISTR some\ncomplaints by people with weird setups).\n\nFetch respects --no-progress, but I don't think it actually works right.\nThe transport code seems to only understand \"want progress\" or \"don't\nknow, guess on isatty\". But it's not really related to this topic, so I\ndidn't investigate too far tonight.\n\n git-pull.sh |    6 ++++--\n 1 files changed, 4 insertions(+), 2 deletions(-)\n\ndiff --git a/git-pull.sh b/git-pull.sh\nindex f6b7b84..63b063a 100755\n--- a/git-pull.sh\n+++ b/git-pull.sh\n@@ -53,6 +53,8 @@ do\n \t\tverbosity=\"$verbosity -v\" ;;\n \t--progress)\n \t\tprogress=--progress ;;\n+\t--no-progress)\n+\t\tprogress=--no-progress ;;\n \t-n|--no-stat|--no-summary)\n \t\tdiffstat=--no-stat ;;\n \t--stat|--summary)\n@@ -293,8 +295,8 @@ true)\n \t;;\n *)\n \teval=\"git-merge $diffstat $no_commit $squash $no_ff $ff_only\"\n-\teval=\"$eval  $log_arg $strategy_args $merge_args\"\n-\teval=\"$eval \\\"\\$merge_name\\\" HEAD $merge_head $verbosity\"\n+\teval=\"$eval  $log_arg $strategy_args $merge_args $verbosity $progress\"\n+\teval=\"$eval \\\"\\$merge_name\\\" HEAD $merge_head\"\n \t;;\n esac\n eval \"exec $eval\"\n-- \n1.7.4.1.26.g5e991\n"},{"id":"161745","messageId":"20110220100424.GB988@sigill.intra.peff.net","threadId":"26552","inReplyTo":"AANLkTi=s6FxnYVv3UinV4Q+4C_9dX9rCLQQoCFD7kG9S@mail.gmail.com","subject":"Re: [RFC/PATCH 4/4] inexact rename detection eye candy","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2011-02-20T10:04:24Z","receivedAt":"2011-02-20T10:04:24Z","isPatch":true,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Sat, Feb 19, 2011 at 11:29:33AM -0500, Martin Langhoff wrote:\n\n> On Sat, Feb 19, 2011 at 5:25 AM, Jeff King <peff@peff.net> wrote:\n> > During a merge, we might spend many seconds doing inexact\n> > rename detection. It's nice to let the user know that\n> > something is actually happening.\n> \n> Given that we're doing costly work, could the results of rename\n> matching be recorded in a .git/MERGE_RENAMES file?\n> \n> If that file's available, the simple merge-helper script I've\n> described is possible...\n\nYeah we can if it's useful, but I'd rather see a prototype merge helper\nfirst. In particular, your merge helper seems like it would help in two\nsituations:\n\n  1. In a rename/rename conflict, you have X becoming Y on one branch\n     and Z on the other. You want to say \"X should have become Y\", and\n     then have it 3-way merge X, Y, and Z, storing the result under the\n     name Y.\n\n     For that do we actually want the rename list, or just the list of\n     rename/rename conflicts?\n\n  2. If git misses a rename, then by definition won't that rename not be\n     in the MERGE_RENAMES list? I guess it may have picked up the rename\n     on one half of history, and you could use that?\n\n     I'm not quite clear on the exact usage.\n\nSo it makes sense to me to get a working merge helper that just uses\n\"git diff\" as appropriate to look up old renames, and then we'll know\nwhat information is required.\n\n-Peff\n"},{"id":"161746","messageId":"20110220101024.GC988@sigill.intra.peff.net","threadId":"26552","inReplyTo":"AANLkTim2ETC0T8=8D5pWstZRZ95eo5V0+Pr62mNXRBH1@mail.gmail.com","subject":"Re: [PATCH 2/4] bump rename limit defaults (again)","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2011-02-20T10:10:24Z","receivedAt":"2011-02-20T10:10:24Z","isPatch":true,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Sat, Feb 19, 2011 at 06:54:47PM +0100, Piotr Krukowiecki wrote:\n\n> On Sat, Feb 19, 2011 at 11:21 AM, Jeff King <peff@peff.net> wrote:\n> > Let's bump the diff/merge default limits from 200/500 to\n> > 400/1000. Those are 2 seconds and 10 seconds respectively on\n> > my modern hardware.\n> \n> Just curious - what is your modern hardware?\n\nIt's a Core i7 840QM. So it's a measly 1.8GHz, but it turboboosts up to\n3.2GHz on a single-threaded process.  The multiple cores are irrelevant\nfor this timing, as it's very single-threaded[1].\n\nThe process used in the tens of megabytes of memory, so I think memory\nsize is irrelevant. It was very CPU bound in my tests.\n\n-Peff\n\n[1] Actually, this is one place where multi-threading the algorithm\nwould be very easy. It's literally an m*n double-loop, seeing which of\neach \"n\" sources matches each \"m\" destination best, and the expensive\nbit is comparing a source and dest. So you could parallelize score\ncalculation up to m*n ways, then just sort the result for each dest.\n"},{"id":"161747","messageId":"20110220101247.GD988@sigill.intra.peff.net","threadId":"26552","inReplyTo":"AANLkTi=aWC_sXK1VBspsiVAwhkC4pP8Z+uq7QceLMopS@mail.gmail.com","subject":"Re: [PATCH 2/4] bump rename limit defaults (again)","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2011-02-20T10:12:47Z","receivedAt":"2011-02-20T10:12:47Z","isPatch":true,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Sat, Feb 19, 2011 at 09:12:49PM +0100, Ævar Arnfjörð Bjarmason wrote:\n\n> On Sat, Feb 19, 2011 at 11:21, Jeff King <peff@peff.net> wrote:\n> \n> > Let's bump the diff/merge default limits from 200/500 to\n> > 400/1000. Those are 2 seconds and 10 seconds respectively on\n> > my modern hardware.\n> \n> This is somewhat outside the scope of your patch, but rather than\n> making these decisions for the user and compiling them into Git\n> wouldn't it be better to expose it somehow?\n\nWhat do you mean by expose it? We already have config options; this is\njust for the defaults.\n\nOr do you mean allowing setting merge.renamelimits to \"30s\" and cutting\noff detection after 30 seconds? I think that would be a reasonable idea,\nthough part of me doesn't like non-deterministic answers based on how\nloaded the system is.\n\n> E.g. when you do a git-merge show a progress bar (similar to\n> git-clone) showing that we're trying to do rename detection, show that\n> we stopped, and when we fail point out that the user could run\n> git-merge with some switch that would make Git try harder.\n\nEr, isn't that exactly what patches 1/4 and 4/4 in my series do?\n\n-Peff\n"},{"id":"161750","messageId":"20110220103703.GA1527@sigill.intra.peff.net","threadId":"26552","inReplyTo":"20110220094803.GA988@sigill.intra.peff.net","subject":"Re: [RFC/PATCH 4/4] inexact rename detection eye candy","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2011-02-20T10:37:03Z","receivedAt":"2011-02-20T10:37:03Z","isPatch":true,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Sun, Feb 20, 2011 at 04:48:04AM -0500, Jeff King wrote:\n\n> (Actually that is not quite accurate, as we may skip some calls for\n> destinations already found, but it's not worth the effort to figure\n> out how many calls we'll actually make ahead of time).\n\nHmm. Actually, we do calculate the exact amount of work a few lines\nabove. So we could be more accurate. I'm still not sure it really\nmatters.\n\n-Peff\n"},{"id":"161757","messageId":"AANLkTinO0h+WF-UKM=g=n_Vk43EnGCLtvj=4pCcSzVYh@mail.gmail.com","threadId":"26552","inReplyTo":"20110220100424.GB988@sigill.intra.peff.net","subject":"Re: [RFC/PATCH 4/4] inexact rename detection eye candy","fromName":"Martin Langhoff","fromEmail":"martin.langhoff@gmail.com","sentAt":"2011-02-20T13:16:41Z","receivedAt":"2011-02-20T13:16:41Z","isPatch":true,"sender":{"key":"martin.langhoff@gmail.com","avatar":"https://gravatar.com/avatar/1e3f311b6c4c15836501901ca58f8c0b0667246488084ba524d8bc9867e22fd9?d=mp&s=160"},"body":"On Sun, Feb 20, 2011 at 5:04 AM, Jeff King <peff@peff.net> wrote:\n> Yeah we can if it's useful, but I'd rather see a prototype merge helper\n> first. In particular, your merge helper seems like it would help in two\n> situations:\n\nKind of circular -- the helper depends on\n\n - What unresolved cases do we have?\n\n - What data can you store from the diff machinery?\n\ncheers,\n\n\n\nm\n-- \n martin.langhoff@gmail.com\n martin@laptop.org -- Software Architect - OLPC\n - ask interesting questions\n - don't get distracted with shiny stuff  - working code first\n - http://wiki.laptop.org/go/User:Martinlanghoff\n"},{"id":"161844","messageId":"7vzkppgdz9.fsf@alter.siamese.dyndns.org","threadId":"26552","inReplyTo":"20110219102051.GA22508@sigill.intra.peff.net","subject":"Re: [PATCH 1/4] merge: improve inexact rename limit warning","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2011-02-21T23:33:14Z","receivedAt":"2011-02-21T23:33:14Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Jeff King <peff@peff.net> writes:\n\n> The warning is generated deep in the diffcore code, which\n> means that it will come first, followed possibly by a spew\n> of conflicts, making it hard to see.\n>\n> Instead, let's have diffcore pass back the information about\n> how big the rename limit would needed to have been, and then\n> the caller can provide a more appropriate message (and at a\n> more appropriate time).\n>\n> No refactoring of other non-merge callers is necessary,\n> because nobody else was even using the warn_on_rename_limit\n> feature.\n>\n> Signed-off-by: Jeff King <peff@peff.net>\n\nThanks.\n\nThis conflicts with 2840824 (diffcore-rename: fall back to -C when -C -C\nbusts the rename limit, 2011-01-06) on 'pu', unfortunately.\n"},{"id":"161887","messageId":"20110222153909.GB27178@sigill.intra.peff.net","threadId":"26552","inReplyTo":"7vzkppgdz9.fsf@alter.siamese.dyndns.org","subject":"Re: [PATCH 1/4] merge: improve inexact rename limit warning","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2011-02-22T15:39:09Z","receivedAt":"2011-02-22T15:39:09Z","isPatch":true,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Mon, Feb 21, 2011 at 03:33:14PM -0800, Junio C Hamano wrote:\n\n> > Instead, let's have diffcore pass back the information about\n> > how big the rename limit would needed to have been, and then\n> > the caller can provide a more appropriate message (and at a\n> > more appropriate time).\n> [...]\n> This conflicts with 2840824 (diffcore-rename: fall back to -C when -C -C\n> busts the rename limit, 2011-01-06) on 'pu', unfortunately.\n\nDo you want to do the merge, or do you want me to rebase on top of\n2840824?\n\n-Peff\n"},{"id":"161903","messageId":"7vsjvfdhzy.fsf@alter.siamese.dyndns.org","threadId":"26552","inReplyTo":"20110222153909.GB27178@sigill.intra.peff.net","subject":"Re: [PATCH 1/4] merge: improve inexact rename limit warning","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2011-02-22T18:46:57Z","receivedAt":"2011-02-22T18:46:57Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Jeff King <peff@peff.net> writes:\n\n> On Mon, Feb 21, 2011 at 03:33:14PM -0800, Junio C Hamano wrote:\n> ...\n>> This conflicts with 2840824 (diffcore-rename: fall back to -C when -C -C\n>> busts the rename limit, 2011-01-06) on 'pu', unfortunately.\n>\n> Do you want to do the merge, or do you want me to rebase on top of\n> 2840824?\n\nPlease check the resolution queued on 'pu'; I suspect that your series\nshould graduate before the fall-back-to-c-from-c-c topic, so I'd rather\nnot to see you rebase this.\n"},{"id":"162006","messageId":"20110223080232.GA2724@sigill.intra.peff.net","threadId":"26552","inReplyTo":"7vsjvfdhzy.fsf@alter.siamese.dyndns.org","subject":"Re: [PATCH 1/4] merge: improve inexact rename limit warning","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2011-02-23T08:02:33Z","receivedAt":"2011-02-23T08:02:33Z","isPatch":true,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Tue, Feb 22, 2011 at 10:46:57AM -0800, Junio C Hamano wrote:\n\n> >> This conflicts with 2840824 (diffcore-rename: fall back to -C when -C -C\n> >> busts the rename limit, 2011-01-06) on 'pu', unfortunately.\n> >\n> > Do you want to do the merge, or do you want me to rebase on top of\n> > 2840824?\n> \n> Please check the resolution queued on 'pu'; I suspect that your series\n> should graduate before the fall-back-to-c-from-c-c topic, so I'd rather\n> not to see you rebase this.\n\nIt looks sane to me. I was going to say we should also have\nmerge-recursive check for the fallback-to-c flag and tell the user what\nhappened, but I don't think merge-recursive ever has FIND_COPIES_HARDER\nenabled.\n\nWhich I think means in 2840824 itself, the warning about fallback can\nnever be triggered (because merge-recursive is the only caller who\nturned on warn_on_too_large_rename).\n\n-Peff\n"}]}