{"thread":{"id":"9803","subject":"What's cooking in git.git (topics)","startedAt":"2007-09-06T08:52:57Z","lastAt":"2007-10-04T07:10:48Z","messageCount":44,"participants":["Junio C Hamano","Shawn O. Pearce","Johannes Schindelin","Carlos Rica","Tom Clarke","Jeff King","David Kastrup","Daniel Barkalow","Steven Grimm","Steffen Prohaska","Matthieu Moy","David Kågedal","Linus Torvalds"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"52714","messageId":"7v1wdcch06.fsf@gitster.siamese.dyndns.org","threadId":"9803","inReplyTo":null,"subject":"What's cooking in git.git (topics)","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2007-09-06T08:52:57Z","receivedAt":"2007-09-06T08:52:57Z","isPatch":false,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Here are the topics that have been cooking. Commits prefixed\nwith '-' are only in 'pu' while commits prefixed with '+' are\nin 'next'. The topics list the commits in reverse chronological\norder.\n\n* js/tag (Mon Sep 3 17:51:43 2007 +0100) 1 commit\n + verify-tag: also grok CR/LFs in the tag signature\n\nLooks correct. Merge to 'master' this weekend.\n\n* lh/svn-first-parent (Wed Sep 5 11:35:29 2007 +0200) 1 commit\n + git-svn: add support for --first-parent\n\nQueued to 'next' with Eric's blessing. Perhaps merge to\n'master' by the end of next week unless there are issues.\n\n* rs/archive (Mon Sep 3 20:08:01 2007 +0200) 3 commits\n + Remove unused function convert_sha1_file()\n + archive: specfile support (--pretty=format: in archive files)\n + Export format_commit_message()\n\nWaiting for the \"$Format: ...$\" updates.\n\n* js/remote (Sun Sep 2 21:10:14 2007 +0100) 1 commit\n + Teach \"git remote\" a mirror mode\n\nWaiting for tests. We should resurrect earlier \"git remote rm\"\nand add tests for it as well.\n\n* jc/stash-create (Mon Jul 9 00:51:23 2007 -0700) 2 commits\n - rebase: allow starting from a dirty tree.\n - stash: implement \"stash create\"\n\nA quick hack to allow starting \"git rebase\" in a dirty work tree\nby automatically stashing the changes first, and unstashing them\nafter rebase is done. Needs tests and documentation.\n\n* np/delta (Thu Sep 6 02:13:11 2007 -0400) 4 commits\n - basic threaded delta search\n - rearrange delta search progress reporting\n - localize window memory usage accounting\n - straighten the list of objects to deltify\n\nI do not know where Nico's \"threaded pack generation\" would lead\nus to yet, so they are parked on 'pu' for now. The first in the\nseries should be applicable to 'next', though.\n\n* jc/pack (Sat Sep 1 23:53:47 2007 -0700) 1 commit\n + Keep last used delta base in the delta window\n\nWould need to straighten out the implementation from the one\nthat is suited for the original FIFO usage to another that is\nmore appropriate for LRU.\n\n* jc/autogc (Wed Sep 5 14:59:59 2007 -0700) 2 commits\n - Invoke \"git gc --auto\" from commit, merge, am and rebase.\n - Implement git gc --auto\n\nThis has been updated since the ones I sent to the list earlier\nin the day. It detects a situation where the user has too much\ncruft in the repository that too many loose objects are left\nunpruned, and issues a warning. Also 'rebase' is covered by\nrunning \"git gc --auto\" from either merge or am.\n\n* ph/strbuf (Wed Sep 5 21:18:43 2007 +0200) 7 commits\n - Use strbuf in cache-tree.c\n - Use strbuf in buitin-rerere.c\n - Use strbuf in apply, blame, commit-tree and diff\n - mktree: Simplify write_tree() using strbuf's.\n - fast-import: Use strbuf API, and simplify cmd_data()\n - Simplify strbuf uses in archive-tar.c using strbuf API\n - Rework strbuf API and semantics.\n\nThe idea is good, and removes more code than it adds, but I find\nit not 'next' material yet. I haven't checked every single line\nyet, and this series needs that kind of vetting.\n\n* jc/pathspec (Tue Sep 4 02:47:25 2007 -0700) 1 commit\n - tree-diff.c: split out a function to match a single pattern.\n\nJust started and not even started to cause breakage yet ;-).\nI'd want to fix pathspec semantics of \"diff-tree\", \"log\" and\n\"ls-tree\" so that they understand globs in addition to leading\ndirectory prefix, just like \"ls-files\", \"diff-files\",\n\"diff-index\" and \"grep\" does.\n\n* jc/diff (Mon Dec 25 01:08:50 2006 -0800) 2 commits\n - test-para: combined diff between HEAD, index and working tree.\n - para-walk: walk n trees, index and working tree in parallel\n\nHave been on hold for a long time. This is about traversing the\nindex, work tree and zero or more trees in parallel, which is\none way to rewrite the merge backend. I may end up reusing\nmerge-tree.c implementation which would make this series\nunnecessary.\n"},{"id":"53043","messageId":"20070914183028.GV3099@spearce.org","threadId":"9803","inReplyTo":"7v1wd1d0le.fsf@gitster.siamese.dyndns.org","subject":"Re: What's cooking in git.git (topics)","fromName":"Shawn O. Pearce","fromEmail":"spearce@spearce.org","sentAt":"2007-09-14T18:30:28Z","receivedAt":"2007-09-14T18:30:28Z","isPatch":false,"sender":{"key":"spearce@spearce.org","avatar":"https://avatars.githubusercontent.com/u/34844?v=4"},"body":"Junio C Hamano wrote:\n> * db/fetch-pack (Fri Sep 14 03:31:25 2007 -0400) 22 commits\n...\n> This is Daniel's fetch-pack in C plus fixups from Shawn.\n> Unfortunately the fixups breaks t3200 (\"*** glibc detected ***\n> fetch: free(): invalid pointer xxx ***\"), which I haven't looked\n> into yet.\n\nDoesn't crash out on my Mac OS X system but I am getting the\nabove failure on my amd64 Linux system. I'm debugging it now.\nI'll have to quit in about an hour and pick it up later, so don't\nexpect a patch immediately. But I'll certainly send something soon.\nClearly I made a change in my fixups that I shouldn't have. ;-)\n\n-- \nShawn.\n"},{"id":"53084","messageId":"Pine.LNX.4.64.0709150047030.28586@racer.site","threadId":"9803","inReplyTo":"7v1wd1d0le.fsf@gitster.siamese.dyndns.org","subject":"Re: What's cooking in git.git (topics)","fromName":"Johannes Schindelin","fromEmail":"johannes.schindelin@gmx.de","sentAt":"2007-09-14T23:47:53Z","receivedAt":"2007-09-14T23:47:53Z","isPatch":false,"sender":{"key":"johannes.schindelin@gmx.de","avatar":"https://avatars.githubusercontent.com/u/127790?v=4"},"body":"Hi,\n\nOn Fri, 14 Sep 2007, Junio C Hamano wrote:\n\n> * cr/reset (Fri Sep 14 01:19:30 2007 -0700) 5 commits\n> + Simplify cache API\n> + An additional test for \"git-reset -- path\"\n> + Make \"git reset\" a builtin.\n> + Move make_cache_entry() from merge-recursive.c into read-cache.c\n> + Add tests for documented features of \"git reset\".\n> \n> I found \"git reset commit paths...\" had problem in this series,\n> which was why jc/cachetree is merged into this topic to fix it.\n> Hopefully we can put this in 'master' soon, after giving it\n> another and final round of eyeballing.\n\nI think this is my bug. The initial reset-with-paths functionality in the \nbuiltin reset came out of my feather... Sorry!\n\nCiao,\nDscho\n"},{"id":"53100","messageId":"20070915074759.GA3099@spearce.org","threadId":"9803","inReplyTo":"20070914183028.GV3099@spearce.org","subject":"Re: db/fetch-pack topic (was: What's cooking in git.git (topics))","fromName":"Shawn O. Pearce","fromEmail":"spearce@spearce.org","sentAt":"2007-09-15T07:47:59Z","receivedAt":"2007-09-15T07:47:59Z","isPatch":false,"sender":{"key":"spearce@spearce.org","avatar":"https://avatars.githubusercontent.com/u/34844?v=4"},"body":"\"Shawn O. Pearce\" wrote:\n> Junio C Hamano wrote:\n> > * db/fetch-pack (Fri Sep 14 03:31:25 2007 -0400) 22 commits\n> ...\n> > This is Daniel's fetch-pack in C plus fixups from Shawn.\n> > Unfortunately the fixups breaks t3200 (\"*** glibc detected ***\n> > fetch: free(): invalid pointer xxx ***\"), which I haven't looked\n> > into yet.\n> \n> Doesn't crash out on my Mac OS X system but I am getting the\n> above failure on my amd64 Linux system.\n\nOK, so in addition to the above fixups Junio mentions I have\nalso sent two more series today:\n\n 2 patch \"fixup of the fixup\" to resolve the t3200 crash\n 5 patch \"fixup + cleanup\" of http support\n\nand now I just discovered that pushing to yourself is probably also\nbroken by this series:\n\n $ git push . jc/maint:gfi-maint\n updating 'refs/heads/gfi-maint' using 'refs/remotes/jc/maint'\n from 05cc2ffc572f05e8aeec495a9ab9bc9609863491\n to 8419d2ee9ba8b375186a5c1019df8dfbce610aba\n Also local refs/heads/gfi-maint\n Generating pack...\n Done counting 0 objects.\n Writing 0 objects...\n Unpacking 0 objects...\n error: Ref refs/heads/gfi-maint is at 8419d2ee9ba8b375186a5c1019df8dfbce610aba but expected 05cc2ffc572f05e8aeec495a9ab9bc9609863491\n error: failed to lock refs/heads/gfi-maint\n Total 0 (delta 0), reused 0 (delta 0)\n ng refs/heads/gfi-maint failed to lock\n error: failed to push to '.'\n\nWhat's really exciting is we actually updated the ref 'gfi-maint',\neven though it was \"ng\" and we failed to push. Yup. More work\nfor me to look at tomorrow. Right now I think I'm all memory\ncorruptioned out for *** brain detected *** Shawn(): nextword():\ntoo tired, try sleep ***\n\n;-)\n\n-- \nShawn.\n"},{"id":"53166","messageId":"20070916040321.GC3099@spearce.org","threadId":"9803","inReplyTo":"20070915074759.GA3099@spearce.org","subject":"Re: db/fetch-pack topic (was: What's cooking in git.git (topics))","fromName":"Shawn O. Pearce","fromEmail":"spearce@spearce.org","sentAt":"2007-09-16T04:03:21Z","receivedAt":"2007-09-16T04:03:21Z","isPatch":false,"sender":{"key":"spearce@spearce.org","avatar":"https://avatars.githubusercontent.com/u/34844?v=4"},"body":"> \"Shawn O. Pearce\" wrote:\n> > Junio C Hamano wrote:\n> > > * db/fetch-pack (Fri Sep 14 03:31:25 2007 -0400) 22 commits\n> > ...\n> > > This is Daniel's fetch-pack in C plus fixups from Shawn.\n> > > Unfortunately the fixups breaks t3200 (\"*** glibc detected ***\n> > > fetch: free(): invalid pointer xxx ***\"), which I haven't looked\n> > > into yet.\n\nWith all of my fixups applied I took some performance measurements\ntoday from two of my larger day-job repositories:\n\n A.git (356 branches):\n Current Fetch New \"Builtin\" Fetch\n ------------------ --------------------\n real 3m19.921s real 0m9.969s\n user 1m58.050s user 0m1.449s\n sys 2m4.747s sys 0m4.465s\n\n B.git (24 branches):\n Current Fetch New \"Builtin\" Fetch\n ------------------ --------------------\n real 0m21.250s real 0m4.735s\n user 0m10.981s user 0m0.510s\n sys 0m12.387s sys 0m1.481s\n\nAll runs were a no-op fetch over SSH on the LAN. The system running\ngit-fetch was a Cygwin/Windows 2000 box and the server was Solaris 9.\nJust starting an SSH connection (`ssh solsrv echo hi`) takes me on\naverage 1 second so a 4.7s no-op fetch on the smaller repository\nis very respectable.\n\n3m19s vs. 9s? I'll take 9s, thankyouverymuch. Even if I have\nmore testing and debugging to do. Especially since the two repos\nabove are only a sampling of the actual set I have to deal with on\na daily basis.\n\n-- \nShawn.\n"},{"id":"54114","messageId":"7vfy11yyxk.fsf@gitster.siamese.dyndns.org","threadId":"9803","inReplyTo":"7v1wd1d0le.fsf@gitster.siamese.dyndns.org","subject":"What's cooking in git.git (topics)","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2007-09-26T20:05:59Z","receivedAt":"2007-09-26T20:05:59Z","isPatch":false,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Here are the topics that have been cooking. Commits prefixed\nwith '-' are only in 'pu' while commits prefixed with '+' are\nin 'next'. The topics list the commits in reverse chronological\norder.\n\n* jk/diff-rename (Tue Sep 25 15:29:42 2007 -0400) 1 commit\n + diffcore-rename: cache file deltas\n\nParked in 'next' for now but is 'master' material.\n\n* mv/unknown (Tue Sep 25 16:38:46 2007 +0200) 1 commit\n + Don't use \"\" for placeholders and suppress printing of\n empty user formats.\n\nParked in 'next'; I was already burned by it not passing one of\nthe test cases, and I am not absolutely certain what else this\nsubtly breaks. Hopefully minor.\n\n* jb/remote-rm (Sun Sep 23 22:29:12 2007 -0700) 3 commits\n + git-remote rm: add tests and minor fix-ups\n + remote: document the 'rm' subcommand\n + remote: add 'rm' subcommand\n\nShould be Ok to push out to 'master'.\n\n* ml/submodule (Sun Sep 23 22:19:42 2007 -0400) 1 commit\n + git-submodule - allow a relative path as the subproject url\n\nShould be Ok to push out to 'master'.\n\n* lh/merge (Mon Sep 24 00:51:45 2007 +0200) 6 commits\n + git-merge: add --ff and --no-ff options\n + git-merge: add support for --commit and --no-squash\n + git-merge: add support for branch..mergeoptions\n + git-merge: refactor option parsing\n + git-merge: fix faulty SQUASH_MSG\n + Add test-script for git-merge porcelain\n\nComments? I personally never felt need for --no-ff but the\nseries is reasonably clean so I do not see strong objection\nagainst this series either.\n\n* sv/svn (Fri Sep 21 15:27:01 2007 +1200) 3 commits\n + git-svn: handle changed svn command-line syntax\n + git-svn: fix test for trunk svn (transaction out of date)\n + git-svn: fix test for trunk svn (commit message not needed)\n\nWill merge to 'master' this weekend.\n\n* js/rebase-i (Tue Sep 25 16:43:15 2007 +0100) 1 commit\n + rebase -i: work on a detached HEAD\n\nWaiting for autogc change as this textually interacts with it,\nand the additional convenience can wait.\n\n* jc/autogc (Mon Sep 17 00:55:13 2007 -0700) 10 commits\n + git-gc --auto: run \"repack -A -d -l\" as necessary.\n + git-gc --auto: restructure the way \"repack\" command line is built.\n + git-gc --auto: protect ourselves from accumulated cruft\n + git-gc --auto: add documentation.\n + git-gc --auto: move threshold check to need_to_gc() function.\n + repack -A -d: use --keep-unreachable when repacking\n + pack-objects --keep-unreachable\n + Export matches_pack_name() and fix its return value\n + Invoke \"git gc --auto\" from commit, merge, am and rebase.\n + Implement git gc --auto\n\nI think the only remaining thing left with this thing is to\nprevent more than one instances of it from running at the same\ntime. Any takers?\n\n* ph/strbuf (Tue Sep 25 10:22:44 2007 +0200) 37 commits\n + Small cache_tree_write refactor.\n + Make builtin-rerere use of strbuf nicer and more efficient.\n + Add strbuf_cmp.\n + strbuf_setlen(): do not barf on setting length of an empty buffer\n to 0\n + sq_quote_argv and add_to_string rework with strbuf's.\n + Full rework of quote_c_style and write_name_quoted.\n + ...\n\nI had to make a small fix-up to strbuf_setlen() last night to\nthis series; this should be ready for 'master'.\n\nAnd it is better to push this out early, as the series touches\neverywhere and conflicts with peoples' patches.\n\n* db/fetch-pack (Tue Sep 25 00:13:25 2007 -0400) 45 commits\n + Prevent send-pack from segfaulting when a branch doesn't match\n + Cleanup unnecessary break in remote.c\n + Cleanup style nit of 'x == NULL' in remote.c\n + Fix memory leaks when disconnecting transport instances\n + Ensure builtin-fetch honors {fetch,transfer}.unpackLimit\n + ...\n\nTwo issues known to me are:\n\n - \"rsync\" transport is not supported yet;\n\n - regresses \"git pull \" using .git/remotes/; does\n not merge the first refspec when branch..merge is not\n set.\n\nThere may be others but some people apparently use this in\nproduction (including me) and I do not expect major breakages in\nthe really essential part.\n\n* ss/svnimport (Mon Sep 24 12:57:40 2007 +0200) 1 commit\n + Fix pool handling in git-svnimport to avoid memory leaks.\n\nThis is meant to eventually go to 'maint' as well but with\ndiminishing user base of svnimport it is getting harder to get\ngood \"tested successfully, seen improvements\" reports.\n\n* jc/stash-create (Mon Jul 9 00:51:23 2007 -0700) 2 commits\n + rebase: allow starting from a dirty tree.\n + stash: implement \"stash create\"\n\nI think \"stash create\" is going in a good direction, but I do\nnot think rebase should unstash unconditionally on the resulting\nwork tree. A good compromise might be not to unstash if the\nuser asked to switch branches first and to unstash if he didn't.\n\n* kh/commit (Mon Sep 17 20:06:48 2007 -0400) 7 commits\n - Implement git commit as a builtin command.\n - Export rerere() and launch_editor().\n - Add strbuf_read_file().\n - Clean up stripspace a bit, use strbuf even more.\n - Introduce entry point for launching add--interactive.\n - Enable wt-status to run against non-standard index file.\n - Enable wt-status output to a given FILE pointer.\n\nThere were a few updates/replacements to the list I missed;\n\n* gr/smtp (Tue Sep 25 17:27:54 2007 -0700) 2 commits\n - [TO BE SQUASHED] Fix-up after review\n - Add ability to specify SMTP server port when using git-send-email.\n\nWill be in 'next'.\n"},{"id":"54118","messageId":"1b46aba20709261407g51fa83e2la96e712ffdd905c2@mail.gmail.com","threadId":"9803","inReplyTo":"Pine.LNX.4.64.0709150047030.28586@racer.site","subject":"Re: What's cooking in git.git (topics)","fromName":"Carlos Rica","fromEmail":"jasampler@gmail.com","sentAt":"2007-09-26T21:07:56Z","receivedAt":"2007-09-26T21:07:56Z","isPatch":false,"sender":{"key":"jasampler@gmail.com","avatar":null},"body":"2007/9/15, Johannes Schindelin :\n> On Fri, 14 Sep 2007, Junio C Hamano wrote:\n>\n> > * cr/reset (Fri Sep 14 01:19:30 2007 -0700) 5 commits\n> > + Simplify cache API\n> > + An additional test for \"git-reset -- path\"\n> > + Make \"git reset\" a builtin.\n> > + Move make_cache_entry() from merge-recursive.c into read-cache.c\n> > + Add tests for documented features of \"git reset\".\n> >\n> > I found \"git reset commit paths...\" had problem in this series,\n> > which was why jc/cachetree is merged into this topic to fix it.\n> > Hopefully we can put this in 'master' soon, after giving it\n> > another and final round of eyeballing.\n>\n> I think this is my bug. The initial reset-with-paths functionality in the\n> builtin reset came out of my feather... Sorry!\n\nI'm sorry for not being able to review that code, but now it was\na bit too hard for my current knowledge of the git's internals.\nThese days I'm very busy here and I cannot continue with that work,\nbut I will be soon in git to learn more. Please, be free to rework the\ncode of builtin-reset.c without waiting for me if anyone is interested\nin get it done ASAP.\n\n--\nCarlos\n"},{"id":"54121","messageId":"Pine.LNX.4.64.0709262159430.28395@racer.site","threadId":"9803","inReplyTo":"7vfy11yyxk.fsf@gitster.siamese.dyndns.org","subject":"Re: What's cooking in git.git (topics)","fromName":"Johannes Schindelin","fromEmail":"johannes.schindelin@gmx.de","sentAt":"2007-09-26T21:44:46Z","receivedAt":"2007-09-26T21:44:46Z","isPatch":false,"sender":{"key":"johannes.schindelin@gmx.de","avatar":"https://avatars.githubusercontent.com/u/127790?v=4"},"body":"Hi,\n\nOn Wed, 26 Sep 2007, Junio C Hamano wrote:\n\n> * mv/unknown (Tue Sep 25 16:38:46 2007 +0200) 1 commit\n> + Don't use \"\" for placeholders and suppress printing of\n> empty user formats.\n> \n> Parked in 'next'; I was already burned by it not passing one of the test \n> cases, and I am not absolutely certain what else this subtly breaks. \n> Hopefully minor.\n\nI guess a few scripts could maybe rely on this behaviour. We should \nadvertise it as such.\n\n> * lh/merge (Mon Sep 24 00:51:45 2007 +0200) 6 commits\n> + git-merge: add --ff and --no-ff options\n> + git-merge: add support for --commit and --no-squash\n> + git-merge: add support for branch..mergeoptions\n> + git-merge: refactor option parsing\n> + git-merge: fix faulty SQUASH_MSG\n> + Add test-script for git-merge porcelain\n> \n> Comments? I personally never felt need for --no-ff but the series is \n> reasonably clean so I do not see strong objection against this series \n> either.\n\nTogether with a resubmitted git-merge-rebase.sh (hint, hint), the \nmergeOptions would be quite useful for a workflow where you want to rebase \non top of an upstream quite often.\n\n> * js/rebase-i (Tue Sep 25 16:43:15 2007 +0100) 1 commit\n> + rebase -i: work on a detached HEAD\n> \n> Waiting for autogc change as this textually interacts with it, and the \n> additional convenience can wait.\n\nSure. I never used it anyway, but you specifically requested it ;-) BTW \nthanks for merging the rest; especially the progress meter was a sore \npoint for me since long.\n\n> * jc/autogc (Mon Sep 17 00:55:13 2007 -0700) 10 commits\n> + git-gc --auto: run \"repack -A -d -l\" as necessary.\n> + git-gc --auto: restructure the way \"repack\" command line is built.\n> + git-gc --auto: protect ourselves from accumulated cruft\n> + git-gc --auto: add documentation.\n> + git-gc --auto: move threshold check to need_to_gc() function.\n> + repack -A -d: use --keep-unreachable when repacking\n> + pack-objects --keep-unreachable\n> + Export matches_pack_name() and fix its return value\n> + Invoke \"git gc --auto\" from commit, merge, am and rebase.\n> + Implement git gc --auto\n> \n> I think the only remaining thing left with this thing is to prevent more \n> than one instances of it from running at the same time. Any takers?\n\nYou mean, just creating a throw-away lock file?\n\n> * ph/strbuf (Tue Sep 25 10:22:44 2007 +0200) 37 commits\n> + Small cache_tree_write refactor.\n> + Make builtin-rerere use of strbuf nicer and more efficient.\n> + Add strbuf_cmp.\n> + strbuf_setlen(): do not barf on setting length of an empty buffer\n> to 0\n> + sq_quote_argv and add_to_string rework with strbuf's.\n> + Full rework of quote_c_style and write_name_quoted.\n> + ...\n> \n> I had to make a small fix-up to strbuf_setlen() last night to this \n> series; this should be ready for 'master'.\n> \n> And it is better to push this out early, as the series touches \n> everywhere and conflicts with peoples' patches.\n\nHehe. Indeed, I had to fix the notes series after rebasing it...\n\n> * db/fetch-pack (Tue Sep 25 00:13:25 2007 -0400) 45 commits\n> + Prevent send-pack from segfaulting when a branch doesn't match\n> + Cleanup unnecessary break in remote.c\n> + Cleanup style nit of 'x == NULL' in remote.c\n> + Fix memory leaks when disconnecting transport instances\n> + Ensure builtin-fetch honors {fetch,transfer}.unpackLimit\n> + ...\n> \n> Two issues known to me are:\n> \n> - \"rsync\" transport is not supported yet;\n\nI promised to do this, and so I will today.\n\n> * jc/stash-create (Mon Jul 9 00:51:23 2007 -0700) 2 commits\n> + rebase: allow starting from a dirty tree.\n> + stash: implement \"stash create\"\n> \n> I think \"stash create\" is going in a good direction, but I do not think \n> rebase should unstash unconditionally on the resulting work tree. A \n> good compromise might be not to unstash if the user asked to switch \n> branches first and to unstash if he didn't.\n\nSounds like a sensible change to me; maybe a little warning after the \nrebase?\n\nI have no idea if I come around to do the same for rebase--interactive any \ntime soon, though.\n\nCiao,\nDscho\n"},{"id":"54122","messageId":"550f9510709261453l6ff50017s9145ee234f39a07@mail.gmail.com","threadId":"9803","inReplyTo":"Pine.LNX.4.64.0709262159430.28395@racer.site","subject":"Re: What's cooking in git.git (topics)","fromName":"Tom Clarke","fromEmail":"tom@u2i.com","sentAt":"2007-09-26T21:53:38Z","receivedAt":"2007-09-26T21:53:38Z","isPatch":false,"sender":{"key":"tom@u2i.com","avatar":null},"body":"On 9/26/07, Johannes Schindelin wrote:\n> Together with a resubmitted git-merge-rebase.sh (hint, hint), the\n> mergeOptions would be quite useful for a workflow where you want to rebase\n> on top of an upstream quite often.\n\nI'll resubmit merge rebase tomorrow :-)\n\n-Tom\n"},{"id":"54127","messageId":"20070927023633.GA28902@coredump.intra.peff.net","threadId":"9803","inReplyTo":"7vfy11yyxk.fsf@gitster.siamese.dyndns.org","subject":"Re: What's cooking in git.git (topics)","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2007-09-27T02:36:33Z","receivedAt":"2007-09-27T02:36:33Z","isPatch":false,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Wed, Sep 26, 2007 at 01:05:59PM -0700, Junio C Hamano wrote:\n\n> * jk/diff-rename (Tue Sep 25 15:29:42 2007 -0400) 1 commit\n> + diffcore-rename: cache file deltas\n> \n> Parked in 'next' for now but is 'master' material.\n\nMy tests after this patch show that spanhash_find is responsible for\na large portion of the processing time in large renames, so I am going\nto look into speeding that up.\n\n> * lh/merge (Mon Sep 24 00:51:45 2007 +0200) 6 commits\n> + git-merge: add --ff and --no-ff options\n> + git-merge: add support for --commit and --no-squash\n> + git-merge: add support for branch..mergeoptions\n> + git-merge: refactor option parsing\n> + git-merge: fix faulty SQUASH_MSG\n> + Add test-script for git-merge porcelain\n> \n> Comments? I personally never felt need for --no-ff but the\n> series is reasonably clean so I do not see strong objection\n> against this series either.\n\nI like it. I know that --no-ff is frowned upon, but I think previous\ndiscussions have mentioned workflows where it might be used\nintelligently. Since the patch is unlikely to break anything for\ntraditional workflows, I think it is a nice way to let people experiment\nwith alternative workflows that use --no-ff. Maybe something interesting\nwill come of it.\n\n-Peff\n"},{"id":"54137","messageId":"854phgfxn7.fsf@lola.goethe.zz","threadId":"9803","inReplyTo":"20070927023633.GA28902@coredump.intra.peff.net","subject":"Re: What's cooking in git.git (topics)","fromName":"David Kastrup","fromEmail":"dak@gnu.org","sentAt":"2007-09-27T06:08:44Z","receivedAt":"2007-09-27T06:08:44Z","isPatch":false,"sender":{"key":"dak@gnu.org","avatar":"https://avatars.githubusercontent.com/u/52141349?v=4"},"body":"Jeff King writes:\n\n> On Wed, Sep 26, 2007 at 01:05:59PM -0700, Junio C Hamano wrote:\n>\n>> * jk/diff-rename (Tue Sep 25 15:29:42 2007 -0400) 1 commit\n>> + diffcore-rename: cache file deltas\n>> \n>> Parked in 'next' for now but is 'master' material.\n>\n> My tests after this patch show that spanhash_find is responsible for\n> a large portion of the processing time in large renames, so I am going\n> to look into speeding that up.\n\nIn itself, it does not look like there is all too much room for\noptimization. One can remove the temporary pointer \"optimization\" and\nsee whether this makes strength reduction possible for the compiler.\nMaking this an endless loop wrapped around a loop on bucket might also\nhelp the compiler in that effect.\n\nBut there is really not all too much leeway, and it might be better\nspent in the caller. For example, the search will take something like\nr/(1-r) iterations on average where r is the fill ratio of the hash\narray. So one would not want to, say, let r grow above 0.75 or\nsomething like that.\n\n-- \nDavid Kastrup, Kriemhildstr. 15, 44793 Bochum\n"},{"id":"54142","messageId":"85sl50ehh9.fsf@lola.goethe.zz","threadId":"9803","inReplyTo":"854phgfxn7.fsf@lola.goethe.zz","subject":"Re: What's cooking in git.git (topics)","fromName":"David Kastrup","fromEmail":"dak@gnu.org","sentAt":"2007-09-27T06:43:14Z","receivedAt":"2007-09-27T06:43:14Z","isPatch":false,"sender":{"key":"dak@gnu.org","avatar":"https://avatars.githubusercontent.com/u/52141349?v=4"},"body":"David Kastrup writes:\n\n> Jeff King writes:\n>\n>> On Wed, Sep 26, 2007 at 01:05:59PM -0700, Junio C Hamano wrote:\n>>\n>>> * jk/diff-rename (Tue Sep 25 15:29:42 2007 -0400) 1 commit\n>>> + diffcore-rename: cache file deltas\n>>> \n>>> Parked in 'next' for now but is 'master' material.\n>>\n>> My tests after this patch show that spanhash_find is responsible for\n>> a large portion of the processing time in large renames, so I am going\n>> to look into speeding that up.\n>\n> In itself, it does not look like there is all too much room for\n> optimization. One can remove the temporary pointer \"optimization\" and\n> see whether this makes strength reduction possible for the compiler.\n> Making this an endless loop wrapped around a loop on bucket might also\n> help the compiler in that effect.\n>\n> But there is really not all too much leeway, and it might be better\n> spent in the caller. For example, the search will take something like\n> r/(1-r) iterations on average where r is the fill ratio of the hash\n> array. So one would not want to, say, let r grow above 0.75 or\n> something like that.\n\nOk, here is some suggestion:\n\nHere is the inner loop for this stuff:\n\n\tfor (i = 0; i < ssz; i++) {\n\t\tstruct spanhash *s = &(src_count->data[i]);\n\t\tstruct spanhash *d;\n\t\tunsigned dst_cnt, src_cnt;\n\t\tif (!s->cnt)\n\t\t\tcontinue;\n\t\tsrc_cnt = s->cnt;\n\t\td = spanhash_find(dst_count, s->hashval);\n\t\tdst_cnt = d ? d->cnt : 0;\n\t\tif (src_cnt < dst_cnt) {\n\t\t\tla += dst_cnt - src_cnt;\n\t\t\tsc += src_cnt;\n\t\t}\n\t\telse\n\t\t\tsc += dst_cnt;\n\t}\n\nNow here is how one could optimize the data structures: The hash\nstructures are with linear probing, and we try to find any hash\nmatches from source to destination. If we sort all hashes indexed to\na given first hash bucket by their full hash value, then one could\nbasically use passes similar to list merges for figuring the 1:1\nrelations. That cuts down the O(l n) cost (where n is the number of\nelements and l their average run length) to O(n).\n\nOf course, making l close to 1 by keeping the hash utilization\nreasonably low is much simpler.\n\n-- \nDavid Kastrup, Kriemhildstr. 15, 44793 Bochum\n"},{"id":"54167","messageId":"20070927133056.GA29674@coredump.intra.peff.net","threadId":"9803","inReplyTo":"854phgfxn7.fsf@lola.goethe.zz","subject":"Re: What's cooking in git.git (topics)","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2007-09-27T13:30:56Z","receivedAt":"2007-09-27T13:30:56Z","isPatch":false,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Thu, Sep 27, 2007 at 08:08:44AM +0200, David Kastrup wrote:\n\n> In itself, it does not look like there is all too much room for\n> optimization. One can remove the temporary pointer \"optimization\" and\n> see whether this makes strength reduction possible for the compiler.\n> Making this an endless loop wrapped around a loop on bucket might also\n> help the compiler in that effect.\n\nI am considering reworking the data structure to be a hash table whose\nbuckets never overflow. However, Junio indicated that he tried something\nsimilar at one point and was not successful. So we will see. I haven't\nhad time to play with it yet, but I will post numbers when I do.\n\n-Peff\n"},{"id":"54171","messageId":"868x6sgr1g.fsf@lola.quinscape.zz","threadId":"9803","inReplyTo":"20070927133056.GA29674@coredump.intra.peff.net","subject":"Re: What's cooking in git.git (topics)","fromName":"David Kastrup","fromEmail":"dak@gnu.org","sentAt":"2007-09-27T13:46:03Z","receivedAt":"2007-09-27T13:46:03Z","isPatch":false,"sender":{"key":"dak@gnu.org","avatar":"https://avatars.githubusercontent.com/u/52141349?v=4"},"body":"Jeff King writes:\n\n> On Thu, Sep 27, 2007 at 08:08:44AM +0200, David Kastrup wrote:\n>\n>> In itself, it does not look like there is all too much room for\n>> optimization. One can remove the temporary pointer \"optimization\" and\n>> see whether this makes strength reduction possible for the compiler.\n>> Making this an endless loop wrapped around a loop on bucket might also\n>> help the compiler in that effect.\n>\n> I am considering reworking the data structure to be a hash table\n> whose buckets never overflow. However, Junio indicated that he tried\n> something similar at one point and was not successful. So we will\n> see. I haven't had time to play with it yet, but I will post numbers\n> when I do.\n\nLinear probing is pretty efficient with regard to keeping memory\naccess locality. With a reasonable table filling ratio (not more than\nsomething like 75%, for which it is necessary to know the maximum\nnumber of hashable entries in advance), there is no gain to be\nexpected in either speed or even memory usage (the waste of 25% is\noffset by not needing space for link pointers) with escape lists.\nLinear probing hashes are quite hard to resize: if the maximum member\ncount is _not_ to be guessed in advance, things might look different.\n\nI don't have the time to look at the code right now, so I don't know\nwhether resizing or unknown maximum size is a relevant factor.\n\n-- \nDavid Kastrup\n"},{"id":"54224","messageId":"Pine.LNX.4.64.0709272153450.5926@iabervon.org","threadId":"9803","inReplyTo":"7vfy11yyxk.fsf@gitster.siamese.dyndns.org","subject":"Re: What's cooking in git.git (topics)","fromName":"Daniel Barkalow","fromEmail":"barkalow@iabervon.org","sentAt":"2007-09-28T03:24:47Z","receivedAt":"2007-09-28T03:24:47Z","isPatch":false,"sender":{"key":"barkalow@iabervon.org","avatar":"https://avatars.githubusercontent.com/u/55364219?v=4"},"body":"On Wed, 26 Sep 2007, Junio C Hamano wrote:\n\n> * db/fetch-pack (Tue Sep 25 00:13:25 2007 -0400) 45 commits\n> + Prevent send-pack from segfaulting when a branch doesn't match\n> + Cleanup unnecessary break in remote.c\n> + Cleanup style nit of 'x == NULL' in remote.c\n> + Fix memory leaks when disconnecting transport instances\n> + Ensure builtin-fetch honors {fetch,transfer}.unpackLimit\n> + ...\n> \n> Two issues known to me are:\n> \n> - \"rsync\" transport is not supported yet;\n> \n> - regresses \"git pull \" using .git/remotes/; does\n> not merge the first refspec when branch..merge is not\n> set.\n\nActually, I think it's only that it doesn't merge anything if the first \nrefspec doesn't match the first fetched ref from the server. I'm running \ntests on a patch that should fix it, but I'm not sure how to write a test \nfor this particular case; I think the fetch tests try to test this, but \npass by chance.\n\n\t-Daniel\n*This .sig left intentionally blank*\n"},{"id":"54525","messageId":"20071002041652.GA32133@coredump.intra.peff.net","threadId":"9803","inReplyTo":"20070927023633.GA28902@coredump.intra.peff.net","subject":"Re: What's cooking in git.git (topics)","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2007-10-02T04:16:52Z","receivedAt":"2007-10-02T04:16:52Z","isPatch":false,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Wed, Sep 26, 2007 at 10:36:33PM -0400, Jeff King wrote:\n\n> > * jk/diff-rename (Tue Sep 25 15:29:42 2007 -0400) 1 commit\n> > + diffcore-rename: cache file deltas\n> > \n> > Parked in 'next' for now but is 'master' material.\n> \n> My tests after this patch show that spanhash_find is responsible for\n> a large portion of the processing time in large renames, so I am going\n> to look into speeding that up.\n\nJust to update, I tried using a non-colliding hash for this (at the\nexpense of much memory), and I wasn't able to get things much faster\n(and certainly not worth the explosion in memory), short of reducing the\nsize of the hash (which is going to reduce the quality of the output).\nSo I am giving up for the time being, but if others are interested in\ntrying to speed things up, I would be happy to discuss ideas.\n\n-Peff\n"},{"id":"54526","messageId":"7vsl4up0tf.fsf@gitster.siamese.dyndns.org","threadId":"9803","inReplyTo":"20071002041652.GA32133@coredump.intra.peff.net","subject":"Re: What's cooking in git.git (topics)","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2007-10-02T05:01:16Z","receivedAt":"2007-10-02T05:01:16Z","isPatch":false,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Jeff King writes:\n\n> Just to update, I tried using a non-colliding hash for this (at the\n> expense of much memory), and I wasn't able to get things much faster\n> (and certainly not worth the explosion in memory), short of reducing the\n> size of the hash (which is going to reduce the quality of the output).\n> So I am giving up for the time being, but if others are interested in\n> trying to speed things up, I would be happy to discuss ideas.\n\nBummer. You are giving up at the same place I gave up the last\ntime. I was somehow hoping that other people are more clever\nand determined than I was ;-).\n\nThanks for trying.\n"},{"id":"54527","messageId":"20071002050820.GA4261@coredump.intra.peff.net","threadId":"9803","inReplyTo":"7vsl4up0tf.fsf@gitster.siamese.dyndns.org","subject":"Re: What's cooking in git.git (topics)","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2007-10-02T05:08:21Z","receivedAt":"2007-10-02T05:08:21Z","isPatch":false,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Mon, Oct 01, 2007 at 10:01:16PM -0700, Junio C Hamano wrote:\n\n> > Just to update, I tried using a non-colliding hash for this (at the\n> > expense of much memory), and I wasn't able to get things much faster\n> > (and certainly not worth the explosion in memory), short of reducing the\n> > size of the hash (which is going to reduce the quality of the output).\n> > So I am giving up for the time being, but if others are interested in\n> > trying to speed things up, I would be happy to discuss ideas.\n> \n> Bummer. You are giving up at the same place I gave up the last\n> time. I was somehow hoping that other people are more clever\n> and determined than I was ;-).\n> \n> Thanks for trying.\n\nWhat was so discouraging is that I literally simplified the process to\n\n for(i = 0; i < HASH_SIZE; i++)\n if(src[i] < dst[i])\n ...\n\nand it spent all of the time on that one conditional.\n\nOne approach which I haven't tried but might be promising is to actually\nkeep each list sorted, and then do a \"merge\" of the two lists, comparing\nas you go. We don't really need to do arbitrary lookups in the hash; we\njust need to compare two hash tables at a time. My approach was to be\nsimple, but have O(HASH_SIZE) comparisons (where HASH_SIZE is on the\norder of 2^17), and that's clearly just too big. But with a list merge,\nit should be O(n), where n is the actual number of lines in the files\n(or binary chunks for the binary case).\n\n-Peff\n"},{"id":"54528","messageId":"20071002051332.GA4462@coredump.intra.peff.net","threadId":"9803","inReplyTo":"20071002050820.GA4261@coredump.intra.peff.net","subject":"Re: What's cooking in git.git (topics)","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2007-10-02T05:13:32Z","receivedAt":"2007-10-02T05:13:32Z","isPatch":false,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Tue, Oct 02, 2007 at 01:08:20AM -0400, Jeff King wrote:\n\n> One approach which I haven't tried but might be promising is to actually\n> keep each list sorted, and then do a \"merge\" of the two lists, comparing\n> as you go. We don't really need to do arbitrary lookups in the hash; we\n> just need to compare two hash tables at a time. My approach was to be\n> simple, but have O(HASH_SIZE) comparisons (where HASH_SIZE is on the\n> order of 2^17), and that's clearly just too big. But with a list merge,\n> it should be O(n), where n is the actual number of lines in the files\n> (or binary chunks for the binary case).\n\nBTW, I don't want to steal credit for this idea...it comes from thinking\nabout what David Kastrup said earlier in the thread, though I think he\nwas proposing sorting just inside buckets.\n\n-Peff\n"},{"id":"54530","messageId":"7v3awunjup.fsf@gitster.siamese.dyndns.org","threadId":"9803","inReplyTo":"7vfy11yyxk.fsf@gitster.siamese.dyndns.org","subject":"What's cooking in git.git (topics)","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2007-10-02T05:53:02Z","receivedAt":"2007-10-02T05:53:02Z","isPatch":false,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Here are the topics that have been cooking. Commits prefixed\nwith '-' are only in 'pu' while commits prefixed with '+' are\nin 'next'. The topics list the commits in reverse chronological\norder.\n\n----------------------------------------------------------------\n\n* ph/strbuf (Thu Sep 27 13:33:19 2007 +0200) 44 commits\n + Make read_patch_file work on a strbuf.\n + strbuf_read_file enhancement, and use it.\n + strbuf change: be sure ->buf is never ever NULL.\n + double free in builtin-update-index.c\n + Clean up stripspace a bit, use strbuf even more.\n + Add strbuf_read_file().\n ...\n\nWill be in 'master' soon. We've seen nice linecount reduction\nand the end result is rather pleasant to read.\n\n* jc/am-quiet (Mon Oct 1 00:27:51 2007 -0700) 2 commits\n + git-am: fix typo in the previous one.\n + git-am: make the output quieter.\n\nResponse against recent \"rebase being too chatty\" complaints.\nThis should be a 'master' material.\n\n* ap/dateformat (Fri Sep 28 15:17:45 2007 +0100) 3 commits\n + Make for-each-ref's grab_date() support per-atom formatting\n + Make for-each-ref allow atom names like \":\"\n + parse_date_format(): convert a format name to an enum date_mode\n\nWith some test suite additions, this could go to 'master' soon.\nYes, that's a HINT, people ;-).\n\n* je/hooks (Wed Sep 26 15:31:01 2007 -0600) 1 commit\n + post-checkout hook, tests, and docs\n\nWill be in 'master' soon.\n\n* db/fetch-pack (Mon Oct 1 00:59:39 2007 +0100) 49 commits\n + fetch/push: readd rsync support\n + Introduce remove_dir_recursively()\n + bundle transport: fix an alloc_ref() call\n + Allow abbreviations in the first refspec to be merged\n + Prevent send-pack from segfaulting when a branch doesn't match\n + Cleanup unnecessary break in remote.c\n ...\n\nHas been cooking for quite long time.\n\nThere was a regression that made me quite unhappy about the\nrewrite, but Daniel fixed it, so I should be happy. There is\nanother usability regression: http transport is now totally\nsilent. Even when you fetch daily, if the other end frequently\nrepacks everything into one big ball of wax like repo.or.cz\ndoes, you will end up transferring quite a large pack every\ntime, and the total lack of progress report is unacceptably\nunnerving. At least we should reinstate \"Fetching blah from URL\nusing http\", and preferrably \"walk $object_name\" lines. The\nlatter could be replaced with just series of CR + \"walked N\ncommits...\" if we do not like many output from the current \"walk\n$object_name\" lines scrolling the other information away.\n\nI am not sure the quality of \"rsync\" transport near the tip,\neither, but at least the change should not affect other\ntransports. Nobody should using about rsync transport these\ndays anyway. Perhaps we should put a deprecation notice in the\nrelease notes to 1.5.4, and remove it three months later.\n\n* jc/autogc (Mon Sep 17 00:55:13 2007 -0700) 10 commits\n + git-gc --auto: run \"repack -A -d -l\" as necessary.\n + git-gc --auto: restructure the way \"repack\" command line is built.\n + git-gc --auto: protect ourselves from accumulated cruft\n + git-gc --auto: add documentation.\n + git-gc --auto: move threshold check to need_to_gc() function.\n + repack -A -d: use --keep-unreachable when repacking\n + pack-objects --keep-unreachable\n + Export matches_pack_name() and fix its return value\n + Invoke \"git gc --auto\" from commit, merge, am and rebase.\n + Implement git gc --auto\n\nI think this one is reasonably sane, but I was the one who wrote\nit so people should take that with a grain of salt. What it is\nand isn't:\n\n - \"gc --auto\" is a way to prevent you from keeping your\n repository _grossly_ inefficient. Ideally, if you ever\n rapacked your repository once, and do the regular repository\n maintenance (\"'git gc' before you leave for lunch every other\n day\"), it should never trigger.\n\n - \"gc --auto\" is not something you can background. We do not\n want to lock the repository and worry about associated stale\n lock, expiry etc. The complexity is not worth it, compared\n to the stated purpose above (I suspect it might already be\n safe to run multiple instances at the same time, but the\n effort to analyze if it is is not even worth, compared to the\n stated purpose above. Just let it run synchronously if it\n triggers, but it should not trigger for you).\n\n* js/rebase-i (Tue Sep 25 16:43:15 2007 +0100) 1 commits\n + rebase -i: work on a detached HEAD\n\nWill be in 'master', together with \"gc --auto\", soon.\n\n* mv/unknown (Tue Sep 25 16:38:46 2007 +0200) 1 commit\n + Don't use \"\" for placeholders and suppress printing of\n empty user formats.\n\nWill be in 'master' soon.\n\n* lh/merge (Mon Sep 24 00:51:45 2007 +0200) 6 commits\n + git-merge: add --ff and --no-ff options\n + git-merge: add support for --commit and --no-squash\n + git-merge: add support for branch..mergeoptions\n + git-merge: refactor option parsing\n + git-merge: fix faulty SQUASH_MSG\n + Add test-script for git-merge porcelain\n\nWill be in 'master' soon.\n\n* jc/stash-create (Mon Jul 9 00:51:23 2007 -0700) 2 commits\n + rebase: allow starting from a dirty tree.\n + stash: implement \"stash create\"\n\nInstead of refusing to rebase, telling you that your work tree\nis dirty, this stashes your local changes, runs rebase and then\nunstashes automatically. That _sounds_ nicer and easier to use,\nbut I am not sure if it is a wise/sane thing to do. We may want\nto revert the \"autostash\" from rebase. Opinions?\n\n* kh/commit (Mon Sep 17 20:06:47 2007 -0400)\n + Export rerere() and launch_editor().\n + Introduce entry point add_interactive and add_files_to_cache\n + Clean up stripspace a bit, use strbuf even more.\n + Add strbuf_read_file().\n + rerere: Fix use of an empty strbuf.buf\n + Small cache_tree_write refactor.\n ...\n\nStalled.\n\n* jc/pathspec (Thu Sep 13 13:38:19 2007 -0700) 3 commits\n - pathspec_can_match(): move it from builtin-ls-tree.c to tree.c\n - ls-tree.c: refactor show_recursive() and rename it.\n - tree-diff.c: split out a function to match a single pattern.\n\nStalled. This is about my pet-peeve that log (diff-tree) family\nhas much limited pathspec semantics. It should learn to glob\nlike ls-files and grep do.\n"},{"id":"54531","messageId":"86ejge6o8b.fsf@lola.quinscape.zz","threadId":"9803","inReplyTo":"20071002051332.GA4462@coredump.intra.peff.net","subject":"Re: What's cooking in git.git (topics)","fromName":"David Kastrup","fromEmail":"dak@gnu.org","sentAt":"2007-10-02T06:10:28Z","receivedAt":"2007-10-02T06:10:28Z","isPatch":false,"sender":{"key":"dak@gnu.org","avatar":"https://avatars.githubusercontent.com/u/52141349?v=4"},"body":"Jeff King writes:\n\n> On Tue, Oct 02, 2007 at 01:08:20AM -0400, Jeff King wrote:\n>\n>> One approach which I haven't tried but might be promising is to actually\n>> keep each list sorted, and then do a \"merge\" of the two lists, comparing\n>> as you go. We don't really need to do arbitrary lookups in the hash; we\n>> just need to compare two hash tables at a time. My approach was to be\n>> simple, but have O(HASH_SIZE) comparisons (where HASH_SIZE is on the\n>> order of 2^17), and that's clearly just too big. But with a list merge,\n>> it should be O(n), where n is the actual number of lines in the files\n>> (or binary chunks for the binary case).\n>\n> BTW, I don't want to steal credit for this idea...it comes from thinking\n> about what David Kastrup said earlier in the thread, though I think he\n> was proposing sorting just inside buckets.\n\nYes: my proposal was about a microoptimization: work with the\nbasically existing data structures and put the already contained\ninformation to best use.\n\nI have not actually looked at the actual task that the structures are\ngoing to be used in, and whether \"reusing\" the information is likely\nto be worth the trouble.\n\nWhen we are talking about buzzword compliance, \"keep sorted\" with the\nmeaning of \"maintain sorted across modifications\" has an O(n^2) or at\nleast O(nm) ring to it. However, if it is possible to sort it just\nonce, and then then only merge with other lists...\n\nI am actually quite a fan of merge sort and have even posted a small\nand quite efficient version to this list once. However, merge sorts\nwere really greatest at the time when cache memory was unusual to\nhave. Nowadays, quicksort or similar could be faster due to better\nlocality of memory accesses. I think the glibc qsort more or less\nuses an array-based merge into a separate memory area (unless it runs\nout of memory in which case it resorts to regular quicksort).\n\n-- \nDavid Kastrup\n"},{"id":"54536","messageId":"4701E80B.3030007@midwinter.com","threadId":"9803","inReplyTo":"7v3awunjup.fsf@gitster.siamese.dyndns.org","subject":"Re: What's cooking in git.git (topics)","fromName":"Steven Grimm","fromEmail":"koreth@midwinter.com","sentAt":"2007-10-02T06:41:15Z","receivedAt":"2007-10-02T06:41:15Z","isPatch":false,"sender":{"key":"koreth@midwinter.com","avatar":"https://gravatar.com/avatar/71b4d2e8b62f168bdc9e9205341159e3567003b4f9e2127c617c5fa0a1f5bad2?d=mp&s=160"},"body":"Junio C Hamano wrote:\n> * jc/stash-create (Mon Jul 9 00:51:23 2007 -0700) 2 commits\n> + rebase: allow starting from a dirty tree.\n> + stash: implement \"stash create\"\n>\n> Instead of refusing to rebase, telling you that your work tree\n> is dirty, this stashes your local changes, runs rebase and then\n> unstashes automatically. That _sounds_ nicer and easier to use,\n> but I am not sure if it is a wise/sane thing to do. We may want\n> to revert the \"autostash\" from rebase. Opinions?\n> \n\nI can say that for people coming from svn (who are often using \"git svn \nrebase\" rather than directly running \"git rebase\") this is a nice \nworkflow improvement. It eliminates one more \"Why is this more of a pain \nto do in git than in svn?\" complaint.\n\nI don't see any circumstance in my use of git -- either in a git-svn \ncontext or not -- where this wouldn't be an improvement over the \nexisting behavior. However, I don't claim to be using git in any \nparticularly interesting way, so I suppose it's possible that this will \nbreak someone's workflow horribly.\n\n-Steve\n"},{"id":"54537","messageId":"49137EAF-BB73-40DC-973C-E60C14B3FA7F@zib.de","threadId":"9803","inReplyTo":"7v3awunjup.fsf@gitster.siamese.dyndns.org","subject":"Re: What's cooking in git.git (topics)","fromName":"Steffen Prohaska","fromEmail":"prohaska@zib.de","sentAt":"2007-10-02T06:44:04Z","receivedAt":"2007-10-02T06:44:04Z","isPatch":false,"sender":{"key":"prohaska@zib.de","avatar":"https://avatars.githubusercontent.com/u/217580?v=4"},"body":"\nOn Oct 2, 2007, at 7:53 AM, Junio C Hamano wrote:\n\n>\n> * jc/stash-create (Mon Jul 9 00:51:23 2007 -0700) 2 commits\n> + rebase: allow starting from a dirty tree.\n> + stash: implement \"stash create\"\n>\n> Instead of refusing to rebase, telling you that your work tree\n> is dirty, this stashes your local changes, runs rebase and then\n> unstashes automatically. That _sounds_ nicer and easier to use,\n> but I am not sure if it is a wise/sane thing to do. We may want\n> to revert the \"autostash\" from rebase. Opinions?\n\nWhat would happen if 'git stash' fails to work? Could this bring\nthe repo in a state that is hard to recover from? Especially if\n'stash' commands were run automatically for you. Maybe if you had\na choice you would not choose to use stash but would commit your\nchanges, or would bring your work tree in a clean state by other means.\n\nI'm a bit concerned because 'git stash' still doesn't work for me\nwhen the work tree is dirty because of a changed subproject (in\nmsysgit with git 1.5.3). After I run 'git stash' the work tree stays\ndirty. How would \"autostash\" behave?\n\nBTW, I run 'git submodule update' to bring the tree into a clean\nstate and later manually check out the previous head in the submodule.\nQuite annoying, but not directly related to the discussion above.\n\n\tSteffen\n"},{"id":"54538","messageId":"vpq641qroae.fsf@bauges.imag.fr","threadId":"9803","inReplyTo":"49137EAF-BB73-40DC-973C-E60C14B3FA7F@zib.de","subject":"Re: What's cooking in git.git (topics)","fromName":"Matthieu Moy","fromEmail":"matthieu.moy@imag.fr","sentAt":"2007-10-02T07:03:37Z","receivedAt":"2007-10-02T07:03:37Z","isPatch":false,"sender":{"key":"git@matthieu-moy.fr","avatar":"https://avatars.githubusercontent.com/u/14709?v=4"},"body":"Steffen Prohaska writes:\n\n> On Oct 2, 2007, at 7:53 AM, Junio C Hamano wrote:\n>\n>>\n>> * jc/stash-create (Mon Jul 9 00:51:23 2007 -0700) 2 commits\n>> + rebase: allow starting from a dirty tree.\n>> + stash: implement \"stash create\"\n>>\n>> Instead of refusing to rebase, telling you that your work tree\n>> is dirty, this stashes your local changes, runs rebase and then\n>> unstashes automatically. That _sounds_ nicer and easier to use,\n>> but I am not sure if it is a wise/sane thing to do. We may want\n>> to revert the \"autostash\" from rebase. Opinions?\n>\n> What would happen if 'git stash' fails to work? Could this bring\n> the repo in a state that is hard to recover from? Especially if\n> 'stash' commands were run automatically for you. Maybe if you had\n> a choice you would not choose to use stash but would commit your\n> changes, or would bring your work tree in a clean state by other means.\n\nI'm also concerned about the possibility of stash/unstash to fail:\nthat means the user has to deal with two kinds of conflicts (rebase\ncan conflict, unstash can also), which can be confusing.\n\nBut the current behavior can be greatly improved by just making the\nerror message better. Currently, I have this:\n\n$ git rebase branch\nfoo.txt: needs update\n$ _\n\nI'd prefer something like\n\n$ git rebase branch\ncannot rebase: the working tree is not clean.\nfoo.txt: Changed but not updated\nCommit your changes, or put them appart with \"git stash\" and retry.\n$ _\n\n-- \nMatthieu\n"},{"id":"54539","messageId":"7vlkamm16s.fsf@gitster.siamese.dyndns.org","threadId":"9803","inReplyTo":"vpq641qroae.fsf@bauges.imag.fr","subject":"Re: What's cooking in git.git (topics)","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2007-10-02T07:21:31Z","receivedAt":"2007-10-02T07:21:31Z","isPatch":false,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Matthieu Moy writes:\n\n> I'm also concerned about the possibility of stash/unstash to fail:\n> that means the user has to deal with two kinds of conflicts (rebase\n> can conflict, unstash can also), which can be confusing.\n\nUnstash should be invoked automatically _only_ after rebase\ncompletes, so I do not forsee such confusion.\n\nBut the trouble I have with the auto unstashing is more at the\nconceptual and workflow level. You start rebasing a branch, and\nyour work tree is dirty. What branch should the local\nmodification belong to?\n\nLogically, it is with the branch you were on when you typed \"git\nrebase\" command.\n\nBut when \"rebase\" successfully concludes (either with or without\nmanual fix-ups), you can be either on your original branch (if\nyou said \"git rebase base\") or something totally unrelated (if\nyou said \"git rebase base other\"). Currently we auto-unstash in\nboth cases. I _think_ the former case should auto-unstash, but\nthe latter shouldn't.\n\nHowever, this auto-stash uses the new \"git stash create\" command\nthat does not update the reflog for \"refs/stash\", specifically\nso that the auto-stashing does _not_ interfere with the normal\nstash the end user uses. Which means that it is a bit\ncumbersome to give the autostashed state back to the user if we\ndo _not_ unstash upon completion of rebase.\n\nPerhaps a good compromise would be to\n\n * not to do the autostash, but fail as we have always done,\n when \"git rebase base other\" form was used, first instructing\n rebase to switch to another branch;\n\n * autostash when \"git rebase base\" form was used, and auto\n unstash upon completion.\n\n> But the current behavior can be greatly improved by just making the\n> error message better. Currently, I have this:\n>\n> $ git rebase branch\n> foo.txt: needs update\n> $ _\n>\n> I'd prefer something like\n>\n> $ git rebase branch\n> cannot rebase: the working tree is not clean.\n> foo.txt: Changed but not updated\n> Commit your changes, or put them appart with \"git stash\" and retry.\n> $ _\n\nYou forgot 'needs merge' case, so that would not fly very well,\nbut something like this might be a good starting point.\n\n---\n git-rebase.sh | 5 ++++-\n 1 files changed, 4 insertions(+), 1 deletions(-)\n\ndiff --git a/git-rebase.sh b/git-rebase.sh\nindex 058fcac..4f8aeb9 100755\n--- a/git-rebase.sh\n+++ b/git-rebase.sh\n@@ -252,7 +252,10 @@ else\n fi\n \n # The tree must be really really clean.\n-git update-index --refresh || exit\n+o=$(git update-index -q --refresh) || {\n+\tprintf \"cannot rebase: the work tree is not clean.\\n%s\\n\" \"$o\"\n+\texit 1\n+}\n diff=$(git diff-index --cached --name-status -r HEAD)\n case \"$diff\" in\n ?*)\techo \"cannot rebase: your index is not up-to-date\"\n"},{"id":"54541","messageId":"878x6mc5d6.fsf@morpheus.local","threadId":"9803","inReplyTo":"7vlkamm16s.fsf@gitster.siamese.dyndns.org","subject":"Re: What's cooking in git.git (topics)","fromName":"David Kågedal","fromEmail":"davidk@lysator.liu.se","sentAt":"2007-10-02T08:01:25Z","receivedAt":"2007-10-02T08:01:25Z","isPatch":false,"sender":{"key":"davidk@lysator.liu.se","avatar":"https://avatars.githubusercontent.com/u/60530?v=4"},"body":"Junio C Hamano writes:\n\n> * not to do the autostash, but fail as we have always done,\n> when \"git rebase base other\" form was used, first instructing\n> rebase to switch to another branch;\n\nI don't like the idea of automatic stashing on a rebase. It makes it\nharder to understand what is happening, and figuring out were things\nwent if everything wasn't successful.\n\n-- \nDavid Kågedal\n"},{"id":"54542","messageId":"vpqr6keos6e.fsf@bauges.imag.fr","threadId":"9803","inReplyTo":"7vlkamm16s.fsf@gitster.siamese.dyndns.org","subject":"Re: What's cooking in git.git (topics)","fromName":"Matthieu Moy","fromEmail":"matthieu.moy@imag.fr","sentAt":"2007-10-02T08:07:53Z","receivedAt":"2007-10-02T08:07:53Z","isPatch":false,"sender":{"key":"git@matthieu-moy.fr","avatar":"https://avatars.githubusercontent.com/u/14709?v=4"},"body":"Junio C Hamano writes:\n\n> Matthieu Moy writes:\n>\n>> I'm also concerned about the possibility of stash/unstash to fail:\n>> that means the user has to deal with two kinds of conflicts (rebase\n>> can conflict, unstash can also), which can be confusing.\n>\n> Unstash should be invoked automatically _only_ after rebase\n> completes, so I do not forsee such confusion.\n\nWell, if rebase itself conflicts, it will stop and tell you about\nconflicts, then you'll have to \"rebase --continue\". If unstash fails,\nyou'll have to resolve the conflicts, and probably do \"commit\", or\ncontinue working afterwards. You don't have to deal with both at the\nsame time, but both do exist, and they have to be taken care of\ndifferently.\n\nFor an advanced user with good understanding of the flow, that's OK,\nbut I'm still afraid of the confusion for not-so-advanced users.\n\nBut that's not a strong argument against auto-stash, just one\ndownside.\n\n> But the trouble I have with the auto unstashing is more at the\n> conceptual and workflow level. You start rebasing a branch, and\n> your work tree is dirty. What branch should the local\n> modification belong to?\n\nYou're in a better position than me to juge on that point.\n\n> # The tree must be really really clean.\n> -git update-index --refresh || exit\n> +o=$(git update-index -q --refresh) || {\n> +\tprintf \"cannot rebase: the work tree is not clean.\\n%s\\n\" \"$o\"\n> +\texit 1\n> +}\n> diff=$(git diff-index --cached --name-status -r HEAD)\n> case \"$diff\" in\n> ?*)\techo \"cannot rebase: your index is not up-to-date\"\n\nThat alone would already be a real improvement.\n\nI'd add this to be consistant with \"git status\". I find the \"needs\nupdate\" really short, and especially confusing for centralized systems\nusers, for whom \"needs update\" would probably mean \"new version\navailable, please run '$VCS update'\".\n\ndiff --git a/read-cache.c b/read-cache.c\nindex 2e40a34..3745a48 100644\n--- a/read-cache.c\n+++ b/read-cache.c\n@@ -869,7 +869,7 @@ int refresh_index(struct index_state *istate, unsigned int flags, const char **p\n }\n if (quiet)\n continue;\n- printf(\"%s: needs update\\n\", ce->name);\n+ printf(\"%s: Changed but not updated\\n\", ce->name);\n has_errors = 1;\n continue;\n }\n\n\n-- \nMatthieu\n"},{"id":"54560","messageId":"Pine.LNX.4.64.0710021350510.28395@racer.site","threadId":"9803","inReplyTo":"7v3awunjup.fsf@gitster.siamese.dyndns.org","subject":"Re: What's cooking in git.git (topics)","fromName":"Johannes Schindelin","fromEmail":"johannes.schindelin@gmx.de","sentAt":"2007-10-02T12:52:02Z","receivedAt":"2007-10-02T12:52:02Z","isPatch":false,"sender":{"key":"johannes.schindelin@gmx.de","avatar":"https://avatars.githubusercontent.com/u/127790?v=4"},"body":"Hi,\n\nOn Mon, 1 Oct 2007, Junio C Hamano wrote:\n\n> I am not sure the quality of \"rsync\" transport near the tip, either, but \n> at least the change should not affect other transports. Nobody should \n> using about rsync transport these days anyway. Perhaps we should put a \n> deprecation notice in the release notes to 1.5.4, and remove it three \n> months later.\n\nWhy not keep it? It's not like it hurts somebody, and in some \ncircumstances (lacking git on the remote side, where it was served via \nhttp) I found it really convenient.\n\nCiao,\nDscho\n"},{"id":"54567","messageId":"20071002161114.GC6828@coredump.intra.peff.net","threadId":"9803","inReplyTo":"86ejge6o8b.fsf@lola.quinscape.zz","subject":"Re: What's cooking in git.git (topics)","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2007-10-02T16:11:14Z","receivedAt":"2007-10-02T16:11:14Z","isPatch":false,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Tue, Oct 02, 2007 at 08:10:28AM +0200, David Kastrup wrote:\n\n> I have not actually looked at the actual task that the structures are\n> going to be used in, and whether \"reusing\" the information is likely\n> to be worth the trouble.\n\nThe algorithm is something like this: We have N files, and we want to\nfind \"similar\" candidates. So we go through each file and generate a\ntable of fingperint hashes (diffcore-rename.c:hash_chars), and then\ncompare each file with every other file, using the hash tables to do the\ncomparison.\n\nSo the comparison step for two files is currently something like:\n\n for each hash in file1\n hash2 = look up hash in file2\n compare hash and hash2\n\nand if they were sorted, perhaps we could do something merge-like:\n\n while hashes are left to compare\n compare file1.next, file2.next\n advance file1, file2, or both (depending on comparison)\n\n> When we are talking about buzzword compliance, \"keep sorted\" with the\n> meaning of \"maintain sorted across modifications\" has an O(n^2) or at\n> least O(nm) ring to it. However, if it is possible to sort it just\n> once, and then then only merge with other lists...\n\nIt would be sort once. I.e.,:\n\n for each file\n generate file.hashes\n sort file.hashes\n for each file1\n for each file2\n compare file1.hashes to file2.hashes\n\nwhere that 'compare' step is taking most of the CPU time (for the\nobvious reason that we call it in an O(n^2) loop).\n\nI will try to implement this as time permits, but if you want to tinker\nwith it in the meantime, feel free.\n\n-Peff\n"},{"id":"54568","messageId":"86ve9p32cp.fsf@lola.quinscape.zz","threadId":"9803","inReplyTo":"20071002161114.GC6828@coredump.intra.peff.net","subject":"Re: What's cooking in git.git (topics)","fromName":"David Kastrup","fromEmail":"dak@gnu.org","sentAt":"2007-10-02T16:31:18Z","receivedAt":"2007-10-02T16:31:18Z","isPatch":false,"sender":{"key":"dak@gnu.org","avatar":"https://avatars.githubusercontent.com/u/52141349?v=4"},"body":"Jeff King writes:\n\n> On Tue, Oct 02, 2007 at 08:10:28AM +0200, David Kastrup wrote:\n\n[...]\n\n> The algorithm is something like this: We have N files, and we want\n> to find \"similar\" candidates. So we go through each file and\n> generate a table of fingperint hashes\n> (diffcore-rename.c:hash_chars), and then compare each file with\n> every other file, using the hash tables to do the comparison.\n>\n> So the comparison step for two files is currently something like:\n>\n> for each hash in file1\n> hash2 = look up hash in file2\n> compare hash and hash2\n>\n> and if they were sorted, perhaps we could do something merge-like:\n>\n> while hashes are left to compare\n> compare file1.next, file2.next\n> advance file1, file2, or both (depending on comparison)\n>\n>> When we are talking about buzzword compliance, \"keep sorted\" with\n>> the meaning of \"maintain sorted across modifications\" has an O(n^2)\n>> or at least O(nm) ring to it. However, if it is possible to sort\n>> it just once, and then then only merge with other lists...\n>\n> It would be sort once. I.e.,:\n>\n> for each file\n> generate file.hashes\n> sort file.hashes\n> for each file1\n> for each file2\n> compare file1.hashes to file2.hashes\n>\n> where that 'compare' step is taking most of the CPU time (for the\n> obvious reason that we call it in an O(n^2) loop).\n>\n> I will try to implement this as time permits, but if you want to\n> tinker with it in the meantime, feel free.\n\nThis does not actually require an actual merge _sort_ AFAICS: do the\n\"sort file.hashed\" step using qsort. The comparison step does not\nactually need to produce merged output, but merely advances through\ntwo hash arrays and generates statistics.\n\nThis should already beat the pants off the current implementation,\neven when the hash array is sparse, simply because our inner loop then\nhas perfect hash coherence.\n\nGetting rid of this outer O(n^2) remains an interesting challenge,\nthough. One way would be the following: fill a _single_ array with\nentries containing _both_ hash and file number. Sort this, and then\ngather the statistics of hash runs by making a single pass through.\nThat reduces the O(n^2) behavior to only those parts with actual hash\ncollisions.\n\n-- \nDavid Kastrup\n"},{"id":"54569","messageId":"Pine.LNX.4.64.0710021212310.9321@iabervon.org","threadId":"9803","inReplyTo":"7v3awunjup.fsf@gitster.siamese.dyndns.org","subject":"Re: What's cooking in git.git (topics)","fromName":"Daniel Barkalow","fromEmail":"barkalow@iabervon.org","sentAt":"2007-10-02T17:00:01Z","receivedAt":"2007-10-02T17:00:01Z","isPatch":false,"sender":{"key":"barkalow@iabervon.org","avatar":"https://avatars.githubusercontent.com/u/55364219?v=4"},"body":"On Mon, 1 Oct 2007, Junio C Hamano wrote:\n\n> * db/fetch-pack (Mon Oct 1 00:59:39 2007 +0100) 49 commits\n> + fetch/push: readd rsync support\n> + Introduce remove_dir_recursively()\n> + bundle transport: fix an alloc_ref() call\n> + Allow abbreviations in the first refspec to be merged\n> + Prevent send-pack from segfaulting when a branch doesn't match\n> + Cleanup unnecessary break in remote.c\n> ...\n> \n> Has been cooking for quite long time.\n> \n> There was a regression that made me quite unhappy about the\n> rewrite, but Daniel fixed it, so I should be happy. There is\n> another usability regression: http transport is now totally\n> silent.\n\nI think this is due to passing through equal verbosity levels, when the \nnon-verbose case for the native protocols was a lot less silent than the \nnon-verbose case for http. \"git fetch -v -v\" does show everything. I think \njust replacing \"transport->verbose\" with \"1\" on line 347 of transport.c \nwould give the old default behavior, but fetch probably needs a more quiet \nsetting than the default, as well as the current more verbose than \ndefault. I'll put together a patch for this when I get a chance.\n\n> Even when you fetch daily, if the other end frequently\n> repacks everything into one big ball of wax like repo.or.cz\n> does, you will end up transferring quite a large pack every\n> time, and the total lack of progress report is unacceptably\n> unnerving. At least we should reinstate \"Fetching blah from URL\n> using http\", and preferrably \"walk $object_name\" lines. The\n> latter could be replaced with just series of CR + \"walked N\n> commits...\" if we do not like many output from the current \"walk\n> $object_name\" lines scrolling the other information away.\n\nThe right thing for now is probably to match the old git-fetch's behavior, \nonce I can remember what it is. (I've been using my C version for my \npersonal use long enough that I can't remember everything the shell \nversion did. My new version should be sufficiently flexible to accomodate \nmost things without too much trouble, but I've lost my ability to notice \ndifferences without prompting.)\n\nAfter the implementation change is in, we can look at improving \nuser-visible things. I think a display like the \"counting objects\" display \n(number that counts up in place), plus progress bars for big downloads, would \nbe ideal.\n\n> I am not sure the quality of \"rsync\" transport near the tip,\n> either, but at least the change should not affect other\n> transports. Nobody should using about rsync transport these\n> days anyway. Perhaps we should put a deprecation notice in the\n> release notes to 1.5.4, and remove it three months later.\n\nI think that rsync should be kept until we've got sftp in place, which \nshould cover the same cases and be better overall. \n\n\t-Daniel\n*This .sig left intentionally blank*\n"},{"id":"54570","messageId":"20071002173941.GA7187@coredump.intra.peff.net","threadId":"9803","inReplyTo":"86ve9p32cp.fsf@lola.quinscape.zz","subject":"Re: What's cooking in git.git (topics)","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2007-10-02T17:39:41Z","receivedAt":"2007-10-02T17:39:41Z","isPatch":false,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Tue, Oct 02, 2007 at 06:31:18PM +0200, David Kastrup wrote:\n\n> This does not actually require an actual merge _sort_ AFAICS: do the\n> \"sort file.hashed\" step using qsort. The comparison step does not\n> actually need to produce merged output, but merely advances through\n> two hash arrays and generates statistics.\n\nRight, that's why I used \"merge\" in quotes. The sort used in the O(n)\nstep is irrelevant, but we are doing a merge-sort-like behavior in the\nsecond step (except instead of actually merging into a new list, we are\nsummarizing the comparisons in a numeric \"difference\" variable). But I\nthink we are on the same page.\n\n> This should already beat the pants off the current implementation,\n> even when the hash array is sparse, simply because our inner loop then\n> has perfect hash coherence.\n\nYes, I hope so. We'll see. :)\n\n> Getting rid of this outer O(n^2) remains an interesting challenge,\n> though. One way would be the following: fill a _single_ array with\n> entries containing _both_ hash and file number. Sort this, and then\n> gather the statistics of hash runs by making a single pass through.\n> That reduces the O(n^2) behavior to only those parts with actual hash\n> collisions.\n\nInteresting. Care to take a stab at implementing it?\n\n-Peff\n"},{"id":"54572","messageId":"7vd4vxmmxr.fsf@gitster.siamese.dyndns.org","threadId":"9803","inReplyTo":"vpqr6keos6e.fsf@bauges.imag.fr","subject":"Re: What's cooking in git.git (topics)","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2007-10-02T17:44:00Z","receivedAt":"2007-10-02T17:44:00Z","isPatch":false,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Matthieu Moy writes:\n\n> I'd add this to be consistant with \"git status\". I find the \"needs\n> update\" really short, and especially confusing for centralized systems\n> users, for whom \"needs update\" would probably mean \"new version\n> available, please run '$VCS update'\".\n\nYeah, and \"needs merge\" solicits 'SCM merge'.\n\n> diff --git a/read-cache.c b/read-cache.c\n> index 2e40a34..3745a48 100644\n> --- a/read-cache.c\n> +++ b/read-cache.c\n> @@ -869,7 +869,7 @@ int refresh_index(struct index_state *istate, unsigned int flags, const char **p\n> }\n> if (quiet)\n> continue;\n> - printf(\"%s: needs update\\n\", ce->name);\n> + printf(\"%s: Changed but not updated\\n\", ce->name);\n> has_errors = 1;\n> continue;\n> }\n\nI tried to stay away from touching that part on purpose. Doing\nthis unconditionally may break people's existing scripts that\nuse update-index --refresh plumbing.\n\nWe could introduce a new option to \"update-index --refresh\" that\nmakes the output more machine readable by either NUL terminating\nor c_quoting ce->name to protect the caller from potential\nspaces and newlines in the name, and use that from the caller.\n\nBut for this particular case, I think a much simpler alternative\nwould be to do it this way:\n\n> # The tree must be really really clean.\n> -git update-index --refresh || exit\n> +git update-index -q --refresh || {\n+\tgit status\n> +\tprintf \"cannot rebase: the work tree is not clean.\\n\"\n> +\texit 1\n> +}\n> diff=$(git diff-index --cached --name-status -r HEAD)\n> case \"$diff\" in\n> ?*)\techo \"cannot rebase: your index is not up-to-date\"\n"},{"id":"54591","messageId":"85sl4tz78x.fsf@lola.goethe.zz","threadId":"9803","inReplyTo":"20071002173941.GA7187@coredump.intra.peff.net","subject":"Re: What's cooking in git.git (topics)","fromName":"David Kastrup","fromEmail":"dak@gnu.org","sentAt":"2007-10-02T18:44:30Z","receivedAt":"2007-10-02T18:44:30Z","isPatch":false,"sender":{"key":"dak@gnu.org","avatar":"https://avatars.githubusercontent.com/u/52141349?v=4"},"body":"Jeff King writes:\n\n> On Tue, Oct 02, 2007 at 06:31:18PM +0200, David Kastrup wrote:\n>\n>> This does not actually require an actual merge _sort_ AFAICS: do the\n>> \"sort file.hashed\" step using qsort. The comparison step does not\n>> actually need to produce merged output, but merely advances through\n>> two hash arrays and generates statistics.\n>\n> Right, that's why I used \"merge\" in quotes.\n>\n>> This should already beat the pants off the current implementation,\n>> even when the hash array is sparse, simply because our inner loop\n>> then has perfect hash coherence.\n>\n> Yes, I hope so. We'll see. :)\n>\n>> Getting rid of this outer O(n^2) remains an interesting challenge,\n>> though. One way would be the following: fill a _single_ array with\n>> entries containing _both_ hash and file number. Sort this, and\n>> then gather the statistics of hash runs by making a single pass\n>> through. That reduces the O(n^2) behavior to only those parts with\n>> actual hash collisions.\n>\n> Interesting. Care to take a stab at implementing it?\n\nI actually have worked through the last night on the day job, have\nurgent stuff piling up in my freelance work queue, and the next thing\nI need to finish for git is some smart stuff for delta packing.\n\nSo it's unlikely I'll get to _that_ anytime soon. However, I had a\nhilarious idea on the way home that kept me rather amused (perhaps my\nprogrammer's humour is affected by sleep deprivation).\n\nI was annoyed at needing double the space because of having to keep\nscore of both hash and file number. So I came up with a rather cute\nmanner to avoid this: first do all files in isolation with full\nprecision, but store the resulting list of hash as difference to the\nlast value. When merging the data of 2^k and 2^k (or somewhat less)\nfiles, we multiply the values by two (this will not carry except for\nutterly improbable cases or very small data sets which we can do\ndifferently) and add one bit of identification. When we have just a\nsingle sequence remaining, undeltafying will tell us about collisions\nin the high bits, and the affected files in the low bits.\n\nOf course, using a merge-like algorithm means that we temporarily need\ndouble space anyway. Which takes some of the fun.\n\n-- \nDavid Kastrup, Kriemhildstr. 15, 44793 Bochum\n"},{"id":"54640","messageId":"alpine.LFD.0.999.0710021916080.3579@woody.linux-foundation.org","threadId":"9803","inReplyTo":"86ve9p32cp.fsf@lola.quinscape.zz","subject":"Re: What's cooking in git.git (topics)","fromName":"Linus Torvalds","fromEmail":"torvalds@linux-foundation.org","sentAt":"2007-10-03T02:28:19Z","receivedAt":"2007-10-03T02:28:19Z","isPatch":false,"sender":{"key":"torvalds@linux-foundation.org","avatar":"https://avatars.githubusercontent.com/u/1024025?v=4"},"body":"\n[ This is the discussed stupid approach - just sort the dang hash array, \n so that we can use a linear scan over the src/dst ]\n\nOn Tue, 2 Oct 2007, David Kastrup wrote:\n> \n> This does not actually require an actual merge _sort_ AFAICS: do the\n> \"sort file.hashed\" step using qsort. The comparison step does not\n> actually need to produce merged output, but merely advances through\n> two hash arrays and generates statistics.\n> \n> This should already beat the pants off the current implementation,\n> even when the hash array is sparse, simply because our inner loop then\n> has perfect hash coherence.\n\nSadly, that's not the case. It *does* seem to beat the current \nimplementation, but it's not \"beat the pants off\". It looks like an \nimprovement of about 15%, which is nothing to sneeze at, but it's not an \norder-of-magnitude improvement either.\n\nHere's a test-patch. I don't guarantee anything, except that when I did \nthe timings I also did a \"wc\" on the result, and they matched..\n\nBefore:\n\t[torvalds@woody linux]$ time git diff -l0 --stat -C v2.6.22.. | wc\n\t 7104 28574 438020\n\n\treal 0m10.526s\n\tuser 0m10.401s\n\tsys 0m0.136s\n\nAfter:\n\t[torvalds@woody linux]$ time ~/git/git diff -l0 --stat -C v2.6.22.. | wc\n\t 7104 28574 438020\n\t\n\treal 0m8.876s\n\tuser 0m8.761s\n\tsys 0m0.128s\n\nbut the diff is fairly simple, so if somebody will go over it and say \nwhether it's likely to be *correct* too, that 15% may well be worth it.\n\n[ Side note, without rename detection, that diff takes just under three \n seconds for me, so in that sense the improvement to the rename detection \n itself is larger than the overall 15% - it brings the cost of just \n rename detection from 7.5s to 5.9s, which would be on the order of just \n over a 20% performance improvement. ]\n\nHmm. The patch depends on half-way subtle issues like the fact that the \nhashtables are guaranteed to not be full => we're guaranteed to have zero \ncounts at the end => we don't need to do any steenking iterator count in \nthe loop. A few comments might in order.\n\n\t\tLinus\n\n---\n diffcore-delta.c | 54 ++++++++++++++++++++++++++++++------------------------\n 1 files changed, 30 insertions(+), 24 deletions(-)\n\ndiff --git a/diffcore-delta.c b/diffcore-delta.c\nindex d9729e5..6d65697 100644\n--- a/diffcore-delta.c\n+++ b/diffcore-delta.c\n@@ -46,22 +46,6 @@ struct spanhash_top {\n \tstruct spanhash data[FLEX_ARRAY];\n };\n \n-static struct spanhash *spanhash_find(struct spanhash_top *top,\n-\t\t\t\t unsigned int hashval)\n-{\n-\tint sz = 1 << top->alloc_log2;\n-\tint bucket = hashval & (sz - 1);\n-\twhile (1) {\n-\t\tstruct spanhash *h = &(top->data[bucket++]);\n-\t\tif (!h->cnt)\n-\t\t\treturn NULL;\n-\t\tif (h->hashval == hashval)\n-\t\t\treturn h;\n-\t\tif (sz <= bucket)\n-\t\t\tbucket = 0;\n-\t}\n-}\n-\n static struct spanhash_top *spanhash_rehash(struct spanhash_top *orig)\n {\n \tstruct spanhash_top *new;\n@@ -122,6 +106,20 @@ static struct spanhash_top *add_spanhash(struct spanhash_top *top,\n \t}\n }\n \n+static int spanhash_cmp(const void *_a, const void *_b)\n+{\n+\tconst struct spanhash *a = _a;\n+\tconst struct spanhash *b = _b;\n+\n+\t/* A count of zero compares at the end.. */\n+\tif (!a->cnt)\n+\t\treturn !b->cnt ? 0 : 1;\n+\tif (!b->cnt)\n+\t\treturn -1;\n+\treturn a->hashval < b->hashval ? -1 :\n+\t\ta->hashval > b->hashval ? 1 : 0;\n+}\n+\n static struct spanhash_top *hash_chars(struct diff_filespec *one)\n {\n \tint i, n;\n@@ -158,6 +156,10 @@ static struct spanhash_top *hash_chars(struct diff_filespec *one)\n \t\tn = 0;\n \t\taccum1 = accum2 = 0;\n \t}\n+\tqsort(hash->data,\n+\t\t1ul << hash->alloc_log2,\n+\t\tsizeof(hash->data[0]),\n+\t\tspanhash_cmp);\n \treturn hash;\n }\n \n@@ -169,7 +171,7 @@ int diffcore_count_changes(struct diff_filespec *src,\n \t\t\t unsigned long *src_copied,\n \t\t\t unsigned long *literal_added)\n {\n-\tint i, ssz;\n+\tstruct spanhash *s, *d;\n \tstruct spanhash_top *src_count, *dst_count;\n \tunsigned long sc, la;\n \n@@ -190,22 +192,26 @@ int diffcore_count_changes(struct diff_filespec *src,\n \t}\n \tsc = la = 0;\n \n-\tssz = 1 << src_count->alloc_log2;\n-\tfor (i = 0; i < ssz; i++) {\n-\t\tstruct spanhash *s = &(src_count->data[i]);\n-\t\tstruct spanhash *d;\n+\ts = src_count->data;\n+\td = dst_count->data;\n+\tfor (;;) {\n \t\tunsigned dst_cnt, src_cnt;\n \t\tif (!s->cnt)\n-\t\t\tcontinue;\n+\t\t\tbreak;\n+\t\twhile (d->cnt) {\n+\t\t\tif (d->hashval >= s->hashval)\n+\t\t\t\tbreak;\n+\t\t\td++;\n+\t\t}\n \t\tsrc_cnt = s->cnt;\n-\t\td = spanhash_find(dst_count, s->hashval);\n-\t\tdst_cnt = d ? d->cnt : 0;\n+\t\tdst_cnt = d->hashval == s->hashval ? d->cnt : 0;\n \t\tif (src_cnt < dst_cnt) {\n \t\t\tla += dst_cnt - src_cnt;\n \t\t\tsc += src_cnt;\n \t\t}\n \t\telse\n \t\t\tsc += dst_cnt;\n+\t\ts++;\n \t}\n \n \tif (!src_count_p)\n"},{"id":"54658","messageId":"7vejgchijf.fsf_-_@gitster.siamese.dyndns.org","threadId":"9803","inReplyTo":"7vd4vxmmxr.fsf@gitster.siamese.dyndns.org","subject":"[PATCH] rebase: make the warning more useful when the work tree is unclean.","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2007-10-03T05:30:12Z","receivedAt":"2007-10-03T05:30:12Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Instead of letting \"update-index --refresh\" report paths needing\nupdates and merges, use git-status to give more useful output.\n\nSigned-off-by: Junio C Hamano \n---\n\n * I won't be a good judge of the updated behaviour, as I never\n start rebase in an unclear tree. Running git-status in a\n large tree may be too expensive to be worth changing the\n output.\n\n git-rebase.sh | 6 +++++-\n 1 files changed, 5 insertions(+), 1 deletions(-)\n\ndiff --git a/git-rebase.sh b/git-rebase.sh\nindex 1583402..93e3b3c 100755\n--- a/git-rebase.sh\n+++ b/git-rebase.sh\n@@ -254,7 +254,11 @@ else\n fi\n \n # The tree must be really really clean.\n-git update-index --refresh || exit\n+git update-index -q --refresh || {\n+\tgit status\n+\tprintf \"cannot rebase: the work tree is not clean.\\n\"\n+\texit 1\n+}\n diff=$(git diff-index --cached --name-status -r HEAD)\n case \"$diff\" in\n ?*)\techo \"cannot rebase: your index is not up-to-date\"\n-- \n1.5.3.3.1144.gf10f2\n"},{"id":"54667","messageId":"20071003065414.GA13737@coredump.intra.peff.net","threadId":"9803","inReplyTo":"alpine.LFD.0.999.0710021916080.3579@woody.linux-foundation.org","subject":"Re: What's cooking in git.git (topics)","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2007-10-03T06:54:15Z","receivedAt":"2007-10-03T06:54:15Z","isPatch":false,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Tue, Oct 02, 2007 at 07:28:19PM -0700, Linus Torvalds wrote:\n\n> Sadly, that's not the case. It *does* seem to beat the current \n> implementation, but it's not \"beat the pants off\". It looks like an \n> improvement of about 15%, which is nothing to sneeze at, but it's not an \n> order-of-magnitude improvement either.\n> \n> Here's a test-patch. I don't guarantee anything, except that when I did \n> the timings I also did a \"wc\" on the result, and they matched..\n\nI get slightly better speedups with my pathological case (around 30%):\n\nBefore:\n $ /usr/bin/time git-diff --raw -M -l0 06d288^ 06d288 >/dev/null\n 105.38user 3.65system 2:14.90elapsed 80%CPU (0avgtext+0avgdata 0maxresident)k\n 0inputs+0outputs (15432major+542627minor)pagefaults 0swaps\n\nAfter:\n $ /usr/bin/time git-diff --raw -M -l0 06d288^ 06d288 >/dev/null\n 71.70user 3.47system 1:40.43elapsed 74%CPU (0avgtext+0avgdata 0maxresident)k\n 0inputs+0outputs (15065major+551778minor)pagefaults 0swaps\n\nBut yes, it's not the order of magnitude we were looking for.\n\n> \t[torvalds@woody linux]$ time git diff -l0 --stat -C v2.6.22.. | wc\n\nI found less noise in the timing by using --raw, since the patch\ncomputation takes an appreciable amount of time.\n\n> but the diff is fairly simple, so if somebody will go over it and say \n> whether it's likely to be *correct* too, that 15% may well be worth it.\n\nPatch looks correct, and it produces correct results on my (admittedly\nlimited) test data.\n\nI think it's worth applying (though I agree that a comment on the\nassumption of a zero \"cnt\" at the end is worth adding) unless some\ndrastically different solution comes along (e.g., David's idea to try\navoiding the outer O(n^2) loop). But I don't think there is much more to\nbe gained from a different approach to comparing the two hash tables.\n\n-Peff\n"},{"id":"54679","messageId":"85641oy5ge.fsf@lola.goethe.zz","threadId":"9803","inReplyTo":"alpine.LFD.0.999.0710021916080.3579@woody.linux-foundation.org","subject":"Re: What's cooking in git.git (topics)","fromName":"David Kastrup","fromEmail":"dak@gnu.org","sentAt":"2007-10-03T08:20:49Z","receivedAt":"2007-10-03T08:20:49Z","isPatch":false,"sender":{"key":"dak@gnu.org","avatar":"https://avatars.githubusercontent.com/u/52141349?v=4"},"body":"Linus Torvalds writes:\n\n> [ This is the discussed stupid approach - just sort the dang hash array, \n> so that we can use a linear scan over the src/dst ]\n>\n> On Tue, 2 Oct 2007, David Kastrup wrote:\n>> \n>> This does not actually require an actual merge _sort_ AFAICS: do the\n>> \"sort file.hashed\" step using qsort. The comparison step does not\n>> actually need to produce merged output, but merely advances through\n>> two hash arrays and generates statistics.\n>> \n>> This should already beat the pants off the current implementation,\n>> even when the hash array is sparse, simply because our inner loop then\n>> has perfect hash coherence.\n>\n> Sadly, that's not the case. It *does* seem to beat the current \n> implementation, but it's not \"beat the pants off\".\n\nPart of the reason is that it is not actually what I had in mind. Why\ncreate the hash array as a hash array? Filling the hash array in\nbasically random order, then sort+compressing it is what is causing\nmuch of the costs. My idea was to just fill the \"hash array\"\nlinearly. It is quite pointless (and certainly very inefficient with\nregard to cache poisoning) to do it in hash order when we are going to\nsort it anyway.\n\n-- \nDavid Kastrup, Kriemhildstr. 15, 44793 Bochum\n"},{"id":"54683","messageId":"vpq1wcca7w2.fsf@bauges.imag.fr","threadId":"9803","inReplyTo":"7vejgchijf.fsf_-_@gitster.siamese.dyndns.org","subject":"Re: [PATCH] rebase: make the warning more useful when the work tree is unclean.","fromName":"Matthieu Moy","fromEmail":"matthieu.moy@imag.fr","sentAt":"2007-10-03T09:02:05Z","receivedAt":"2007-10-03T09:02:05Z","isPatch":true,"sender":{"key":"git@matthieu-moy.fr","avatar":"https://avatars.githubusercontent.com/u/14709?v=4"},"body":"Junio C Hamano writes:\n\n> Instead of letting \"update-index --refresh\" report paths needing\n> updates and merges, use git-status to give more useful output.\n>\n> Signed-off-by: Junio C Hamano \n> ---\n>\n> * I won't be a good judge of the updated behaviour, as I never\n> start rebase in an unclear tree. Running git-status in a\n> large tree may be too expensive to be worth changing the\n> output.\n\nI see your patch as an improvement too. status is a bit expansive, but\nyou hit this portion of code only when trying \"rebase\" by mistake, so\nit's acceptable to let git take a bit of time to explain your mistake.\n\nThat said, I think it's still worth making the messages of\n\"update-index\" a little more verbose and consistant with \"status\" (my\nprevious patch), since I think there are other occurences of\nuser-visible output of update-index in porcelain git.\n\nIdeally, update-index would be a C function returning a struct\ncontaining all the information about the status, with a function\nis_clean(...) and another print_as_status(...), to allow the same\nfunctionality with better performances, but I won't have time to do\nthis.\n\n-- \nMatthieu\n"},{"id":"54723","messageId":"alpine.LFD.0.999.0710030912410.3579@woody.linux-foundation.org","threadId":"9803","inReplyTo":"20071003065414.GA13737@coredump.intra.peff.net","subject":"Re: What's cooking in git.git (topics)","fromName":"Linus Torvalds","fromEmail":"torvalds@linux-foundation.org","sentAt":"2007-10-03T16:13:49Z","receivedAt":"2007-10-03T16:13:49Z","isPatch":false,"sender":{"key":"torvalds@linux-foundation.org","avatar":"https://avatars.githubusercontent.com/u/1024025?v=4"},"body":"\n\nOn Wed, 3 Oct 2007, Jeff King wrote:\n> \n> I get slightly better speedups with my pathological case (around 30%):\n\nOk, 30% is definitely \"worth doing\". Even if your performance still sucks, \nand 71 seconds is just way out of line for anything like this (of course, \nthese days you need that \"-l0\" to ever trigger that case, but it would be \nnice if we could speed things up so much that we no longer care).\n\n\t\tLinus\n"},{"id":"54728","messageId":"20071003165927.GA25055@coredump.intra.peff.net","threadId":"9803","inReplyTo":"85641oy5ge.fsf@lola.goethe.zz","subject":"Re: What's cooking in git.git (topics)","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2007-10-03T16:59:27Z","receivedAt":"2007-10-03T16:59:27Z","isPatch":false,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Wed, Oct 03, 2007 at 10:20:49AM +0200, David Kastrup wrote:\n\n> Part of the reason is that it is not actually what I had in mind. Why\n> create the hash array as a hash array? Filling the hash array in\n> basically random order, then sort+compressing it is what is causing\n> much of the costs. My idea was to just fill the \"hash array\"\n> linearly. It is quite pointless (and certainly very inefficient with\n> regard to cache poisoning) to do it in hash order when we are going to\n> sort it anyway.\n\nTry profiling the code, and you will see that the creation of the hashes\nis totally dwarfed by the comparisons. So yes, you might be able to\nspeed up the creation code, but it's going to have a minimal impact on\nthe overall run time.\n\n-Peff\n"},{"id":"54732","messageId":"alpine.LFD.0.999.0710031007340.3579@woody.linux-foundation.org","threadId":"9803","inReplyTo":"20071003165927.GA25055@coredump.intra.peff.net","subject":"Re: What's cooking in git.git (topics)","fromName":"Linus Torvalds","fromEmail":"torvalds@linux-foundation.org","sentAt":"2007-10-03T17:53:27Z","receivedAt":"2007-10-03T17:53:27Z","isPatch":false,"sender":{"key":"torvalds@linux-foundation.org","avatar":"https://avatars.githubusercontent.com/u/1024025?v=4"},"body":"\n\nOn Wed, 3 Oct 2007, Jeff King wrote:\n>\n> Try profiling the code, and you will see that the creation of the hashes\n> is totally dwarfed by the comparisons. So yes, you might be able to\n> speed up the creation code, but it's going to have a minimal impact on\n> the overall run time.\n\nYeah. Oprofile is your friend.\n\nThe biggest win would be to avoid calling diffcore_count_changes() in the \nfirst place, and we actually do that in the caller by looking at the size \nof the files. However, while that prunes out the *really* obvious cases, \nit's not nearly enough, since there tends to be very limited filesizes \nanyway.\n\nWhat we could also do is to pass in the minimum similarity score, and use \nthat to at least exit early. We currently pass in \"delta_limit\" which is \nclose, but we never use it, and we really probably would be better off \npassing in the minimum score itself and perhaps be able to exit early from \ndiffcore_count_changes.\n\nHowever, while I did think about doing that, I came to the conclusion that \nwe'd still always end up having to look at least at *half* the hash data \n(for the default 50% score), so while it would help, it again wouldn't be \na matter of orders-of-magnitudes, and it looked like the end result would \nbe unnecessarily complex too.\n\nThe best option, of course, would be to use a similarity hash to make the \ninitial choice. I think we had one at one point, but it wasn't \nfine-grained enough. But it might be interesting to do that as a filter in \n*front* of the more expensive diffcore_count_changes() call.\n\nWe had some \"similarity fingerprint\" code at some point using a Rabin \npolynomial. It wasn't reliable enough to be used as a direct score, but \nmaybe it can be used as a first-line \"we know this isn't even worth doing \nthe expensive stuff on\".\n\n\t\t\tLinus\n"},{"id":"54736","messageId":"85wsu4qddo.fsf@lola.goethe.zz","threadId":"9803","inReplyTo":"alpine.LFD.0.999.0710031007340.3579@woody.linux-foundation.org","subject":"Re: What's cooking in git.git (topics)","fromName":"David Kastrup","fromEmail":"dak@gnu.org","sentAt":"2007-10-03T18:09:07Z","receivedAt":"2007-10-03T18:09:07Z","isPatch":false,"sender":{"key":"dak@gnu.org","avatar":"https://avatars.githubusercontent.com/u/52141349?v=4"},"body":"Linus Torvalds writes:\n\n> On Wed, 3 Oct 2007, Jeff King wrote:\n>>\n>> Try profiling the code, and you will see that the creation of the hashes\n>> is totally dwarfed by the comparisons. So yes, you might be able to\n>> speed up the creation code, but it's going to have a minimal impact on\n>> the overall run time.\n>\n> Yeah. Oprofile is your friend.\n\nWell, and if -Oprofile has no strong opinion, I'd let wc -l pitch in.\n\nWhen we are not actually going to use the hash tables as hash tables,\nwhy create them as such? If the first thing that actually looks at\nthe values of the hashes (except possibly for the optimization of not\nstoring the same hash twice in succession) is the sort, then there is\nno code that can go out of whack when confronted with degenerate data.\n\nMaybe it's not much of an optimization, but it certainly should be a\ncleanup.\n\n-- \nDavid Kastrup, Kriemhildstr. 15, 44793 Bochum\n"},{"id":"54793","messageId":"7vmyuzbbif.fsf@gitster.siamese.dyndns.org","threadId":"9803","inReplyTo":"alpine.LFD.0.999.0710021916080.3579@woody.linux-foundation.org","subject":"Re: What's cooking in git.git (topics)","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2007-10-04T07:10:48Z","receivedAt":"2007-10-04T07:10:48Z","isPatch":false,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Linus Torvalds writes:\n\n> Hmm. The patch depends on half-way subtle issues like the fact that the \n> hashtables are guaranteed to not be full => we're guaranteed to have zero \n> counts at the end => we don't need to do any steenking iterator count in \n> the loop. A few comments might in order.\n\nThe patch actually is quite readable. That double-loop finding\nthe matching hashval in destination hash was simply silly to\nbegin with, so even if this is not \"orders of magnitude\"\nimprovement, I think your patch is worth doing.\n"}]}