{"thread":{"id":"52947","subject":"Git Merge 2020 slides and reproducibility","startedAt":"2020-03-06T15:00:50Z","lastAt":"2020-03-10T14:36:55Z","messageCount":8,"participants":["Elijah Newren","Derrick Stolee","Matheus Tavares Bernardino","Konstantin Tokarev"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"392904","messageId":"CABPp-BHk0TyxEgudMX_-zzpFsUPHCmRkvZezN_49J2ivi2-N+w@mail.gmail.com","threadId":"52947","inReplyTo":null,"subject":"Git Merge 2020 slides and reproducibility","fromName":"Elijah Newren","fromEmail":"newren@gmail.com","sentAt":"2020-03-06T15:00:37Z","receivedAt":"2020-03-06T15:00:50Z","isPatch":false,"sender":{"key":"newren@gmail.com","avatar":"https://avatars.githubusercontent.com/u/5455730?v=4"},"body":"Hi,\n\nHad a few different folks ask me at Git Merge about slides for my\ntalk. I said I'd post them on github somewhere, but in case you were\none of the folks and have a hard time finding it...they are up at\nhttps://github.com/newren/presentations/blob/pdfs/merge-performance/merge-performance-slides.pdf\nand steps to reproduce the speedups I got can be found at\nhttps://github.com/newren/git/blob/git-merge-2020-demo/README.md\n(though be forewarned that the code is has lots of fixmes & ifdefs &\nother problems, has awful commit messages, etc.; I will be cleaning it\nup soon).\n\nI know the \"suggested\" way to make this stuff available was on\nTwitter, but I don't really have any much of any social media presence\n(I can't even access the blog I once had) and don't want to make a\ntwitter account just for this. (If someone else wants to repost my\nslides, feel free.)\n\nElijah\n"},{"id":"392908","messageId":"14db3e6f-6919-aa58-7084-e4404452820c@gmail.com","threadId":"52947","inReplyTo":"CABPp-BHk0TyxEgudMX_-zzpFsUPHCmRkvZezN_49J2ivi2-N+w@mail.gmail.com","subject":"Re: Git Merge 2020 slides and reproducibility","fromName":"Derrick Stolee","fromEmail":"stolee@gmail.com","sentAt":"2020-03-06T16:40:33Z","receivedAt":"2020-03-06T16:40:40Z","isPatch":false,"sender":{"key":"stolee@gmail.com","avatar":"https://avatars.githubusercontent.com/u/570044?v=4"},"body":"On 3/6/2020 10:00 AM, Elijah Newren wrote:\n> Had a few different folks ask me at Git Merge about slides for my\n> talk. I said I'd post them on github somewhere, but in case you were\n> one of the folks and have a hard time finding it...they are up at\n> https://github.com/newren/presentations/blob/pdfs/merge-performance/merge-performance-slides.pdf\n\nThanks! I guess I can post mine, too:\n\nhttps://stolee.dev/docs/git-merge-2020.pdf\n\n> and steps to reproduce the speedups I got can be found at\n> https://github.com/newren/git/blob/git-merge-2020-demo/README.md\n> (though be forewarned that the code is has lots of fixmes & ifdefs &\n> other problems, has awful commit messages, etc.; I will be cleaning it\n> up soon).\n> \n> I know the \"suggested\" way to make this stuff available was on\n> Twitter, but I don't really have any much of any social media presence\n> (I can't even access the blog I once had) and don't want to make a\n> twitter account just for this. (If someone else wants to repost my\n> slides, feel free.)\n\nDone: https://twitter.com/stolee/status/1235968445637771265?s=20\n\nThanks!\n-Stolee\n"},{"id":"392916","messageId":"CAHd-oW4P34aAoMfyDHDS1Kv9YpJ8ejU-GZpqtoHL8YLaJuEOeQ@mail.gmail.com","threadId":"52947","inReplyTo":"14db3e6f-6919-aa58-7084-e4404452820c@gmail.com","subject":"Re: Git Merge 2020 slides and reproducibility","fromName":"Matheus Tavares Bernardino","fromEmail":"matheus.bernardino@usp.br","sentAt":"2020-03-06T18:25:33Z","receivedAt":"2020-03-06T18:25:50Z","isPatch":false,"sender":{"key":"matheus.tavb@gmail.com","avatar":"https://avatars.githubusercontent.com/u/12701583?v=4"},"body":"On Fri, Mar 6, 2020 at 8:40 AM Derrick Stolee wrote:\n>\n> On 3/6/2020 10:00 AM, Elijah Newren wrote:\n> > Had a few different folks ask me at Git Merge about slides for my\n> > talk. I said I'd post them on github somewhere, but in case you were\n> > one of the folks and have a hard time finding it...they are up at\n> > https://github.com/newren/presentations/blob/pdfs/merge-performance/merge-performance-slides.pdf\n>\n> Thanks! I guess I can post mine, too:\n>\n> https://stolee.dev/docs/git-merge-2020.pdf\n\nThank you both for making your slides available. And for the great\npresentations, as well!\n\n---\nMatheus\n"},{"id":"392936","messageId":"3165171583586403@sas1-2bf44b70450e.qloud-c.yandex.net","threadId":"52947","inReplyTo":"CABPp-BHk0TyxEgudMX_-zzpFsUPHCmRkvZezN_49J2ivi2-N+w@mail.gmail.com","subject":"Re: Git Merge 2020 slides and reproducibility","fromName":"Konstantin Tokarev","fromEmail":"annulen@yandex.ru","sentAt":"2020-03-07T13:38:44Z","receivedAt":"2020-03-07T13:45:50Z","isPatch":false,"sender":{"key":"annulen@yandex.ru","avatar":null},"body":"\n\n06.03.2020, 18:00, \"Elijah Newren\" :\n> Hi,\n>\n> Had a few different folks ask me at Git Merge about slides for my\n> talk. I said I'd post them on github somewhere, but in case you were\n> one of the folks and have a hard time finding it...they are up at\n> https://github.com/newren/presentations/blob/pdfs/merge-performance/merge-performance-slides.pdf\n> and steps to reproduce the speedups I got can be found at\n> https://github.com/newren/git/blob/git-merge-2020-demo/README.md\n> (though be forewarned that the code is has lots of fixmes & ifdefs &\n> other problems, has awful commit messages, etc.; I will be cleaning it\n> up soon).\n\nHello, I've just tried your branch on my repository and it seems like it can\nbe a salvation from all rename-related pain that I'm regularly facing when\ndoing merges and cherry-picks! Thank you very much, I hope it will be\nintegrated into mainline soon.\n\nHowever, when testing my previous merges which had to be done with helper \nscript, I've encountered case of\n\nCONFLICT (directory rename split)\n\nIs there any way to prevent conflict in this case if files are the same, and\nmerge their contents if there are differences? I think it would be reasonable\nto assume that move done in newest commit should win, and allow user\nto change strategy via command line option, provide explicit hint where files\nshould be moved, or maybe even decide it interactively.\n\n-- \nRegards,\nKonstantin\n\n"},{"id":"392938","messageId":"CABPp-BECOarg+G-_oz83i0EuKuypJQA=wyjnfG4U0heG=0L0hg@mail.gmail.com","threadId":"52947","inReplyTo":"3165171583586403@sas1-2bf44b70450e.qloud-c.yandex.net","subject":"Re: Git Merge 2020 slides and reproducibility","fromName":"Elijah Newren","fromEmail":"newren@gmail.com","sentAt":"2020-03-07T16:03:30Z","receivedAt":"2020-03-07T16:03:44Z","isPatch":false,"sender":{"key":"newren@gmail.com","avatar":"https://avatars.githubusercontent.com/u/5455730?v=4"},"body":"On Sat, Mar 7, 2020 at 5:38 AM Konstantin Tokarev wrote:\n>\n> 06.03.2020, 18:00, \"Elijah Newren\" :\n> > Hi,\n> >\n> > Had a few different folks ask me at Git Merge about slides for my\n> > talk. I said I'd post them on github somewhere, but in case you were\n> > one of the folks and have a hard time finding it...they are up at\n> > https://github.com/newren/presentations/blob/pdfs/merge-performance/merge-performance-slides.pdf\n> > and steps to reproduce the speedups I got can be found at\n> > https://github.com/newren/git/blob/git-merge-2020-demo/README.md\n> > (though be forewarned that the code is has lots of fixmes & ifdefs &\n> > other problems, has awful commit messages, etc.; I will be cleaning it\n> > up soon).\n>\n> Hello, I've just tried your branch on my repository and it seems like it can\n> be a salvation from all rename-related pain that I'm regularly facing when\n> doing merges and cherry-picks! Thank you very much, I hope it will be\n> integrated into mainline soon.\n\nWow, thanks for trying it out. Please note that while it _might_ be\nokay to use for real work, I am not that confident that it is. There\nare a number of factors making the 'demo' label I gave it a rather\nfitting one:\n\n * I only started using it personally on a real world repository (or\ntwo) about a week and a half ago. (Before then, I knew merge-ort\ndidn't work.)\n * The second real world repo I used it on uncovered a bug in my code\nthat the testsuite didn't catch[1]\n * Although I've tested with two real world repos now, that testing\nwas very minimal; I was focused on getting the demo ready and\nimplementing as many optimizations as I could.\n * While the outer merge, rebase, and cherry-pick commands will\naccept a bunch of merge-machinery options and pass them along,\nmerge-ort flat ignores them all.\n * merge-ort is hardcoded for merge.directoryRenames=true, when the\ndefault should be merge.directoryRenames=conflict\n * it has a bunch of FIXMEs, some of which are code cleanliness\nissues but some of which represent minor bugs\n\n[1] https://lore.kernel.org/git/911de63afa274b0791e4d4252934a5e9b0031f10.1582762465.git.gitgitgadget@gmail.com/\n\nAlso...\n\n> However, when testing my previous merges which had to be done with helper\n> script, I've encountered case of\n>\n> CONFLICT (directory rename split)\n>\n> Is there any way to prevent conflict in this case if files are the same, and\n> merge their contents if there are differences? I think it would be reasonable\n> to assume that move done in newest commit should win, and allow user\n> to change strategy via command line option, provide explicit hint where files\n> should be moved, or maybe even decide it interactively.\n\nThis conflict message is known to trigger in some cases where it\nshouldn't; it may be that you're just experiencing annoyance from\nthat. Let me fix that issue before worrying about workarounds.\n\n\nAlso, if you try out the 'fast-rebase' builtin from that branch (which\nis a demo only and not meant to become a real command), note that its\nusage message is really helpful:\n$ git fast-rebase -h\nfatal: usage: read the code, figure out how to use it, then do so\n\nIt's the kind of thing you put in code when you're trying to get it\nworking the night before you'll include its results in your talk (and\nfinish getting it to work the morning of)...\n\n\n\nAnyway, thank you very much for giving it a whirl and reporting, just\nplease be cautious about depending on it since it's still work in\nprogress.\n\nElijah\n"},{"id":"392939","messageId":"3207561583597253@iva2-fa9fd5fad11f.qloud-c.yandex.net","threadId":"52947","inReplyTo":"CABPp-BECOarg+G-_oz83i0EuKuypJQA=wyjnfG4U0heG=0L0hg@mail.gmail.com","subject":"Re: Git Merge 2020 slides and reproducibility","fromName":"Konstantin Tokarev","fromEmail":"annulen@yandex.ru","sentAt":"2020-03-07T19:38:36Z","receivedAt":"2020-03-07T19:38:43Z","isPatch":false,"sender":{"key":"annulen@yandex.ru","avatar":null},"body":"\n\n07.03.2020, 19:03, \"Elijah Newren\" :\n> On Sat, Mar 7, 2020 at 5:38 AM Konstantin Tokarev wrote:\n>>  06.03.2020, 18:00, \"Elijah Newren\" :\n>>  > Hi,\n>>  >\n>>  > Had a few different folks ask me at Git Merge about slides for my\n>>  > talk. I said I'd post them on github somewhere, but in case you were\n>>  > one of the folks and have a hard time finding it...they are up at\n>>  > https://github.com/newren/presentations/blob/pdfs/merge-performance/merge-performance-slides.pdf\n>>  > and steps to reproduce the speedups I got can be found at\n>>  > https://github.com/newren/git/blob/git-merge-2020-demo/README.md\n>>  > (though be forewarned that the code is has lots of fixmes & ifdefs &\n>>  > other problems, has awful commit messages, etc.; I will be cleaning it\n>>  > up soon).\n>>\n>>  Hello, I've just tried your branch on my repository and it seems like it can\n>>  be a salvation from all rename-related pain that I'm regularly facing when\n>>  doing merges and cherry-picks! Thank you very much, I hope it will be\n>>  integrated into mainline soon.\n>\n> Wow, thanks for trying it out. Please note that while it _might_ be\n> okay to use for real work, I am not that confident that it is.\n\nDo not worry, I've made full copy of repo before trying anything.\n\n> There\n> are a number of factors making the 'demo' label I gave it a rather\n> fitting one:\n>\n>   * I only started using it personally on a real world repository (or\n> two) about a week and a half ago. (Before then, I knew merge-ort\n> didn't work.)\n>   * The second real world repo I used it on uncovered a bug in my code\n> that the testsuite didn't catch[1]\n>   * Although I've tested with two real world repos now, that testing\n> was very minimal; I was focused on getting the demo ready and\n> implementing as many optimizations as I could.\n>   * While the outer merge, rebase, and cherry-pick commands will\n> accept a bunch of merge-machinery options and pass them along,\n> merge-ort flat ignores them all.\n>   * merge-ort is hardcoded for merge.directoryRenames=true, when the\n> default should be merge.directoryRenames=conflict\n\ndirectoryRenames=true is actually one of features which I was badly\nmissing and somehow overlooked.\n\n>   * it has a bunch of FIXMEs, some of which are code cleanliness\n> issues but some of which represent minor bugs\n>\n> [1] https://lore.kernel.org/git/911de63afa274b0791e4d4252934a5e9b0031f10.1582762465.git.gitgitgadget@gmail.com/\n>\n> Also...\n>\n>>  However, when testing my previous merges which had to be done with helper\n>>  script, I've encountered case of\n>>\n>>  CONFLICT (directory rename split)\n>>\n>>  Is there any way to prevent conflict in this case if files are the same, and\n>>  merge their contents if there are differences? I think it would be reasonable\n>>  to assume that move done in newest commit should win, and allow user\n>>  to change strategy via command line option, provide explicit hint where files\n>>  should be moved, or maybe even decide it interactively.\n>\n> This conflict message is known to trigger in some cases where it\n> shouldn't; it may be that you're just experiencing annoyance from\n> that. Let me fix that issue before worrying about workarounds.\n\nWell, in my case a directory of files was moved path A in one of merged heads\nand to path B in another, so I guess it was legitimate.\n\nAre you going to continue development in the same branch?\nWhen do you expect it to be ready for review?\n-- \nRegards,\nKonstantin\n\n\n"},{"id":"392966","messageId":"CABPp-BGyz2uRtmw05uCFVACq9aXS9fwcLwEEvw4EU9toixwf2w@mail.gmail.com","threadId":"52947","inReplyTo":"3207561583597253@iva2-fa9fd5fad11f.qloud-c.yandex.net","subject":"Re: Git Merge 2020 slides and reproducibility","fromName":"Elijah Newren","fromEmail":"newren@gmail.com","sentAt":"2020-03-09T15:50:47Z","receivedAt":"2020-03-09T15:50:57Z","isPatch":false,"sender":{"key":"newren@gmail.com","avatar":"https://avatars.githubusercontent.com/u/5455730?v=4"},"body":"On Sat, Mar 7, 2020 at 11:38 AM Konstantin Tokarev wrote:\n>\n> 07.03.2020, 19:03, \"Elijah Newren\" :\n> > On Sat, Mar 7, 2020 at 5:38 AM Konstantin Tokarev wrote:\n...\n> >> However, when testing my previous merges which had to be done with helper\n> >> script, I've encountered case of\n> >>\n> >> CONFLICT (directory rename split)\n> >>\n> >> Is there any way to prevent conflict in this case if files are the same, and\n> >> merge their contents if there are differences? I think it would be reasonable\n> >> to assume that move done in newest commit should win, and allow user\n> >> to change strategy via command line option, provide explicit hint where files\n> >> should be moved, or maybe even decide it interactively.\n> >\n> > This conflict message is known to trigger in some cases where it\n> > shouldn't; it may be that you're just experiencing annoyance from\n> > that. Let me fix that issue before worrying about workarounds.\n>\n> Well, in my case a directory of files was moved path A in one of merged heads\n> and to path B in another, so I guess it was legitimate.\n\nThe point of directory rename detection is to allow new paths on the\nunrenamed side of history to follow the directory rename. So, while\nthere may have been an ambiguous directory rename, if there were no\nnew paths to be moved by it, then that directory rename is irrelevant\nand shouldn't be reported as a problem. (If you did have new paths on\nthe unrenamed side in that directory, then yes, it's legitimate.)\n\n> Are you going to continue development in the same branch?\n\nNope, the branch exists for reproducibility of the demo. Right now,\nmy plan is to work on the 'ort' branch (which the git-merge-2020-demo\nbranch was a snapshot of), but I reserve the right at any time to push\nup code to that branch that doesn't even compile or is known to be\nhorribly broken.\n\n> When do you expect it to be ready for review?\n\nGood question. There's other work I've been pushing off with the\nexcuse of preparing for the Git Merge 2020 conference, and working on\nthose other things may limit my time on this and make it harder to\ngive good guestimates.\n\nI'm hoping that _parts_ of it will be ready to review a week or two\nafter 2.26 is released. That will not mean I'm done with development\nat that time, just that I'm trying to get feedback in parallel with\ndoing further development. Besides competing priorities, there's\nanother reason to be somewhat cautious about the timeline: I don't\nwant us to replace one area of the code that only one person is\nwilling to touch with a different scary beast that no one wants to\ntouch. So, I need to put some work into high level algorithm and data\nstructure documentation, splitting up patches nicely, etc. And the\npurpose of writing those documents isn't to put the design in stone,\nbut rather to make review easier -- at which point I expect at least\none big change or two (and dozens of small changes) to be requested\nfor maintenance/performance/API-design reasons. I'll be disappointed\nif I don't get that kind of feedback, as I'll be worried we're just\nputting a new black box into place.\n\nI happen to think that the basics of the new module are nicer than the\nold merge-recursive module I'm replacing, but the performance work\ncomplicated things a fair amount and I want to make it more\napproachable. So, we'll see.\n\nI know this is horribly vague. Sorry.\n\nElijah\n"},{"id":"392996","messageId":"6997681583850484@myt6-887fb48a9c29.qloud-c.yandex.net","threadId":"52947","inReplyTo":"CABPp-BGyz2uRtmw05uCFVACq9aXS9fwcLwEEvw4EU9toixwf2w@mail.gmail.com","subject":"Re: Git Merge 2020 slides and reproducibility","fromName":"Konstantin Tokarev","fromEmail":"annulen@yandex.ru","sentAt":"2020-03-10T14:36:49Z","receivedAt":"2020-03-10T14:36:55Z","isPatch":false,"sender":{"key":"annulen@yandex.ru","avatar":null},"body":"\n\n09.03.2020, 18:50, \"Elijah Newren\" :\n> On Sat, Mar 7, 2020 at 11:38 AM Konstantin Tokarev wrote:\n>>  07.03.2020, 19:03, \"Elijah Newren\" :\n>>  > On Sat, Mar 7, 2020 at 5:38 AM Konstantin Tokarev wrote:\n>\n> ...\n>>  >> However, when testing my previous merges which had to be done with helper\n>>  >> script, I've encountered case of\n>>  >>\n>>  >> CONFLICT (directory rename split)\n>>  >>\n>>  >> Is there any way to prevent conflict in this case if files are the same, and\n>>  >> merge their contents if there are differences? I think it would be reasonable\n>>  >> to assume that move done in newest commit should win, and allow user\n>>  >> to change strategy via command line option, provide explicit hint where files\n>>  >> should be moved, or maybe even decide it interactively.\n>>  >\n>>  > This conflict message is known to trigger in some cases where it\n>>  > shouldn't; it may be that you're just experiencing annoyance from\n>>  > that. Let me fix that issue before worrying about workarounds.\n>>\n>>  Well, in my case a directory of files was moved path A in one of merged heads\n>>  and to path B in another, so I guess it was legitimate.\n>\n> The point of directory rename detection is to allow new paths on the\n> unrenamed side of history to follow the directory rename. So, while\n> there may have been an ambiguous directory rename, if there were no\n> new paths to be moved by it, then that directory rename is irrelevant\n> and shouldn't be reported as a problem. (If you did have new paths on\n> the unrenamed side in that directory, then yes, it's legitimate.)\n\nIn my case, both sides have different renames, but files in subject directory are\nmostly unchanged. It would even work for me if merge placed it to wrong\ndirectory in the end, just to have it merge files contents automatically.\n\n>\n>>  Are you going to continue development in the same branch?\n>\n> Nope, the branch exists for reproducibility of the demo. Right now,\n> my plan is to work on the 'ort' branch (which the git-merge-2020-demo\n> branch was a snapshot of), but I reserve the right at any time to push\n> up code to that branch that doesn't even compile or is known to be\n> horribly broken.\n>\n>>  When do you expect it to be ready for review?\n>\n> Good question. There's other work I've been pushing off with the\n> excuse of preparing for the Git Merge 2020 conference, and working on\n> those other things may limit my time on this and make it harder to\n> give good guestimates.\n>\n> I'm hoping that _parts_ of it will be ready to review a week or two\n> after 2.26 is released. That will not mean I'm done with development\n> at that time, just that I'm trying to get feedback in parallel with\n> doing further development. Besides competing priorities, there's\n> another reason to be somewhat cautious about the timeline: I don't\n> want us to replace one area of the code that only one person is\n> willing to touch with a different scary beast that no one wants to\n> touch. So, I need to put some work into high level algorithm and data\n> structure documentation, splitting up patches nicely, etc. And the\n> purpose of writing those documents isn't to put the design in stone,\n> but rather to make review easier -- at which point I expect at least\n> one big change or two (and dozens of small changes) to be requested\n> for maintenance/performance/API-design reasons. I'll be disappointed\n> if I don't get that kind of feedback, as I'll be worried we're just\n> putting a new black box into place.\n>\n> I happen to think that the basics of the new module are nicer than the\n> old merge-recursive module I'm replacing, but the performance work\n> complicated things a fair amount and I want to make it more\n> approachable. So, we'll see.\n\n/me personally would at any time prefer correct renames detection over speed,\neven if things become _slower_, just to resolve less conflicts manually.\nHowever, I guess planning all optimizations up front may be necessary to choose\noptimal data structures.\n\n>\n> I know this is horribly vague. Sorry.\n\nNo problem, thanks a lot for your work and this information!\n\n>\n> Elijah\n\n-- \nRegards,\nKonstantin\n\n"}]}