{"thread":{"id":"13977","subject":"git-rerere observations and feature suggestions","startedAt":"2008-06-16T11:01:13Z","lastAt":"2008-06-23T15:22:48Z","messageCount":45,"participants":["Ingo Molnar","Mike Hommey","David Kastrup","Theodore Tso","Pierre Habouzit","Sverre Rabbelier","Junio C Hamano","Jakub Narebski","Karl Hasselström","Johannes Schindelin","Miklos Vajna","Peter Zijlstra","Jeff King"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"80016","messageId":"20080616110113.GA22945@elte.hu","threadId":"13977","inReplyTo":null,"subject":"git-rerere observations and feature suggestions","fromName":"Ingo Molnar","fromEmail":"mingo@elte.hu","sentAt":"2008-06-16T11:01:13Z","receivedAt":"2008-06-16T11:01:13Z","isPatch":false,"sender":{"key":"mingo@elte.hu","avatar":null},"body":"We are running a rather complex Git tree with heavy use of git-rerere \n(the -tip kernel tree, with more than 80 topic branches). git-rerere is \nreally nice in that it caches conflict resolutions, but there are a few \nareas where it would be nice to have improvements:\n\n - Fixing resolutions: currently, when i do an incorrect conflict\n resolution, and fix it on the next run, git-rerere does not pick up\n the new resolution but uses the old (buggy) one on the next run. To\n fix it up i have to find the right entries in .git/rr-cache/* and\n manually erase them. Would be nice to have \"git-rerere gc \"\n to flush out a single bad resolution.\n\n - File deletion: would be nice if git-rerere picked up git-rm\n resolutions. We hit this every now and then and right now i know \n which ones need an extra git-rm pass.\n\n - Automation: would be nice to have a git-rerere modus operandi where\n it would auto-commit things if and only if all conflicting files were \n resolved.\n\n - Sharing .git/rr-cache. It's quite a PITA to share the .git/rr-cache\n amongst -tip maintainers right now. It seems to have dependencies on \n the index file, so if we want to share the conflict resolution data, \n we have to copy our index file (which is dangerous anyway and assumes \n very similar repositories).\n\n It would be much nicer if we could share conflict resolutions with \n each other - and with others as well. For example linux-next could \n re-use our conflict resolution data as well - often Stephen Rothwell \n has to re-do the same conflict resolution as well, creating \n duplicated work.\n\n ( Also, it's a GPL nitpicky issue: the conflict resolution database \n can be argued to be part of \"source code\" and as such it should be \n shared with everyone who asks. With trivial merges the data is\n probably not copyrightable hence probably falls outside the scope \n of the GPL, but with a complex topic tree like -tip with dozens of \n conflict resolutions, the boundary is perhaps more blurred. )\n\n\tIngo\n"},{"id":"80017","messageId":"20080616110918.GA30856@glandium.org","threadId":"13977","inReplyTo":"20080616110113.GA22945@elte.hu","subject":"Re: git-rerere observations and feature suggestions","fromName":"Mike Hommey","fromEmail":"mh@glandium.org","sentAt":"2008-06-16T11:09:18Z","receivedAt":"2008-06-16T11:09:18Z","isPatch":false,"sender":{"key":"mh@glandium.org","avatar":"https://avatars.githubusercontent.com/u/1038527?v=4"},"body":"- At least, compress the data in the rr-cache. It can grow big quite\n easily. Also, I wonder if keeping the entire files is not overkill...\n\nMike\n"},{"id":"80018","messageId":"85bq21fwnp.fsf@lola.goethe.zz","threadId":"13977","inReplyTo":"20080616110113.GA22945@elte.hu","subject":"Re: git-rerere observations and feature suggestions","fromName":"David Kastrup","fromEmail":"dak@gnu.org","sentAt":"2008-06-16T11:26:34Z","receivedAt":"2008-06-16T11:26:34Z","isPatch":false,"sender":{"key":"dak@gnu.org","avatar":"https://avatars.githubusercontent.com/u/52141349?v=4"},"body":"Ingo Molnar writes:\n\n> ( Also, it's a GPL nitpicky issue: the conflict resolution database \n> can be argued to be part of \"source code\" and as such it should be \n> shared with everyone who asks.\n\nI don't think that interpretation holds water. Not even the version\ncontrol history is part of the _corresponding_ source code AFAICT. If\nit were, GPLed software distributions would be a nightmare since you\nwould have to deliver everything with complete history.\n\nOnly very nonstandard usage of version control might make the\n_corresponding_ source code be contained in more than HEAD of the\nrelease branch.\n\n-- \nDavid Kastrup, Kriemhildstr. 15, 44793 Bochum\n"},{"id":"80020","messageId":"20080616112709.GG12260@mit.edu","threadId":"13977","inReplyTo":"20080616110113.GA22945@elte.hu","subject":"Re: git-rerere observations and feature suggestions","fromName":"Theodore Tso","fromEmail":"tytso@mit.edu","sentAt":"2008-06-16T11:27:09Z","receivedAt":"2008-06-16T11:27:09Z","isPatch":false,"sender":{"key":"tytso@mit.edu","avatar":"https://avatars.githubusercontent.com/u/51416?v=4"},"body":"On Mon, Jun 16, 2008 at 01:01:13PM +0200, Ingo Molnar wrote:\n> ( Also, it's a GPL nitpicky issue: the conflict resolution database \n> can be argued to be part of \"source code\" and as such it should be \n> shared with everyone who asks. With trivial merges the data is\n> probably not copyrightable hence probably falls outside the scope \n> of the GPL, but with a complex topic tree like -tip with dozens of \n> conflict resolutions, the boundary is perhaps more blurred. )\n\nFor a more complex merge resolution, granted that it rises to the\nlevel of being \"copyrightable\", but I think it would be a huge stretch\nto call the rr-cache the \"preferred form for modifications\"! :-)\n\n \t \t \t \t \t - Ted\n"},{"id":"80027","messageId":"8563s9ftce.fsf@lola.goethe.zz","threadId":"13977","inReplyTo":"20080616112709.GG12260@mit.edu","subject":"Re: git-rerere observations and feature suggestions","fromName":"David Kastrup","fromEmail":"dak@gnu.org","sentAt":"2008-06-16T12:38:09Z","receivedAt":"2008-06-16T12:38:09Z","isPatch":false,"sender":{"key":"dak@gnu.org","avatar":"https://avatars.githubusercontent.com/u/52141349?v=4"},"body":"Theodore Tso writes:\n\n> On Mon, Jun 16, 2008 at 01:01:13PM +0200, Ingo Molnar wrote:\n>> ( Also, it's a GPL nitpicky issue: the conflict resolution database \n>> can be argued to be part of \"source code\" and as such it should be \n>> shared with everyone who asks. With trivial merges the data is\n>> probably not copyrightable hence probably falls outside the scope \n>> of the GPL, but with a complex topic tree like -tip with dozens of \n>> conflict resolutions, the boundary is perhaps more blurred. )\n>\n> For a more complex merge resolution, granted that it rises to the\n> level of being \"copyrightable\", but I think it would be a huge stretch\n> to call the rr-cache the \"preferred form for modifications\"! :-)\n\nThe GPL just calls for all \"corresponding\" source code, not all\n\"interesting\" source code.\n\n-- \nDavid Kastrup, Kriemhildstr. 15, 44793 Bochum\n"},{"id":"80034","messageId":"20080616154851.GA6938@artemis.madism.org","threadId":"13977","inReplyTo":"20080616110918.GA30856@glandium.org","subject":"Re: git-rerere observations and feature suggestions","fromName":"Pierre Habouzit","fromEmail":"madcoder@debian.org","sentAt":"2008-06-16T15:48:51Z","receivedAt":"2008-06-16T15:48:51Z","isPatch":false,"sender":{"key":"madcoder@debian.org","avatar":"https://avatars.githubusercontent.com/u/44708?v=4"},"body":"On Mon, Jun 16, 2008 at 11:09:18AM +0000, Mike Hommey wrote:\n> - At least, compress the data in the rr-cache. It can grow big quite\n> easily. Also, I wonder if keeping the entire files is not overkill...\n\n Actually it would be rather straightforward to put it in the usual git\nstore, and represent the current rr-cache with a flat file that points\nto the in-git preimage/postimages, and make git-gc aware of those.\n\n This would deal with the huge number of files + compression quite\neasily. I'm quite sure it's pretty straightforward actually :)\n\n-- \n·O· Pierre Habouzit\n··O madcoder@debian.org\nOOO http://www.madism.org\n"},{"id":"80038","messageId":"20080616155747.GB6938@artemis.madism.org","threadId":"13977","inReplyTo":"20080616154851.GA6938@artemis.madism.org","subject":"Re: git-rerere observations and feature suggestions","fromName":"Pierre Habouzit","fromEmail":"madcoder@debian.org","sentAt":"2008-06-16T15:57:47Z","receivedAt":"2008-06-16T15:57:47Z","isPatch":false,"sender":{"key":"madcoder@debian.org","avatar":"https://avatars.githubusercontent.com/u/44708?v=4"},"body":"On Mon, Jun 16, 2008 at 03:48:51PM +0000, Pierre Habouzit wrote:\n> On Mon, Jun 16, 2008 at 11:09:18AM +0000, Mike Hommey wrote:\n> > - At least, compress the data in the rr-cache. It can grow big quite\n> > easily. Also, I wonder if keeping the entire files is not overkill...\n> \n> Actually it would be rather straightforward to put it in the usual git\n> store, and represent the current rr-cache with a flat file that points\n> to the in-git preimage/postimages, and make git-gc aware of those.\n\n Actually, this is probably a required step in the direction of sharing\nsuch things btw.\n\n-- \n·O· Pierre Habouzit\n··O madcoder@debian.org\nOOO http://www.madism.org\n"},{"id":"80044","messageId":"bd6139dc0806160918w1eea96f3r6f676d7cf186652d@mail.gmail.com","threadId":"13977","inReplyTo":"20080616155747.GB6938@artemis.madism.org","subject":"Re: git-rerere observations and feature suggestions","fromName":"Sverre Rabbelier","fromEmail":"alturin@gmail.com","sentAt":"2008-06-16T16:18:40Z","receivedAt":"2008-06-16T16:18:40Z","isPatch":false,"sender":{"key":"alturin@gmail.com","avatar":null},"body":"On Mon, Jun 16, 2008 at 5:57 PM, Pierre Habouzit wrote:\n> On Mon, Jun 16, 2008 at 03:48:51PM +0000, Pierre Habouzit wrote:\n>> Actually it would be rather straightforward to put it in the usual git\n>> store, and represent the current rr-cache with a flat file that points\n>> to the in-git preimage/postimages, and make git-gc aware of those.\n>\n> Actually, this is probably a required step in the direction of sharing\n> such things btw.\n\nPerhaps an approach similar to the 'notes' implementation can be used,\nin which a separate branch is created to contain the notes. This way\nthe rerere information (being the 'rerere' branch) can be shared\neasily (by just pulling the branch), and as said we get free\ncompression. Another advantage would be that you automagically get the\nability to unlearn a bad rerere by simply (partially) reverting a\ncommit on the rerere branch!\n\n-- \nCheers,\n\nSverre Rabbelier\n"},{"id":"80055","messageId":"7vej6xb4lr.fsf@gitster.siamese.dyndns.org","threadId":"13977","inReplyTo":"20080616110113.GA22945@elte.hu","subject":"Re: git-rerere observations and feature suggestions","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2008-06-16T18:46:08Z","receivedAt":"2008-06-16T18:46:08Z","isPatch":false,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Ingo Molnar writes:\n\n> We are running a rather complex Git tree with heavy use of git-rerere \n> (the -tip kernel tree, with more than 80 topic branches). git-rerere is \n> really nice in that it caches conflict resolutions, but there are a few \n> areas where it would be nice to have improvements:\n>\n> - Fixing resolutions: currently, when i do an incorrect conflict\n> resolution, and fix it on the next run, git-rerere does not pick up\n> the new resolution but uses the old (buggy) one on the next run. To\n> fix it up i have to find the right entries in .git/rr-cache/* and\n> manually erase them. Would be nice to have \"git-rerere gc \"\n> to flush out a single bad resolution.\n\nI agree this is a real issue (I sometimes know that the resolution is iffy\nand say \"rerere clear\" to choose not to record it, but that is working\naround the issue with a perfect foresight and is not a solution).\n\nI think (and I think you would agree) \"gc\" is not the right word but\nrather you would want to more actively discard the wrong one.\n\nI agree that it is the right UI to do this to specify paths right after\nyou found that a bad resolution that was recorded previously was used by\nrerere (I think that is what you are suggesting). Upon such a request, we\nshould undo the bad resolution and bring the working tree copy to the\noriginal conflicted state, and clear the bad rerere entry.\n\n> - File deletion: would be nice if git-rerere picked up git-rm\n> resolutions. We hit this every now and then and right now i know \n> which ones need an extra git-rm pass.\n\nI originally did not have need for anything other than three-way conflict\nresolving to a result. I do not know how safe reapplying a removal to\ndifferent context, though.\n\n> - Automation: would be nice to have a git-rerere modus operandi where\n> it would auto-commit things if and only if all conflicting files were \n> resolved.\n\nI am not sure how safe this is. rerere as originally designed does not\neven update the index with merge results so that the application of\nearlier resolution can be manually inspected, and this is exactly because\nI consider a blind textual reapplication of previous resolution always\niffy, even though I invented the whole mechanism.\n"},{"id":"80057","messageId":"20080616190911.GA7047@elte.hu","threadId":"13977","inReplyTo":"7vej6xb4lr.fsf@gitster.siamese.dyndns.org","subject":"Re: git-rerere observations and feature suggestions","fromName":"Ingo Molnar","fromEmail":"mingo@elte.hu","sentAt":"2008-06-16T19:09:11Z","receivedAt":"2008-06-16T19:09:11Z","isPatch":false,"sender":{"key":"mingo@elte.hu","avatar":null},"body":"\n* Junio C Hamano wrote:\n\n> > - Automation: would be nice to have a git-rerere modus operandi where\n> > it would auto-commit things if and only if all conflicting files were \n> > resolved.\n> \n> I am not sure how safe this is. rerere as originally designed does \n> not even update the index with merge results so that the application \n> of earlier resolution can be manually inspected, and this is exactly \n> because I consider a blind textual reapplication of previous \n> resolution always iffy, even though I invented the whole mechanism.\n\nWe use a 'safe, lazy integration' method in -tip, that basically has \nexternal checks against any integration bugs.\n\nBasically, we integrate only about once a day, and we advance the topic \nbranches but do not reintegrate on every topic merge. We merge commits \n_both_ to their target topic branches, and to the (previous) integration \nbranch.\n\nThen once a day (or every second day) we 'reintegrate': we propagate the \ntopic branches to the linux-next auto-*-next branches [recreating them \nfrom scratch] and flush out the messy criss-cross merges from the \nintegration tree.\n\nBut that is always an identity transformation as far as the integration \nresult is concerned: the result of the integration run must be exactly \nthe same content (obviously it results in a very different tree \nstructure) as the previous one. We only run it on a perfectly tested \ntree so we know none of our previous merges were wrong, and we want the \ngit-rerere result to be the same. We repeat the integration until the \nend result matches.\n\nIn fact sometimes git-rerere is able to pick up a conflict resolution \nfrom our 'messy' delta-merge into the integration tree, which is an \nadded bonus. (this doesnt always work if the merge order differs from \nintegration order)\n\nAnyway, the gist is that in this workflow it does not hurt at all if \ngit-rerere is \"unsafe\", and we'd love to have the integration as fast as \npossible. Right now most of my manual overhead is in making sure that \ngit-rerere has not missed some file.\n\nAt a ~100 conflicting files tracked, that is rather error-prone, and i'd \nlove to have further automation here besides a rather lame method of \ngrepping for:\n\n \"Resolved 'kernel/Makefile' using previous resolution.\"\n\ntype of patterns in git-merge output.\n\nSo i'd not mind if git-rerere was safe by default, but it would be nice \nto have some knob to turn it into something fast and automatic. For us \nit would be much _safer_, because right now most of our manual energy is \nspent on checking something that could be automated.\n\nWe could in theory avoid git-rerere altogether by creating separate \nconflict resolution branches, and automated their handling - but we \nthought git-rerere was pretty nice as well and kept the branch count \ndown.\n\nAnd while asking for an arm i'd also like to ask for a leg, if i may: \ni'd love it if a \"slightly conflicting\" octopus merge of 85 topic trees \nwould not result in one huge conflict commit that merges together 1000 \ncommits into a single commit ;-)\n\nSo right now in our -tip scripts work around this issue: we 'serialize' \nthe topic merges despite having very nice opportunities for higher-order \noctopus merges. The integration would be a lot faster if we could use \noctopus merges and automated git-rerere. (Octopus merges would look much \nnicer as well in graphical representation as well, which counts too :-) )\n\n\tIngo\n"},{"id":"80058","messageId":"7vabhlb3ho.fsf@gitster.siamese.dyndns.org","threadId":"13977","inReplyTo":"7vej6xb4lr.fsf@gitster.siamese.dyndns.org","subject":"Re: git-rerere observations and feature suggestions","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2008-06-16T19:10:11Z","receivedAt":"2008-06-16T19:10:11Z","isPatch":false,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Junio C Hamano writes:\n\n> Ingo Molnar writes:\n> ...\n>> - Automation: would be nice to have a git-rerere modus operandi where\n>> it would auto-commit things if and only if all conflicting files were \n>> resolved.\n>\n> I am not sure how safe this is. rerere as originally designed does not\n> even update the index with merge results so that the application of\n> earlier resolution can be manually inspected, and this is exactly because\n> I consider a blind textual reapplication of previous resolution always\n> iffy, even though I invented the whole mechanism.\n\nBy the way, this safety is not a theoretical issue but has been a real\none. I had two topics that changed the calling convention of the same\nfunction in different ways, and when they were merged to 'pu', the\ndeclaration, definition, and call sites existed on both of these branches\nwere handled beautifully by rerere.\n\nRecording autoresolution would have been a wrong thing to do. One of the\nbranches added a new call site to a file that was not among the ones that\nconflicted in the merge between the two branches. That call site, that\nuses the calling convention of one branch, needed to be adjusted to\naccomodate the change of calling convention from the other branch (from\ntextual merge's point of view, this has to be an evil merge). I had to\nmake and keep a mental note about that new call site until both topics\ngraduated to 'master' (similar to your need to remember a particular merge\nis resolved to removal right now).\n\nTo safely automate reapplication of such a merge, rerere needs to become\nmuch more clever.\n\nThe conflicts rerere notices and records are strictly per blob. A\nconflicted merge to a blob is inspected and a \"conflict signature\", which\nbecomes the directory name under rr-cache, is computed. We record the\nconflicted blob as a whole as the preimage, and your hand resolution as a\nwhoe as the postimage. Next time when you have a conflicted merge to a\nblob, and the conflict has the exact same conflict signature, we run\nthree-way merge between the recorded preimage, postimage and the new\nconflicted result.\n\nIf we want to handle new call sites added only on a single side, you\nshould be able to express something like \"when a merge has a conflicted\nblob with this conflict signature, look in the whole tree, even outside\nthe set of conflicted paths, and change this text to that\". This is too\nmuch automation and I somehow think the potential for errors (both from\nthe tool and from the user) is too high.\n"},{"id":"80059","messageId":"20080616194415.GA11447@elte.hu","threadId":"13977","inReplyTo":"7vabhlb3ho.fsf@gitster.siamese.dyndns.org","subject":"Re: git-rerere observations and feature suggestions","fromName":"Ingo Molnar","fromEmail":"mingo@elte.hu","sentAt":"2008-06-16T19:44:15Z","receivedAt":"2008-06-16T19:44:15Z","isPatch":false,"sender":{"key":"mingo@elte.hu","avatar":null},"body":"\n* Junio C Hamano wrote:\n\n> > Ingo Molnar writes:\n> > ...\n> >> - Automation: would be nice to have a git-rerere modus operandi where\n> >> it would auto-commit things if and only if all conflicting files were \n> >> resolved.\n> >\n> > I am not sure how safe this is. rerere as originally designed does \n> > not even update the index with merge results so that the application \n> > of earlier resolution can be manually inspected, and this is exactly \n> > because I consider a blind textual reapplication of previous \n> > resolution always iffy, even though I invented the whole mechanism.\n> \n> By the way, this safety is not a theoretical issue but has been a real \n> one. I had two topics that changed the calling convention of the same \n> function in different ways, and when they were merged to 'pu', the \n> declaration, definition, and call sites existed on both of these \n> branches were handled beautifully by rerere.\n> \n> Recording autoresolution would have been a wrong thing to do. One of \n> the branches added a new call site to a file that was not among the \n> ones that conflicted in the merge between the two branches. That call \n> site, that uses the calling convention of one branch, needed to be \n> adjusted to accomodate the change of calling convention from the other \n> branch (from textual merge's point of view, this has to be an evil \n> merge). I had to make and keep a mental note about that new call site \n> until both topics graduated to 'master' (similar to your need to \n> remember a particular merge is resolved to removal right now).\n> \n> To safely automate reapplication of such a merge, rerere needs to \n> become much more clever.\n\nin our workflow, we dont ever do any semantic things during the \nintegration run. I.e. we dont put more complex merge changes into the \nintegration merge commits.\n\nSuch integration effects do come up occasionally (especially when a \ntopic changes some widely used infrastructure), and we handle them via \nseparate merge branches. The current ones in -tip are \ntip/tracing/ftrace-mergefixups and tip/tracing/mmiotrace-mergefixups.\n\nThey are one or two orders of magnitude more rare than regular \nconflicts, and they show up immediately during testing. (or we \nanticipate them beforehand)\n\ni.e. we'd like to have a 'dumb' phase of integration, as much cached and \nautomated as possible. Things that need more thought need to go into \nseparate branches anyway, for better reviewability - merge commits are \nrather hard to debug as they hide their true contents, so we try to keep \nthem simple and contextual only.\n\n\tIngo\n"},{"id":"80061","messageId":"20080616195252.GA18848@elte.hu","threadId":"13977","inReplyTo":"20080616112709.GG12260@mit.edu","subject":"Re: git-rerere observations and feature suggestions","fromName":"Ingo Molnar","fromEmail":"mingo@elte.hu","sentAt":"2008-06-16T19:52:52Z","receivedAt":"2008-06-16T19:52:52Z","isPatch":false,"sender":{"key":"mingo@elte.hu","avatar":null},"body":"\n* Theodore Tso wrote:\n\n> On Mon, Jun 16, 2008 at 01:01:13PM +0200, Ingo Molnar wrote:\n> > ( Also, it's a GPL nitpicky issue: the conflict resolution database \n> > can be argued to be part of \"source code\" and as such it should be \n> > shared with everyone who asks. With trivial merges the data is\n> > probably not copyrightable hence probably falls outside the scope \n> > of the GPL, but with a complex topic tree like -tip with dozens of \n> > conflict resolutions, the boundary is perhaps more blurred. )\n> \n> For a more complex merge resolution, granted that it rises to the \n> level of being \"copyrightable\", but I think it would be a huge stretch \n> to call the rr-cache the \"preferred form for modifications\"! :-)\n\nyeah - i'm not really arguing any detail of the GPL here. I'm arguing \nthe principle: there should be no technical assymetry between maintainer \nand contributor. So if i am able to run an effort-free integration of 85 \ntopic branches, i'd like contributors (who will eventually grow up into \nco-maintainer roles in the future) to be able to do the same, if they \nwant to do so.\n\nright now that is simply not possible technically - it's even very hard \nto share a .git/rr-cache with a co-maintainer whom i can trust with my \nindex file. (which is an otherwise unsafe private binary cache that i'd \nnot put into a public repository as it could in theory contain lots of \nunrelated data and is not endian-safe, etc.)\n\n\tIngo\n"},{"id":"80064","messageId":"m3y755nq99.fsf@localhost.localdomain","threadId":"13977","inReplyTo":"20080616110113.GA22945@elte.hu","subject":"Re: git-rerere observations and feature suggestions","fromName":"Jakub Narebski","fromEmail":"jnareb@gmail.com","sentAt":"2008-06-16T20:11:10Z","receivedAt":"2008-06-16T20:11:10Z","isPatch":false,"sender":{"key":"jnareb@gmail.com","avatar":"https://avatars.githubusercontent.com/u/2706?v=4"},"body":"Ingo Molnar writes:\n\n> We are running a rather complex Git tree with heavy use of git-rerere \n> (the -tip kernel tree, with more than 80 topic branches). git-rerere is \n> really nice in that it caches conflict resolutions, but there are a few \n> areas where it would be nice to have improvements:\n[...]\n\n> - File deletion: would be nice if git-rerere picked up git-rm\n> resolutions. We hit this every now and then and right now i know \n> which ones need an extra git-rm pass.\n\n>From what I remember some time ago on git mailing list there was idea\nfor git-rerere2, which would record resolutions on tree level,\ni.e. record file renames. It could probably record file deletion as\nwell... would someone implement it, and didn't it stay loose idea.\n\n-- \nJakub Narebski\nPoland\nShadeHawk on #git\n"},{"id":"80066","messageId":"7v4p7tb00l.fsf@gitster.siamese.dyndns.org","threadId":"13977","inReplyTo":"20080616195252.GA18848@elte.hu","subject":"Re: git-rerere observations and feature suggestions","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2008-06-16T20:25:14Z","receivedAt":"2008-06-16T20:25:14Z","isPatch":false,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Ingo Molnar writes:\n\n> right now that is simply not possible technically - it's even very hard \n> to share a .git/rr-cache with a co-maintainer whom i can trust with my \n> index file. (which is an otherwise unsafe private binary cache that i'd \n> not put into a public repository as it could in theory contain lots of \n> unrelated data and is not endian-safe, etc.)\n\nWhere did you get the idea that .git/index is involved in any way, I\nwonder...\n"},{"id":"80070","messageId":"20080616204630.GA552@elte.hu","threadId":"13977","inReplyTo":"7v4p7tb00l.fsf@gitster.siamese.dyndns.org","subject":"Re: git-rerere observations and feature suggestions","fromName":"Ingo Molnar","fromEmail":"mingo@elte.hu","sentAt":"2008-06-16T20:46:30Z","receivedAt":"2008-06-16T20:46:30Z","isPatch":false,"sender":{"key":"mingo@elte.hu","avatar":null},"body":"\n* Junio C Hamano wrote:\n\n> Ingo Molnar writes:\n> \n> > right now that is simply not possible technically - it's even very \n> > hard to share a .git/rr-cache with a co-maintainer whom i can trust \n> > with my index file. (which is an otherwise unsafe private binary \n> > cache that i'd not put into a public repository as it could in \n> > theory contain lots of unrelated data and is not endian-safe, etc.)\n> \n> Where did you get the idea that .git/index is involved in any way, I \n> wonder...\n\nso it's only the rr-cache metadata that is involved? We had a few cases \nwhere git-rerere sessions were not repeatable by copying the \n.git/rr-cache, so i just assumed that there's some extra metadata in the \nindex file. When that happened i took a look at git/builtin-rerere.c:\n\n static int find_conflict(struct path_list *conflict)\n {\n int i;\n if (read_cache() < 0)\n return error(\"Could not read index\");\n\nand (mistakenly) assumed that git-rerere depends on having something in \nthe index file - but on a second look it just checks out the conflicting \nfile(s) from the index file, right?\n\n\tIngo\n"},{"id":"80071","messageId":"7vskvd9kai.fsf@gitster.siamese.dyndns.org","threadId":"13977","inReplyTo":"20080616190911.GA7047@elte.hu","subject":"Re: git-rerere observations and feature suggestions","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2008-06-16T20:50:13Z","receivedAt":"2008-06-16T20:50:13Z","isPatch":false,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Ingo Molnar writes:\n\n> * Junio C Hamano wrote:\n>\n>> > - Automation: would be nice to have a git-rerere modus operandi where\n>> > it would auto-commit things if and only if all conflicting files were \n>> > resolved.\n>> \n>> I am not sure how safe this is. rerere as originally designed does \n>> not even update the index with merge results so that the application \n>> of earlier resolution can be manually inspected, and this is exactly \n>> because I consider a blind textual reapplication of previous \n>> resolution always iffy, even though I invented the whole mechanism.\n> ...\n> So i'd not mind if git-rerere was safe by default, but it would be nice \n> to have some knob to turn it into something fast and automatic. For us \n> it would be much _safer_, because right now most of our manual energy is \n> spent on checking something that could be automated.\n\nOh, \"unsafe switch\" that is off by default will not hurt anybody, and I do\nnot mind it as a new feature. We are in agreement in that sense.\n\nPerhaps the way forward would be (and this is independent of the issue of\nrecording removal as a possible form of resolution):\n\n (1) Introduce a new configuration rerere.autoupdate that is off by\n default, but when it is on, paths cleanly resolved by rerere will\n also be updated in the index (if we have capability to record\n removal, this may remove such a path from the index as the result).\n\n (2) The callers of rerere that expects rerere to resolve needs to be\n changed to see if the resulting index after rerere is fully merged,\n and continue. Currently the callers are \"merge\", \"rebase\" and \"am\",\n I think. This step might be a bit more involved than you might\n think, as rerere currently happens in the codepath that knows the\n caller does _not_ go further than leaving the failed conflict to be\n sorted out by the user (rerere is designed as merely a way to help).\n\n Also you _might_ want a separate configuration rerere.autocommit to\n control this --- the user (but not you) might be willing to allow\n autoupdate but you may still want to eyeball the result.\n\nIndependent of the above, we have two potential new features:\n\n * Introduce \"git rerere revert paths...\" that brings the index and\n working tree back to the conflicted state after a previous resolution\n is applied, because that resolution is incorrect. The old resolution\n cached in rr-cache is also removed.\n\n This however will become much less useful if you allow autoresolution\n to be committed automatically, as the caller will move ahead without\n giving you a chance to say \"oh, that one is bad -- do not proceed\".\n\n * Somehow record the fact that the resolution for a particular conflict\n signature is to remove the resulting path.\n"},{"id":"80076","messageId":"7viqw99i2z.fsf@gitster.siamese.dyndns.org","threadId":"13977","inReplyTo":"20080616204630.GA552@elte.hu","subject":"Re: git-rerere observations and feature suggestions","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2008-06-16T21:37:56Z","receivedAt":"2008-06-16T21:37:56Z","isPatch":false,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Ingo Molnar writes:\n\n> * Junio C Hamano wrote:\n>\n>> Ingo Molnar writes:\n>> \n>> > right now that is simply not possible technically - it's even very \n>> > hard to share a .git/rr-cache with a co-maintainer whom i can trust \n>> > with my index file. (which is an otherwise unsafe private binary \n>> > cache that i'd not put into a public repository as it could in \n>> > theory contain lots of unrelated data and is not endian-safe, etc.)\n>> \n>> Where did you get the idea that .git/index is involved in any way, I \n>> wonder...\n>\n> so it's only the rr-cache metadata that is involved?\n\nThe binary part of the index should be in network byte order and endian\nsafe. But it is not necessary to share the index. Well, if you think\nabout it, it would be mighty silly if index had any long term effect on\nthe operation of rerere, which is all about \"I've done many conflict\nresolutions in the past. My work tree state (including the index) came\nback to a state similar to the conflicted state I saw some time ago.\nLet's reuse the previous resolution if we can.\" You might have switched\nbranches, ran \"reset --hard\" and did 47 thousands different things to your\nindex since you resolved the conflict you are about to re-resolve ;-).\n\nThe replay and conflict recoding codepath of rerere goes like this:\n\n * read the index, list the paths that have conflicts;\n\n * inspect the conflicted blob to compute the conflict signature $sig and\n store the sig and path in MERGE_RR;\n\n * look into rr-cache/$sig; does it have already a conflict resolution\n recorded?\n\n - If so, modify the file in the working tree the same way to bring\n rr-cache/$sig/preimage to rr-cache/$sig/postimage by 3-way merge.\n\n - if not, record the file in the working tree as rr-cache/$sig/preimage\n\nThe resolution recording codepath goes like:\n\n * see if any paths listed in MERGE_RR is resolved in the index;\n\n * look into rr-cache/$sig for such resolved path. Does it already record\n a resolution?\n\n - If not, we have a new resolution we can use. Record it as\n rr-cache/$sig/postimage for later use.\n\nSo rerere _does_ look at the index to decide what entries in rr-cache are\nrelevant and applicable. But other than that, it is not used. I do not\nthink there is no reason copy index to be able to reuse rr-cache.\n"},{"id":"80114","messageId":"20080617073714.GB5346@diana.vm.bytemark.co.uk","threadId":"13977","inReplyTo":"bd6139dc0806160918w1eea96f3r6f676d7cf186652d@mail.gmail.com","subject":"Re: git-rerere observations and feature suggestions","fromName":"Karl Hasselström","fromEmail":"kha@treskal.com","sentAt":"2008-06-17T07:37:14Z","receivedAt":"2008-06-17T07:37:14Z","isPatch":false,"sender":{"key":"kha@treskal.com","avatar":"https://gravatar.com/avatar/f0120c734b5279b345075a28521e1ac66acb20c9913ffe9bf6ae97e53f7f3f13?d=mp&s=160"},"body":"On 2008-06-16 18:18:40 +0200, Sverre Rabbelier wrote:\n\n> On Mon, Jun 16, 2008 at 5:57 PM, Pierre Habouzit wrote:\n>\n> > On Mon, Jun 16, 2008 at 03:48:51PM +0000, Pierre Habouzit wrote:\n> >\n> > > Actually it would be rather straightforward to put it in the\n> > > usual git store, and represent the current rr-cache with a flat\n> > > file that points to the in-git preimage/postimages, and make\n> > > git-gc aware of those.\n> >\n> > Actually, this is probably a required step in the direction of\n> > sharing such things btw.\n>\n> Perhaps an approach similar to the 'notes' implementation can be\n> used, in which a separate branch is created to contain the notes.\n> This way the rerere information (being the 'rerere' branch) can be\n> shared easily (by just pulling the branch), and as said we get free\n> compression. Another advantage would be that you automagically get\n> the ability to unlearn a bad rerere by simply (partially) reverting\n> a commit on the rerere branch!\n\nFWIW, StGit is well on its way to store its patch metadata in a git\nbranch, for much the same reasons.\n\n-- \nKarl Hasselström, kha@treskal.com\n www.treskal.com/kalle\n"},{"id":"80126","messageId":"alpine.DEB.1.00.0806171122130.6439@racer","threadId":"13977","inReplyTo":"20080616110113.GA22945@elte.hu","subject":"Re: git-rerere observations and feature suggestions","fromName":"Johannes Schindelin","fromEmail":"johannes.schindelin@gmx.de","sentAt":"2008-06-17T10:24:35Z","receivedAt":"2008-06-17T10:24:35Z","isPatch":false,"sender":{"key":"johannes.schindelin@gmx.de","avatar":"https://avatars.githubusercontent.com/u/127790?v=4"},"body":"Hi,\n\nOn Mon, 16 Jun 2008, Ingo Molnar wrote:\n>\n> - Sharing .git/rr-cache. It's quite a PITA to share the .git/rr-cache \n> amongst -tip maintainers right now. It seems to have dependencies on \n> the index file, so if we want to share the conflict resolution data, \n> we have to copy our index file (which is dangerous anyway and assumes \n> very similar repositories).\n\nI was dreaming about having \"git rerere infer-from \". This \nwould be\n\n- more versatile, as you do not have to ask the guy to share the cache,\n\n- would avoid transmitting lots of data that can be inferred from the \n data,\n\n- would avoid relying on the honesty of the person sharing the cache, and\n\n- it would put all license wieners^Wissues at rest.\n\nFWIW this is in my TODO list, but I am unlikely to get to it, least of all \nbefore 1.5.6 comes out.\n\nCiao,\nDscho\n"},{"id":"80213","messageId":"20080618105731.GA9242@elte.hu","threadId":"13977","inReplyTo":"20080616190911.GA7047@elte.hu","subject":"Re: git-rerere observations and feature suggestions","fromName":"Ingo Molnar","fromEmail":"mingo@elte.hu","sentAt":"2008-06-18T10:57:31Z","receivedAt":"2008-06-18T10:57:31Z","isPatch":false,"sender":{"key":"mingo@elte.hu","avatar":null},"body":"\n* Ingo Molnar wrote:\n\n> And while asking for an arm i'd also like to ask for a leg, if i may: \n> i'd love it if a \"slightly conflicting\" octopus merge of 85 topic \n> trees would not result in one huge conflict commit that merges \n> together 1000 commits into a single commit ;-)\n> \n> So right now in our -tip scripts work around this issue: we \n> 'serialize' the topic merges despite having very nice opportunities \n> for higher-order octopus merges. The integration would be a lot faster \n> if we could use octopus merges and automated git-rerere. (Octopus \n> merges would look much nicer as well in graphical representation as \n> well, which counts too :-) )\n\njust to demonstrate it, i tried today to do an octopus merge of 87 topic \nbranches:\n\ngit-merge build checkme core/checkme core/debugobjects core/futex-64bit \ncore/iter-div core/kill-the-BKL core/locking core/misc core/percpu \ncore/printk core/rcu core/rodata core/softirq core/softlockup \ncore/stacktrace core/topology core/urgent cpus4096 genirq kmemcheck \nkmemcheck2 mm/xen out-of-tree pci-for-jesse safe-poison-pointers sched \nsched-devel scratch stackprotector timers/clockevents timers/hpet \ntimers/hrtimers timers/nohz timers/posixtimers tip tracing/ftrace \ntracing/ftrace-mergefixups tracing/immediates tracing/markers \ntracing/mmiotrace tracing/mmiotrace-mergefixups tracing/nmisafe \ntracing/sched_markers tracing/stopmachine-allcpus tracing/sysprof \ntracing/textedit x86/apic x86/apm x86/bitops x86/build x86/checkme \nx86/cleanups x86/cpa x86/cpu x86/defconfig x86/delay x86/gart x86/i8259 \nx86/idle x86/intel x86/irq x86/irqstats x86/kconfig x86/ldt x86/mce \nx86/memtest x86/mmio x86/mpparse x86/nmi x86/numa x86/numa-fixes x86/pat \nx86/pebs x86/ptemask x86/resumetrace x86/scratch x86/setup x86/smpboot \nx86/threadinfo x86/timers x86/urgent x86/urgent-undo-ioapic x86/uv \nx86/vdso x86/xen x86/xsave\n\nit failed miserably:\n\n warning: ignoring 066519068ad2fbe98c7f45552b1f592903a9c8c8; cannot \n handle more than 25 refs\n [...]\n fatal: merge program failed\n Automated merge did not work.\n Should not be doing an Octopus.\n Merge with strategy octopus failed.\n\nthis wasnt even for purposes of an integration run: all i wanted to do \nwas to pick up 2-3 new commits i have queued into 2-3 topic branches, \ninto the (throw-away) integration branch. All the other branches were \nunmodified and already merged into the integration branch.\n\nHence i believe that the suggestions above by Git that i'm doing \nsomething wrong are ... wrong :-)\n\nMy scripting around this would be a lot faster (less than 10 seconds \nruntime versus a minute currently) and more robust if we could do such \nhigher-order octopus merges.\n\n\tIngo\n"},{"id":"80215","messageId":"20080618112931.GY29404@genesis.frugalware.org","threadId":"13977","inReplyTo":"20080618105731.GA9242@elte.hu","subject":"Re: git-rerere observations and feature suggestions","fromName":"Miklos Vajna","fromEmail":"vmiklos@frugalware.org","sentAt":"2008-06-18T11:29:31Z","receivedAt":"2008-06-18T11:29:31Z","isPatch":false,"sender":{"key":"vmiklos@frugalware.org","avatar":"https://gravatar.com/avatar/401c1cbbb3a5d13e650c691a2c71d6fd0b80df1a01bc74d9f1972675dd58f2bd?d=mp&s=160"},"body":"On Wed, Jun 18, 2008 at 12:57:31PM +0200, Ingo Molnar wrote:\n> just to demonstrate it, i tried today to do an octopus merge of 87 topic \n> branches:\n> \n> git-merge build checkme core/checkme core/debugobjects core/futex-64bit \n> core/iter-div core/kill-the-BKL core/locking core/misc core/percpu \n> core/printk core/rcu core/rodata core/softirq core/softlockup \n> core/stacktrace core/topology core/urgent cpus4096 genirq kmemcheck \n> kmemcheck2 mm/xen out-of-tree pci-for-jesse safe-poison-pointers sched \n> sched-devel scratch stackprotector timers/clockevents timers/hpet \n> timers/hrtimers timers/nohz timers/posixtimers tip tracing/ftrace \n> tracing/ftrace-mergefixups tracing/immediates tracing/markers \n> tracing/mmiotrace tracing/mmiotrace-mergefixups tracing/nmisafe \n> tracing/sched_markers tracing/stopmachine-allcpus tracing/sysprof \n> tracing/textedit x86/apic x86/apm x86/bitops x86/build x86/checkme \n> x86/cleanups x86/cpa x86/cpu x86/defconfig x86/delay x86/gart x86/i8259 \n> x86/idle x86/intel x86/irq x86/irqstats x86/kconfig x86/ldt x86/mce \n> x86/memtest x86/mmio x86/mpparse x86/nmi x86/numa x86/numa-fixes x86/pat \n> x86/pebs x86/ptemask x86/resumetrace x86/scratch x86/setup x86/smpboot \n> x86/threadinfo x86/timers x86/urgent x86/urgent-undo-ioapic x86/uv \n> x86/vdso x86/xen x86/xsave\n> \n> it failed miserably:\n> \n> warning: ignoring 066519068ad2fbe98c7f45552b1f592903a9c8c8; cannot \n> handle more than 25 refs\n\nThe upcoming builtin-merge won't have this problem. I have added a\ntestcase for this in my working branch:\n\nhttp://repo.or.cz/w/git/vmiklos.git?a=commit;h=7eef40b3cd772692c6eb7520686300533f35f10c\n"},{"id":"80217","messageId":"20080618113605.GA6590@elte.hu","threadId":"13977","inReplyTo":"20080618105731.GA9242@elte.hu","subject":"Re: git-rerere observations and feature suggestions","fromName":"Ingo Molnar","fromEmail":"mingo@elte.hu","sentAt":"2008-06-18T11:36:05Z","receivedAt":"2008-06-18T11:36:05Z","isPatch":false,"sender":{"key":"mingo@elte.hu","avatar":null},"body":"\n* Ingo Molnar wrote:\n\n> just to demonstrate it, i tried today to do an octopus merge of 87 \n> topic branches:\n> \n> git-merge build checkme core/checkme core/debugobjects core/futex-64bit \n> core/iter-div core/kill-the-BKL core/locking core/misc core/percpu \n> core/printk core/rcu core/rodata core/softirq core/softlockup \n> core/stacktrace core/topology core/urgent cpus4096 genirq kmemcheck \n> kmemcheck2 mm/xen out-of-tree pci-for-jesse safe-poison-pointers sched \n> sched-devel scratch stackprotector timers/clockevents timers/hpet \n> timers/hrtimers timers/nohz timers/posixtimers tip tracing/ftrace \n> tracing/ftrace-mergefixups tracing/immediates tracing/markers \n> tracing/mmiotrace tracing/mmiotrace-mergefixups tracing/nmisafe \n> tracing/sched_markers tracing/stopmachine-allcpus tracing/sysprof \n> tracing/textedit x86/apic x86/apm x86/bitops x86/build x86/checkme \n> x86/cleanups x86/cpa x86/cpu x86/defconfig x86/delay x86/gart x86/i8259 \n> x86/idle x86/intel x86/irq x86/irqstats x86/kconfig x86/ldt x86/mce \n> x86/memtest x86/mmio x86/mpparse x86/nmi x86/numa x86/numa-fixes x86/pat \n> x86/pebs x86/ptemask x86/resumetrace x86/scratch x86/setup x86/smpboot \n> x86/threadinfo x86/timers x86/urgent x86/urgent-undo-ioapic x86/uv \n> x86/vdso x86/xen x86/xsave\n> \n> it failed miserably:\n> \n> warning: ignoring 066519068ad2fbe98c7f45552b1f592903a9c8c8; cannot \n> handle more than 25 refs\n> [...]\n> fatal: merge program failed\n> Automated merge did not work.\n> Should not be doing an Octopus.\n> Merge with strategy octopus failed.\n> \n> this wasnt even for purposes of an integration run: all i wanted to do \n> was to pick up 2-3 new commits i have queued into 2-3 topic branches, \n> into the (throw-away) integration branch. All the other branches were \n> unmodified and already merged into the integration branch.\n> \n> Hence i believe that the suggestions above by Git that i'm doing \n> something wrong are ... wrong :-)\n> \n> My scripting around this would be a lot faster (less than 10 seconds \n> runtime versus a minute currently) and more robust if we could do such \n> higher-order octopus merges.\n\nsome hard numbers. Doing a scripted loop of 80 git-merges is 16.2 \nseconds:\n\n earth4:~/tip> time ( for N in $(cat 11 12 13 14); do git-merge $N; done )\n [...]\n Already up-to-date.\n\n real 0m16.211s\n user 0m10.719s\n sys 0m5.604s\n\ndoing the octopus merge of 4x 20 branch octopus merges is 11.6 seconds:\n\n earth4:~/tip> time ( for N in 1 2 3 4; do git-merge $(cat 1$N); done )\n Already up-to-date. Yeeah!\n Already up-to-date. Yeeah!\n Already up-to-date. Yeeah!\n Already up-to-date. Yeeah!\n\n real 0m11.580s\n user 0m8.617s\n sys 0m2.895s\n\na 40% speedup - and would be another 10% faster with an order-of-80 \nmerge as well i think. Not to be sniffed at.\n\n\tIngo\n"},{"id":"80236","messageId":"20080618184329.GB25707@elte.hu","threadId":"13977","inReplyTo":"20080618112931.GY29404@genesis.frugalware.org","subject":"Re: git-rerere observations and feature suggestions","fromName":"Ingo Molnar","fromEmail":"mingo@elte.hu","sentAt":"2008-06-18T18:43:29Z","receivedAt":"2008-06-18T18:43:29Z","isPatch":false,"sender":{"key":"mingo@elte.hu","avatar":null},"body":"\n* Miklos Vajna wrote:\n\n> On Wed, Jun 18, 2008 at 12:57:31PM +0200, Ingo Molnar wrote:\n> > just to demonstrate it, i tried today to do an octopus merge of 87 topic \n> > branches:\n> > \n> > git-merge build checkme core/checkme core/debugobjects core/futex-64bit \n> > core/iter-div core/kill-the-BKL core/locking core/misc core/percpu \n> > core/printk core/rcu core/rodata core/softirq core/softlockup \n> > core/stacktrace core/topology core/urgent cpus4096 genirq kmemcheck \n> > kmemcheck2 mm/xen out-of-tree pci-for-jesse safe-poison-pointers sched \n> > sched-devel scratch stackprotector timers/clockevents timers/hpet \n> > timers/hrtimers timers/nohz timers/posixtimers tip tracing/ftrace \n> > tracing/ftrace-mergefixups tracing/immediates tracing/markers \n> > tracing/mmiotrace tracing/mmiotrace-mergefixups tracing/nmisafe \n> > tracing/sched_markers tracing/stopmachine-allcpus tracing/sysprof \n> > tracing/textedit x86/apic x86/apm x86/bitops x86/build x86/checkme \n> > x86/cleanups x86/cpa x86/cpu x86/defconfig x86/delay x86/gart x86/i8259 \n> > x86/idle x86/intel x86/irq x86/irqstats x86/kconfig x86/ldt x86/mce \n> > x86/memtest x86/mmio x86/mpparse x86/nmi x86/numa x86/numa-fixes x86/pat \n> > x86/pebs x86/ptemask x86/resumetrace x86/scratch x86/setup x86/smpboot \n> > x86/threadinfo x86/timers x86/urgent x86/urgent-undo-ioapic x86/uv \n> > x86/vdso x86/xen x86/xsave\n> > \n> > it failed miserably:\n> > \n> > warning: ignoring 066519068ad2fbe98c7f45552b1f592903a9c8c8; cannot \n> > handle more than 25 refs\n> \n> The upcoming builtin-merge won't have this problem. I have added a \n> testcase for this in my working branch:\n> \n> http://repo.or.cz/w/git/vmiklos.git?a=commit;h=7eef40b3cd772692c6eb7520686300533f35f10c\n\ncool, thanks a ton!\n\nstupid question: does this mean that if i install the latest Git devel \nsnapshot (v1.5.6-rc3-21-g8c6b578 or later), i'll be able to experiment \naround with it right now?\n\n\tIngo\n"},{"id":"80246","messageId":"20080618195353.GG29404@genesis.frugalware.org","threadId":"13977","inReplyTo":"20080618184329.GB25707@elte.hu","subject":"Re: git-rerere observations and feature suggestions","fromName":"Miklos Vajna","fromEmail":"vmiklos@frugalware.org","sentAt":"2008-06-18T19:53:53Z","receivedAt":"2008-06-18T19:53:53Z","isPatch":false,"sender":{"key":"vmiklos@frugalware.org","avatar":"https://gravatar.com/avatar/401c1cbbb3a5d13e650c691a2c71d6fd0b80df1a01bc74d9f1972675dd58f2bd?d=mp&s=160"},"body":"On Wed, Jun 18, 2008 at 08:43:29PM +0200, Ingo Molnar wrote:\n> cool, thanks a ton!\n> \n> stupid question: does this mean that if i install the latest Git devel \n> snapshot (v1.5.6-rc3-21-g8c6b578 or later), i'll be able to experiment \n> around with it right now?\n\nNope. It is currently in the 'builtin-merge' branch of\ngit://repo.or.cz/git/vmiklos.git. And I'm working on to be merged after\n1.5.6 will be out.\n"},{"id":"80252","messageId":"m33anao11u.fsf@localhost.localdomain","threadId":"13977","inReplyTo":"20080618105731.GA9242@elte.hu","subject":"Re: git-rerere observations and feature suggestions","fromName":"Jakub Narebski","fromEmail":"jnareb@gmail.com","sentAt":"2008-06-18T22:01:24Z","receivedAt":"2008-06-18T22:01:24Z","isPatch":false,"sender":{"key":"jnareb@gmail.com","avatar":"https://avatars.githubusercontent.com/u/2706?v=4"},"body":"Ingo Molnar writes:\n\n> * Ingo Molnar wrote:\n> \n> > And while asking for an arm i'd also like to ask for a leg, if i may: \n> > i'd love it if a \"slightly conflicting\" octopus merge of 85 topic \n> > trees would not result in one huge conflict commit that merges \n> > together 1000 commits into a single commit ;-)\n> > \n> > So right now in our -tip scripts work around this issue: we \n> > 'serialize' the topic merges despite having very nice opportunities \n> > for higher-order octopus merges. The integration would be a lot faster \n> > if we could use octopus merges and automated git-rerere. (Octopus \n> > merges would look much nicer as well in graphical representation as \n> > well, which counts too :-) )\n> \n> just to demonstrate it, i tried today to do an octopus merge of 87 topic \n> branches:\n> \n> git-merge build checkme core/checkme core/debugobjects core/futex-64bit \n> core/iter-div core/kill-the-BKL core/locking core/misc core/percpu \n> core/printk core/rcu core/rodata core/softirq core/softlockup \n> core/stacktrace core/topology core/urgent cpus4096 genirq kmemcheck \n> kmemcheck2 mm/xen out-of-tree pci-for-jesse safe-poison-pointers sched \n> sched-devel scratch stackprotector timers/clockevents timers/hpet \n> timers/hrtimers timers/nohz timers/posixtimers tip tracing/ftrace \n> tracing/ftrace-mergefixups tracing/immediates tracing/markers \n> tracing/mmiotrace tracing/mmiotrace-mergefixups tracing/nmisafe \n> tracing/sched_markers tracing/stopmachine-allcpus tracing/sysprof \n> tracing/textedit x86/apic x86/apm x86/bitops x86/build x86/checkme \n> x86/cleanups x86/cpa x86/cpu x86/defconfig x86/delay x86/gart x86/i8259 \n> x86/idle x86/intel x86/irq x86/irqstats x86/kconfig x86/ldt x86/mce \n> x86/memtest x86/mmio x86/mpparse x86/nmi x86/numa x86/numa-fixes x86/pat \n> x86/pebs x86/ptemask x86/resumetrace x86/scratch x86/setup x86/smpboot \n> x86/threadinfo x86/timers x86/urgent x86/urgent-undo-ioapic x86/uv \n> x86/vdso x86/xen x86/xsave\n> \n> it failed miserably:\n> \n> warning: ignoring 066519068ad2fbe98c7f45552b1f592903a9c8c8; cannot \n> handle more than 25 refs\n> [...]\n> fatal: merge program failed\n> Automated merge did not work.\n> Should not be doing an Octopus.\n> Merge with strategy octopus failed.\n> \n> this wasnt even for purposes of an integration run: all i wanted to do \n> was to pick up 2-3 new commits i have queued into 2-3 topic branches, \n> into the (throw-away) integration branch. All the other branches were \n> unmodified and already merged into the integration branch.\n> \n> Hence i believe that the suggestions above by Git that i'm doing \n> something wrong are ... wrong :-)\n> \n> My scripting around this would be a lot faster (less than 10 seconds \n> runtime versus a minute currently) and more robust if we could do such \n> higher-order octopus merges.\n\nAs a part of patch series introducing new fast-forward strategies\n(--ff=never, --ff=only) there was patch which did merge reduction\nbefore selecting merge strategy, by Sverre Hvammen Johansen\n \"[PATCH 4/5] Head reduction before selecting merge strategy\"\n http://thread.gmane.org/gmane.comp.version-control.git/80288/focus=80335\n(I'm not sure if the link above is to nevest version of patch series).\n\nIt is now part of 'pu' branch, as commit 59171adb9c. It didn't make\ninto 'next' as it conflict with builtin merge by Miklos Vajna, which\n(as he wrote) also includes head reduction.\n\nSo you either would have to compile git from builtin-merge repository,\ncompile git from 'pu' or just use git-merge.sh from 'pu' branch, or\napply or cherry pick appropriate commit and compile git.\n\n-- \nJakub Narebski\nPoland\nShadeHawk on #git\n"},{"id":"80259","messageId":"20080618223821.GJ29404@genesis.frugalware.org","threadId":"13977","inReplyTo":"m33anao11u.fsf@localhost.localdomain","subject":"Re: git-rerere observations and feature suggestions","fromName":"Miklos Vajna","fromEmail":"vmiklos@frugalware.org","sentAt":"2008-06-18T22:38:21Z","receivedAt":"2008-06-18T22:38:21Z","isPatch":false,"sender":{"key":"vmiklos@frugalware.org","avatar":"https://gravatar.com/avatar/401c1cbbb3a5d13e650c691a2c71d6fd0b80df1a01bc74d9f1972675dd58f2bd?d=mp&s=160"},"body":"On Wed, Jun 18, 2008 at 03:01:24PM -0700, Jakub Narebski wrote:\n> As a part of patch series introducing new fast-forward strategies\n> (--ff=never, --ff=only) there was patch which did merge reduction\n> before selecting merge strategy, by Sverre Hvammen Johansen\n> \"[PATCH 4/5] Head reduction before selecting merge strategy\"\n> http://thread.gmane.org/gmane.comp.version-control.git/80288/focus=80335\n> (I'm not sure if the link above is to nevest version of patch series).\n\nSide note: builtin-merge does not have problem with merging 25+ refs\neven in case every ref contains \"new\" commits.\n\nThe patch by Sverre Hvammen Johansen is useful if some of the refs has\nno \"new\" commits, so it will help here, but I think it does not help in\nall cases.\n"},{"id":"80291","messageId":"20080619072308.GA12727@diana.vm.bytemark.co.uk","threadId":"13977","inReplyTo":"20080618223821.GJ29404@genesis.frugalware.org","subject":"Re: git-rerere observations and feature suggestions","fromName":"Karl Hasselström","fromEmail":"kha@treskal.com","sentAt":"2008-06-19T07:23:08Z","receivedAt":"2008-06-19T07:23:08Z","isPatch":false,"sender":{"key":"kha@treskal.com","avatar":"https://gravatar.com/avatar/f0120c734b5279b345075a28521e1ac66acb20c9913ffe9bf6ae97e53f7f3f13?d=mp&s=160"},"body":"On 2008-06-19 00:38:21 +0200, Miklos Vajna wrote:\n\n> On Wed, Jun 18, 2008 at 03:01:24PM -0700, Jakub Narebski\n> wrote:\n>\n> > As a part of patch series introducing new fast-forward strategies\n> > (--ff=never, --ff=only) there was patch which did merge reduction\n> > before selecting merge strategy, by Sverre Hvammen Johansen\n> > \"[PATCH 4/5] Head reduction before selecting merge strategy\"\n> > http://thread.gmane.org/gmane.comp.version-control.git/80288/focus=80335\n> > (I'm not sure if the link above is to nevest version of patch\n> > series).\n>\n> Side note: builtin-merge does not have problem with merging 25+ refs\n> even in case every ref contains \"new\" commits.\n\nSo how many parents can a commit have, exactly? Is there a hard limit\nsomewhere, or just a point beyond which some git tools will start\nbehaving strangely?\n\n-- \nKarl Hasselström, kha@treskal.com\n www.treskal.com/kalle\n"},{"id":"80293","messageId":"20080619072908.GM29404@genesis.frugalware.org","threadId":"13977","inReplyTo":"20080619072308.GA12727@diana.vm.bytemark.co.uk","subject":"Re: git-rerere observations and feature suggestions","fromName":"Miklos Vajna","fromEmail":"vmiklos@frugalware.org","sentAt":"2008-06-19T07:29:08Z","receivedAt":"2008-06-19T07:29:08Z","isPatch":false,"sender":{"key":"vmiklos@frugalware.org","avatar":"https://gravatar.com/avatar/401c1cbbb3a5d13e650c691a2c71d6fd0b80df1a01bc74d9f1972675dd58f2bd?d=mp&s=160"},"body":"On Thu, Jun 19, 2008 at 09:23:08AM +0200, Karl Hasselström wrote:\n> > Side note: builtin-merge does not have problem with merging 25+ refs\n> > even in case every ref contains \"new\" commits.\n> \n> So how many parents can a commit have, exactly? Is there a hard limit\n> somewhere, or just a point beyond which some git tools will start\n> behaving strangely?\n\nAFAIK there is no limit at a core level. git-show-branch has a limit of\n25 refs (it can't show more then 25 refs at one time) and git-merge.sh\nuses show-branch, while builtin-merge does not.\n"},{"id":"80294","messageId":"7v7iclx4nw.fsf@gitster.siamese.dyndns.org","threadId":"13977","inReplyTo":"20080619072308.GA12727@diana.vm.bytemark.co.uk","subject":"Re: git-rerere observations and feature suggestions","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2008-06-19T07:30:43Z","receivedAt":"2008-06-19T07:30:43Z","isPatch":false,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Karl Hasselström writes:\n\n> So how many parents can a commit have, exactly? Is there a hard limit\n> somewhere, or just a point beyond which some git tools will start\n> behaving strangely?\n\nThere is no hard limit at the data structure level.\n\ngit-commit-tree has a hard limit of accepting 16 parents. git-blame has\nthe same 16-parent limit while following the history (but the one in\n'next' has lifted the latter limitation).\n\nBut that is purely academic. Anybody who does an octopus with more than 8\nlegs should get his head examined ;-).\n"},{"id":"80298","messageId":"20080619082156.GB12727@diana.vm.bytemark.co.uk","threadId":"13977","inReplyTo":"7v7iclx4nw.fsf@gitster.siamese.dyndns.org","subject":"Re: git-rerere observations and feature suggestions","fromName":"Karl Hasselström","fromEmail":"kha@treskal.com","sentAt":"2008-06-19T08:21:56Z","receivedAt":"2008-06-19T08:21:56Z","isPatch":false,"sender":{"key":"kha@treskal.com","avatar":"https://gravatar.com/avatar/f0120c734b5279b345075a28521e1ac66acb20c9913ffe9bf6ae97e53f7f3f13?d=mp&s=160"},"body":"On 2008-06-19 00:30:43 -0700, Junio C Hamano wrote:\n\n> Karl Hasselström writes:\n>\n> > So how many parents can a commit have, exactly? Is there a hard\n> > limit somewhere, or just a point beyond which some git tools will\n> > start behaving strangely?\n>\n> There is no hard limit at the data structure level.\n>\n> git-commit-tree has a hard limit of accepting 16 parents. git-blame\n> has the same 16-parent limit while following the history (but the\n> one in 'next' has lifted the latter limitation).\n\nThanks.\n\n> But that is purely academic. Anybody who does an octopus with more\n> than 8 legs should get his head examined ;-).\n\nCatalin and I are tossing ideas around for how to represent the\nhistory of an StGit patch stack (using a git commit for each log\nentry). One complication is that we have to keep references to all\nunapplied patches so that gc will leave them alone (and so that they\nwill get carried along during a pull, in the future). And the number\nof unapplied patches is potentially large, so I thought we'd be going\nto have to make a tree of \"merge\" commits to connect them all up.\n\n(What we'd really like, of course, is a way to refer to a set of\ncommits such that they are guaranteed to be reachable (in the gc and\npull sense), but not considered \"parents\".)\n\n-- \nKarl Hasselström, kha@treskal.com\n www.treskal.com/kalle\n"},{"id":"80299","messageId":"20080619083356.GN29404@genesis.frugalware.org","threadId":"13977","inReplyTo":"20080619082156.GB12727@diana.vm.bytemark.co.uk","subject":"Re: git-rerere observations and feature suggestions","fromName":"Miklos Vajna","fromEmail":"vmiklos@frugalware.org","sentAt":"2008-06-19T08:33:56Z","receivedAt":"2008-06-19T08:33:56Z","isPatch":false,"sender":{"key":"vmiklos@frugalware.org","avatar":"https://gravatar.com/avatar/401c1cbbb3a5d13e650c691a2c71d6fd0b80df1a01bc74d9f1972675dd58f2bd?d=mp&s=160"},"body":"On Thu, Jun 19, 2008 at 10:21:56AM +0200, Karl Hasselström wrote:\n> Catalin and I are tossing ideas around for how to represent the\n> history of an StGit patch stack (using a git commit for each log\n> entry). One complication is that we have to keep references to all\n> unapplied patches so that gc will leave them alone (and so that they\n> will get carried along during a pull, in the future). And the number\n> of unapplied patches is potentially large, so I thought we'd be going\n> to have to make a tree of \"merge\" commits to connect them all up.\n> \n> (What we'd really like, of course, is a way to refer to a set of\n> commits such that they are guaranteed to be reachable (in the gc and\n> pull sense), but not considered \"parents\".)\n\nI had a similar problem in git/vmiklos.git on repo.or.cz, while working\non builtin-rebase: I squash several patches using rebase -i before\nsending a series, but it's nice to have the old long list of small\npatches in case I would need them later.\n\nWhat I did is to have a rebase-history branch: each commit in it is an\noctopus merge:\n\n- The first parent is the previous rebase-history ref\n\n- The second is the old HEAD\n\n- The third is the new HEAD\n\nThis way I can use git rebase -i without worrying about loosing history,\neven if reflogs are not shared among machines.\n\n(It may or may not be a good idea to do something like this in StGit, I\njust though I share this idea here.)\n"},{"id":"80303","messageId":"20080619091903.GA14415@diana.vm.bytemark.co.uk","threadId":"13977","inReplyTo":"20080619083356.GN29404@genesis.frugalware.org","subject":"Re: git-rerere observations and feature suggestions","fromName":"Karl Hasselström","fromEmail":"kha@treskal.com","sentAt":"2008-06-19T09:19:03Z","receivedAt":"2008-06-19T09:19:03Z","isPatch":false,"sender":{"key":"kha@treskal.com","avatar":"https://gravatar.com/avatar/f0120c734b5279b345075a28521e1ac66acb20c9913ffe9bf6ae97e53f7f3f13?d=mp&s=160"},"body":"On 2008-06-19 10:33:56 +0200, Miklos Vajna wrote:\n\n> On Thu, Jun 19, 2008 at 10:21:56AM +0200, Karl Hasselström\n> wrote:\n>\n> > Catalin and I are tossing ideas around for how to represent the\n> > history of an StGit patch stack (using a git commit for each log\n> > entry). One complication is that we have to keep references to all\n> > unapplied patches so that gc will leave them alone (and so that\n> > they will get carried along during a pull, in the future). And the\n> > number of unapplied patches is potentially large, so I thought\n> > we'd be going to have to make a tree of \"merge\" commits to connect\n> > them all up.\n> >\n> > (What we'd really like, of course, is a way to refer to a set of\n> > commits such that they are guaranteed to be reachable (in the gc\n> > and pull sense), but not considered \"parents\".)\n>\n> I had a similar problem in git/vmiklos.git on repo.or.cz, while\n> working on builtin-rebase: I squash several patches using rebase -i\n> before sending a series, but it's nice to have the old long list of\n> small patches in case I would need them later.\n>\n> What I did is to have a rebase-history branch: each commit in it is\n> an octopus merge:\n>\n> - The first parent is the previous rebase-history ref\n>\n> - The second is the old HEAD\n>\n> - The third is the new HEAD\n>\n> This way I can use git rebase -i without worrying about loosing\n> history, even if reflogs are not shared among machines.\n>\n> (It may or may not be a good idea to do something like this in\n> StGit, I just though I share this idea here.)\n\nWhat you're describing is pretty much what we're thinking about doing\n-- have a log branch where each commit contains enough metadata to\nrecreate the complete patch stack state at that point in time, and has\nall the parents it needs to be safe from gc.\n\nThe particular problem I'm asking about here is that due to StGit's\nconcept of \"unapplied\" patches that are per definition not reachable\nfrom the current branch head, a given log entry might have to keep an\nunbounded number of commits from being gc'ed. Thus my question about\nwhat would blow up if we were to make a commit with 50 parents. Or\n100. Or 1000, if our users are crazy enough. (The alternative being,\nof course, to make a tree of octopuses with a fixed maximum fan-out.)\n\n-- \nKarl Hasselström, kha@treskal.com\n www.treskal.com/kalle\n"},{"id":"80305","messageId":"20080619100637.GO29404@genesis.frugalware.org","threadId":"13977","inReplyTo":"20080619091903.GA14415@diana.vm.bytemark.co.uk","subject":"Re: git-rerere observations and feature suggestions","fromName":"Miklos Vajna","fromEmail":"vmiklos@frugalware.org","sentAt":"2008-06-19T10:06:37Z","receivedAt":"2008-06-19T10:06:37Z","isPatch":false,"sender":{"key":"vmiklos@frugalware.org","avatar":"https://gravatar.com/avatar/401c1cbbb3a5d13e650c691a2c71d6fd0b80df1a01bc74d9f1972675dd58f2bd?d=mp&s=160"},"body":"On Thu, Jun 19, 2008 at 11:19:03AM +0200, Karl Hasselström wrote:\n> What you're describing is pretty much what we're thinking about doing\n> -- have a log branch where each commit contains enough metadata to\n> recreate the complete patch stack state at that point in time, and has\n> all the parents it needs to be safe from gc.\n> \n> The particular problem I'm asking about here is that due to StGit's\n> concept of \"unapplied\" patches that are per definition not reachable\n> from the current branch head, a given log entry might have to keep an\n> unbounded number of commits from being gc'ed. Thus my question about\n> what would blow up if we were to make a commit with 50 parents. Or\n> 100. Or 1000, if our users are crazy enough. (The alternative being,\n> of course, to make a tree of octopuses with a fixed maximum fan-out.)\n\nI may miss something, but you have (at least) two options to store\n\"patches\".\n\nYou can store them as a blob, make a tree of them and make a commit in\nthe log branch point to the tree. This one has the advantage of being\nable to do a 'git log' on a particular patch of the patch set.\n\nThe other one is to create n+1 trees (and commits, where the first\ncommit has no parent) for n patches, and point to the last commit from\nthe log branch.\n"},{"id":"80308","messageId":"20080619103521.GC14415@diana.vm.bytemark.co.uk","threadId":"13977","inReplyTo":"20080619100637.GO29404@genesis.frugalware.org","subject":"Re: git-rerere observations and feature suggestions","fromName":"Karl Hasselström","fromEmail":"kha@treskal.com","sentAt":"2008-06-19T10:35:21Z","receivedAt":"2008-06-19T10:35:21Z","isPatch":false,"sender":{"key":"kha@treskal.com","avatar":"https://gravatar.com/avatar/f0120c734b5279b345075a28521e1ac66acb20c9913ffe9bf6ae97e53f7f3f13?d=mp&s=160"},"body":"On 2008-06-19 12:06:37 +0200, Miklos Vajna wrote:\n\n> You can store them as a blob, make a tree of them and make a commit\n> in the log branch point to the tree. This one has the advantage of\n> being able to do a 'git log' on a particular patch of the patch set.\n\nIf I don't store the pre or post tree in its entirety, I lose the\nability to do patch application by three-way merge. (The current StGit\ndesign assumes that we can always make a three-way merge as a last\nresort when applying patches. Basically, StGit is just a fancy way to\nrebase.)\n\nBut yes, this is a viable idea. (Though once I have to store one of\nthe trees, I believe it's actually simpler and cheaper to just store\nthe other tree as well, instead of having to compute the diff and\nstore that in a blob.)\n\n> The other one is to create n+1 trees (and commits, where the first\n> commit has no parent) for n patches, and point to the last commit\n> from the log branch.\n\nThere's actually no point in making more than one commit. A tree can\neasily hold a lot of sub-trees.\n\nI have an existing implementation that stores the pre and post tree\nfor each patch, plus some metadata (message, author). The issue with\nthis format is that every time we write a new log entry (that is, for\nevery StGit command), we have to call git multiple times in order to\nwrite several new trees and blobs.\n\nStGit normally represents each patch by a commit object, so it should\nbe faster to simply write a single new commit to the log that has some\nmetadata in its commit message and just refers to all the patches'\ncommit objects (by having them as parents). Which is why I was\ninquiring about the maximum number of parents of a commit object.\n\n( Some background: At a given point in time, your StGit stack consists\n of a few applied patches, and a few unapplied patches. The applied\n patches are just a linear sequence of commits at the top of your\n current branch, so we can trivially save them all from the garbage\n collector by making the stack top a parent of our log commit. The\n unapplied patches, however, are commits that are not reachable from\n the stack top -- they can be \"pushed\" onto the stack by rebasing, at\n which point they become applied, but until then we can't make any\n assumptions about them being ancestors of anything. So a log commit\n potentially has to have _every_ unapplied patch as a parent. (If we\n know that the commit of an unapplied patch used to be applied, we\n know that it's reachable from previous log commits, but we don't\n always know that.) )\n\n-- \nKarl Hasselström, kha@treskal.com\n www.treskal.com/kalle\n"},{"id":"80598","messageId":"7vod5tol6r.fsf@gitster.siamese.dyndns.org","threadId":"13977","inReplyTo":"7vskvd9kai.fsf@gitster.siamese.dyndns.org","subject":"[PATCH 1/5] rerere: rerere_created_at() and has_resolution() abstraction","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2008-06-22T09:47:40Z","receivedAt":"2008-06-22T09:47:40Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"There were too many places in the code how an entry in the rerere database\nlooks like, and the garbage_collect() function that iterates over\nsubdirectories of the rr-cache directory was the worse offender.\n\nIntroduce two helper functions, rerere_created_at() and has_resolution(),\nto abstract out the logic a bit better.\n\nIncidentally this fixes a small memory leak in garbage_collect()\nfunction. The path list to collect the entries to be pruned were defined\nto strdup the paths but the caller was feeding a path after doing an extra\ncopy. Because the list does not have to be sorted by conflict signature\nhash, we use path_list_append() instead of path_list_insert().\n\nWhile we are at it, make a conflicted hunk comparision in handle_file() a\nbit easier to read.\n\nSigned-off-by: Junio C Hamano \n---\n * So this is a series to handle the first point in the message I am\n replying to.\n\n builtin-rerere.c | 57 ++++++++++++++++++++++++-----------------------------\n 1 files changed, 26 insertions(+), 31 deletions(-)\n\ndiff --git a/builtin-rerere.c b/builtin-rerere.c\nindex 85222d9..610b96a 100644\n--- a/builtin-rerere.c\n+++ b/builtin-rerere.c\n@@ -23,6 +23,18 @@ static const char *rr_path(const char *name, const char *file)\n \treturn git_path(\"rr-cache/%s/%s\", name, file);\n }\n \n+static time_t rerere_created_at(const char *name)\n+{\n+\tstruct stat st;\n+\treturn stat(rr_path(name, \"preimage\"), &st) ? (time_t) 0 : st.st_mtime;\n+}\n+\n+static int has_resolution(const char *name)\n+{\n+\tstruct stat st;\n+\treturn !stat(rr_path(name, \"postimage\"), &st);\n+}\n+\n static void read_rr(struct path_list *rr)\n {\n \tunsigned char sha1[20];\n@@ -98,13 +110,10 @@ static int handle_file(const char *path,\n \t\telse if (!prefixcmp(buf, \"=======\"))\n \t\t\thunk = 2;\n \t\telse if (!prefixcmp(buf, \">>>>>>> \")) {\n-\t\t\tint cmp = strbuf_cmp(&one, &two);\n-\n+\t\t\tif (strbuf_cmp(&one, &two) > 0)\n+\t\t\t\tstrbuf_swap(&one, &two);\n \t\t\thunk_no++;\n \t\t\thunk = 0;\n-\t\t\tif (cmp > 0) {\n-\t\t\t\tstrbuf_swap(&one, &two);\n-\t\t\t}\n \t\t\tif (out) {\n \t\t\t\tfputs(\"<<<<<<<\\n\", out);\n \t\t\t\tfwrite(one.buf, one.len, 1, out);\n@@ -201,33 +210,24 @@ static void unlink_rr_item(const char *name)\n static void garbage_collect(struct path_list *rr)\n {\n \tstruct path_list to_remove = { NULL, 0, 0, 1 };\n-\tchar buf[1024];\n \tDIR *dir;\n \tstruct dirent *e;\n-\tint len, i, cutoff;\n+\tint i, cutoff;\n \ttime_t now = time(NULL), then;\n \n-\tstrlcpy(buf, git_path(\"rr-cache\"), sizeof(buf));\n-\tlen = strlen(buf);\n-\tdir = opendir(buf);\n-\tstrcpy(buf + len++, \"/\");\n+\tdir = opendir(git_path(\"rr-cache\"));\n \twhile ((e = readdir(dir))) {\n \t\tconst char *name = e->d_name;\n-\t\tstruct stat st;\n-\t\tif (name[0] == '.' && (name[1] == '\\0' ||\n-\t\t\t\t\t(name[1] == '.' && name[2] == '\\0')))\n+\t\tif (name[0] == '.' &&\n+\t\t (name[1] == '\\0' || (name[1] == '.' && name[2] == '\\0')))\n \t\t\tcontinue;\n-\t\ti = snprintf(buf + len, sizeof(buf) - len, \"%s\", name);\n-\t\tstrlcpy(buf + len + i, \"/preimage\", sizeof(buf) - len - i);\n-\t\tif (stat(buf, &st))\n+\t\tthen = rerere_created_at(name);\n+\t\tif (!then)\n \t\t\tcontinue;\n-\t\tthen = st.st_mtime;\n-\t\tstrlcpy(buf + len + i, \"/postimage\", sizeof(buf) - len - i);\n-\t\tcutoff = stat(buf, &st) ? cutoff_noresolve : cutoff_resolve;\n-\t\tif (then < now - cutoff * 86400) {\n-\t\t\tbuf[len + i] = '\\0';\n-\t\t\tpath_list_insert(xstrdup(name), &to_remove);\n-\t\t}\n+\t\tcutoff = (has_resolution(name)\n+\t\t\t ? cutoff_resolve : cutoff_noresolve);\n+\t\tif (then < now - cutoff * 86400)\n+\t\t\tpath_list_append(name, &to_remove);\n \t}\n \tfor (i = 0; i < to_remove.nr; i++)\n \t\tunlink_rr_item(to_remove.items[i].path);\n@@ -306,13 +306,11 @@ static int do_plain_rerere(struct path_list *rr, int fd)\n \t */\n \n \tfor (i = 0; i < rr->nr; i++) {\n-\t\tstruct stat st;\n \t\tint ret;\n \t\tconst char *path = rr->items[i].path;\n \t\tconst char *name = (const char *)rr->items[i].util;\n \n-\t\tif (!stat(rr_path(name, \"preimage\"), &st) &&\n-\t\t\t\t!stat(rr_path(name, \"postimage\"), &st)) {\n+\t\tif (has_resolution(name)) {\n \t\t\tif (!merge(name, path)) {\n \t\t\t\tfprintf(stderr, \"Resolved '%s' using \"\n \t\t\t\t\t\t\"previous resolution.\\n\", path);\n@@ -410,11 +408,8 @@ int cmd_rerere(int argc, const char **argv, const char *prefix)\n \t\treturn do_plain_rerere(&merge_rr, fd);\n \telse if (!strcmp(argv[1], \"clear\")) {\n \t\tfor (i = 0; i < merge_rr.nr; i++) {\n-\t\t\tstruct stat st;\n \t\t\tconst char *name = (const char *)merge_rr.items[i].util;\n-\t\t\tif (!stat(git_path(\"rr-cache/%s\", name), &st) &&\n-\t\t\t\t\tS_ISDIR(st.st_mode) &&\n-\t\t\t\t\tstat(rr_path(name, \"postimage\"), &st))\n+\t\t\tif (!has_resolution(name))\n \t\t\t\tunlink_rr_item(name);\n \t\t}\n \t\tunlink(merge_rr_path);\n-- \n1.5.6.12.g73f03\n"},{"id":"80597","messageId":"7viqw1ol6l.fsf@gitster.siamese.dyndns.org","threadId":"13977","inReplyTo":"7vskvd9kai.fsf@gitster.siamese.dyndns.org","subject":"[PATCH 2/5] git-rerere: detect unparsable conflicts","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2008-06-22T09:47:46Z","receivedAt":"2008-06-22T09:47:46Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"rerere did not detect the case where <<< === >>> markers did not match.\n\nSigned-off-by: Junio C Hamano \n---\n builtin-rerere.c | 5 +++++\n 1 files changed, 5 insertions(+), 0 deletions(-)\n\ndiff --git a/builtin-rerere.c b/builtin-rerere.c\nindex 610b96a..addc5c7 100644\n--- a/builtin-rerere.c\n+++ b/builtin-rerere.c\n@@ -144,6 +144,11 @@ static int handle_file(const char *path,\n \t\tfclose(out);\n \tif (sha1)\n \t\tSHA1_Final(sha1, &ctx);\n+\tif (hunk) {\n+\t\tif (output)\n+\t\t\tunlink(output);\n+\t\treturn error(\"Could not parse conflict hunks in %s\", path);\n+\t}\n \treturn hunk_no;\n }\n \n-- \n1.5.6.12.g73f03\n"},{"id":"80599","messageId":"7vd4m9ol6f.fsf@gitster.siamese.dyndns.org","threadId":"13977","inReplyTo":"7vskvd9kai.fsf@gitster.siamese.dyndns.org","subject":"[PATCH 3/5] rerere: remove dubious \"tail_optimization\"","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2008-06-22T09:47:52Z","receivedAt":"2008-06-22T09:47:52Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"It is dubious if it is cheaper to shift entries repeatedly using memmove()\nto collect entries that needs to be written out in front of an array than\nsimply marking the entries to be skipped. In addition, the label called this\n\"tail optimization\", but this obviously is not what people usually call\nwith that name.\n\nSigned-off-by: Junio C Hamano \n---\n builtin-rerere.c | 19 +++++++++----------\n 1 files changed, 9 insertions(+), 10 deletions(-)\n\ndiff --git a/builtin-rerere.c b/builtin-rerere.c\nindex addc5c7..0eec1f9 100644\n--- a/builtin-rerere.c\n+++ b/builtin-rerere.c\n@@ -66,8 +66,12 @@ static int write_rr(struct path_list *rr, int out_fd)\n {\n \tint i;\n \tfor (i = 0; i < rr->nr; i++) {\n-\t\tconst char *path = rr->items[i].path;\n-\t\tint length = strlen(path) + 1;\n+\t\tconst char *path;\n+\t\tint length;\n+\t\tif (!rr->items[i].util)\n+\t\t\tcontinue;\n+\t\tpath = rr->items[i].path;\n+\t\tlength = strlen(path) + 1;\n \t\tif (write_in_full(out_fd, rr->items[i].util, 40) != 40 ||\n \t\t write_in_full(out_fd, \"\\t\", 1) != 1 ||\n \t\t write_in_full(out_fd, path, length) != length)\n@@ -319,7 +323,7 @@ static int do_plain_rerere(struct path_list *rr, int fd)\n \t\t\tif (!merge(name, path)) {\n \t\t\t\tfprintf(stderr, \"Resolved '%s' using \"\n \t\t\t\t\t\t\"previous resolution.\\n\", path);\n-\t\t\t\tgoto tail_optimization;\n+\t\t\t\tgoto mark_resolved;\n \t\t\t}\n \t\t}\n \n@@ -330,13 +334,8 @@ static int do_plain_rerere(struct path_list *rr, int fd)\n \n \t\tfprintf(stderr, \"Recorded resolution for '%s'.\\n\", path);\n \t\tcopy_file(rr_path(name, \"postimage\"), path, 0666);\n-tail_optimization:\n-\t\tif (i < rr->nr - 1)\n-\t\t\tmemmove(rr->items + i,\n-\t\t\t\trr->items + i + 1,\n-\t\t\t\tsizeof(rr->items[0]) * (rr->nr - i - 1));\n-\t\trr->nr--;\n-\t\ti--;\n+\tmark_resolved:\n+\t\trr->items[i].util = NULL;\n \t}\n \n \treturn write_rr(rr, fd);\n-- \n1.5.6.12.g73f03\n"},{"id":"80600","messageId":"7v7ichol63.fsf@gitster.siamese.dyndns.org","threadId":"13977","inReplyTo":"7vskvd9kai.fsf@gitster.siamese.dyndns.org","subject":"[PATCH 4/5] t4200: fix rerere test","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2008-06-22T09:48:04Z","receivedAt":"2008-06-22T09:48:04Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"The test used \"diff-files -q\" which is not about reporting if there is\na difference at all. Instead, make sure that the path remains as\nconflicting in the index after rerere autoresolves it, as we will be\nadding rerere.autoupdate configuration with the next patch.\n\nSigned-off-by: Junio C Hamano \n---\n t/t4200-rerere.sh | 6 +++---\n 1 files changed, 3 insertions(+), 3 deletions(-)\n\ndiff --git a/t/t4200-rerere.sh b/t/t4200-rerere.sh\nindex 85d7e3e..afb3e3d 100755\n--- a/t/t4200-rerere.sh\n+++ b/t/t4200-rerere.sh\n@@ -193,9 +193,9 @@ test_expect_success 'resolution was recorded properly' '\n \techo Bello > file3 &&\n \tgit add file3 &&\n \tgit commit -m version2 &&\n-\t! git merge fifth &&\n-\tgit diff-files -q &&\n-\ttest Cello = \"$(cat file3)\"\n+\ttest_must_fail git merge fifth &&\n+\ttest Cello = \"$(cat file3)\" &&\n+\ttest 0 != $(git ls-files -u | wc -l)\n '\n \n test_done\n-- \n1.5.6.12.g73f03\n"},{"id":"80601","messageId":"7v1w2pol5x.fsf@gitster.siamese.dyndns.org","threadId":"13977","inReplyTo":"7vskvd9kai.fsf@gitster.siamese.dyndns.org","subject":"[PATCH 5/5] rerere.autoupdate","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2008-06-22T09:48:10Z","receivedAt":"2008-06-22T09:48:10Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"When this configuration is set, paths that are autoresolved by git-rerere\nare updated in the index as well.\n\nSigned-off-by: Junio C Hamano \n---\n Documentation/config.txt | 5 +++++\n builtin-rerere.c | 37 +++++++++++++++++++++++++++++++++++++\n t/t4200-rerere.sh | 10 ++++++++++\n 3 files changed, 52 insertions(+), 0 deletions(-)\n\ndiff --git a/Documentation/config.txt b/Documentation/config.txt\nindex 5331b45..0c7cf61 100644\n--- a/Documentation/config.txt\n+++ b/Documentation/config.txt\n@@ -650,6 +650,11 @@ gc.rerereunresolved::\n \tkept for this many days when `git rerere gc` is run.\n \tThe default is 15 days. See linkgit:git-rerere[1].\n \n+rerere.autoupdate::\n+\tWhen set to true, `git-rerere` updates the index with the\n+\tresulting contents after it cleanly resolves conflicts using\n+\tpreviously recorded resolution. Defaults to false.\n+\n rerere.enabled::\n \tActivate recording of resolved conflicts, so that identical\n \tconflict hunks can be resolved automatically, should they\ndiff --git a/builtin-rerere.c b/builtin-rerere.c\nindex 0eec1f9..839b26e 100644\n--- a/builtin-rerere.c\n+++ b/builtin-rerere.c\n@@ -16,6 +16,9 @@ static int cutoff_resolve = 60;\n /* if rerere_enabled == -1, fall back to detection of .git/rr-cache */\n static int rerere_enabled = -1;\n \n+/* automatically update cleanly resolved paths to the index */\n+static int rerere_autoupdate;\n+\n static char *merge_rr_path;\n \n static const char *rr_path(const char *name, const char *file)\n@@ -276,9 +279,36 @@ static int diff_two(const char *file1, const char *label1,\n \treturn 0;\n }\n \n+static struct lock_file index_lock;\n+\n+static int update_paths(struct path_list *update)\n+{\n+\tint i;\n+\tint fd = hold_locked_index(&index_lock, 0);\n+\tint status = 0;\n+\n+\tif (fd < 0)\n+\t\treturn -1;\n+\n+\tfor (i = 0; i < update->nr; i++) {\n+\t\tstruct path_list_item *item = &update->items[i];\n+\t\tif (add_file_to_cache(item->path, ADD_CACHE_IGNORE_ERRORS))\n+\t\t\tstatus = -1;\n+\t}\n+\n+\tif (!status && active_cache_changed) {\n+\t\tif (write_cache(fd, active_cache, active_nr) ||\n+\t\t commit_locked_index(&index_lock))\n+\t\t\tdie(\"Unable to write new index file\");\n+\t} else if (fd >= 0)\n+\t\trollback_lock_file(&index_lock);\n+\treturn status;\n+}\n+\n static int do_plain_rerere(struct path_list *rr, int fd)\n {\n \tstruct path_list conflict = { NULL, 0, 0, 1 };\n+\tstruct path_list update = { NULL, 0, 0, 1 };\n \tint i;\n \n \tfind_conflict(&conflict);\n@@ -323,6 +353,8 @@ static int do_plain_rerere(struct path_list *rr, int fd)\n \t\t\tif (!merge(name, path)) {\n \t\t\t\tfprintf(stderr, \"Resolved '%s' using \"\n \t\t\t\t\t\t\"previous resolution.\\n\", path);\n+\t\t\t\tif (rerere_autoupdate)\n+\t\t\t\t\tpath_list_insert(path, &update);\n \t\t\t\tgoto mark_resolved;\n \t\t\t}\n \t\t}\n@@ -338,6 +370,9 @@ static int do_plain_rerere(struct path_list *rr, int fd)\n \t\trr->items[i].util = NULL;\n \t}\n \n+\tif (update.nr)\n+\t\tupdate_paths(&update);\n+\n \treturn write_rr(rr, fd);\n }\n \n@@ -349,6 +384,8 @@ static int git_rerere_config(const char *var, const char *value, void *cb)\n \t\tcutoff_noresolve = git_config_int(var, value);\n \telse if (!strcmp(var, \"rerere.enabled\"))\n \t\trerere_enabled = git_config_bool(var, value);\n+\telse if (!strcmp(var, \"rerere.autoupdate\"))\n+\t\trerere_autoupdate = git_config_bool(var, value);\n \telse\n \t\treturn git_default_config(var, value, cb);\n \treturn 0;\ndiff --git a/t/t4200-rerere.sh b/t/t4200-rerere.sh\nindex afb3e3d..a64727d 100755\n--- a/t/t4200-rerere.sh\n+++ b/t/t4200-rerere.sh\n@@ -193,9 +193,19 @@ test_expect_success 'resolution was recorded properly' '\n \techo Bello > file3 &&\n \tgit add file3 &&\n \tgit commit -m version2 &&\n+\tgit tag version2 &&\n \ttest_must_fail git merge fifth &&\n \ttest Cello = \"$(cat file3)\" &&\n \ttest 0 != $(git ls-files -u | wc -l)\n '\n \n+test_expect_success 'rerere.autoupdate' '\n+\tgit config rerere.autoupdate true\n+\tgit reset --hard &&\n+\tgit checkout version2 &&\n+\ttest_must_fail git merge fifth &&\n+\ttest 0 = $(git ls-files -u | wc -l)\n+\n+'\n+\n test_done\n-- \n1.5.6.12.g73f03\n"},{"id":"80687","messageId":"20080623094906.GA8284@elte.hu","threadId":"13977","inReplyTo":"7vej6xb4lr.fsf@gitster.siamese.dyndns.org","subject":"Re: git-rerere observations and feature suggestions","fromName":"Ingo Molnar","fromEmail":"mingo@elte.hu","sentAt":"2008-06-23T09:49:06Z","receivedAt":"2008-06-23T09:49:06Z","isPatch":false,"sender":{"key":"mingo@elte.hu","avatar":null},"body":"\nanother git-rerere observation: occasionally it happens that i \naccidentally commit a merge marker into the source code.\n\nThat's obviously stupid, and it normally gets found by testing quickly, \nbut still it would be a really useful avoid-shoot-self-in-foot feature \nif git-commit could warn about such stupidities of mine.\n\n( and if i could configure git-commit to outright reject a commit like \n that - i never want to commit lines with <<<<<< or >>>>> markers)\n\nAnother merge conflict observation is that Git is much worse at figuring \nout the right merge resolution than our previous Quilt based workflow \nwas. I eventually found it to be mainly due to the following detail: \nsometimes it's more useful to first apply the merged branch and then \nattempt to merge HEAD, as a patch.\n\nI've got a script for that which also combines it with the \"rej\" tool, \nand in about 70%-80% of the cases where Git is unable to resolve a merge \nautomatically it figures things out. ('rej' is obviously a more relaxed \nmerge utility, but it's fairly robust in my experience, with a very low \nfalse positive rate.)\n\nThe ad-hoc \"tip-mergetool\" script we are using is attached below. It's \nreally just for demonstration purposes - it doesnt work when there's a \nrename related conflict, etc.\n\nPeter Zijstra also wrote a git-mergetool extension for the 'rej' tool \nbtw., he might want to post that patch. I've attached Chris Mason's rej \ntool too.\n\n\tIngo\n\n\n[ \"$#\" = 0 ] && {\n SRC=`git-ls-files -u | cut -f2 | head -1`\n} || {\n SRC=`git-ls-files -u | grep $1 | cut -f2 | head -1`\n}\n\n[ \"$SRC\" = \"\" -o ! -f \"$SRC\" ] && { echo \"$1 has no conflicts!\"; exit -1; }\n\nSRC_SED=`echo $SRC | sed 's/\\//\\\\\\\\\\//g'`\n\nSHA_1=`git-ls-files -u | grep $SRC | grep '^.* .* 1\\>' | cut -d' ' -f2`\nSHA_2=`git-ls-files -u | grep $SRC | grep '^.* .* 2\\>' | cut -d' ' -f2`\nSHA_3=`git-ls-files -u | grep $SRC | grep '^.* .* 3\\>' | cut -d' ' -f2`\n\nmv -b $SRC $SRC.automerge || { echo error1; exit -1; }\n\ngit-diff $SHA_1 $SHA_2 | sed \"s/$SHA_1/$SRC_SED/g\" |\n sed \"s/$SHA_2/$SRC_SED/g\" > $SRC.diff-v1 || { echo error2; exit -1; }\n\ngit-diff $SHA_1 $SHA_3 | sed \"s/$SHA_1/$SRC_SED/g\" |\n sed \"s/$SHA_3/$SRC_SED/g\" > $SRC.diff-v2 || { echo error2; exit -1; }\n\ngit-cat-file -p $SHA_1 > $SRC || { echo error4; exit -1; }\n\nls -l $SRC.automerge $SRC $SRC.diff-v1 $SRC.diff-v2\n\npatch -p1 < $SRC.diff-v2 || { echo error5; exit -1; }\npatch -p1 < $SRC.diff-v1 || {\n echo \"reject file ...\"\n ls -l $SRC.rej\n echo \"trying auto-merge ...\"\n OK=$(rej -dry-run $SRC.rej 2>&1 | grep ', 0 conflicts remain')\n [ \"$OK\" = \"\" ] && { echo \"$OK\"; exit -1; }\n rej -a $SRC.rej\n}\n\necho \"adding $SRC to the commit\"\ngit-add $SRC\necho 'if happy with the result, do: git-commit -m \"Manual merge of conflicts.\"'\n\necho \"merge successful!\"\nexit 0\n\n\n\n#!/usr/bin/perl\n# Reject merging program, released under GPLv2\n# Contact Chris Mason with bugs or patches\n#\n\nuse strict;\nuse Getopt::Long qw(:config no_ignore_case);\nuse File::Temp;\nuse IO::File;\nuse POSIX \":sys_wait_h\";\n\nmy @file; \t\t\t# array with the contents of the file\nmy @merged_file; \t\t# temporary copy of the file for merging\nmy $diff_mode = 0;\t\t# running a multi-file diff instead of a reject\nmy @hunks;\t\t\t# array of hashes for all the hunks\nmy $hunk_num = 0; \t\t# total number of hunks\nmy $rejfh;\t\t\t# handle for the reject file\nmy $filefh;\t\t\t# handle for the file\nmy $mergefh;\t\t\t# handle for the merge stream\nmy $VERSION = \"0.15\";\nmy $context = 0;\t\t# prefer context from the reject?\nmy $merge_prog = \"gvimdiff\"; # merge program to run\nmy $source_file;\t\t# name of the source file\nmy $reject_file;\t\t# name of the reject file\nmy $orig_reject_file;\t\t# original name of the reject file\nmy $output_file;\t\t# name of merge output (skips merge prog)\nmy $auto;\t\t\t# auto mode, put results into source file\nmy $interactive = 0;\t\t# go into command mode\nmy $editor = \"gvim\";\t\t# selected editor for opening the reject\nmy $open_reject = 0;\t\t# should the reject be opened?\nmy $report_only = 0;\t\t# only try to place the hunks\nmy $exit_value = 0;\t\t# our return value;\nmy $fully_matched_hunks = 0;\t# lines that match the -/+ lines in hunk\nmy $strip_level = 0;\t\t# how many path components to strip in diffmode\nmy $last_diff_line;\t\t# for buffering input lines\nmy $total_hunks;\t\t# total hunks found\nmy $reverse_patch = 0;\t\t# should we reverse the input patch?\nmy $no_forward_search = 0; # only try to match reverse hunks\nmy $quick = 0;\t\t\t# stop trying after the first conflict\nmy $skip_merge = 0;\t\t# don't run the merge prog at all\nmy $global_matched_hunks = 0; # total for whole diff\nmy $global_reject_hunks = 0;\t# \nmy $global_conflicts = 0;\t# number of conflicts for the last rej run\n\n\n# string equality ignoring whitespace\n# the first string has a leading control char from the reject,\n# either ' ', '-' or '+'\nsub fuzzy_eq($$) {\n my ($hunk, $file) = @_;\n my $line;\n\n $line = $hunk;\n $line =~ s/^.//;\n\n if ($line eq $file) {\n return 1;\n }\n # strip all whitespace and try agin\n $line =~ s/\\s//g;\n $file =~ s/\\s//g;\n if ($line eq $file) {\n return 1;\n }\n}\n\n# places the aligned hunk into the merge stream.\n# in context mode, or if this was a forward match\n# all context is taken from the reject file\n#\n# otherwise, this tries to take as much context as possible from the source\n# file.\nsub place_hunk($$$) {\n my ($hunk, $start, $end) = @_;\n my $fhunk = \\@{$hunk->{'fwhunk'}};\n my $rhunk = $hunk->{'aligned_hunk'};\n my $size = scalar(@file);\n my $i;\n my $k;\n my $merged_index = 0;\n my $hunk_index = 0;\n my $hunk_len = scalar(@$fhunk);\n my $line;\n my $file_hunk = $hunk->{'file_hunk'};\n my @tmp_hunk;\n my $file_index;\n my $c;\n my $fline;\n\n @merged_file = ();\n # check for a change that is already applied. print a warning as well\n # since this is not a 100% reliable check.\n #\n if ($hunk->{'method'} eq \"forward\" && \n $hunk->{'hunk_match'} == count_hunk_changed($hunk, \"forward\")) {\n\tprint STDERR \"WARNING: skipping hunk already applied: $hunk->{'desc'}\";\n return;\n }\n # try to maximize context from the reject file \n # if we matched the forward hunk, it means the file\n # already had some form of the change applied. Don't\n # try to be smart, just use the context from the reject\n if ($hunk->{'options'} =~ m/context/ || $context || \n $hunk->{'method'} eq \"forward\") {\n @merged_file = @file [ 0 .. $start - 1];\n\t\n\tforeach $line (@$fhunk) {\n\t $line =~ s/^.//;\n\t push @merged_file, $line;\n\t}\n\tpush @merged_file, @file[$end+1 .. scalar(@file)];\n\t@file = @merged_file;\n\treturn;\n }\n # align to the start of the reverse hunk\n while($hunk_index < scalar(@$rhunk)) {\n $line = $rhunk->[$hunk_index]; \n\t$fline = $file_hunk->[0];\n\t$fline =~ s/^.//;\n\tlast if (fuzzy_eq($line, $fline));\n\t$hunk_index++;\n }\n\n $file_index = 0;\n while($file_index < scalar(@$file_hunk)) {\n $line = $rhunk->[$hunk_index]; \n\t$fline = $file_hunk->[$file_index];\n\tif ($line =~ m/^\\|/) {\n\t $fline =~ s/^./\\+/;\n\t push @tmp_hunk, $fline;\n\t} elsif ($line =~ m/^ /) {\n\t push @tmp_hunk, $fline;\n\t} elsif ($line =~ m/^-/) {\n\t my $tline = $fline;\n\t $tline =~ s/^.//;\n\t if (!fuzzy_eq($line, $tline)) {\n\t\tmy $cline = $line;\n\t\t$cline =~ s/^.//;\n\t push @tmp_hunk, \"+<<<<<<< delete $cline\";\n\t }\n\t}\n\t$file_index++;\n\t$hunk_index++;\n\tif ($hunk_index > scalar(@$rhunk)) {\n\t print STDERR \"warning: hunk_index is $hunk_index, limit \" . \n\t scalar(@$rhunk) . \"\\n\";\n\t}\n }\n @$file_hunk = @tmp_hunk;\n @tmp_hunk = ();\n $size = scalar(@$file_hunk);\n $hunk_index = 0;\n $file_index = 0;\n while($hunk_index < scalar(@$fhunk)) {\n $line = $fhunk->[$hunk_index]; \n\t$fline = $file_hunk->[0];\n\tif (!($fline =~ m/^\\|/)) {\n\t $fline =~ s/^.//;\n\t last if (fuzzy_eq($line, $fline));\n\t}\n\tlast if ($line =~ m/^\\+/) ;\n\t$hunk_index++;\n }\n while($file_index < $size || $hunk_index < scalar(@$fhunk)) {\n\t$fline = $file_hunk->[$file_index];\n\tif ($fline =~ m/^\\+/ || $hunk_index >= scalar(@$fhunk)) {\n\t $fline =~ s/^.//;\n\t push @tmp_hunk, $fline;\n\t $file_index++;\n\t next;\n\t}\n $line = $fhunk->[$hunk_index]; \n\tif ($line =~ m/^\\+/) {\n\t $line =~ s/^.//;\n\t push @tmp_hunk, $line;\n\t} elsif ($file_index < scalar(@$file_hunk)) {\n\t if ($fline =~ m/^ /) {\n\t\t$fline =~ s/^.//;\n\t\tpush @tmp_hunk, $fline;\n\t }\n\t $file_index++;\n\t}\n\t$hunk_index++;\n }\n @merged_file = @file[ 0 .. $start -1];\n push @merged_file, @tmp_hunk;\n push @merged_file, @file [ $end+1 .. scalar(@file) ];\n @file = @merged_file;\n}\n\n# try to find the next line in the file or the hunk that\n# match. The entire hunk is searched for a line matching the\n# file, 20 lines forward are searched in the file.\n#\n# $aligned_hunk and $file_hunk are pointers to arrays, a line\n# containing \"|\\n\" is inserted to indicate missing lines in the\n# file or hunk:\n#\n# FILE\t\t\t\tHUNK:\n# A\t\t\t\t A\n# |\t\t\t\t B\n# C\t\t\t\t C\n# D\t\t\t\t|\n# There is a leading control char on each line in each of the aligned\n# arrays\n#\nsub align_match($$$$$$) {\n my ($hunk, $aligned_hunk, $file_hunk, $hindex, $findex, $hunk_changed) = @_;\n my $hunk_len = scalar(@$hunk);\n my $i;\n my $hline;\n my $fline;\n my $s;\n my $limit = 20;\n if ($$findex + $limit > scalar(@file)) {\n $limit = scalar(@file) - $$findex;\n }\n\n # look forward in the hunk for this line from the file\n $fline = $file[$$findex];\n for ($i = $$hindex; $i < $hunk_len; $i++) {\n if ($i != $$hindex && $hunk->[$i] =~ m/^\\S/) {\n\t $hunk_changed++;\n\t}\n\tif (fuzzy_eq($hunk->[$i], $fline)) {\n\t for ($s = 0 ; $s < $i - $$hindex ; $s++) {\n\t\tpush @$file_hunk, \"|\\n\";\n\t\tpush @$aligned_hunk, $hunk->[$$hindex + $s];\t\t\n\t }\t \n\t push @$file_hunk, \" $fline\";\n\t push @$aligned_hunk, $hunk->[$i];\n\t $$hindex = $i;\n\t return 1;\n\t}\n }\n # look forward in the file for this line from the hunk\n $hline = $hunk->[$$hindex];\n for ($i = $$findex; $i < $$findex + $limit; $i++) {\n\tif (fuzzy_eq($hline, $file[$i])) {\n\t for ($s = 0 ; $s < $i - $$findex ; $s++) {\n\t\tpush @$aligned_hunk, \"|\\n\";\n\t\tpush @$file_hunk, \" $file[$$findex + $s]\";\n\t }\n\t push @$file_hunk, \" $file[$i]\";\n\t push @$aligned_hunk, $hline;\n\t $$findex = $i;\n\t return 1;\n\t}\n }\n return 0;\n}\n\n# once a matching line is found between the hunk and the file,\n# test match walks through both trying to find out how many total\n# matching lines there are.\nsub test_match($$$$) {\n my ($hunk, $hunk_line, $file_line, $direction) = @_;\n my $line;\n my $i;\n my $file_len = scalar(@file);\n my $hunk_len = scalar(@$hunk);\n my @file_hunk = ();\n my @aligned_hunk = ();\n my $match_len = 0;\n my $last_match_line = -1;\n my $score = 0;\n my $consec = 0;\n my $ws_match = 0;\n my $hunk_match = 0;\n my $hunk_changed = 0;\n while($hunk_line < $hunk_len) {\n\t$line = $hunk->[$hunk_line];\n\tif ($line =~ m/^\\S/) {\n\t $hunk_changed++;\n\t}\n\tif (fuzzy_eq($line,$file[$file_line])) {\n\t # each line in the file hunk has one char for \n\t # special chars\n\t #\n\t push @file_hunk, \" $file[$file_line]\";\n\t push @aligned_hunk, $line;\n\t $match_len++;\n\t $last_match_line = $file_line;\n\t # bump the score if we're matching a non-context line\n\t # or if this is a consecutive match\n\t if ($line =~ m/^\\S/) {\n\t $hunk_match++;\n\t\t$score++;\n\t }\n\t if ($consec) {\n\t $score++;\n\t }\n\t $consec = 1;\n\t # count matches that are whitespace alone. These are\n\t # very unreliable.\n\t if ($file[$file_line] =~ m/^\\s*$/) {\n\t $ws_match++;\n\t }\n\t} else {\n\t # walk the hunk and the file trying to find the next match\n\t $consec = 0;\n\t if (align_match($hunk, \\@aligned_hunk, \\@file_hunk, \n\t \\$hunk_line, \\$file_line, \\$hunk_changed)) {\n\t\t$match_len++;\n\t\tif ($hunk->[$hunk_line] =~ m/^\\S/) {\n\t\t $score++;\n\t\t $hunk_match++;\n\t\t}\n\t } else {\n\t\tif ($file_line < scalar(@file)) {\n\t\t push @file_hunk, \" $file[$file_line]\";\n\t\t} else {\n\t\t push @file_hunk, \"|\\n\";\n\t\t}\n\t\tpush @aligned_hunk, $line;\n\t }\n\t $last_match_line = $file_line;\n\t}\n\t$file_line++;\n\t$hunk_line++;\n }\n \n if ($match_len == $ws_match) {\n $match_len = 0;\n\t$score = 0;\n\t$last_match_line = -1;\n }\n \n return ($match_len, $hunk_match, $score, $last_match_line, \n \\@aligned_hunk, \\@file_hunk)\n}\n\nsub count_hunk_changed($$) {\n my ($hunk, $method) = @_;\n my $fhunk;\n my $changed = 0;\n \n if ($method eq \"forward\") {\n\t$fhunk = \\@{$hunk->{'fwhunk'}};\n } else {\n\t$fhunk = \\@{$hunk->{'revhunk'}};\n }\n foreach my $l (@$fhunk) {\n\tif ($l =~ m/^\\S/) {\n\t $changed++;\n\t}\n }\n return $changed;\n}\n# start of the fuzzy matching engine, try to find matching lines\n# between the hunk and the file.\nsub _find_hunk($$$) {\n my ($struct, $hunk, $direction) = @_;\n my $hunk_len = scalar(@$hunk);\n my $file_len = scalar(@file);\n my $i;\n my $k;\n my $hunk_line;\n\n # for each line in the hunk, try to fuzz it into the file\n for ($i = 0 ; $i < $file_len; $i++) {\n for ($k = 0; $k < $hunk_len; $k++) {\n\t $hunk_line = $hunk->[$k];\n\t # if the hunk lines remaining are less then the best match\n\t # so far, we're done\n\t #\n\t if (scalar(@$hunk) - $k < $struct->{'match_count'}) {\n\t last;\n\t }\n\t # don't try too far into the hunk, after a while there's\n\t # no chance we'll find any useful match\n\t if ($k > 10) {\n\t last;\n\t }\n\t if (fuzzy_eq($hunk_line, $file[$i])) {\n\t\tmy ($match_len, $hunk_match, $score, $file_end, \n\t\t $aligned_hunk, $file_hunk) = test_match($hunk, $k, $i, \n\t\t \t\t\t\t\t $direction);\n\t\tif ($match_len > $struct->{'match_count'} || \n\t\t ($match_len == $struct->{'match_count'} && \n\t\t $score > $struct->{'score'})) {\n\t\t # don't pick the forward hunk over the reverse hunk\n\t\t # if the reverse hunk made any matches to non-context lines\n\t\t # and we don't have a perfect forward match\n\t\t if ($direction eq \"forward\" && $struct->{'method'} eq\n\t\t \"reverse\") {\n\t\t\tif ($hunk_match != \n\t\t\t count_hunk_changed($struct,\"forward\")) {\n\t\t\t my $rch = count_hunk_changed($struct, \"reverse\");\n\t\t\t if ($rch > 0 && $struct->{'hunk_match'} > 0) {\n\t\t\t\tnext;\n\t\t\t } elsif ($rch == 0 && $struct->{'match_count'} > 2){\n\t\t\t next;\n\t\t\t } elsif ($rch == $struct->{'hunk_match'}) {\n\t\t\t next;\n\t\t\t }\n\t\t\t}\n\n\t\t }\n\t\t $struct->{'match_count'} = $match_len;\n\t\t $struct->{'score'} = $score;\n\t\t $struct->{'start'} = $i;\n\t\t $struct->{'end'} = $file_end;\n\t\t $struct->{'method'} = $direction;\n\t\t $struct->{'aligned_hunk'} = $aligned_hunk;\n\t\t $struct->{'file_hunk'} = $file_hunk;\n\t\t $struct->{'hunk_match'} = $hunk_match;\n\t\t}\n\t\t# when deciding we've perfectly placed the change\n\t\t# allow for fuzz of two on either side, unless\n\t\t# there are no non-context lines in this part of the patch.\n\t\t# in that case, be fuzz free.\n\t\t# this code needs a little more work, disabled for now\n\t\t#my $fuzz = 4;\n\t\t#if ($struct->{'hunk_match'} == 0) {\n\t\t# $fuzz = 0;\n\t\t#}\n\t\t#if ($struct->{'match_count'} >= (scalar(@$hunk) - $fuzz) &&\n\t\t# $struct->{'hunk_match'} >= \n\t\t# count_hunk_changed($struct,$direction)) {\n\t\t# #return;\n\t\t#}\n\t\tlast;\n\t }\n\t}\n }\n}\n\n# figures out where the hunk should go into the file, and\n# inserts it into the merge stream. If no suitable location is found\n# the hunk is merged at the top.\n#\nsub find_hunk($) {\n my ($hunk) = @_;\n my $fhunk = \\@{$hunk->{'fwhunk'}};\n my $hc;\n my $ret = 0;\n if (!($hunk->{'options'} =~ m/forward/)) {\n _find_hunk($hunk, \\@{$hunk->{'revhunk'}}, \"reverse\");\n }\n if (!$no_forward_search && !($hunk->{'options'} =~ m/reverse/)) {\n _find_hunk($hunk, $fhunk, \"forward\");\n }\n $hc = count_hunk_changed($hunk, \"reverse\");\n if ($hunk->{'method'} eq \"reverse\" &&\n $hunk->{'hunk_match'} == count_hunk_changed($hunk, \"reverse\")) {\n\t# if hunk_mach is zero, then our patch is only adding new lines.\n\t# make sure we've found some reasonable context in the patch\n\t# before calling it a fully matched hunk\n\tif ($hunk->{'hunk_match'} > 0 || $hunk->{'match_count'} > 2) {\n $fully_matched_hunks++;\n\t $ret = 1;\n }\n }\n if ($hunk->{'match_count'} >= 2) {\n place_hunk($hunk, $hunk->{'start'}, $hunk->{'end'});\n } else {\n\t$hunk->{'method'} = \"forward\";\n place_hunk($hunk, 0, 0);\n }\n return $ret;\n}\n\nsub print_interactive_usage {\n print \"[c]ontext: toggle the context command line parameter off/on\\n\";\n print \"[d]one: exit interactive mode\\n\";\n print \"[h]elp: this screen\\n\";\n print \"[m]erge: run the merge program again\\n\";\n print \"[p]rocess: process the hunks again. This will write over the output file\\n\";\n print \"[r]eject: open in the reject in \\$REJEDITOR or \\$EDITOR\\n\";\n print \"\\t\\$REJEDITOR is used first if it exists\\n\";\n print \"[t]empreject: copy the reject to a temp file and open for editing\\n\";\n print \"\\tany later process commands will use the temp file\\n\";\n print \"restore: restore backup copy of source file in auto mode\\n\";\n print \"\\n\";\n}\nsub run_interactive($$$) {\n my ($pid, $file, $file2) = @_;\n my $editor_pid = 0;\n my $tfh;\n\n print \">> rej $VERSION interactive mode (type help for help)\\n\";\n print \">> \";\n while() {\n chomp;\n\tif (m/^(h|help)$/) {\n\t print_interactive_usage();\n\t} elsif (m/^restore$/) {\n\t if ($auto) {\n\t `cp $source_file.mergebackup $source_file`;\n\t\tif ($?) {\n\t\t print \"cp $source_file.mergebackup $source_file \\n\";\n\t\t print \"exited with \" . $? >> 8 . \"\\n\";\n\t\t}\n\t }\n\t} elsif (m/^(r|reject)$/) {\n\t open_reject();\n\t} elsif (m/^(t|tempreject)$/) {\n\t $rejfh = new IO::File;\n\t $rejfh->open(\"<$orig_reject_file\") || \n\t die \"Unable to open $orig_reject_file\";\n\t $tfh = new IO::File;\n\t $tfh->open(\">$orig_reject_file.tmp\") || \n\t die \"Unable to open $orig_reject_file.tmp\";\n\t while(<$rejfh>) {\n\t print $tfh $_; \n\t }\n\t close($rejfh);\n\t close($tfh);\n\t $reject_file = \"$orig_reject_file.tmp\";\n\t print \"Switching reject to $orig_reject_file.tmp\";\n\t open_reject();\n\t} elsif (m/^(c|context)$/) {\n\t $context = ($context + 1) % 2;\n\t print \"context mode toggled to $context\\n\";\n\t} elsif (m/^(p|process)$/) {\n\t print \"processing reject again\\n\";\n\t process_reject();\n\t} elsif (m/^(m|mergewindow)$/) {\n\t waitpid(-1, WNOHANG);\n\t $pid = fork();\n\t if (!$pid) {\n\t _run_merge($file, $file2);\n\t\texit 0;\n\t }\n\t} elsif (m/^(d|done|quit)$/) {\n\t last;\n\t}\n\tprint \">> \";\n }\n print \"waiting for merge windows\\n\";\n wait;\n if (!$auto) {\n unlink($file2);\n }\n}\n\nsub open_reject {\n if (defined($editor)) {\n\tprint \"opening reject $reject_file in $editor\\n\";\n\tsystem(\"$editor $reject_file\");\n } else {\n\tprint \"please define \\$EDITOR or \\$REJEDITOR in your env\\n\";\n }\n}\n\nsub _run_merge($$) {\n my ($file, $file2) = @_;\n my $ret;\n my $pid;\n\n if ($merge_prog eq \"kdiff3\") {\n\t$ret = system(\"kdiff3 -o $file $file $file2\");\n } elsif ($merge_prog eq \"tkdiff\") {\n $ret = system(\"tkdiff -o $file $file $file2\");\n } elsif ($merge_prog =~ m/vimdiff/) {\n $ret = system(\"$merge_prog -f $file $file2\");\n } else {\n $ret = system(\"$merge_prog $file $file2\");\n }\n if ($ret) {\n $ret = $ret >> 8;\n\tprint STDERR \"warning: $merge_prog exited with $ret\\n\";\n }\n}\n\n# run the merge program, with a little customization for each kind\n#\nsub run_merge($$) {\n my ($file, $file2) = @_;\n my $ret;\n my $pid;\n\n if ($interactive) {\n $pid = fork();\n\tif ($pid) {\n\t run_interactive($pid, $file, $file2);\n\t return;\n\t}\n }\n _run_merge($file, $file2);\n\n if (!$auto && !$interactive) {\n unlink $file2;\n }\n if ($interactive) {\n exit 0;\n }\n}\n\n# look forward in the hunk for three more consecutive context lines\n# this is used to split a large hunk into smaller ones\nsub three_more_context($$$$) {\n my ($rev, $fw, $rindex, $findex) = @_;\n my $revctx = 0;\n my $fwctx = 0;\n \n while($rindex < scalar(@$rev) && $findex < scalar(@$fw)) {\n if ($rev->[$rindex++] =~ m/^ /) {\n\t $revctx++;\n\t} else {\n\t $revctx = 0;\n\t}\n if ($fw->[$findex++] =~ m/^ /) {\n\t $fwctx++;\n\t} else {\n\t $fwctx = 0;\n\t}\n\tlast if($revctx > 3 && $fwctx > 3);\n }\n return ($revctx > 3 && $fwctx > 3);\n}\n\n# walk a hunk and divide it into smaller pieces. The smaller pieces should\n# be easier to place in the file.\n#\nsub split_and_push_hunk($$) {\n my ($hunks, $hunk) = @_;\n my $rev;\n my $fw;\n my $line;\n my $i;\n my @tmp;\n my $tmp_hash;\n my $fw_index;\n my $context;\n my $nonctx;\nagain:\n @tmp = ();\n $tmp_hash = {};\n $fw_index = 0;\n $context = 0;\n $i = 0;\n $rev = \\@{$hunk->{'revhunk'}};\n $fw = \\@{$hunk->{'fwhunk'}};\n $nonctx = 0;\n\n while($i < scalar(@$rev) && $fw_index < scalar(@$fw)) {\n\t$line = $rev->[$i];\n if ($line =~ m/^ / && $fw->[$fw_index] =~ m/^ /) {\n\t $context++;\n\t} else {\n\t $nonctx = 1;\n\t # walk both arrays forward until we get to the next next bit\n\t # of context in both\n\t while($fw_index < scalar(@$fw) && !($fw->[$fw_index] =~ m/^ /)) {\n\t $fw_index++;\n\t }\n\t while($i < scalar(@$rev) && !($rev->[$i] =~ m/^ /)) {\n\t $i++;\n\t }\n\t $context = 1;\n\t}\n\n\t# split the hunk if we've seen a non-context line,\n\t# we've seen three context lines already, and the hunk\n\t# still has three context lines in a row later on.\n\tif ($nonctx && $context >= 3 && \n\t three_more_context($rev, $fw, $i, $fw_index)) {\n\t my $t;\n\t # for both the rev and forward arrays, copy the \n\t # first part of the hunk to a tmp array and assign it\n\t # back into the old hunk. \n\t #\n\t # Then make a new hunk comprised\n\t # of the remaining parts of both arrays.\n\t # Make sure the context we've found is put into both\n\t # old and new hunks.\n\t for ($t = $i - $context + 1 ; $t < scalar(@$rev); $t++) {\n\t push @tmp, $rev->[$t];\n\t }\n\t $tmp_hash->{'revhunk'} = [@tmp];\n\t @tmp = ();\n\t for ($t = 0; $t <= $i; $t++) {\n\t push @tmp, $rev->[$t];\n\t }\n\t $hunk->{'revhunk'} = [@tmp];\n\t $tmp_hash->{'desc'} = $tmp[0];\n\n\t @tmp = ();\n\t for ($t = $fw_index - $context + 1 ; $t < scalar(@$fw); $t++) {\n\t push @tmp, $fw->[$t];\n\t }\n\t $tmp_hash->{'fwhunk'} = [@tmp];\n\t @tmp = ();\n\t for ($t = 0; $t <= $fw_index; $t++) {\n\t push @tmp, $fw->[$t];\n\t }\n\t $hunk->{'fwhunk'} = [@tmp];\n\t push @$hunks, $hunk;\n\t $hunk = {%$tmp_hash};\n\t goto again;\n\t}\n\t$fw_index++;\n\t$i++;\n }\n push @$hunks, $hunk;\n}\n\nsub print_usage {\n print STDERR \"usage: rej [-acdeiFMqrR] [-p num] [-o file] [-m prog] file file.rej\\n\";\n print STDERR \"\\t-a replace file with merged result. Use with care!\\n\";\n print STDERR \"\\t\\tbackup of original file created as file.mergebackup\\n\";\n print STDERR \"\\t-c maximize context from the reject. This makes it\\n\";\n print STDERR \"\\t\\teasier to figure out stubborn rejets\\n\";\n print STDERR \"\\t-dry-run check for conflicts, but do nothing\\n\";\n print STDERR \"\\t-i interactive mode\\n\";\n print STDERR \"\\t-F don't check for already applied changes\\n\";\n print STDERR \"\\t-M don't run the external merge program at all\\n\";\n print STDERR \"\\t-p num: strip num levels off the paths found in the diff\\n\";\n print STDERR \"\\t-q in dry-run mode, exit once a conflict is found\\n\";\n print STDERR \"\\t\\totherwise, only run merge prog when there are conflicts\\n\";\n print STDERR \"\\t-r open the reject file in \\$EDITOR\\n\";\n print STDERR \"\\t-R reverse the diff or reject\\n\";\n print STDERR \"\\t-o file specify output file for merge\\n\";\n print STDERR \"\\t\\tThe merge program will not be run in this case\\n\";\n print STDERR \"\\t-m prog specify merge progam. You can use:\\n\";\n print STDERR \"\\t\\t[g]vimdiff (default), kdiff3, tkdiff and meld\\n\";\n print STDERR \"\\t\\tothers called as: mergeprog foo.c foo.c.tmp\\n\";\n print STDERR \"\\t\\tThe REJMERGE environment var specifies the merge program as well\\n\";\n print STDERR \"\\t-r open the reject with \\$REJEDITOR or \\$EDITOR\\n\";\n print STDERR \"\\n\";\n exit 1;\n}\n\nsub strip_path($) {\n my ($p) = @_;\n\n #if ($p =~ m/(.*\\/){$strip_level}?(.*)/) {\n if ($p =~ m/([^\\/]+\\/){$strip_level}?(.*)/) {\n $p = $2;\n\treturn $p;\n }\n return undef;\n}\n\nsub reverse_line($) {\n my ($line) = @_;\n my $orig_line = $line;\n if (!$reverse_patch) {\n return $line;\n }\n if ($line =~ s/^-([^-].*)/\\+$1/) {\n return $line; \n } elsif ($line =~ s/^\\+([^\\+].*)/-$1/) {\n return $line;\n }\n return $line;\n}\n\nsub process_reject {\n if (!defined($rejfh) || !$diff_mode) {\n\t$rejfh = new IO::File;\n\t$rejfh->open(\"<$reject_file\") || die \"Unable to open $reject_file\";\n }\n # in diff mode, find the first file indicator\n if ($diff_mode) {\n\tif ($rejfh->eof()) {\n\t return 1;\n\t}\n while(defined($last_diff_line) || !$rejfh->eof()) {\n\t if (defined($last_diff_line)) {\n\t $_ = $last_diff_line;\n\t\tundef $last_diff_line;\n\t } else {\n\t $_ = <$rejfh>;\n\t }\n\t if (m/^--- (\\S*)/) {\n\t my $fname = $1;\n\t\t$fname = strip_path($fname);\n\t\tif ( -f $fname) {\n\t\t $source_file = $fname;\n\t\t last;\n\t\t} else {\n\t\t my $t = <$rejfh>;\n\t\t if ($t =~ m/^\\+\\+\\+ (\\S*)/) {\n\t\t $fname = strip_path($1);\n\t\t\tif (-f $fname) {\n\t\t\t $source_file = $fname;\n\t\t\t last;\n\t\t\t}\n\t\t }\n\t\t}\n\t }\n\t}\n }\n\n $filefh = new IO::File;\n $filefh->open(\"<$source_file\") || die \"Unable to open $source_file\";\n\n # struct hunk {\n # @fwhunk;\n # @revhunk;\n # $hunk_offset; offset in hunk where matching started\n # $start; \n # $end;\n # $score;\n # $match_count;\n # $desc; @@ line\n #\t $options; # string with the special options for this hunk\n # $aligned_hunk; points to array of @revhunk aligned with file\n # $file_hunk; points to array of file lines aligned with aligned_hunk\n # $method; \"forward\" or \"reverse\" defines how the hunk was matched\n # }\n\n #build the arrays of hunks\n my $hunk;\n my @fw;\n my @rev;\n my $more;\n my $last = 0;\n my $exclude = 0;\n my $hunk_opt = \"\";\n @hunks = ();\n $fully_matched_hunks = 0;\n $total_hunks = 0;\n $hunk_num = 0;\n while(<$rejfh>) {\n\tif ($diff_mode && m/^--- /) {\n\t $last_diff_line = $_;\n\t last;\n\t}\n\t$_ = reverse_line($_);\n\tif (m/^(@@|\\*\\*\\* )/) {\nagain:\n\t last if ($last);\n\t # special options string\n\t $hunk_opt = \"\";\n\t if (m/###(.*)/) {\n\t $hunk_opt = $1;\n\t\tif ($hunk_opt =~ m/only/) {\n\t\t @hunks = ();\n\t\t $hunk_num = 0;\n\t\t $last = 1;\n\t\t} \n\t\tif ($hunk_opt =~ m/exclude/) {\n\t\t while(<$rejfh>) {\n\t\t\tif ($diff_mode && m/^--- /) {\n\t\t\t $last_diff_line = $_;\n\t\t\t goto read_file;\n\t\t\t}\n\t\t\tif (m/^(@@|\\*\\*\\* )/) {\n\t\t\t goto again;\n\t\t\t}\n\t\t }\n\t\t} \n\t\tif ($hunk_opt =~ m/last/) {\n\t\t $last = 1;\n\t\t}\n\t }\n\t if ($hunk_num > 0) {\n\t\t$hunk->{'fwhunk'} = [@fw];\n\t\t$hunk->{'revhunk'} = [@rev];\n\t\tsplit_and_push_hunk(\\@hunks, $hunk);\n\t }\n\t @fw = ();\n\t @rev = ();\n\t $hunk = {};\n\t $hunk_num++;\n\t $hunk->{'desc'} = $_;\n\t $hunk->{'options'} = $hunk_opt;\n\t # not a unified diff?\n\t # process the whole thing right here\n\t if (!m/^@@/) {\n\t\tif ($diff_mode) {\n\t\t die \"Unable to handle multi file context diffs\";\n\t\t}\n\t\t$more = 0;\n\t\twhile(<$rejfh>) {\n\t\t $_ = reverse_line($_);\n\t\t last if (m/^--- \\d+,\\d+ ---/);\n\t\t s/(^.)./$1/;\n\t\t if ($reverse_patch) {\n\t\t\tpush @fw, $_;\n\t\t } else {\n\t\t\tpush @rev, $_;\n\t\t }\n\t\t}\n\t\twhile(<$rejfh>) {\n\t\t $_ = reverse_line($_);\n\t\t if (m/^\\*\\*\\* /) {\n\t\t\t$more = 1;\n\t\t\tlast;\n\t\t }\n\t\t if (!(m/^\\*/)) {\n\t\t\ts/(^.)./$1/;\n\t\t\tif ($reverse_patch) {\n\t\t\t push @rev, $_;\n\t\t\t} else {\n\t\t\t push @fw, $_;\n\t\t\t}\n\t\t }\n\t\t}\n\t\tif ($more) {\n\t\t goto again;\n\t\t}\n\t }\n\t} elsif (m/^-[^-]/) {\n\t push @rev, $_;\n\t} elsif (m/^\\+[^\\+]/) {\n\t push @fw, $_;\n\t} elsif (m/^ /) {\n\t push @fw, $_;\n\t push @rev, $_;\n\t}\n }\nread_file:\n if (!$diff_mode) {\n\t$rejfh->close();\n }\n\n # push any leftover hunks from the loop above\n if (defined($hunk->{'desc'})) {\n\t$hunk->{'fwhunk'} = [@fw];\n\t$hunk->{'revhunk'} = [@rev];\n\tsplit_and_push_hunk(\\@hunks, $hunk);\n }\n\n @file = ();\n # build the file array\n while(<$filefh>) {\n\tpush @file, $_;\n }\n $filefh->close();\n\n $total_hunks += scalar(@hunks);\n # try to place each hunk into the file\n my $ret;\n foreach my $href (@hunks) {\n\t$ret = find_hunk($href);\n\tif ($report_only && $quick && $ret == 0) {\n\t last;\n\t}\n }\n my $conflicts = $total_hunks - $fully_matched_hunks;\n $global_matched_hunks += $fully_matched_hunks;\n $global_reject_hunks += $total_hunks;\n $global_conflicts = $conflicts;\n if ($conflicts > 0) {\n $exit_value = 1;\n }\n if ($report_only && $quick && $conflicts > 0) {\n $conflicts = \"some\";\n }\n print STDERR \"\\t$source_file: $fully_matched_hunks matched, $conflicts conflicts remain\\n\";\n if ($report_only) {\n\tif ($quick && $exit_value) {\n\t return 1;\n\t}\n return 0;\n }\n\n if (!defined($mergefh)) {\n\t# from here down either copies the merge result to $output_file or\n\t# runs the merge program\n\tif (defined($auto)) {\n\t my $ret;\n\t $ret = rename $source_file, \"$source_file.mergebackup\";\n\t if (!$ret) {\n\t\tdie \"Unable to rename $source_file to $source_file.mergebackup\";\n\t }\n\t $output_file = $source_file;\n\t}\n\tif (defined($output_file)) {\n\t $mergefh = new IO::File;\n\t $mergefh->open(\">$output_file\") || \n\t die \"Unable to open $output_file\";\n\t} else {\n\t $mergefh = new File::Temp(TEMPLATE => \"$source_file.XXXXX\", \n\t\t\t\t UNLINK => 0) ||\n\t\t\t\t die \"Unable to create temp file\";\n\t}\n } else {\n\t# mergefh is only defined when we're reloading. \n\t# Just truncate and seek to 0\n\tif ($output_file) {\n\t $mergefh->open(\">$output_file\")||die \"Unable to open $output_file\";\n\t} else {\n\t $mergefh->truncate(0); \n\t seek $mergefh, 0, SEEK_SET;\n\t}\n }\n foreach my $l (@file) {\n\tprint $mergefh $l;\n }\n $mergefh->flush();\n if ($output_file) {\n $mergefh->close();\n }\n return 0;\n}\n\nif (defined($ENV{'REJMERGE'})) {\n $merge_prog = $ENV{'REJMERGE'};\n}\nif (defined($ENV{'REJEDITOR'})) {\n $editor = $ENV{'REJEDITOR'};\n} elsif (defined($ENV{'EDITOR'})) {\n $editor = $ENV{'EDITOR'};\n}\n\nGetOptions(\"context\" => \\$context,\n\t \"auto\" => \\$auto,\n\t \"dry-run\" => \\$report_only,\n\t \"out=s\" => \\$output_file,\n\t \"F|no-forward\" => \\$no_forward_search,\n\t \"interactive\" => \\$interactive,\n\t \"reject\" => \\$open_reject,\n\t \"Reverse\" => \\$reverse_patch,\n\t \"p|strip-level=s\" => \\$strip_level,\n\t \"quick\" => \\$quick,\n\t \"M|no-merge\" => \\$skip_merge,\n \"merge=s\" => \\$merge_prog) || print_usage();;\n\n$source_file = $ARGV[0];\n$reject_file = $ARGV[1];\nif (scalar(@ARGV) < 2) {\n if ($ARGV[0] =~ m/\\.rej$/) {\n $reject_file = $ARGV[0];\n\t$source_file = $reject_file;\n\t$source_file =~ s/\\.rej$//;\n } elsif (-f $source_file && -f \"$source_file.rej\") {\n $reject_file = \"$source_file.rej\";\n } elsif (-f $source_file) {\n\t$reject_file = $source_file;\n\tundef($source_file);\n $diff_mode = 1;\n } else {\n print_usage();\n }\n}\n\n$orig_reject_file = $reject_file;\n\nif (!$diff_mode) {\n foreach my $f ($source_file, $reject_file) {\n\tif (! -f $f) {\n\t print STDERR \"Unable to find $f\\n\";\n\t exit 1;\n\t}\n }\n}\n\nwhile(1) {\n if (process_reject()) {\n if ($diff_mode) {\n\t print STDERR \"$reject_file: total of $global_matched_hunks / $global_reject_hunks matched\\n\";\n\t}\n last;\n }\n if (!$report_only) {\n\tif ($open_reject) {\n\t open_reject();\n\t}\n\tif (!$skip_merge && (!$quick || $global_conflicts > 0)) {\n\t if (!defined($output_file)) {\n\t\trun_merge($source_file, $mergefh);\n\t } elsif ($auto) {\n\t\trun_merge($source_file, \"$source_file.mergebackup\");\n\t }\n\t}\n }\n if ($diff_mode) {\n undef($mergefh);\n } else {\n last;\n }\n}\n\nexit $exit_value;\n"},{"id":"80704","messageId":"1214230796.3223.326.camel@lappy.programming.kicks-ass.net","threadId":"13977","inReplyTo":"20080623094906.GA8284@elte.hu","subject":"Re: git-rerere observations and feature suggestions","fromName":"Peter Zijlstra","fromEmail":"a.p.zijlstra@chello.nl","sentAt":"2008-06-23T14:19:55Z","receivedAt":"2008-06-23T14:19:55Z","isPatch":false,"sender":{"key":"a.p.zijlstra@chello.nl","avatar":null},"body":"On Mon, 2008-06-23 at 11:49 +0200, Ingo Molnar wrote:\n> another git-rerere observation: occasionally it happens that i \n> accidentally commit a merge marker into the source code.\n> \n> That's obviously stupid, and it normally gets found by testing quickly, \n> but still it would be a really useful avoid-shoot-self-in-foot feature \n> if git-commit could warn about such stupidities of mine.\n> \n> ( and if i could configure git-commit to outright reject a commit like \n> that - i never want to commit lines with <<<<<< or >>>>> markers)\n> \n> Another merge conflict observation is that Git is much worse at figuring \n> out the right merge resolution than our previous Quilt based workflow \n> was. I eventually found it to be mainly due to the following detail: \n> sometimes it's more useful to first apply the merged branch and then \n> attempt to merge HEAD, as a patch.\n> \n> I've got a script for that which also combines it with the \"rej\" tool, \n> and in about 70%-80% of the cases where Git is unable to resolve a merge \n> automatically it figures things out. ('rej' is obviously a more relaxed \n> merge utility, but it's fairly robust in my experience, with a very low \n> false positive rate.)\n> \n> The ad-hoc \"tip-mergetool\" script we are using is attached below. It's \n> really just for demonstration purposes - it doesnt work when there's a \n> rename related conflict, etc.\n> \n> Peter Zijstra also wrote a git-mergetool extension for the 'rej' tool \n> btw., he might want to post that patch. I've attached Chris Mason's rej \n> tool too.\n\nThis is what I run with.\n\nI added the cp to the 3-way merge tools because I think its stupid to\nsee the messed up merge markers instead of the original file.\n\nThe rej target basically takes the local version and takes the diff\nbetween base and remote and applies that as a patch, upon failure it\ninvokes rej to fix up the mess.\n\n--- /usr/bin/git-mergetool\t2008-04-08 19:01:37.000000000 +0200\n+++ git-mergetool\t2008-06-02 19:00:55.000000000 +0200\n@@ -214,12 +214,14 @@ merge_file () {\n \t ;;\n \tmeld|vimdiff)\n \t touch \"$BACKUP\"\n+\t cp -- \"$BASE\" \"$path\"\n \t \"$merge_tool_path\" -- \"$LOCAL\" \"$path\" \"$REMOTE\"\n \t check_unchanged\n \t save_backup\n \t ;;\n \tgvimdiff)\n \t\ttouch \"$BACKUP\"\n+\t\tcp -- \"$BASE\" \"$path\"\n \t\t\"$merge_tool_path\" -f -- \"$LOCAL\" \"$path\" \"$REMOTE\"\n \t\tcheck_unchanged\n \t\tsave_backup\n@@ -271,6 +273,13 @@ merge_file () {\n \t status=$?\n \t save_backup\n \t ;;\n+ rej)\n+\t touch \"$BACKUP\"\n+\t cp -- \"$LOCAL\" \"$path\"\n+\t diff -up \"$BASE\" \"$REMOTE\" | patch \"$path\" || rej \"$path\"\n+\t check_unchanged\n+\t save_backup\n+\t ;;\n esac\n if test \"$status\" -ne 0; then\n \techo \"merge of $path failed\" 1>&2\n@@ -311,7 +320,7 @@ done\n \n valid_tool() {\n \tcase \"$1\" in\n-\t\tkdiff3 | tkdiff | xxdiff | meld | opendiff | emerge | vimdiff | gvimdiff | ecmerge)\n+\t\tkdiff3 | tkdiff | xxdiff | meld | opendiff | emerge | vimdiff | gvimdiff | ecmerge | rej)\n \t\t\t;; # happy\n \t\t*)\n \t\t\treturn 1\n"},{"id":"80705","messageId":"1214231197.3223.332.camel@lappy.programming.kicks-ass.net","threadId":"13977","inReplyTo":"1214230796.3223.326.camel@lappy.programming.kicks-ass.net","subject":"Re: git-rerere observations and feature suggestions","fromName":"Peter Zijlstra","fromEmail":"peterz@infradead.org","sentAt":"2008-06-23T14:26:36Z","receivedAt":"2008-06-23T14:26:36Z","isPatch":false,"sender":{"key":"peterz@infradead.org","avatar":null},"body":"On Mon, 2008-06-23 at 16:20 +0200, Peter Zijlstra wrote:\n> On Mon, 2008-06-23 at 11:49 +0200, Ingo Molnar wrote:\n> > another git-rerere observation: occasionally it happens that i \n> > accidentally commit a merge marker into the source code.\n> > \n> > That's obviously stupid, and it normally gets found by testing quickly, \n> > but still it would be a really useful avoid-shoot-self-in-foot feature \n> > if git-commit could warn about such stupidities of mine.\n> > \n> > ( and if i could configure git-commit to outright reject a commit like \n> > that - i never want to commit lines with <<<<<< or >>>>> markers)\n> > \n> > Another merge conflict observation is that Git is much worse at figuring \n> > out the right merge resolution than our previous Quilt based workflow \n> > was. I eventually found it to be mainly due to the following detail: \n> > sometimes it's more useful to first apply the merged branch and then \n> > attempt to merge HEAD, as a patch.\n> > \n> > I've got a script for that which also combines it with the \"rej\" tool, \n> > and in about 70%-80% of the cases where Git is unable to resolve a merge \n> > automatically it figures things out. ('rej' is obviously a more relaxed \n> > merge utility, but it's fairly robust in my experience, with a very low \n> > false positive rate.)\n> > \n> > The ad-hoc \"tip-mergetool\" script we are using is attached below. It's \n> > really just for demonstration purposes - it doesnt work when there's a \n> > rename related conflict, etc.\n> > \n> > Peter Zijstra also wrote a git-mergetool extension for the 'rej' tool \n> > btw., he might want to post that patch. I've attached Chris Mason's rej \n> > tool too.\n> \n> This is what I run with.\n> \n> I added the cp to the 3-way merge tools because I think its stupid to\n> see the messed up merge markers instead of the original file.\n\nWhile we're on the subject, I only found one tool that 'digs' these\nmerge markers and that is xxdiff --unmerge.\n\nOne would think more tools understand these merge markers, but I\ncouldn't find any.\n"},{"id":"80708","messageId":"20080623151201.GB20902@sigill.intra.peff.net","threadId":"13977","inReplyTo":"20080623094906.GA8284@elte.hu","subject":"Re: git-rerere observations and feature suggestions","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2008-06-23T15:12:02Z","receivedAt":"2008-06-23T15:12:02Z","isPatch":false,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Mon, Jun 23, 2008 at 11:49:06AM +0200, Ingo Molnar wrote:\n\n> another git-rerere observation: occasionally it happens that i \n> accidentally commit a merge marker into the source code.\n> \n> That's obviously stupid, and it normally gets found by testing quickly, \n> but still it would be a really useful avoid-shoot-self-in-foot feature \n> if git-commit could warn about such stupidities of mine.\n> \n> ( and if i could configure git-commit to outright reject a commit like \n> that - i never want to commit lines with <<<<<< or >>>>> markers)\n\nThe right place for this is in a pre-commit hook, which can look at what\nyou are about to commit and decide if it is OK. In fact, the default\npre-commit hook that ships with git performs this exact check. You just\nneed to turn it on with:\n\n chmod +x .git/hooks/pre-commit\n\n-Peff\n"},{"id":"80711","messageId":"20080623152248.GB28394@elte.hu","threadId":"13977","inReplyTo":"20080623151201.GB20902@sigill.intra.peff.net","subject":"Re: git-rerere observations and feature suggestions","fromName":"Ingo Molnar","fromEmail":"mingo@elte.hu","sentAt":"2008-06-23T15:22:48Z","receivedAt":"2008-06-23T15:22:48Z","isPatch":false,"sender":{"key":"mingo@elte.hu","avatar":null},"body":"\n* Jeff King wrote:\n\n> > ( and if i could configure git-commit to outright reject a commit like \n> > that - i never want to commit lines with <<<<<< or >>>>> markers)\n> \n> The right place for this is in a pre-commit hook, which can look at \n> what you are about to commit and decide if it is OK. In fact, the \n> default pre-commit hook that ships with git performs this exact check. \n> You just need to turn it on with:\n> \n> chmod +x .git/hooks/pre-commit\n\ncool, thanks :-)\n\n\tIngo\n"}]}