{"thread":{"id":"3324","subject":"maildir / read-tree trivial merging getting in the way?","startedAt":"2006-02-14T02:18:19Z","lastAt":"2006-02-14T02:36:21Z","messageCount":5,"participants":["Ben Clifford","Junio C Hamano","Linus Torvalds"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"16076","messageId":"Pine.LNX.4.60.0602140217380.19093@mundungus.clifford.ac","threadId":"3324","inReplyTo":null,"subject":"maildir / read-tree trivial merging getting in the way?","fromName":"Ben Clifford","fromEmail":"benc@hawaga.org.uk","sentAt":"2006-02-14T02:18:19Z","receivedAt":"2006-02-14T02:18:19Z","isPatch":false,"sender":{"key":"benc@hawaga.org.uk","avatar":"https://gravatar.com/avatar/c7ce083471287f8e77b69dd147f757799d1efdd740727fd7e3003f33e88be898?d=mp&s=160"},"body":"\nI've spent a few hours playing round with maildir-aware merging.\n\nThe basic idea I'm trying to implement is to flip the index round so that \ninstead of looking at how the content has changed for a particular filename, \nI'm looking at how the filenames have changed for a particular content.\n\nSo I'm using git read-tree -m to populate the index with entries for the \nbranches to merge so that I can then diddle round with those.\n\nBut the read-tree trivial merge logic seems to be getting in the way a bit.\n\nIn my test repo, I have two branches ('master' and 'red') forked from the base \npoint 'base':\n\nin 'base':\n\n$ ls\nA    fish one\n\nin 'red':\n\n$ ls\nB         billygoat one\n\nin 'master'\n\n$ ls\nA    lion two\n\n> From base, I renamed and cg add / cg rm'd to change A to B, one to two, \nand fish to billygoat and lion to give the above.\n\nWhen I read in the tree I get automatic resolving (down to stage 0) for the \nadded files. But actually in the output of my merge, I'm not always going to \nwant that to happen: In the A->B case, I do want to keep B (and need to remove \nA), likewise in the one->two case.\n\nBut for fish->{billygoat,lion}, I only want one file to end up at stage 0, and \nit might not be called either billygoat or lion - in maildir, the filenames are \nmore structured, and given a filename like\nfoo:2,SR and foo:2,SF I would want to compose the filenames to give me \nfoo:2,SRF.\n\n\n$ git read-tree -m base master red\n\n$ git ls-files  --stage\n100644 40e0a6f540b1b457c61315f3ccf2f5ed628e2f36 1       A\n100644 40e0a6f540b1b457c61315f3ccf2f5ed628e2f36 2       A\n100644 40e0a6f540b1b457c61315f3ccf2f5ed628e2f36 0       B\n100644 a8150e61a3a4c9941d29169ee639396547f40de2 0       billygoat\n100644 a8150e61a3a4c9941d29169ee639396547f40de2 1       fish\n100644 a8150e61a3a4c9941d29169ee639396547f40de2 0       lion\n100644 b67e17aeb5938def7ee105c2afe9fbb30a28a872 1       one\n100644 b67e17aeb5938def7ee105c2afe9fbb30a28a872 3       one\n100644 b67e17aeb5938def7ee105c2afe9fbb30a28a872 0       two\n\nNow, I think maybe I can just look at what has made it to stage 0 and play \nround with those, but it makes me feel a little dirty - if anything, the index \nindicates that a bunch of stuff has been correctly merged (by being at stage 0) \nwhen in fact it hasn't.\n\nSo basically my question is: should I feel dirty about doing this and diddle \nread-tree so that there's a flag to not do the trivial merges automatically?\n\n-- \n"},{"id":"16078","messageId":"7vy80ewu6n.fsf@assigned-by-dhcp.cox.net","threadId":"3324","inReplyTo":"Pine.LNX.4.60.0602140217380.19093@mundungus.clifford.ac","subject":"Re: maildir / read-tree trivial merging getting in the way?","fromName":"Junio C Hamano","fromEmail":"junkio@cox.net","sentAt":"2006-02-14T02:28:00Z","receivedAt":"2006-02-14T02:28:00Z","isPatch":false,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Ben Clifford <benc@hawaga.org.uk> writes:\n\n> So basically my question is: should I feel dirty about doing this and\n> diddle read-tree so that there's a flag to not do the trivial merges\n> automatically?\n\nI am mildly negative about touching read-tree for this kind of\nnon-SCM'ish usage.\n\nIf you are doing read-tree without doing any trivial merge, then\nyou would use ls-files to inspect each stage, decide what the\nfinal shape of the tree you want, and construct such a tree in\nthe index.\n\nThat would be more naturally done by writing that thing in a\nmore reasonable scripting language (not shell, but Perl or\nPython), call ls-tree three times, do whatever merge to come up\nwith the final shape of the tree, and then construct the tree\nwith a single invocation of \"update-index --index-info\", maybe\neven starting from an empty index file.\n"},{"id":"16079","messageId":"Pine.LNX.4.64.0602131820490.3691@g5.osdl.org","threadId":"3324","inReplyTo":"Pine.LNX.4.60.0602140217380.19093@mundungus.clifford.ac","subject":"Re: maildir / read-tree trivial merging getting in the way?","fromName":"Linus Torvalds","fromEmail":"torvalds@osdl.org","sentAt":"2006-02-14T02:32:35Z","receivedAt":"2006-02-14T02:32:35Z","isPatch":false,"sender":{"key":"torvalds@linux-foundation.org","avatar":"https://avatars.githubusercontent.com/u/1024025?v=4"},"body":"\n\nOn Tue, 14 Feb 2006, Ben Clifford wrote:\n> \n> I've spent a few hours playing round with maildir-aware merging.\n> \n> The basic idea I'm trying to implement is to flip the index round so that\n> instead of looking at how the content has changed for a particular filename,\n> I'm looking at how the filenames have changed for a particular content.\n> \n> So I'm using git read-tree -m to populate the index with entries for the\n> branches to merge so that I can then diddle round with those.\n> \n> But the read-tree trivial merge logic seems to be getting in the way a bit.\n\nYou are much better off working with \"git-ls-tree\", or perhaps \n\"git-diff-tree\".\n\nThe latter in particular will show you what got added and what got \ndeleted, but will quickly ignore any common entries (which is probably \nexactly what you want).\n\n> So basically my question is: should I feel dirty about doing this and diddle\n> read-tree so that there's a flag to not do the trivial merges automatically?\n\nYou should try to avoid git-read-tree entirely, I suspect.\n\nAll the things git-read-tree does are wrong for you. Notably, it very much \non purpose will match things up name-by-name, and it does a lot of extra \nwork to create a sorted version of the index to do the trivial merges \nquickly. The thing is, it doesn't even do that the smart way.\n\nNow, git-read-tree actually does a _great_ job - don't get me wrong. It's \njust that the job it does isn't really suitable for your usage, and it's \ndoing some things the \"simple and stupid\" way instead of being very smart \nabout them, just because they aren't that important under normal loads.\n\nFor example, in a three-way merge (with an index), it will basically have \nfour sorted inputs that it needs to interleave. Now, there's a _smart_ way \nto interleave sorted input, and there's a stupid one. The smart way is to \nread the sources all together, and just pick the right sorted order, and \ninterleave them all together.\n\nThat's not what git-read-tree does.\n\ngit-read-tree will read them one by one, and use \"insertion sort\" to \nmaintain the result in sorted order. Now, insertion sort isn't totally \nidiotic (it's not doing a bogo-sort, at least), but it _is_ pretty damn \nsilly when all the sources are already sorted and known ahead of time.\n\nSo git-read-tree does some stupid things, and scales badly with really big \ntrees. The good news is that we can fix it - the bad news is that my \nmotivation for it is pretty low, since \"really big\" means \"much bigger \nthan the kernel\" ;)\n\nIn contrast \"git-diff-tree -r a b\" does the _smart_ thing, and scales \nlinearly with tree size _and_ can take advantage of subdirectories not \nchanging (the latter is apparently not an issue for you, but can be one in \nother circumstances).\n\n(The \"raw output\" from git-diff-tree is also very easy to parse. Don't do \nthe \"-p\" (patch) form, the raw \"this is how the SHA's changed\" sounds \nlike it's exactly what you want, assuming you're interested in renames \nwith no content change)\n\n\t\tLinus\n"},{"id":"16081","messageId":"Pine.LNX.4.60.0602140231340.19093@mundungus.clifford.ac","threadId":"3324","inReplyTo":"7vy80ewu6n.fsf@assigned-by-dhcp.cox.net","subject":"Re: maildir / read-tree trivial merging getting in the way?","fromName":"Ben Clifford","fromEmail":"benc@hawaga.org.uk","sentAt":"2006-02-14T02:35:52Z","receivedAt":"2006-02-14T02:35:52Z","isPatch":false,"sender":{"key":"benc@hawaga.org.uk","avatar":"https://gravatar.com/avatar/c7ce083471287f8e77b69dd147f757799d1efdd740727fd7e3003f33e88be898?d=mp&s=160"},"body":"On Mon, 13 Feb 2006, Junio C Hamano wrote:\n\n>\n> That would be more naturally done by writing that thing in a\n> more reasonable scripting language (not shell, but Perl or\n> Python), call ls-tree three times, do whatever merge to come up\n> with the final shape of the tree, and then construct the tree\n> with a single invocation of \"update-index --index-info\", maybe\n> even starting from an empty index file.\n\nyeah, looks like ls-tree x 3 is what I want and quite possibly I'll end up \nconstructing a new index from scratch.\n\n-- \nBen\nhttp://www.hawaga.org.uk/ben/\n"},{"id":"16080","messageId":"Pine.LNX.4.64.0602131833110.3691@g5.osdl.org","threadId":"3324","inReplyTo":"7vy80ewu6n.fsf@assigned-by-dhcp.cox.net","subject":"Re: maildir / read-tree trivial merging getting in the way?","fromName":"Linus Torvalds","fromEmail":"torvalds@osdl.org","sentAt":"2006-02-14T02:36:21Z","receivedAt":"2006-02-14T02:36:21Z","isPatch":false,"sender":{"key":"torvalds@linux-foundation.org","avatar":"https://avatars.githubusercontent.com/u/1024025?v=4"},"body":"\n\nOn Mon, 13 Feb 2006, Junio C Hamano wrote:\n> \n> That would be more naturally done by writing that thing in a\n> more reasonable scripting language (not shell, but Perl or\n> Python), call ls-tree three times, do whatever merge to come up\n> with the final shape of the tree, and then construct the tree\n> with a single invocation of \"update-index --index-info\", maybe\n> even starting from an empty index file.\n\nExactly. Except that it probably makes sense to use \"git-diff-tree\" to try \nto avoid doing lots of unnecessary work in a script, if the normal case is \nthat there's still a lot of stuff that hasn't changed.\n\nSo conceptually you would do three \"git-ls-tree\" invocations, but in \n_practice_ it's probably better to do just one \"git-ls-tree\", and then use \n\"git-diff-tree\" to basically generate the differences from that one \nls-tree to the other cases of interest.\n\nSo start with the merge-base, for example, and then basically generate the \n\"what changed\" between the merge base and the two branch heads. \n\nThat was the plan for doing merges initially, it just turned out that \ndoing them in the index made things easier.\n\n\t\t\tLinus\n"}]}