{"thread":{"id":"53547","subject":"Assessing about commit order in upstream Linux","startedAt":"2020-05-26T06:53:33Z","lastAt":"2020-05-28T20:25:50Z","messageCount":6,"participants":["Eugeniu Rosca","Junio C Hamano","Michal Suchánek"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"398572","messageId":"20200526065320.GA18107@lxhi-065.adit-jv.com","threadId":"53547","inReplyTo":null,"subject":"Assessing about commit order in upstream Linux","fromName":"Eugeniu Rosca","fromEmail":"erosca@de.adit-jv.com","sentAt":"2020-05-26T06:53:20Z","receivedAt":"2020-05-26T06:53:33Z","isPatch":false,"sender":{"key":"erosca@de.adit-jv.com","avatar":null},"body":"Dear Git community,\n\nAssessing about the correct order of upstream commits is essential\nduring the backporting process, since we aim to ensure that the\nbackporting result can be built and bisected at each commit.\n\nHowever, there appear to be at least two ways to compute the\nrelative order of \"mainline\" commits, specifically based on the:\n\n * index/position of commit summary line in the output of\n 'git log --oneline --topo-order upstream/master'\n\n * 'git describe --contains --match=\"v*\" ' of each commit\n\nI've considered both approaches equivalent, until I ran into [A-B].\n\nJudging by the index in the 'git log' output, commit [B] seems to\n(topologically) come first and hence would need to be backported first:\n\n$ git log --reverse --oneline --topo-order v4.14..v4.15 | grep -n \"mm: slabinfo: remove CONFIG_SLABINFO\" | cut -f1 -d:\n7261\n$ git log --reverse --oneline --topo-order v4.14..v4.15 | grep -n \"RDMA/umem: Avoid partial declaration of non-static function\" | cut -f1 -d:\n7029\n\nJudging by the version returned by 'git describe --contains', commit [A]\nseems to (topologically) come first due to '~93' putting it (mentally)\n\"earlier\" in the topological graph compared to '~73':\n\n$ git describe --contains --match=\"v*\" 5b36577109be\n v4.15-rc1~93^2~117\n$ git describe --contains --match=\"v*\" fec99ededf6b\n v4.15-rc1~73^2~56\n\nSo, the two approaches lead to different results. If you see any false\nassumption or mistaken belief, could you please pinpoint that? TIA.\n\n[A] https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=5b36577109be\n (\"mm: slabinfo: remove CONFIG_SLABINFO\")\n[B] https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=fec99ededf6b\n (\"RDMA/umem: Avoid partial declaration of non-static function\")\n\n-- \nBest regards,\nEugeniu Rosca\n"},{"id":"398579","messageId":"xmqqr1v6oh6y.fsf@gitster.c.googlers.com","threadId":"53547","inReplyTo":"20200526065320.GA18107@lxhi-065.adit-jv.com","subject":"Re: Assessing about commit order in upstream Linux","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2020-05-26T15:21:25Z","receivedAt":"2020-05-26T15:21:34Z","isPatch":false,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Eugeniu Rosca writes:\n\n> So, the two approaches lead to different results. If you see any false\n> assumption or mistaken belief, could you please pinpoint that? TIA.\n\nPerhaps the assumption/belief that the set of commits in a history\ncan be totally ordered is the issue? When multiple people work\ntogether on a project, especially in a project where \"pull --no-ff\"\nis not enforced, there can exist only partial order among them?\n\n"},{"id":"398588","messageId":"20200526171443.GE25173@kitsune.suse.cz","threadId":"53547","inReplyTo":"xmqqr1v6oh6y.fsf@gitster.c.googlers.com","subject":"Re: Assessing about commit order in upstream Linux","fromName":"Michal Suchánek","fromEmail":"msuchanek@suse.de","sentAt":"2020-05-26T17:14:43Z","receivedAt":"2020-05-26T17:14:49Z","isPatch":false,"sender":{"key":"msuchanek@suse.de","avatar":"https://avatars.githubusercontent.com/u/787652?v=4"},"body":"On Tue, May 26, 2020 at 08:21:25AM -0700, Junio C Hamano wrote:\n> Eugeniu Rosca writes:\n> \n> > So, the two approaches lead to different results. If you see any false\n> > assumption or mistaken belief, could you please pinpoint that? TIA.\n> \n> Perhaps the assumption/belief that the set of commits in a history\n> can be totally ordered is the issue? When multiple people work\n> together on a project, especially in a project where \"pull --no-ff\"\n> is not enforced, there can exist only partial order among them?\n> \nAs in if you have history with two branches\n\n D\n / \\\n B C\n \\ /\n A\n\ncommits B and C are not comparable. They are both between A and D but\nthe order of B and C is arbitrary. Different renderings of the history\nmay choose different order of B and C. This is a simle example. Linux\nhistory is a spaghetti of tens of branches.\n\nThanks\n\nMichal\n"},{"id":"398754","messageId":"20200528175954.GA9275@lxhi-065.adit-jv.com","threadId":"53547","inReplyTo":"xmqqr1v6oh6y.fsf@gitster.c.googlers.com","subject":"Re: Assessing about commit order in upstream Linux","fromName":"Eugeniu Rosca","fromEmail":"erosca@de.adit-jv.com","sentAt":"2020-05-28T17:59:54Z","receivedAt":"2020-05-28T18:00:20Z","isPatch":false,"sender":{"key":"erosca@de.adit-jv.com","avatar":null},"body":"Hi Junio,\n\nOn Tue, May 26, 2020 at 08:21:25AM -0700, Junio C Hamano wrote:\n> Eugeniu Rosca writes:\n> \n> > So, the two approaches lead to different results. If you see any false\n> > assumption or mistaken belief, could you please pinpoint that? TIA.\n> \n> Perhaps the assumption/belief that the set of commits in a history\n> can be totally ordered is the issue? When multiple people work\n> together on a project, especially in a project where \"pull --no-ff\"\n> is not enforced, there can exist only partial order among them?\n> \n\nIMHO it might be an issue in truly decentralized projects, for which we\ncan't define an upstream and a downstream. But is it an issue for Linux?\n\nHere is a quick attempt to sketch how commits flow into linux/master,\nevery development cycle, again and again, respecting the same pattern.\n\n +-----------------o Linus\n | +-------------o Maintainers\n | | +-o Contributors\n v v v\n\nmaster o\n | +-------------------+\n | | |\n C(M)o---o Same story as \"A\" |\n | | |\n B o +-------------------+\n |\n A(M)o---o A^2 \n | | \n A~1 o o A^2~1\n | | \n A~2 o o A^2~2\n |\n o A^2~3(M)--o A^2~3^2\n | |\n | o A^2~3^2~1\n | |\n | o A^2~3^2~2\n |\n o A^2~4(M)--o A^2~4^2\n | |\n o A^2~5 o A^2~4^2~1\n |\n o A^2~4^2~2\n\nThe order of these commits matter to me because:\n\n - Commits A^2~4^2~2 through A^2~4^2 likely originate from the same\n series, with a well defined topic/scope and inner sequence. It would\n be ideal to mirror this same order during backporting. Otherwise,\n both the product of porting is questionable and the reviewing\n effort is high.\n\n - Likewise, commits A^2~3^2~2 through A^2~3^2 probably come from one\n single series. The reviewers would hugely appreciate if these are\n not scattered during backporting, but are kept together (preferably\n in the exact same succession).\n\n - Any merge commit (marked with '(M)' above) might carry a conflict\n resolution in itself (aka 'evil merge') which might act as dependency\n to any of its children. So, cherry picking commits in no particular\n order may very likely introduce build and runtime failures, whose\n reasons may be difficult to spot in the downstream projects.\n\nHaving said that, I am curious, does anybody resonate with these\nstatements, based on personal experience (in Linux or other projects)?\n\n-- \nBest regards,\nEugeniu Rosca\n"},{"id":"398772","messageId":"20200528181226.GB9275@lxhi-065.adit-jv.com","threadId":"53547","inReplyTo":"20200526171443.GE25173@kitsune.suse.cz","subject":"Re: Assessing about commit order in upstream Linux","fromName":"Eugeniu Rosca","fromEmail":"erosca@de.adit-jv.com","sentAt":"2020-05-28T18:12:26Z","receivedAt":"2020-05-28T18:12:37Z","isPatch":false,"sender":{"key":"erosca@de.adit-jv.com","avatar":null},"body":"Hi Michal,\n\nOn Tue, May 26, 2020 at 07:14:43PM +0200, Michal Suchánek wrote:\n> On Tue, May 26, 2020 at 08:21:25AM -0700, Junio C Hamano wrote:\n> > Eugeniu Rosca writes:\n> > \n> > > So, the two approaches lead to different results. If you see any false\n> > > assumption or mistaken belief, could you please pinpoint that? TIA.\n> > \n> > Perhaps the assumption/belief that the set of commits in a history\n> > can be totally ordered is the issue? When multiple people work\n> > together on a project, especially in a project where \"pull --no-ff\"\n> > is not enforced, there can exist only partial order among them?\n> > \n> As in if you have history with two branches\n> \n> D\n> / \\\n> B C\n> \\ /\n> A\n> \n> commits B and C are not comparable. They are both between A and D but\n> the order of B and C is arbitrary. Different renderings of the history\n> may choose different order of B and C. This is a simle example. Linux\n> history is a spaghetti of tens of branches.\n\nWhile in theory 'B' and 'C' might look equivalent, IMHO in practice\nthere is a clear distinction between the two. It's commonly known that\nGit refers to 'B' as the 'first parent' of 'D'. Git also provides means\nto identify such first parents via 'git log --first-parent'.\n\nA fun fact about first parents is that, unless Linus is on vacation\nand hands over his responsibilities to GKH, you will be quite\nconfident that 'git log --first-parent linux/master' will list\nstuff committed by Linus himself. That's why (I bet) in the minds\nof people involved in Linux development, the diagram looks like:\n\n D\n | \\\n B C\n | /\n A\n\nIMHO the fact that 'A' is the parent of 'C' (IOW 'C' has an appropriate\nbase version) is mostly important to achieve an effortless merge of 'C'\nand later on loses its major significance. So, I would say that\n(contents-wise) the diagram can be further reduced to:\n\n D\n | \\\n B D^2\n |\n A\n\nJust visually, a sane backporting order looks A, B an D^2 (A is assumed\nnon-merge and D is skipped, since cherry picking merges is not common).\n\nI am quite sure people have thought about backporting techniques and\nstrategies long before I started to ask these questions. So, I am\nstill looking forward to seeing various experiences shared.\n\n-- \nBest regards,\nEugeniu Rosca\n"},{"id":"398797","messageId":"20200528202538.GK25173@kitsune.suse.cz","threadId":"53547","inReplyTo":"20200528181226.GB9275@lxhi-065.adit-jv.com","subject":"Re: Assessing about commit order in upstream Linux","fromName":"Michal Suchánek","fromEmail":"msuchanek@suse.de","sentAt":"2020-05-28T20:25:38Z","receivedAt":"2020-05-28T20:25:50Z","isPatch":false,"sender":{"key":"msuchanek@suse.de","avatar":"https://avatars.githubusercontent.com/u/787652?v=4"},"body":"On Thu, May 28, 2020 at 08:12:26PM +0200, Eugeniu Rosca wrote:\n> Hi Michal,\n> \n> On Tue, May 26, 2020 at 07:14:43PM +0200, Michal Suchánek wrote:\n> > On Tue, May 26, 2020 at 08:21:25AM -0700, Junio C Hamano wrote:\n> > > Eugeniu Rosca writes:\n> > > \n> > > > So, the two approaches lead to different results. If you see any false\n> > > > assumption or mistaken belief, could you please pinpoint that? TIA.\n> > > \n> > > Perhaps the assumption/belief that the set of commits in a history\n> > > can be totally ordered is the issue? When multiple people work\n> > > together on a project, especially in a project where \"pull --no-ff\"\n> > > is not enforced, there can exist only partial order among them?\n> > > \n> > As in if you have history with two branches\n> > \n> > D\n> > / \\\n> > B C\n> > \\ /\n> > A\n> > \n> > commits B and C are not comparable. They are both between A and D but\n> > the order of B and C is arbitrary. Different renderings of the history\n> > may choose different order of B and C. This is a simle example. Linux\n> > history is a spaghetti of tens of branches.\n> \n> While in theory 'B' and 'C' might look equivalent, IMHO in practice\n> there is a clear distinction between the two. It's commonly known that\n> Git refers to 'B' as the 'first parent' of 'D'. Git also provides means\n> to identify such first parents via 'git log --first-parent'.\n> \n> A fun fact about first parents is that, unless Linus is on vacation\n> and hands over his responsibilities to GKH, you will be quite\n> confident that 'git log --first-parent linux/master' will list\n> stuff committed by Linus himself. That's why (I bet) in the minds\n> of people involved in Linux development, the diagram looks like:\n> \n> D\n> | \\\n> B C\n> | /\n> A\nAnd that's not the case. Commits B and C will typically com from\ndifferent subsystems, and are truly interchangeable. These subsystems,\nagain, will have number of separate branches that are merged together\nbefore they are submitted to Linus. Often a feature requires cross-merge\nbetween different subsystems which further complicates the history. Even\nif B is a commit authored by Linus and you can infer from that it's on\nthe master branch it says nothing about order of B and C. They are\nstill not comparable. And you may still need to reconcile the changes in\nB and C in D and whatever order you choose for backporting them you will\nneed to reflect D in both B and C in the conflicting case.\n\nHTH\n\nMichal\n"}]}