{"thread":{"id":"53413","subject":"[RFC] subtree: handle unmerged history trees","startedAt":"2020-05-06T14:01:01Z","lastAt":"2020-05-11T11:47:05Z","messageCount":2,"participants":["Claus Schneider","Tom Clarkson"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"397185","messageId":"CA+GP4bou+_bvGjnd3Z+fctXX4V2sE=Ta9ivCnYmbn1+JUxKRQg@mail.gmail.com","threadId":"53413","inReplyTo":"CA+GP4bqEjK2V7fGqXsJMkRURod8zVzZAQQ7woUUtqybxfnmSVg@mail.gmail.com","subject":"[RFC] subtree: handle unmerged history trees","fromName":"Claus Schneider","fromEmail":"claus.schneider@eficode.com","sentAt":"2020-05-06T14:00:47Z","receivedAt":"2020-05-06T14:01:01Z","isPatch":false,"sender":{"key":"claus.schneider@eficode.com","avatar":"https://avatars.githubusercontent.com/u/3347017?v=4"},"body":"Hi..\n\nI would like to enhance git subtree functionality\n\nProblem outline:\nIn a scenario where a team develops a software subsystem and part of\nthe code is internal and parts (source and/or interfaces) should be\ndelivered continuously for others to incorporate into their\nsystem/product.\n\nOne solution is to use submodules structure for both the development\nteam and product which is a hassle for the development team. They need\nto make logical commits across submodule and parent repository which\ncan be a problem with parallel development and verification.\n\nSubtree has the feature of splitting the repository in order to\nachieve this, but there are some constraints that I would fix.\n- In bare mode it pushes changes to a separate branch containing the\nprefix changes which is fine. You get a problem when you run the next\nsplit. Either you re-split all the commits again - Or you add the\n-rejoin parameter with the result that the splitted prefix patches are\npart of your history twice or even more if you have further extracts.\nSo this is either a performance issue or a usability issue.\n- You lose traceability from the extracted subtree commit back to you\noriginal commits,\n\nSolution outline using subtree:\n- Add traceability to each extracted commit in new history\n  - It enables humans to trace from the extracted commit to the\noriginal commit by basic reading, clicking in tools like gitk and\nscripting if desired\n  - Enable subtree itself to utilize the above mentioned traceability\nand simulate the add repository or rejoin merge commit. Subtree can\nthen \"behave\" similarly independent of the method being used.\n  - Add option for rev-list so it can list based on\nprefix/subdirectory. I have not been able to find any error, issues or\nside effects adding the \"-- $dir\" to the rev-list command. All the\nmanual tests, I have done, behave correctly in my total patched git\nrevision. It gives a heck of performance for many-commit repositories.\n - Example: split contrib/subtree out of git repository.\n    Without the option: parsing ~28000 commit\n    With the option: parsing ~100 commits\n\nPatches can be found here:\nhttps://github.com/git/git/compare/master...Praqma:split-append-info-options-master\nCommits related to this:\n* cd712dda39 subtree: use append-info to only extract new commits in the prefix\n* deb2e1cd8b subtree: add append-info option for adding info about\noriginal commit\n* ff73b37e22 subtree: Add option for listing commits based on prefix\n\nBest regards\nClaus Schneider\n"},{"id":"397533","messageId":"57CDDE09-7AF8-4F0B-80A0-53E288455AE5@icloud.com","threadId":"53413","inReplyTo":"CA+GP4bou+_bvGjnd3Z+fctXX4V2sE=Ta9ivCnYmbn1+JUxKRQg@mail.gmail.com","subject":"Re: [RFC] subtree: handle unmerged history trees","fromName":"Tom Clarkson","fromEmail":"tqclarkson@icloud.com","sentAt":"2020-05-11T11:46:59Z","receivedAt":"2020-05-11T11:47:05Z","isPatch":false,"sender":{"key":"tqclarkson@icloud.com","avatar":"https://gravatar.com/avatar/02ea2942fcaf5dc1140ccda995e0dee545ff210ca8adee02d50f35de7205e301?d=mp&s=160"},"body":"\n> On 7 May 2020, at 12:00 am, Claus Schneider <claus.schneider@eficode.com> wrote:\n\n> - In bare mode it pushes changes to a separate branch containing the\n> prefix changes which is fine. You get a problem when you run the next\n> split. Either you re-split all the commits again - Or you add the\n> -rejoin parameter with the result that the splitted prefix patches are\n> part of your history twice or even more if you have further extracts.\n> So this is either a performance issue or a usability issue.\n\nA simpler way to link a split without including both histories would be to add a mainline commit with a git-subtree-split annotation, but without having the subtree commit as a parent. That would give you a reference to a commit not reachable from HEAD though, so plenty of opportunity to shoot yourself in the foot.\n\nPersisting the cache between runs would be enough to avoid any potential performance penalty on subsequent splits, and is just a matter of changing the directory used. My unrelated patch implements that for other reasons, along with letting you specify specific commit mappings from script if that’s what you need.\n\n> - Add traceability to each extracted commit in new history\n>  - It enables humans to trace from the extracted commit to the\n> original commit by basic reading, clicking in tools like gitk and\n> scripting if desired\n>  - Enable subtree itself to utilize the above mentioned traceability\n> and simulate the add repository or rejoin merge commit. Subtree can\n> then \"behave\" similarly independent of the method being used.\n\nHave you considered how your annotations will behave if you import the same subtree repo into two different mainline repos? The subtree history would then have references to a bunch of commits that don’t exist. Adding similar annotations to merge commits on the mainline side seems like a good idea though, and would let you use find_existing_splits to avoid regenerating too many commits.\n\nFor the human readable link from the subtree repo to your original monorepo, perhaps a custom annotation would be a better fit - something like\n\ngit subtree split dir - - annotate-mainline-commit-as=“id-in-monorepo”\n\n>  - Add option for rev-list so it can list based on\n> prefix/subdirectory. I have not been able to find any error, issues or\n> side effects adding the \"-- $dir\" to the rev-list command. All the\n> manual tests, I have done, behave correctly in my total patched git\n> revision. It gives a heck of performance for many-commit repositories.\n\nHave you tested the rev-list dir option against preexisting history without your new annotations or created without split? If any of the new commits has a parent that is not in the rev-list, it will look up that commit individually and recursively. A git-subtree-mainline annotation will shortcut that, but without it the individual lookup is massively slower than working from even a very large rev-list.\n\n"}]}