{"thread":{"id":"41757","subject":"Properties of trees referencing commit objects (mode 160000)?","startedAt":"2016-03-19T22:13:48Z","lastAt":"2016-03-21T15:36:27Z","messageCount":9,"participants":["Josh Triplett","Jeff King","Junio C Hamano"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"281249","messageId":"20160319221348.GA5247@x","threadId":"41757","inReplyTo":null,"subject":"Properties of trees referencing commit objects (mode 160000)?","fromName":"Josh Triplett","fromEmail":"josh@joshtriplett.org","sentAt":"2016-03-19T22:13:48Z","receivedAt":"2016-03-19T22:13:48Z","isPatch":false,"sender":{"key":"josh@joshtriplett.org","avatar":"https://avatars.githubusercontent.com/u/162737?v=4"},"body":"I'm building some tools to track commit objects, and I'm thinking of\nusing submodule-style references to commit objects in tree objects (mode\n160000) to do so.  I'm trying to figure out some of the properties of\nthat.\n\nCan a commit object referenced that way live in the same repository,\nrather than some external repository?\n\nWill git treat such a reference as keeping the commit object (and\neverything recursively referenced by it) live and reachable?  If that\ncommit object is only reachable by the tree, and not by following the\nparents of any commit directly referenced from refs/*, will git discard\nit as unreachable?\n\n- Josh Triplett\n"},{"id":"281255","messageId":"20160320041803.GC18312@sigill.intra.peff.net","threadId":"41757","inReplyTo":"20160319221348.GA5247@x","subject":"Re: Properties of trees referencing commit objects (mode 160000)?","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2016-03-20T04:18:04Z","receivedAt":"2016-03-20T04:18:04Z","isPatch":false,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Sat, Mar 19, 2016 at 03:13:48PM -0700, Josh Triplett wrote:\n\n> I'm building some tools to track commit objects, and I'm thinking of\n> using submodule-style references to commit objects in tree objects (mode\n> 160000) to do so.  I'm trying to figure out some of the properties of\n> that.\n> \n> Can a commit object referenced that way live in the same repository,\n> rather than some external repository?\n\nYes, it can be in the same repository, but...\n\n> Will git treat such a reference as keeping the commit object (and\n> everything recursively referenced by it) live and reachable?  If that\n> commit object is only reachable by the tree, and not by following the\n> parents of any commit directly referenced from refs/*, will git discard\n> it as unreachable?\n\nNo, we do not follow \"gitlinks\" like this for reachability. Neither for\npruning, nor for object transfer via push/fetch. So you'd need to have a\nseparate reference to it (or history containing it).\n\n-Peff\n"},{"id":"281301","messageId":"20160320184524.GA16064@x","threadId":"41757","inReplyTo":"20160320041803.GC18312@sigill.intra.peff.net","subject":"Re: Properties of trees referencing commit objects (mode 160000)?","fromName":"Josh Triplett","fromEmail":"josh@joshtriplett.org","sentAt":"2016-03-20T18:45:24Z","receivedAt":"2016-03-20T18:45:24Z","isPatch":false,"sender":{"key":"josh@joshtriplett.org","avatar":"https://avatars.githubusercontent.com/u/162737?v=4"},"body":"On Sun, Mar 20, 2016 at 12:18:04AM -0400, Jeff King wrote:\n> On Sat, Mar 19, 2016 at 03:13:48PM -0700, Josh Triplett wrote:\n> \n> > I'm building some tools to track commit objects, and I'm thinking of\n> > using submodule-style references to commit objects in tree objects (mode\n> > 160000) to do so.  I'm trying to figure out some of the properties of\n> > that.\n> > \n> > Can a commit object referenced that way live in the same repository,\n> > rather than some external repository?\n> \n> Yes, it can be in the same repository, but...\n\nWill git clone/checkout/etc handle it properly in that case, in the\nabsence of a .gitmodules file?  Or would it only work with custom tools?\n\n> > Will git treat such a reference as keeping the commit object (and\n> > everything recursively referenced by it) live and reachable?  If that\n> > commit object is only reachable by the tree, and not by following the\n> > parents of any commit directly referenced from refs/*, will git discard\n> > it as unreachable?\n> \n> No, we do not follow \"gitlinks\" like this for reachability. Neither for\n> pruning, nor for object transfer via push/fetch. So you'd need to have a\n> separate reference to it (or history containing it).\n\nArgh.  If I have a pile of disconnected commits, is there anything git\n*would* follow to see them, other than a pile of refs?\n\nI suppose I could artificially generate a stack of merge commits with\nthose otherwise disconnect commits as parents, which would let me\nreference them all from a single ref.  Still unsatisfying, though.\n\nAlso, thanks, \"gitlink\" was the term I was trying to think of.\n\n(I'd also be tempted to ask whether a patch to teach git to follow\ngitlinks for reachability and/or object transfer would be acceptable.)\n"},{"id":"281306","messageId":"20160320200724.GC6288@sigill.intra.peff.net","threadId":"41757","inReplyTo":"20160320184524.GA16064@x","subject":"Re: Properties of trees referencing commit objects (mode 160000)?","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2016-03-20T20:07:25Z","receivedAt":"2016-03-20T20:07:25Z","isPatch":false,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Sun, Mar 20, 2016 at 11:45:24AM -0700, Josh Triplett wrote:\n\n> Will git clone/checkout/etc handle it properly in that case, in the\n> absence of a .gitmodules file?  Or would it only work with custom tools?\n\nI think checkout just creates an empty tree for any gitlinks, and waits\nfor the submodule code to fill it in.  So you'd need a custom tool. It's\neasy enough to test, though:\n\n    git init repo &&\n    cd repo &&\n    echo content >file && git add file && git commit -m base &&\n    printf '160000 commit %s\\t%s' \"$(git rev-parse HEAD)\" linked >tree &&\n    tree=$(git mktree <tree) &&\n    commit=$(echo foo | git commit-tree $tree) &&\n    git checkout $commit\n\n> > No, we do not follow \"gitlinks\" like this for reachability. Neither for\n> > pruning, nor for object transfer via push/fetch. So you'd need to have a\n> > separate reference to it (or history containing it).\n> \n> Argh.  If I have a pile of disconnected commits, is there anything git\n> *would* follow to see them, other than a pile of refs?\n\nI don't think so. We follow refs, tag \"object\" fields, and commit\n\"parent\" fields to get to commits, but never tree entries.\n\nAnd I don't think it works to just tweak the mode to 100644 for the\ncommit entry; the checkout code will complain that it was expecting a\nblob and got a commit.\n\n> I suppose I could artificially generate a stack of merge commits with\n> those otherwise disconnect commits as parents, which would let me\n> reference them all from a single ref.  Still unsatisfying, though.\n\nYes, though you can do it all in one single merge commit. The tree of\nthat merge commit wouldn't matter, it could literally just be a pointer\nto all of the parents (we used to have some limits on the number of\nparents, but I don't think we do anymore?).\n\nIf you just threw away that stacked merge and made a new one any time it\nwas updated, I think it wouldn't end up very efficient for fetching. The\nclient would say \"I have giant-merge-commit X\", and the server would say\n\"well, I need to send you giant-merge-commit Y\", but if it does not any\nlonger have X, it cannot realize that Y includes 99% of what is in X.\n\nSo I guess you'd want a history chain of those merges. I'm not quite\nsure what your application is, so I don't know if that would be a pain\nor not.\n\n> (I'd also be tempted to ask whether a patch to teach git to follow\n> gitlinks for reachability and/or object transfer would be acceptable.)\n\nAs always, I reserve all judgement on patches until I see them. :)\n\nBut I tend to think it would end up rather complex. Off the top of my\nhead, these are the three gotchas I can think of:\n\n  1. The reachability rules must be agreed upon between both sides of\n     the transfer. So you would need a protocol extension for \"please\n     consider gitlinks reachable\" so that the sender knows to include\n     them, and that when the client advertises commit X that can reach\n     Y via gitlink, it can assume that the client does not need Y.\n\n  2. There may be some issues with efficiently finding shared history\n     during a transfer.  That process usually just looks at the commits,\n     and then we hand the set of have/want commits to the pack\n     generation code to figure out the full set of required objects (and\n     in particular, we do not comb through all of the \"have\" commits for\n     objects that might have become re-referenced; it's too expensive\n     for too little gain).\n\n     But with the flag from (1), we'd have commits buried inside trees.\n     The packing code would figure out which objects to send, but I\n     suspect you'd end up with some inefficiencies where an internal\n     gitlink bumped from commit X to Y, and we _could_ know that the\n     client already has X, but digging it up doesn't happen with the\n     current code.\n\n     I didn't think it through too hard, though, so perhaps it would all\n     just work.\n\n  3. The reachability is usually an invariant in the repository, but\n     presumably this would be enabled through a config variable (it\n     _can't_ be on all the time, because then true external submodules\n     wouldn't work, and it can't just be \"eh, reachable if you got it\",\n     because then transfers cannot assume the other side has it and\n     would resend every time).\n\n     So flipping that switch in an existing repository with gitlinks\n     would probably put it in a broken state until it fetches all of the\n     gitlinks.\n\n-Peff\n"},{"id":"281312","messageId":"xmqqk2kwok8s.fsf@gitster.mtv.corp.google.com","threadId":"41757","inReplyTo":"20160320184524.GA16064@x","subject":"Re: Properties of trees referencing commit objects (mode 160000)?","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2016-03-20T22:30:27Z","receivedAt":"2016-03-20T22:30:27Z","isPatch":false,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Josh Triplett <josh@joshtriplett.org> writes:\n\n> On Sun, Mar 20, 2016 at 12:18:04AM -0400, Jeff King wrote:\n>> On Sat, Mar 19, 2016 at 03:13:48PM -0700, Josh Triplett wrote:\n>> \n>> > I'm building some tools to track commit objects, and I'm thinking of\n>> > using submodule-style references to commit objects in tree objects (mode\n>> > 160000) to do so.  I'm trying to figure out some of the properties of\n>> > that.\n>> > \n>> > Can a commit object referenced that way live in the same repository,\n>> > rather than some external repository?\n>> \n>> Yes, it can be in the same repository, but...\n>\n> Will git clone/checkout/etc handle it properly in that case, in the\n> absence of a .gitmodules file?  Or would it only work with custom tools?\n\nThat depends on the definition of \"proper\".  The default \"proper\"\nway for the core Git for submodules/gitlinks is to create an empty\ndirectory.  If you want to populate a working tree for that, you'd\nneed \"git submodule\" support, but because you are writing \"some\ntools\" on your own, there probably is a reason why you do not want\nto use \"git submodule\", so I am guessing that your definition of\n\"proper\" does not match either core Git or \"git submodule\" considers\n\"proper\"?  In which case you would need to implement your own\nsemantics (you may not even want to have an empty directory there,\nfor example).\n\n\n>> No, we do not follow \"gitlinks\" like this for reachability. Neither for\n>> pruning, nor for object transfer via push/fetch. So you'd need to have a\n>> separate reference to it (or history containing it).\n>\n> Argh.  If I have a pile of disconnected commits, is there anything git\n> *would* follow to see them, other than a pile of refs?\n\nNo.\n"},{"id":"281313","messageId":"20160320224301.GA20803@x","threadId":"41757","inReplyTo":"xmqqk2kwok8s.fsf@gitster.mtv.corp.google.com","subject":"Re: Properties of trees referencing commit objects (mode 160000)?","fromName":"Josh Triplett","fromEmail":"josh@joshtriplett.org","sentAt":"2016-03-20T22:43:01Z","receivedAt":"2016-03-20T22:43:01Z","isPatch":false,"sender":{"key":"josh@joshtriplett.org","avatar":"https://avatars.githubusercontent.com/u/162737?v=4"},"body":"On Sun, Mar 20, 2016 at 03:30:27PM -0700, Junio C Hamano wrote:\n> Josh Triplett <josh@joshtriplett.org> writes:\n> \n> > On Sun, Mar 20, 2016 at 12:18:04AM -0400, Jeff King wrote:\n> >> On Sat, Mar 19, 2016 at 03:13:48PM -0700, Josh Triplett wrote:\n> >> \n> >> > I'm building some tools to track commit objects, and I'm thinking of\n> >> > using submodule-style references to commit objects in tree objects (mode\n> >> > 160000) to do so.  I'm trying to figure out some of the properties of\n> >> > that.\n> >> > \n> >> > Can a commit object referenced that way live in the same repository,\n> >> > rather than some external repository?\n> >> \n> >> Yes, it can be in the same repository, but...\n> >\n> > Will git clone/checkout/etc handle it properly in that case, in the\n> > absence of a .gitmodules file?  Or would it only work with custom tools?\n> \n> That depends on the definition of \"proper\".  The default \"proper\"\n> way for the core Git for submodules/gitlinks is to create an empty\n> directory.  If you want to populate a working tree for that, you'd\n> need \"git submodule\" support, but because you are writing \"some\n> tools\" on your own, there probably is a reason why you do not want\n> to use \"git submodule\", so I am guessing that your definition of\n> \"proper\" does not match either core Git or \"git submodule\" considers\n> \"proper\"?  In which case you would need to implement your own\n> semantics (you may not even want to have an empty directory there,\n> for example).\n\nI can live with \"creates an empty directory and doesn't choke or\ncomplain\".  I'd kinda hoped that git might notice it already has the\ncommit, and do something like checking out the files of that commit into\nthe working directory, but it isn't critical.\n\n- Josh Triplett\n"},{"id":"281314","messageId":"20160320232202.GB20803@x","threadId":"41757","inReplyTo":"20160320200724.GC6288@sigill.intra.peff.net","subject":"Re: Properties of trees referencing commit objects (mode 160000)?","fromName":"Josh Triplett","fromEmail":"josh@joshtriplett.org","sentAt":"2016-03-20T23:22:02Z","receivedAt":"2016-03-20T23:22:02Z","isPatch":false,"sender":{"key":"josh@joshtriplett.org","avatar":"https://avatars.githubusercontent.com/u/162737?v=4"},"body":"On Sun, Mar 20, 2016 at 04:07:25PM -0400, Jeff King wrote:\n> On Sun, Mar 20, 2016 at 11:45:24AM -0700, Josh Triplett wrote:\n> > > No, we do not follow \"gitlinks\" like this for reachability. Neither for\n> > > pruning, nor for object transfer via push/fetch. So you'd need to have a\n> > > separate reference to it (or history containing it).\n> > \n> > Argh.  If I have a pile of disconnected commits, is there anything git\n> > *would* follow to see them, other than a pile of refs?\n> \n> I don't think so. We follow refs, tag \"object\" fields, and commit\n> \"parent\" fields to get to commits, but never tree entries.\n\nThat does make sense, given your point (2) below about efficiency.\n\n> And I don't think it works to just tweak the mode to 100644 for the\n> commit entry; the checkout code will complain that it was expecting a\n> blob and got a commit.\n\nThat much I assumed. :)\n\n> > I suppose I could artificially generate a stack of merge commits with\n> > those otherwise disconnect commits as parents, which would let me\n> > reference them all from a single ref.  Still unsatisfying, though.\n> \n> Yes, though you can do it all in one single merge commit. The tree of\n> that merge commit wouldn't matter, it could literally just be a pointer\n> to all of the parents (we used to have some limits on the number of\n> parents, but I don't think we do anymore?).\n\nI didn't want to assume that all the software assuming upper bounds on\noctopus merges had disappeared.  And more importantly...\n\n> If you just threw away that stacked merge and made a new one any time it\n> was updated, I think it wouldn't end up very efficient for fetching. The\n> client would say \"I have giant-merge-commit X\", and the server would say\n> \"well, I need to send you giant-merge-commit Y\", but if it does not any\n> longer have X, it cannot realize that Y includes 99% of what is in X.\n\n...that.  And the linearly growing cost of recreating and working with\nthe merge commit as the number of parent commits grows.\n\n> So I guess you'd want a history chain of those merges. I'm not quite\n> sure what your application is, so I don't know if that would be a pain\n> or not.\n\nAs it turns out, that fits fairly naturally.\n\nI want to track the evolution of a patch series or other commit history,\nthrough non-fast-forwarding actions like rebase, rebase -i, or commit\n--amend.  Similar in spirit to reflog, but with intentional commits and\ncommit messages, and most importantly with the ability to share and\ncollaborate on it.  For each version of the patch series, I plan to\ntrack the commit at the end of the series, and optionally the commit at\nthe base of the patch series (to simplify tracking of rebasing); I'll\nuse the tree object associated with the commit to track the cover\nletter, and perhaps meta-changelog entries associated with v2/v3/vN of\nthe series.\n\nPatch series almost always need to evolve through non-fast-forwarding\nhistory.  And I've seen that done in two ways: either pull the patch\nseries out of git, put it in quilt or similar, and track the quilt\nseries with git; or pull the versioning of the patch series out of git\nand track it with branch names like feature-v2, feature-2016-03-20,\nfeature-rebased, and feature-rebased-4.4-fixed-foo-fixed-bar.  That last\none closely matches a real-world example I've recently seen.  And that\nstarts to look a lot like the naming of files and documents in\norganizations that haven't yet discovered the wonders of version\ncontrol.\n\nI'd like to have the best of both worlds: handle the patch series in\ngit, *and* handle the versioning of the patch series in git.\n\n> > (I'd also be tempted to ask whether a patch to teach git to follow\n> > gitlinks for reachability and/or object transfer would be acceptable.)\n> \n> As always, I reserve all judgement on patches until I see them. :)\n\nOf course.  Obviously it's not possible to accept a patch that doesn't\nexist yet, but it's possible to reject a patch that doesn't exist yet\nbased solely on description, saving the trouble of writing it. :)\n\n> But I tend to think it would end up rather complex. Off the top of my\n> head, these are the three gotchas I can think of:\n> \n>   1. The reachability rules must be agreed upon between both sides of\n>      the transfer. So you would need a protocol extension for \"please\n>      consider gitlinks reachable\" so that the sender knows to include\n>      them, and that when the client advertises commit X that can reach\n>      Y via gitlink, it can assume that the client does not need Y.\n> \n>   2. There may be some issues with efficiently finding shared history\n>      during a transfer.  That process usually just looks at the commits,\n>      and then we hand the set of have/want commits to the pack\n>      generation code to figure out the full set of required objects (and\n>      in particular, we do not comb through all of the \"have\" commits for\n>      objects that might have become re-referenced; it's too expensive\n>      for too little gain).\n> \n>      But with the flag from (1), we'd have commits buried inside trees.\n>      The packing code would figure out which objects to send, but I\n>      suspect you'd end up with some inefficiencies where an internal\n>      gitlink bumped from commit X to Y, and we _could_ know that the\n>      client already has X, but digging it up doesn't happen with the\n>      current code.\n> \n>      I didn't think it through too hard, though, so perhaps it would all\n>      just work.\n> \n>   3. The reachability is usually an invariant in the repository, but\n>      presumably this would be enabled through a config variable (it\n>      _can't_ be on all the time, because then true external submodules\n>      wouldn't work, and it can't just be \"eh, reachable if you got it\",\n>      because then transfers cannot assume the other side has it and\n>      would resend every time).\n> \n>      So flipping that switch in an existing repository with gitlinks\n>      would probably put it in a broken state until it fetches all of the\n>      gitlinks.\n\nThis point in particular makes me completely disinclined to go down this\nroad.  It could never be mandatory, and having it optional seems\nerror-prone.\n\nThanks for the feedback and suggestions.\n\n- Josh Triplett\n"},{"id":"281328","messageId":"20160321055712.GA21568@sigill.intra.peff.net","threadId":"41757","inReplyTo":"20160320232202.GB20803@x","subject":"Re: Properties of trees referencing commit objects (mode 160000)?","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2016-03-21T05:57:13Z","receivedAt":"2016-03-21T05:57:13Z","isPatch":false,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Sun, Mar 20, 2016 at 04:22:02PM -0700, Josh Triplett wrote:\n\n> I want to track the evolution of a patch series or other commit history,\n> through non-fast-forwarding actions like rebase, rebase -i, or commit\n> --amend.  Similar in spirit to reflog, but with intentional commits and\n> commit messages, and most importantly with the ability to share and\n> collaborate on it.  For each version of the patch series, I plan to\n> track the commit at the end of the series, and optionally the commit at\n> the base of the patch series (to simplify tracking of rebasing); I'll\n> use the tree object associated with the commit to track the cover\n> letter, and perhaps meta-changelog entries associated with v2/v3/vN of\n> the series.\n> \n> Patch series almost always need to evolve through non-fast-forwarding\n> history.  And I've seen that done in two ways: either pull the patch\n> series out of git, put it in quilt or similar, and track the quilt\n> series with git; or pull the versioning of the patch series out of git\n> and track it with branch names like feature-v2, feature-2016-03-20,\n> feature-rebased, and feature-rebased-4.4-fixed-foo-fixed-bar.  That last\n> one closely matches a real-world example I've recently seen.  And that\n> starts to look a lot like the naming of files and documents in\n> organizations that haven't yet discovered the wonders of version\n> control.\n> \n> I'd like to have the best of both worlds: handle the patch series in\n> git, *and* handle the versioning of the patch series in git.\n\nIt seems like you could represent this in git by storing a merge commit\nfor each revision of the patch series, with one parent as the current\nreal tip of the series, and the other as the merge-commit for the\nprevious revision of the series. The tree would be the same as current\ntip commit's tree. You could store metadata about the series in the\nmerge commits (presumably including some marker to say \"I'm a special\nseries-revision commit, not a regular merge\").\n\nThat's roughly the same as having \"feature-v1\", \"feature-v2\", etc, in a\ntree, except that you have to walk down the parent pointers to discover\neach entry, rather than walking the tree.\n\nThe resulting history would be viewable with naive \"git log\", but you\nwould probably want to use \"--first-parent\" to see just the revisions,\nor some to-be-invented option to see just the most recent commits (e.g.,\nsomething like: if the commit subject is \"magic-token: my-series\", then\nskip this commit, show the second parent, and then walk down the first\nparent chain, skipping commits until we hit a commit that doesn't have\n\"magic-token: my-series\" in it).\n\nAnd it would just work for transferring revisions. You'd probably have\none ref per patch series (pointing to the marker commit of the most\nrecent version), and the \"real\" clean history of the project would never\ninclude any of the revision markers at all (though it could if you\nreally wanted to).\n\nBut I will be the first to admit that I haven't thought too hard on\nthis. It may even be that I just unconsciously regurgitated somebody\nelse's storage scheme. Once upon a time I was interesting in \"topgit\",\n\"guilt\", and other tools, but I haven't looked at them in years.\n\nSo it may well be that somebody tried something like this, and already\ndiscovered its downsides. :)\n\n-Peff\n"},{"id":"281345","messageId":"20160321153626.GA2180@jtriplet-mobl2.jf.intel.com","threadId":"41757","inReplyTo":"20160321055712.GA21568@sigill.intra.peff.net","subject":"Re: Properties of trees referencing commit objects (mode 160000)?","fromName":"Josh Triplett","fromEmail":"josh@joshtriplett.org","sentAt":"2016-03-21T15:36:27Z","receivedAt":"2016-03-21T15:36:27Z","isPatch":false,"sender":{"key":"josh@joshtriplett.org","avatar":"https://avatars.githubusercontent.com/u/162737?v=4"},"body":"On Mon, Mar 21, 2016 at 01:57:13AM -0400, Jeff King wrote:\n> On Sun, Mar 20, 2016 at 04:22:02PM -0700, Josh Triplett wrote:\n> \n> > I want to track the evolution of a patch series or other commit history,\n> > through non-fast-forwarding actions like rebase, rebase -i, or commit\n> > --amend.  Similar in spirit to reflog, but with intentional commits and\n> > commit messages, and most importantly with the ability to share and\n> > collaborate on it.  For each version of the patch series, I plan to\n> > track the commit at the end of the series, and optionally the commit at\n> > the base of the patch series (to simplify tracking of rebasing); I'll\n> > use the tree object associated with the commit to track the cover\n> > letter, and perhaps meta-changelog entries associated with v2/v3/vN of\n> > the series.\n> > \n> > Patch series almost always need to evolve through non-fast-forwarding\n> > history.  And I've seen that done in two ways: either pull the patch\n> > series out of git, put it in quilt or similar, and track the quilt\n> > series with git; or pull the versioning of the patch series out of git\n> > and track it with branch names like feature-v2, feature-2016-03-20,\n> > feature-rebased, and feature-rebased-4.4-fixed-foo-fixed-bar.  That last\n> > one closely matches a real-world example I've recently seen.  And that\n> > starts to look a lot like the naming of files and documents in\n> > organizations that haven't yet discovered the wonders of version\n> > control.\n> > \n> > I'd like to have the best of both worlds: handle the patch series in\n> > git, *and* handle the versioning of the patch series in git.\n> \n> It seems like you could represent this in git by storing a merge commit\n> for each revision of the patch series, with one parent as the current\n> real tip of the series, and the other as the merge-commit for the\n> previous revision of the series. The tree would be the same as current\n> tip commit's tree. You could store metadata about the series in the\n> merge commits (presumably including some marker to say \"I'm a special\n> series-revision commit, not a regular merge\").\n\nThat's exactly what I'm planning to do now, ever since your initial\nresponse said submodules wouldn't work.\n\n> That's roughly the same as having \"feature-v1\", \"feature-v2\", etc, in a\n> tree, except that you have to walk down the parent pointers to discover\n> each entry, rather than walking the tree.\n> \n> The resulting history would be viewable with naive \"git log\", but you\n> would probably want to use \"--first-parent\" to see just the revisions,\n> or some to-be-invented option to see just the most recent commits (e.g.,\n> something like: if the commit subject is \"magic-token: my-series\", then\n> skip this commit, show the second parent, and then walk down the first\n> parent chain, skipping commits until we hit a commit that doesn't have\n> \"magic-token: my-series\" in it).\n\nRather than relying on the numeric index of the commit parents for\nsemantic value, I planned to use the tree object.  I can still use\ngitlinks for named references to commits from within the series, as long\nas the commits linked from those trees have references via parents to\nkeep them alive.  For instance, /series within the commit can refer to\nthe commit object at the top of the series, /base (if present) can refer\nto the base of the series (e.g. v4.4), and /cover (a blob) can contain\nthe cover letter.  Then \"git series format\" automatically knows the base\nto start from, and \"git series log\" or \"git series diff\" will know the\ndifference between \"reordered patches\" and \"rebased on a new base\".\n\nThe working copy and .git/HEAD will point to the last commit in the\ncurrent version of the patch series, so that tools like \"git rebase -i\"\nand similar can Just Work; \"git series\" will manage the separate ref\nindependently.\n\nI'll need to provide a variety of additional tools here for showing what\nhas changed in a patch series; I'd eventually like to support a sensible\n\"git series diff\" that can show things like \"rebased on this base,\nreordered three patches, dropped one, edited another, and changed the\ncover letter\".\n\n> And it would just work for transferring revisions. You'd probably have\n> one ref per patch series (pointing to the marker commit of the most\n> recent version), and the \"real\" clean history of the project would never\n> include any of the revision markers at all (though it could if you\n> really wanted to).\n\nExactly the plan.  And you won't ever actually check out those refs into\nyour working copy; you'll check out the patch series commits they\nreference instead.\n\n> But I will be the first to admit that I haven't thought too hard on\n> this. It may even be that I just unconsciously regurgitated somebody\n> else's storage scheme. Once upon a time I was interesting in \"topgit\",\n> \"guilt\", and other tools, but I haven't looked at them in years.\n> \n> So it may well be that somebody tried something like this, and already\n> discovered its downsides. :)\n\nOne major downside is that creating such a tool doesn't automatically\nmake things like a three-way merge of patch series easy.  It makes\nrecognizing the situation *possible* (unlike before when you couldn't\neasily exchange records of non-fast-forwarding history with others), but\nif your remote and you have both changed the patch series independently,\nmerging the two will still prove challenging.\n\nOne step at a time, though: first I want to make it possible to track\nas a first-class concept, and then we can make better tooling on top\nof that.\n\n- Josh Triplett\n"}]}