{"thread":{"id":"22738","subject":"Storing (hidden) per-commit metadata","startedAt":"2010-02-19T17:11:25Z","lastAt":"2010-02-22T22:13:34Z","messageCount":19,"participants":["Jelmer Vernooij","Ben Gamari","Avery Pennarun","Jeff King","Johannes Schindelin","Gabriel Filion","Dmitry Potapov","Alejandro R. Sedeño"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"135155","messageId":"1266599485.29753.54.camel@ganieda","threadId":"22738","inReplyTo":null,"subject":"Storing (hidden) per-commit metadata","fromName":"Jelmer Vernooij","fromEmail":"jelmer@samba.org","sentAt":"2010-02-19T17:11:25Z","receivedAt":"2010-02-19T17:11:25Z","isPatch":false,"sender":{"key":"jelmer@samba.org","avatar":"https://avatars.githubusercontent.com/u/49032?v=4"},"body":"To allow round-tripping pushes from Bazaar into Git, I'm looking for a\ngood place to store Bazaar semantics that can not be represented in Git\nat the moment. This data should ideally be hidden from the user as much\nas possible; it would e.g. contain mappings from git hashes to Bazaar\nids. \n\nOne option would be to store it (as hg-git does) at the bottom of each\ngit commit message. However, given the amount of data and the its kind,\nit would be annoying to have it displayed by e.g. \"git show\" or \"git\nlog\".\n\nSome people have suggested I use the new git notes to store this\nmetadata, but I haven't quite figured out how to add notes that aren't\ndisplayed by git log/show and are still propagated along with the\nrevision. Is that at all possible using notes, and are they the right\nthing to use here?\n\nThere also doesn't appear to be any documentation on notes in\nDocumentation/technical at the moment. I'm happy to contribute some if\nsomebody can provide pointers.\n\nCheers,\n\nJelmer\n"},{"id":"135168","messageId":"1266687636-sup-7641@ben-laptop","threadId":"22738","inReplyTo":"1266599485.29753.54.camel@ganieda","subject":"Re: Storing (hidden) per-commit metadata","fromName":"Ben Gamari","fromEmail":"bgamari@gmail.com","sentAt":"2010-02-20T17:41:36Z","receivedAt":"2010-02-20T17:41:36Z","isPatch":false,"sender":{"key":"bgamari@gmail.com","avatar":null},"body":"Excerpts from Jelmer Vernooij's message of Fri Feb 19 12:11:25 -0500 2010:\n> To allow round-tripping pushes from Bazaar into Git, I'm looking for a\n> good place to store Bazaar semantics that can not be represented in Git\n> at the moment. This data should ideally be hidden from the user as much\n> as possible; it would e.g. contain mappings from git hashes to Bazaar\n> ids. \n> \nAre you sure you want to hide this? I believe git-svn puts this\ninformation in its commit messages (although I don't know whether it's\nstored elsewhere as well).\n\n- Ben\n"},{"id":"135175","messageId":"32541b131002201057t31fc8a6aydb0942171fe1b8c8@mail.gmail.com","threadId":"22738","inReplyTo":"1266687636-sup-7641@ben-laptop","subject":"Re: Storing (hidden) per-commit metadata","fromName":"Avery Pennarun","fromEmail":"apenwarr@gmail.com","sentAt":"2010-02-20T18:57:31Z","receivedAt":"2010-02-20T18:57:31Z","isPatch":false,"sender":{"key":"apenwarr@gmail.com","avatar":"https://avatars.githubusercontent.com/u/20592?v=4"},"body":"On Sat, Feb 20, 2010 at 12:41 PM, Ben Gamari <bgamari@gmail.com> wrote:\n> Excerpts from Jelmer Vernooij's message of Fri Feb 19 12:11:25 -0500 2010:\n>> To allow round-tripping pushes from Bazaar into Git, I'm looking for a\n>> good place to store Bazaar semantics that can not be represented in Git\n>> at the moment. This data should ideally be hidden from the user as much\n>> as possible; it would e.g. contain mappings from git hashes to Bazaar\n>> ids.\n>>\n> Are you sure you want to hide this? I believe git-svn puts this\n> information in its commit messages (although I don't know whether it's\n> stored elsewhere as well).\n\nNote that git-svn doesn't store *all* the stuff from svn in the git\nrepository.  So you couldn't, for example, regenerate an svn repo\nidentical to the original from its git-svn clone.  This limitation is\nrarely noticed since the stuff git-svn doesn't store is stuff that git\nmostly does differently/automatically/etc.  But that's why git-svn can\nget away with cluttering your commit messages with \"only\" one line of\ngit-svn cruft each.\n\nHowever, this does bring up the question: how important is it *really*\nto be able to \"round trip\" from bzr to git and back without losing\ninformation?  Maybe you only need to store enough information to pull\nfrom bzr and then push back your commits.\n\nAs for git-notes, they sound like they would be useful for this sort\nof thing.  I haven't tried them yet, but my understanding is that\nnotes anywhere other than the \"default\" notes ref are not shown in\ncommit messages, so you can use them for whatever you want.\n\nHave fun,\n\nAvery\n"},{"id":"135226","messageId":"20100221063433.GA2840@coredump.intra.peff.net","threadId":"22738","inReplyTo":"32541b131002201057t31fc8a6aydb0942171fe1b8c8@mail.gmail.com","subject":"Re: Storing (hidden) per-commit metadata","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2010-02-21T06:34:33Z","receivedAt":"2010-02-21T06:34:33Z","isPatch":false,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Sat, Feb 20, 2010 at 01:57:31PM -0500, Avery Pennarun wrote:\n\n> As for git-notes, they sound like they would be useful for this sort\n> of thing.  I haven't tried them yet, but my understanding is that\n> notes anywhere other than the \"default\" notes ref are not shown in\n> commit messages, so you can use them for whatever you want.\n\nI would want to hear more about the actual data being stored. The\nstrength of notes is that you can _change_ them after the commit has\nbeen created. And the price you pay is that they are more annoying to\nmove around, because they are in a totally different ref.\n\nIf this is data that is being generated at the time the commit is\ncreated and then set in stone, then it probably should be part of the\ncommit object.\n\nIf the only problem is that the data is ugly in \"git show\", then perhaps\nwe need a \"suppress these pseudo-headers\" feature for showing logs. It\nkeeps them easily available for inspection or for --grep, but most of\nthe time you would not see them.\n\n-Peff\n"},{"id":"135233","messageId":"alpine.DEB.1.00.1002210945490.20986@pacific.mpi-cbg.de","threadId":"22738","inReplyTo":"20100221063433.GA2840@coredump.intra.peff.net","subject":"Re: Storing (hidden) per-commit metadata","fromName":"Johannes Schindelin","fromEmail":"johannes.schindelin@gmx.de","sentAt":"2010-02-21T08:49:28Z","receivedAt":"2010-02-21T08:49:28Z","isPatch":false,"sender":{"key":"johannes.schindelin@gmx.de","avatar":"https://avatars.githubusercontent.com/u/127790?v=4"},"body":"Hi,\n\nOn Sun, 21 Feb 2010, Jeff King wrote:\n\n> If the only problem is that the data is ugly in \"git show\", then perhaps \n> we need a \"suppress these pseudo-headers\" feature for showing logs. It \n> keeps them easily available for inspection or for --grep, but most of \n> the time you would not see them.\n\nWhoa. Even more processing to do for each commit during a \"git log\" run? \nYou know, other people are working on _accelerating_ git log as we speak!\n\nAnd really, while I can understand that the OP wanted to hide the \ninformation, I am really against that. For example, when I see a log with \ngit-svn footers, it gives me _additional_ information which I actually \nlike (it tells me where these commits really come from). If they do not \nneed bidirectional, they can skip those footers.\n\nBut I do agree that it is better to put the information into the same \nobjects rather than notes, lest the information get out-of-sync.\n\nCiao,\nDscho\n"},{"id":"135230","messageId":"20100221085237.GA31189@coredump.intra.peff.net","threadId":"22738","inReplyTo":"alpine.DEB.1.00.1002210945490.20986@pacific.mpi-cbg.de","subject":"Re: Storing (hidden) per-commit metadata","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2010-02-21T08:52:37Z","receivedAt":"2010-02-21T08:52:37Z","isPatch":false,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Sun, Feb 21, 2010 at 09:49:28AM +0100, Johannes Schindelin wrote:\n\n> > If the only problem is that the data is ugly in \"git show\", then perhaps \n> > we need a \"suppress these pseudo-headers\" feature for showing logs. It \n> > keeps them easily available for inspection or for --grep, but most of \n> > the time you would not see them.\n> \n> Whoa. Even more processing to do for each commit during a \"git log\" run? \n> You know, other people are working on _accelerating_ git log as we speak!\n\nI think this is premature.  You would not need to pay the price for such\na feature if you were not actually using it. On top of which, as it does\nnot yet exist, it has not actually been benchmarked. So any complaints\nshould wait until it is actually implemented.\n\n> And really, while I can understand that the OP wanted to hide the \n> information, I am really against that. For example, when I see a log with \n> git-svn footers, it gives me _additional_ information which I actually \n> like (it tells me where these commits really come from). If they do not \n> need bidirectional, they can skip those footers.\n\nI think it depends on what the information is, which I'm still not clear\non. But most importantly, I think it makes sense to put control of\nwhether that information is seen in the hands of the user who is\ninvoking git.\n\n-Peff\n\nPS I tried to keep this message short. Short enough? :)\n"},{"id":"135239","messageId":"1266754646.12035.23.camel@ganieda","threadId":"22738","inReplyTo":"20100221063433.GA2840@coredump.intra.peff.net","subject":"Re: Storing (hidden) per-commit metadata","fromName":"Jelmer Vernooij","fromEmail":"jelmer@samba.org","sentAt":"2010-02-21T12:17:26Z","receivedAt":"2010-02-21T12:17:26Z","isPatch":false,"sender":{"key":"jelmer@samba.org","avatar":"https://avatars.githubusercontent.com/u/49032?v=4"},"body":"On Sun, 2010-02-21 at 01:34 -0500, Jeff King wrote:\n> On Sat, Feb 20, 2010 at 01:57:31PM -0500, Avery Pennarun wrote:\n> > As for git-notes, they sound like they would be useful for this sort\n> > of thing.  I haven't tried them yet, but my understanding is that\n> > notes anywhere other than the \"default\" notes ref are not shown in\n> > commit messages, so you can use them for whatever you want.\n> I would want to hear more about the actual data being stored. The\n> strength of notes is that you can _change_ them after the commit has\n> been created. And the price you pay is that they are more annoying to\n> move around, because they are in a totally different ref.\n> \n> If this is data that is being generated at the time the commit is\n> created and then set in stone, then it probably should be part of the\n> commit object.\nThis data is supposed to be set in stone, since Bazaar revisions are\nintended to be immutable, like Git commits are.\n\nFor each file we would need to store:\n\n * the Bazaar revision id\n * any Bazaar revision properties. This is typically a list of URLs of\nbugs that were fixed, name of the branch the commit was on, any\nadditional parents, or anything arbitrary set by plugins (e.g. the\nrebase plugin sets 'rebase-of' to the id of the original revision)\n * For each file that was added or moved around in the revision, a path\nto fileid mapping\n * Optionally, a list of ghost parent ids and \"unusual\" revisions for\neach file but these should be rare.\n\nThis is at least a couple of lines of data and in some cases a lot more.\nI would rather avoid confronting git users who don't care about Bazaar\nwith it.\n\n> If the only problem is that the data is ugly in \"git show\", then perhaps\n> we need a \"suppress these pseudo-headers\" feature for showing logs. It\n> keeps them easily available for inspection or for --grep, but most of\n> the time you would not see them.\nThat seems like a sensible thing to do, and would work well for me.\n\nCheers,\n\nJelmer\n"},{"id":"135235","messageId":"4B821202.8070700@gmail.com","threadId":"22738","inReplyTo":"1266599485.29753.54.camel@ganieda","subject":"Re: Storing (hidden) per-commit metadata","fromName":"Gabriel Filion","fromEmail":"lelutin@gmail.com","sentAt":"2010-02-22T05:11:30Z","receivedAt":"2010-02-22T05:11:30Z","isPatch":false,"sender":{"key":"lelutin@gmail.com","avatar":"https://avatars.githubusercontent.com/u/108728?v=4"},"body":"Hello,\n\nOn 2010-02-19 12:11, Jelmer Vernooij wrote:\n> To allow round-tripping pushes from Bazaar into Git, I'm looking for a\n> good place to store Bazaar semantics that can not be represented in Git\n> at the moment. This data should ideally be hidden from the user as much\n> as possible; it would e.g. contain mappings from git hashes to Bazaar\n> ids. \n> \nWhat are you currently using for interacting with Bazaar repositories?\n\nIf you already have code for a remote helper, I would be interested in\nhelping you out. I started a discussion recently on this list about\nstaring such a script. Would you be willing to collaborate on having\nthis implemented?\n\n-- \nGabriel Filion\n"},{"id":"135240","messageId":"20100222051748.GB10191@dpotapov.dyndns.org","threadId":"22738","inReplyTo":"1266754646.12035.23.camel@ganieda","subject":"Re: Storing (hidden) per-commit metadata","fromName":"Dmitry Potapov","fromEmail":"dpotapov@gmail.com","sentAt":"2010-02-22T05:17:48Z","receivedAt":"2010-02-22T05:17:48Z","isPatch":false,"sender":{"key":"dpotapov@gmail.com","avatar":"https://avatars.githubusercontent.com/u/6568595?v=4"},"body":"On Sun, Feb 21, 2010 at 01:17:26PM +0100, Jelmer Vernooij wrote:\n> \n> For each file we would need to store:\n> \n>  * the Bazaar revision id\n>  * any Bazaar revision properties. This is typically a list of URLs of\n> bugs that were fixed, name of the branch the commit was on, any\n> additional parents, or anything arbitrary set by plugins (e.g. the\n> rebase plugin sets 'rebase-of' to the id of the original revision)\n>  * For each file that was added or moved around in the revision, a path\n> to fileid mapping\n>  * Optionally, a list of ghost parent ids and \"unusual\" revisions for\n> each file but these should be rare.\n> \n> This is at least a couple of lines of data and in some cases a lot more.\n> I would rather avoid confronting git users who don't care about Bazaar\n> with it.\n\nThe problem with storying this meta data in the commit object is that\nany newly created commits in Git will not have this information, and you\nprobably have to add it later when you export these commits to Bazaar,\nwhich means that the history in Git should be re-written, and Git users\nwill have to rebase their branches from one commit to another that are\nidentical except this Bazaar-specific information, which you try to hide\nfrom them. So much for don't care about Bazaar!\n\nIn other words, no matter what git-log displays, as long as you put this\nmeta data wherever it changes commit-id, it is visible to Git users, and\ntrying to hide this fact is utterly stupid.\n\nThere are many ways to store Bazaar data in Git without confronting git\nusers who don't care about Bazaar with it. For instance, you can create\na separate branch that will hold this meta data.\n\n   master      bzr/master\n\n      /---------o\n     o          |\n     |          |\n     |/---------o\n     o          |\n     |          |\n\nCommits on bzr/master are fast-forward merges that have the same tree-id\nas corresponding commits on master, but the commit message contains\nBazaar specific information. So, if someone does not care about Bazaar,\nthis is a throw away branch for him. Also, there is no problem to add\nBazaar specific information to any git commit later when it is pushed\nto Bazaar. The only problem is if you try to rebase commits that were\npushed to Bazaar, but AFAIK Bazaar does not support overwriting history,\nso you cannot expect anything good of this attempt anyway. The published\nhistory should not rebased.\n\n\nDmitry\n"},{"id":"135284","messageId":"1266832145.31769.29.camel@ganieda","threadId":"22738","inReplyTo":"4B821202.8070700@gmail.com","subject":"Re: Storing (hidden) per-commit metadata","fromName":"Jelmer Vernooij","fromEmail":"jelmer@samba.org","sentAt":"2010-02-22T09:49:05Z","receivedAt":"2010-02-22T09:49:05Z","isPatch":false,"sender":{"key":"jelmer@samba.org","avatar":"https://avatars.githubusercontent.com/u/49032?v=4"},"body":"On Mon, 2010-02-22 at 00:11 -0500, Gabriel Filion wrote:\n> On 2010-02-19 12:11, Jelmer Vernooij wrote:\n> > To allow round-tripping pushes from Bazaar into Git, I'm looking for a\n> > good place to store Bazaar semantics that can not be represented in Git\n> > at the moment. This data should ideally be hidden from the user as much\n> > as possible; it would e.g. contain mappings from git hashes to Bazaar\n> > ids. \n> What are you currently using for interacting with Bazaar repositories?\nIt's the other way around :-) I'm the developer of the bzr-git plugin,\nwhich allows Bazaar to be used with Git repositories. \n\n> If you already have code for a remote helper, I would be interested in\n> helping you out. I started a discussion recently on this list about\n> staring such a script. Would you be willing to collaborate on having\n> this implemented?\nI'm happy to discuss a remote helper for Bazaar in Git, but not\nparticularly interested in contributing.\n\nCheers,\n\nJelmer\n"},{"id":"135293","messageId":"1266832607.31769.37.camel@ganieda","threadId":"22738","inReplyTo":"20100222051748.GB10191@dpotapov.dyndns.org","subject":"Re: Storing (hidden) per-commit metadata","fromName":"Jelmer Vernooij","fromEmail":"jelmer@samba.org","sentAt":"2010-02-22T09:56:47Z","receivedAt":"2010-02-22T09:56:47Z","isPatch":false,"sender":{"key":"jelmer@samba.org","avatar":"https://avatars.githubusercontent.com/u/49032?v=4"},"body":"On Mon, 2010-02-22 at 08:17 +0300, Dmitry Potapov wrote:\n> On Sun, Feb 21, 2010 at 01:17:26PM +0100, Jelmer Vernooij wrote:\n> > For each file we would need to store:\n> > \n> >  * the Bazaar revision id\n> >  * any Bazaar revision properties. This is typically a list of URLs of\n> > bugs that were fixed, name of the branch the commit was on, any\n> > additional parents, or anything arbitrary set by plugins (e.g. the\n> > rebase plugin sets 'rebase-of' to the id of the original revision)\n> >  * For each file that was added or moved around in the revision, a path\n> > to fileid mapping\n> >  * Optionally, a list of ghost parent ids and \"unusual\" revisions for\n> > each file but these should be rare.\n> > \n> > This is at least a couple of lines of data and in some cases a lot more.\n> > I would rather avoid confronting git users who don't care about Bazaar\n> > with it.\n> The problem with storying this meta data in the commit object is that\n> any newly created commits in Git will not have this information, and you\n> probably have to add it later when you export these commits to Bazaar,\n> which means that the history in Git should be re-written, and Git users\n> will have to rebase their branches from one commit to another that are\n> identical except this Bazaar-specific information, which you try to hide\n> from them. So much for don't care about Bazaar!\n\n> In other words, no matter what git-log displays, as long as you put this\n> meta data wherever it changes commit-id, it is visible to Git users, and\n> trying to hide this fact is utterly stupid.\n> \n> There are many ways to store Bazaar data in Git without confronting git\n> users who don't care about Bazaar with it. For instance, you can create\n> a separate branch that will hold this meta data.\n> \n>    master      bzr/master\n> \n>       /---------o\n>      o          |\n>      |          |\n>      |/---------o\n>      o          |\n>      |          |\n> \n> Commits on bzr/master are fast-forward merges that have the same tree-id\n> as corresponding commits on master, but the commit message contains\n> Bazaar specific information. So, if someone does not care about Bazaar,\n> this is a throw away branch for him. Also, there is no problem to add\n> Bazaar specific information to any git commit later when it is pushed\n> to Bazaar. The only problem is if you try to rebase commits that were\n> pushed to Bazaar, but AFAIK Bazaar does not support overwriting history,\n> so you cannot expect anything good of this attempt anyway. The published\n> history should not rebased.\n\nThere is no need for that data to be added later for revisions that did\nnot originate from Bazaar. All of the metadata that has to be stored\nwill be known at the time the commit is created. Those commits that were\nmade in Git later will not have any metadata that can not be represented\nin Git (they were made with Git, after all). There is no need for\nrebasing/overwriting history for existing revisions to enable access by\nBazaar.\n\nHaving a bzr/master ref means that the extra metadata will not always be\ncopied around (unless git is patched), so if I push my work from Bazaar\ninto Git, somebody works on it in Git and pushes a derived branch and\nthen somebody else clones that derived Git branch into Bazaar again, I\nwill not be able to communicate with that person's branch.\n\nCheers,\n\nJelmer\n"},{"id":"135302","messageId":"20100222112845.GE10191@dpotapov.dyndns.org","threadId":"22738","inReplyTo":"1266832607.31769.37.camel@ganieda","subject":"Re: Storing (hidden) per-commit metadata","fromName":"Dmitry Potapov","fromEmail":"dpotapov@gmail.com","sentAt":"2010-02-22T11:28:45Z","receivedAt":"2010-02-22T11:28:45Z","isPatch":false,"sender":{"key":"dpotapov@gmail.com","avatar":"https://avatars.githubusercontent.com/u/6568595?v=4"},"body":"On Mon, Feb 22, 2010 at 10:56:47AM +0100, Jelmer Vernooij wrote:\n> \n> There is no need for that data to be added later for revisions that did\n> not originate from Bazaar. All of the metadata that has to be stored\n> will be known at the time the commit is created. Those commits that were\n> made in Git later will not have any metadata that can not be represented\n> in Git (they were made with Git, after all).\n\nIf so, I do not see why any metadata should be stored in Git at all. If\nyou can work without them then why do you want to add to Git? And then\nhow about commit that originated in Git then exported to Bazaar and then\nimported back at Git? It is still originated in Git and thus should not\nhave any metadata despite being imported from Bazaar.\n\n> Having a bzr/master ref means that the extra metadata will not always be\n> copied around (unless git is patched), so if I push my work from Bazaar\n> into Git, somebody works on it in Git and pushes a derived branch and\n> then somebody else clones that derived Git branch into Bazaar again, I\n> will not be able to communicate with that person's branch.\n\nNo matter how many times a branch was cloned, it is exactly same branch\n(i.e. it consists of commits having exactly the same id). So, if you can\nwork with the original branch, you can work with any cloned branch. So,\nI see no need to copy this data around for people who do not work with\nBazaar directly.\n\n\nDmitry\n"},{"id":"135306","messageId":"1266839972.4575.38.camel@ganieda","threadId":"22738","inReplyTo":"20100222112845.GE10191@dpotapov.dyndns.org","subject":"Re: Storing (hidden) per-commit metadata","fromName":"Jelmer Vernooij","fromEmail":"jelmer@samba.org","sentAt":"2010-02-22T11:59:32Z","receivedAt":"2010-02-22T11:59:32Z","isPatch":false,"sender":{"key":"jelmer@samba.org","avatar":"https://avatars.githubusercontent.com/u/49032?v=4"},"body":"On Mon, 2010-02-22 at 14:28 +0300, Dmitry Potapov wrote:\n> On Mon, Feb 22, 2010 at 10:56:47AM +0100, Jelmer Vernooij wrote:\n> > \n> > There is no need for that data to be added later for revisions that did\n> > not originate from Bazaar. All of the metadata that has to be stored\n> > will be known at the time the commit is created. Those commits that were\n> > made in Git later will not have any metadata that can not be represented\n> > in Git (they were made with Git, after all).\n> If so, I do not see why any metadata should be stored in Git at all. If\n> you can work without them then why do you want to add to Git? And then\n> how about commit that originated in Git then exported to Bazaar and then\n> imported back at Git? It is still originated in Git and thus should not\n> have any metadata despite being imported from Bazaar.\nCommits that originated in Git do not contain any Bazaar-specific\nmetadata, even if they also lived in Bazaar at some point, because they\ncould not have been set by Git at commit time. \n\nWe would only add the metadata for revisions that did not come out of\nGit originally. \n\nWe'd like to have the extra metadata in Git so that we can push Bazaar\ncommits into a Git repository losslessly. If we can't do this losslessly\nthen the identity of the commit changes just like it does in git if you\naren't able to produce the same tree, blob and commit objects.\n\n> > Having a bzr/master ref means that the extra metadata will not always be\n> > copied around (unless git is patched), so if I push my work from Bazaar\n> > into Git, somebody works on it in Git and pushes a derived branch and\n> > then somebody else clones that derived Git branch into Bazaar again, I\n> > will not be able to communicate with that person's branch.\n> No matter how many times a branch was cloned, it is exactly same branch\n> (i.e. it consists of commits having exactly the same id). So, if you can\n> work with the original branch, you can work with any cloned branch. So,\n> I see no need to copy this data around for people who do not work with\n> Bazaar directly.\nThe original branch is a Bazaar branch here, so that's not true. You can\nonly work with any cloned branch if the matching bzr/ branch is also\naround. If it isn't then you won't be able to find the original commit. \n\nhg-git already does something similar by putting a --HG-- line followed\nby hg-git specific metadata in the commit message when it pushes into\nGit. I'd like to find a place to put this data that's not as intruisive\nfor users.\n\nCheers,\n\nJelmer\n"},{"id":"135307","messageId":"20100222130836.GG10191@dpotapov.dyndns.org","threadId":"22738","inReplyTo":"1266839972.4575.38.camel@ganieda","subject":"Re: Storing (hidden) per-commit metadata","fromName":"Dmitry Potapov","fromEmail":"dpotapov@gmail.com","sentAt":"2010-02-22T13:08:36Z","receivedAt":"2010-02-22T13:08:36Z","isPatch":false,"sender":{"key":"dpotapov@gmail.com","avatar":"https://avatars.githubusercontent.com/u/6568595?v=4"},"body":"On Mon, Feb 22, 2010 at 12:59:32PM +0100, Jelmer Vernooij wrote:\n> \n> We'd like to have the extra metadata in Git so that we can push Bazaar\n> commits into a Git repository losslessly. If we can't do this losslessly\n> then the identity of the commit changes just like it does in git if you\n> aren't able to produce the same tree, blob and commit objects.\n\nbut the problem is that you may want to add some information when you\nimport some Git to Bazaar. For instance, Git does not record file\nrenames explicitly and relies on content of files to detect renames\nautomatically. So, when I use gitk, I can see that what file is renamed.\nIf you work in Bazaar, you probably also want to see renames, but this\nrequires that you add this information when you import commits to\nBazaaar. But if you do that, the export to Git will produce a different\ncommit just because you added this Bazaar-specific data.\n\n> \n> > > Having a bzr/master ref means that the extra metadata will not always be\n> > > copied around (unless git is patched), so if I push my work from Bazaar\n> > > into Git, somebody works on it in Git and pushes a derived branch and\n> > > then somebody else clones that derived Git branch into Bazaar again, I\n> > > will not be able to communicate with that person's branch.\n> > No matter how many times a branch was cloned, it is exactly same branch\n> > (i.e. it consists of commits having exactly the same id). So, if you can\n> > work with the original branch, you can work with any cloned branch. So,\n> > I see no need to copy this data around for people who do not work with\n> > Bazaar directly.\n> The original branch is a Bazaar branch here, so that's not true. You can\n> only work with any cloned branch if the matching bzr/ branch is also\n> around. If it isn't then you won't be able to find the original commit. \n\nObviously bzr/ branch should be around somewhere, but it does not have\nto be in any cloned repo. It is sufficient to have it in one place,\nbecause it refers to commit-id, which does not change when you clone it.\n\n> \n> hg-git already does something similar by putting a --HG-- line followed\n> by hg-git specific metadata in the commit message when it pushes into\n> Git. I'd like to find a place to put this data that's not as intruisive\n> for users.\n\nI still think it is wrong to hide some information in the commit object.\nI am not sure that the commit object is the right place to store that\nmetadata, but hidding this information is even more problematic. Let's\nsuppose that someone cherry-pick your Bazaar originated commit. Now when\nyou try to synchronize with Bazaar, your synchronizer will see that it\nhas some Bazaar revision ID and branch name, but, in fact, it is new\ncommit on a completely different branch...\n\n\nDmitry\n"},{"id":"135311","messageId":"1266846289.4575.69.camel@ganieda","threadId":"22738","inReplyTo":"20100222130836.GG10191@dpotapov.dyndns.org","subject":"Re: Storing (hidden) per-commit metadata","fromName":"Jelmer Vernooij","fromEmail":"jelmer@samba.org","sentAt":"2010-02-22T13:44:49Z","receivedAt":"2010-02-22T13:44:49Z","isPatch":false,"sender":{"key":"jelmer@samba.org","avatar":"https://avatars.githubusercontent.com/u/49032?v=4"},"body":"On Mon, 2010-02-22 at 16:08 +0300, Dmitry Potapov wrote:\n> On Mon, Feb 22, 2010 at 12:59:32PM +0100, Jelmer Vernooij wrote:\n> > We'd like to have the extra metadata in Git so that we can push Bazaar\n> > commits into a Git repository losslessly. If we can't do this losslessly\n> > then the identity of the commit changes just like it does in git if you\n> > aren't able to produce the same tree, blob and commit objects.\n> but the problem is that you may want to add some information when you\n> import some Git to Bazaar. For instance, Git does not record file\n> renames explicitly and relies on content of files to detect renames\n> automatically. So, when I use gitk, I can see that what file is renamed.\n> If you work in Bazaar, you probably also want to see renames, but this\n> requires that you add this information when you import commits to\n> Bazaaar. But if you do that, the export to Git will produce a different\n> commit just because you added this Bazaar-specific data.\nWe can already do the other way around - Bazaar allows storing arbitrary\nrevision properties, so we use that to some things that can not be\nrepresented in Bazaar but exist in Git. An example of this are  the\nunusual file modes created by older versions of git or non-utf8 commit\nmessages. Those extra revision properties are set at the moment that the\nBazaar revision is imported into Git, not afterwards and there is no\nneed to update them later.\n\nThe fact that we have this extra metadata allows us to reproduce the\noriginal Git commit bit for bit so we can actually extract the same\nrevision that went in, with the same git sha1.\n\n> > > > Having a bzr/master ref means that the extra metadata will not always be\n> > > > copied around (unless git is patched), so if I push my work from Bazaar\n> > > > into Git, somebody works on it in Git and pushes a derived branch and\n> > > > then somebody else clones that derived Git branch into Bazaar again, I\n> > > > will not be able to communicate with that person's branch.\n> > > No matter how many times a branch was cloned, it is exactly same branch\n> > > (i.e. it consists of commits having exactly the same id). So, if you can\n> > > work with the original branch, you can work with any cloned branch. So,\n> > > I see no need to copy this data around for people who do not work with\n> > > Bazaar directly.\n> > The original branch is a Bazaar branch here, so that's not true. You can\n> > only work with any cloned branch if the matching bzr/ branch is also\n> > around. If it isn't then you won't be able to find the original commit. \n> Obviously bzr/ branch should be around somewhere, but it does not have\n> to be in any cloned repo. It is sufficient to have it in one place,\n> because it refers to commit-id, which does not change when you clone it.\nIf some other Bazaar user clones that repo, they end up without the\nBazaar specific metadata and thus with different Bazaar commits. If they\nthen try to communicate with the Bazaar user that pushed the revisions\nin, their histories appear unrelated.\n\n> > hg-git already does something similar by putting a --HG-- line followed\n> > by hg-git specific metadata in the commit message when it pushes into\n> > Git. I'd like to find a place to put this data that's not as intruisive\n> > for users.\n> I still think it is wrong to hide some information in the commit object.\nWhat exactly is the problem with doing so? \"encoding\" is already there\nand as far as I can tell not displayed directly to the user.\n\n> I am not sure that the commit object is the right place to store that\n> metadata, but hidding this information is even more problematic. Let's\n> suppose that someone cherry-pick your Bazaar originated commit. Now when\n> you try to synchronize with Bazaar, your synchronizer will see that it\n> has some Bazaar revision ID and branch name, but, in fact, it is new\n> commit on a completely different branch...\nI don't see how the fact that the bzr-git/hg-git data is being hidden is\nthe problem in the scenario you mention.\n\nIt'd be nice if this sort of information was discarded by \"git rebase\",\nbut that's another good reason to treat it in a different way from the\ncommit message instead.\n\nCheers,\n\nJelmer\n"},{"id":"135312","messageId":"20100222142013.GA7863@dpotapov.dyndns.org","threadId":"22738","inReplyTo":"1266846289.4575.69.camel@ganieda","subject":"Re: Storing (hidden) per-commit metadata","fromName":"Dmitry Potapov","fromEmail":"dpotapov@gmail.com","sentAt":"2010-02-22T14:20:13Z","receivedAt":"2010-02-22T14:20:13Z","isPatch":false,"sender":{"key":"dpotapov@gmail.com","avatar":"https://avatars.githubusercontent.com/u/6568595?v=4"},"body":"On Mon, Feb 22, 2010 at 02:44:49PM +0100, Jelmer Vernooij wrote:\n> On Mon, 2010-02-22 at 16:08 +0300, Dmitry Potapov wrote:\n> > I am not sure that the commit object is the right place to store that\n> > metadata, but hidding this information is even more problematic. Let's\n> > suppose that someone cherry-pick your Bazaar originated commit. Now when\n> > you try to synchronize with Bazaar, your synchronizer will see that it\n> > has some Bazaar revision ID and branch name, but, in fact, it is new\n> > commit on a completely different branch...\n> I don't see how the fact that the bzr-git/hg-git data is being hidden is\n> the problem in the scenario you mention.\n\nBecause you can easily remove that information manually when you cherry-pick\nsome commit. It is more difficult to do when it is hidden.\n\n> It'd be nice if this sort of information was discarded by \"git rebase\",\n> but that's another good reason to treat it in a different way from the\n> commit message instead.\n\nWell, I do not see any other place in the commit object aside the commit\nmessage where you can easily put information, and I do not think it is a\ngood idea for \"git rebase\" to edit the commit message automatically.\nMaybe, you should look at git-notes. (I don't know enough about them to\ntell whether they are suitable or not).\n\n\nDmitry\n"},{"id":"135314","messageId":"1266850659.4575.156.camel@ganieda","threadId":"22738","inReplyTo":"20100221063433.GA2840@coredump.intra.peff.net","subject":"Re: Storing (hidden) per-commit metadata","fromName":"Jelmer Vernooij","fromEmail":"jelmer@samba.org","sentAt":"2010-02-22T14:57:39Z","receivedAt":"2010-02-22T14:57:39Z","isPatch":false,"sender":{"key":"jelmer@samba.org","avatar":"https://avatars.githubusercontent.com/u/49032?v=4"},"body":"On Sun, 2010-02-21 at 01:34 -0500, Jeff King wrote:\n> On Sat, Feb 20, 2010 at 01:57:31PM -0500, Avery Pennarun wrote:\n> > As for git-notes, they sound like they would be useful for this sort\n> > of thing.  I haven't tried them yet, but my understanding is that\n> > notes anywhere other than the \"default\" notes ref are not shown in\n> > commit messages, so you can use them for whatever you want.\n> I would want to hear more about the actual data being stored. The\n> strength of notes is that you can _change_ them after the commit has\n> been created. And the price you pay is that they are more annoying to\n> move around, because they are in a totally different ref.\n> \n> If this is data that is being generated at the time the commit is\n> created and then set in stone, then it probably should be part of the\n> commit object.\nThis data is supposed to be set in stone, since Bazaar revisions are\nintended to be immutable, like Git commits are.\n\nFor each file we would need to store:\n\n * the Bazaar revision id\n * any Bazaar revision properties. This is typically a list of URLs of\nbugs that were fixed, name of the branch the commit was on, any\nadditional parents, or anything arbitrary set by plugins (e.g. the\nrebase plugin sets 'rebase-of' to the id of the original revision)\n * For each file that was added or moved around in the revision, a path\nto fileid mapping\n * Optionally, a list of ghost parent ids and \"unusual\" revisions for\neach file but these should be rare.\n\nThis is at least a couple of lines of data and in some cases a lot more.\nI would rather avoid confronting git users who don't care about Bazaar\nwith it.\n\n> If the only problem is that the data is ugly in \"git show\", then perhaps\n> we need a \"suppress these pseudo-headers\" feature for showing logs. It\n> keeps them easily available for inspection or for --grep, but most of\n> the time you would not see them.\nThat seems like a sensible thing to do, and would work well for me.\n\nCheers,\n\nJelmer\n\n"},{"id":"135336","messageId":"1266865993.11527.49.camel@ganieda","threadId":"22738","inReplyTo":"20100222142013.GA7863@dpotapov.dyndns.org","subject":"Re: Storing (hidden) per-commit metadata","fromName":"Jelmer Vernooij","fromEmail":"jelmer@samba.org","sentAt":"2010-02-22T19:13:13Z","receivedAt":"2010-02-22T19:13:13Z","isPatch":false,"sender":{"key":"jelmer@samba.org","avatar":"https://avatars.githubusercontent.com/u/49032?v=4"},"body":"On Mon, 2010-02-22 at 17:20 +0300, Dmitry Potapov wrote:\n> On Mon, Feb 22, 2010 at 02:44:49PM +0100, Jelmer Vernooij wrote:\n> > On Mon, 2010-02-22 at 16:08 +0300, Dmitry Potapov wrote:\n> > > I am not sure that the commit object is the right place to store that\n> > > metadata, but hidding this information is even more problematic. Let's\n> > > suppose that someone cherry-pick your Bazaar originated commit. Now when\n> > > you try to synchronize with Bazaar, your synchronizer will see that it\n> > > has some Bazaar revision ID and branch name, but, in fact, it is new\n> > > commit on a completely different branch...\n> > I don't see how the fact that the bzr-git/hg-git data is being hidden is\n> > the problem in the scenario you mention.\n> Because you can easily remove that information manually when you cherry-pick\n> some commit. It is more difficult to do when it is hidden.\nMy point is that if you don't make it part of the user-visible commit\nmessage there is no need to remove it at all, it'll just disappear by\nitself.\n\n> > It'd be nice if this sort of information was discarded by \"git rebase\",\n> > but that's another good reason to treat it in a different way from the\n> > commit message instead.\n> Well, I do not see any other place in the commit object aside the commit\n> message where you can easily put information, and I do not think it is a\n> good idea for \"git rebase\" to edit the commit message automatically.\n> Maybe, you should look at git-notes. (I don't know enough about them to\n> tell whether they are suitable or not).\nSome other people have suggested putting e.g. a RFC822-style header in\nthe commit message field and using the headers in that to allow custom\nrevision properties, only displaying the body in \"git log\", \"git show\"\netc. What do you think about that?\n\nCheers,\n\nJelmer\n"},{"id":"135347","messageId":"4B83018E.3020608@mit.edu","threadId":"22738","inReplyTo":"1266599485.29753.54.camel@ganieda","subject":"Re: Storing (hidden) per-commit metadata","fromName":"Alejandro R. Sedeño","fromEmail":"asedeno@mit.edu","sentAt":"2010-02-22T22:13:34Z","receivedAt":"2010-02-22T22:13:34Z","isPatch":false,"sender":{"key":"asedeno@mit.edu","avatar":"https://avatars.githubusercontent.com/u/28302?v=4"},"body":"On 02/19/2010 12:11 PM, Jelmer Vernooij wrote:\n> To allow round-tripping pushes from Bazaar into Git, I'm looking for a\n> good place to store Bazaar semantics that can not be represented in Git\n> at the moment. This data should ideally be hidden from the user as much\n> as possible; it would e.g. contain mappings from git hashes to Bazaar\n> ids.\n\nI've been having similar thoughts for git-svn, since I am working with a\nvery large svn repository that uses svn:keywords and svn:externals in a\nfew places. I've written some scripts to parse the git-svn\nunhandled.log, but that does not propagate to other git clones of the\nrepo, and rebuilding the log is about as expensive as cloning the svn\nrepo again. Also, querying for externals is slow, presumably because\ngit-svn needs to talk to svn to fetch them.\n\nSo far, I have been leaning towards having an optional tree associated\nwith commits, in which metadata could be stored. This metadata tree\nwould be propagated by git clone, used by remote helper scripts, and be\nquite ignorable if unneeded. The metadata tree would use sub-trees for\nnamespaces. For instance, a sub-tree called git-svn would contain the\nrevision map, externals, ignores, properties, etc.\n\nThis isn't fully thought out yet, and I'm not even sure if it would\nwork, or be backwards-compatible. However, since the topic came up, I\nfigured I should mention what I had so far.\n\nPreemptive: No, I don't like svn:keywords, but I can't just ignore them.\n\nPreemptive: Yes, I considered notes in a different GIT_NOTES_REF, but I\nfeel those are too loosely coupled to the commits. I could be wrong, and\nhave not completely dismissed them yet.\n\n-Alejandro\n"}]}