{"thread":{"id":"14257","subject":"RFC: grafts generalised","startedAt":"2008-07-02T14:35:19Z","lastAt":"2008-07-07T06:59:33Z","messageCount":36,"participants":["Stephen R. van den Berg","Jakub Narebski","Michael J Gruber","Dmitry Potapov","Mike Hommey","Stephan Beyer","Junio C Hamano","Avery Pennarun","Petr Baudis","Johannes Sixt","Andreas Ericsson"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"81982","messageId":"20080702143519.GA8391@cuci.nl","threadId":"14257","inReplyTo":null,"subject":"RFC: grafts generalised","fromName":"Stephen R. van den Berg","fromEmail":"srb@cuci.nl","sentAt":"2008-07-02T14:35:19Z","receivedAt":"2008-07-02T14:35:19Z","isPatch":false,"sender":{"key":"srb@cuci.nl","avatar":"https://gravatar.com/avatar/f75389059e827634d38e9df2a9b6ecbd50028b5a454442efa1c7205b7ff29c6a?d=mp&s=160"},"body":"I'm in the process of converting and stitching and patching vast amounts\nof initially disjunct CVS and SVN repositories into larger complete\nhistories inside a single git repository. Recreating history as\naccurately as possible.\n\nThe problem I encounter is that any number of times I have to \"edit\"\nhistory in a non-parameterable fashion, in any of the following ways:\n- Change parents.\n- Add merges.\n- Change author, committer, commitdate, authordate.\n- Change the tree (because of conversion errors in the automated\n conversion process) belonging to a single commit.\n- Retrofit a patch which has to ripple through all of history until\n the present.\n\nThe only things which are easily done at the moment are:\nChange parents and add merges. This can be accomplished fairly easily\nusing the grafts file.\nThe other changes are messy at best and need to be parameterised into the\nform of a shell script so that git filter-branch can have a go at it.\nThis parameterisation is doable for author/committer/dates in most cases\n(but not pretty), but is rather (too) convoluted for ripple-through\npatches.\n\nYou have to imagine that the whole tree has lots of interconnects\nalready (merges), and changing the tree at a point in history which has\nto ripple through is a mess, because all references and interconnects\nneed to be rewritten as well.\n\nI propose the following:\n- Extend git fsck to do more sanity checks on the content of the grafts\n file (to make it more difficult to shoot yourself in the foot with\n that file; my feet will be grateful).\n- Extend the grafts file format to support something like the following syntax:\n\ncommit eb03813cdb999f25628784bb4f07b3f4c8bfe3f6\nParent: 7bc72e647d54c2f713160b22e2e08c39d86c7c28\nMerge: 3b3da24960a82a479b9ad64affab50226df02abe 13b8f53e8ccec3b08eeb6515e6a10a2a\nMerge: ac719ed37270558f21d89676fce97eab4469b0f1\nTree: 32fc99814b97322174dbe97ec320cf32314959e2\nAuthor: Foo Bar (FooBar) \nAuthorDate: Sat Jun 6 13:50:44 1998 +0000\nCommit: Foo Bar (FooBar) \nCommitDate: Sat Jun 7 13:50:44 1998 +0000\nLogmessage: First line of logmessage override\nLogmessage: Second line of logmessage override\nLogmessage: Etc.\n \n- Whereas not specified fields default to not altering the commit for\n those fields.\n E.g.\n\ncommit eb03813cdb999f25628784bb4f07b3f4c8bfe3f6\nParent: 7bc72e647d54c2f713160b22e2e08c39d86c7c28\n\n Would alter just the parent, nothing else.\n- Keep backward compatibility with the old format.\n\nObviously, the use case for this is to change the tree as needed, then\nrun git filter-branch to actually get things in permanently, after which\nit becomes clonable.\n-- \nSincerely,\n Stephen R. van den Berg.\n\nYou are confused; but this is your normal state.\n"},{"id":"81996","messageId":"m3lk0kfdo1.fsf@localhost.localdomain","threadId":"14257","inReplyTo":"20080702143519.GA8391@cuci.nl","subject":"Re: RFC: grafts generalised","fromName":"Jakub Narebski","fromEmail":"jnareb@gmail.com","sentAt":"2008-07-02T16:35:17Z","receivedAt":"2008-07-02T16:35:17Z","isPatch":false,"sender":{"key":"jnareb@gmail.com","avatar":"https://avatars.githubusercontent.com/u/2706?v=4"},"body":"\"Stephen R. van den Berg\" writes:\n\n> I'm in the process of converting and stitching and patching vast amounts\n> of initially disjunct CVS and SVN repositories into larger complete\n> histories inside a single git repository. Recreating history as\n> accurately as possible.\n> \n> The problem I encounter is that any number of times I have to \"edit\"\n> history in a non-parameterable fashion, in any of the following ways:\n> - Change parents.\n> - Add merges.\n> - Change author, committer, commitdate, authordate.\n> - Change the tree (because of conversion errors in the automated\n> conversion process) belonging to a single commit.\n> - Retrofit a patch which has to ripple through all of history until\n> the present.\n> \n> The only things which are easily done at the moment are:\n> Change parents and add merges. This can be accomplished fairly easily\n> using the grafts file.\n> The other changes are messy at best and need to be parameterised into the\n> form of a shell script so that git filter-branch can have a go at it.\n[...]\n\n> I propose the following:\n> - Extend git fsck to do more sanity checks on the content of the grafts\n> file (to make it more difficult to shoot yourself in the foot with\n> that file; my feet will be grateful).\n> - Extend the grafts file format to support something like the following syntax:\n> \n> commit eb03813cdb999f25628784bb4f07b3f4c8bfe3f6\n> Parent: 7bc72e647d54c2f713160b22e2e08c39d86c7c28\n> Merge: 3b3da24960a82a479b9ad64affab50226df02abe 13b8f53e8ccec3b08eeb6515e6a10a2a\n> Merge: ac719ed37270558f21d89676fce97eab4469b0f1\n> Tree: 32fc99814b97322174dbe97ec320cf32314959e2\n> Author: Foo Bar (FooBar) \n> AuthorDate: Sat Jun 6 13:50:44 1998 +0000\n> Commit: Foo Bar (FooBar) \n> CommitDate: Sat Jun 7 13:50:44 1998 +0000\n> Logmessage: First line of logmessage override\n> Logmessage: Second line of logmessage override\n> Logmessage: Etc.\n[...]\n\nFirst, if I remember correctly (from KernelTrap and now defunct Kernel\nTraffic and one issue of Git Traffic) the 'graft' mechanizm was\ncreated so it would be possible to \"graft\" (join) historical\nconversion repository with the \"current work\" git repository (started\nfrom zero when git was deemed good enough for Linux kernel\ndevelopment). The same mechanism is used for shallow clone, where one\ngoes in the opposite direction, shortening history instead of joining\ntwo repositories (two histories).\n\nThe fact that git-filter-branch (and earlier cg-admin-rewrite-hist)\nrespects grafts, and rewrites history so that grafts are no-op and are\nnot needed further is a bit of side-effect. So I think that it would\nbe better to provide generic git-filter-branch filter which can\nunderstand this \"generalized grafts\" file format, or rather\n'description of changes' file. Put it in contrib/, and here you\ngo...\n\n-- \nJakub Narebski\nPoland\nShadeHawk on #git\n"},{"id":"81997","messageId":"g4gb7a$ket$1@ger.gmane.org","threadId":"14257","inReplyTo":"m3lk0kfdo1.fsf@localhost.localdomain","subject":"Re: RFC: grafts generalised","fromName":"Michael J Gruber","fromEmail":"michaeljgruber+gmane@fastmail.fm","sentAt":"2008-07-02T16:43:22Z","receivedAt":"2008-07-02T16:43:22Z","isPatch":false,"sender":{"key":"git@grubix.eu","avatar":"https://avatars.githubusercontent.com/u/233215?v=4"},"body":"Jakub Narebski venit, vidit, dixit 02.07.2008 18:35:\n> \"Stephen R. van den Berg\" writes:\n> \n>> I'm in the process of converting and stitching and patching vast amounts\n>> of initially disjunct CVS and SVN repositories into larger complete\n>> histories inside a single git repository. Recreating history as\n>> accurately as possible.\n>>\n>> The problem I encounter is that any number of times I have to \"edit\"\n>> history in a non-parameterable fashion, in any of the following ways:\n>> - Change parents.\n>> - Add merges.\n>> - Change author, committer, commitdate, authordate.\n>> - Change the tree (because of conversion errors in the automated\n>> conversion process) belonging to a single commit.\n>> - Retrofit a patch which has to ripple through all of history until\n>> the present.\n>>\n>> The only things which are easily done at the moment are:\n>> Change parents and add merges. This can be accomplished fairly easily\n>> using the grafts file.\n>> The other changes are messy at best and need to be parameterised into the\n>> form of a shell script so that git filter-branch can have a go at it.\n> [...]\n> \n>> I propose the following:\n>> - Extend git fsck to do more sanity checks on the content of the grafts\n>> file (to make it more difficult to shoot yourself in the foot with\n>> that file; my feet will be grateful).\n>> - Extend the grafts file format to support something like the following syntax:\n>>\n>> commit eb03813cdb999f25628784bb4f07b3f4c8bfe3f6\n>> Parent: 7bc72e647d54c2f713160b22e2e08c39d86c7c28\n>> Merge: 3b3da24960a82a479b9ad64affab50226df02abe 13b8f53e8ccec3b08eeb6515e6a10a2a\n>> Merge: ac719ed37270558f21d89676fce97eab4469b0f1\n>> Tree: 32fc99814b97322174dbe97ec320cf32314959e2\n>> Author: Foo Bar (FooBar) \n>> AuthorDate: Sat Jun 6 13:50:44 1998 +0000\n>> Commit: Foo Bar (FooBar) \n>> CommitDate: Sat Jun 7 13:50:44 1998 +0000\n>> Logmessage: First line of logmessage override\n>> Logmessage: Second line of logmessage override\n>> Logmessage: Etc.\n> [...]\n> \n> First, if I remember correctly (from KernelTrap and now defunct Kernel\n> Traffic and one issue of Git Traffic) the 'graft' mechanizm was\n> created so it would be possible to \"graft\" (join) historical\n> conversion repository with the \"current work\" git repository (started\n> from zero when git was deemed good enough for Linux kernel\n> development). The same mechanism is used for shallow clone, where one\n> goes in the opposite direction, shortening history instead of joining\n> two repositories (two histories).\n> \n> The fact that git-filter-branch (and earlier cg-admin-rewrite-hist)\n> respects grafts, and rewrites history so that grafts are no-op and are\n> not needed further is a bit of side-effect. So I think that it would\n> be better to provide generic git-filter-branch filter which can\n> understand this \"generalized grafts\" file format, or rather\n> 'description of changes' file. Put it in contrib/, and here you\n> go...\n> \n\nMaybe the upcoming git-sequencer could be the appropriate place? It \ntries to achieve just that: edit history by specifying a list of \ncommands. The currently planned set of commands would need to be \namended, but the framework should be in place.\n\nMichael\n"},{"id":"82002","messageId":"37fcd2780807021019t76008bbfq265f8bf15f59c178@mail.gmail.com","threadId":"14257","inReplyTo":"20080702143519.GA8391@cuci.nl","subject":"Re: RFC: grafts generalised","fromName":"Dmitry Potapov","fromEmail":"dpotapov@gmail.com","sentAt":"2008-07-02T17:19:20Z","receivedAt":"2008-07-02T17:19:20Z","isPatch":false,"sender":{"key":"dpotapov@gmail.com","avatar":"https://avatars.githubusercontent.com/u/6568595?v=4"},"body":"On Wed, Jul 02, 2008 at 04:35:19PM +0200, Stephen R. van den Berg wrote:\n>_\n> - Extend the grafts file format to support something like the following syntax:\n>_\n> commit eb03813cdb999f25628784bb4f07b3f4c8bfe3f6\n> Parent: 7bc72e647d54c2f713160b22e2e08c39d86c7c28\n> Merge: 3b3da24960a82a479b9ad64affab50226df02abe 13b8f53e8ccec3b08eeb6515e6a10a2a\n> Merge: ac719ed37270558f21d89676fce97eab4469b0f1\n> Tree: 32fc99814b97322174dbe97ec320cf32314959e2\n> Author: Foo Bar (FooBar) \n> AuthorDate: Sat Jun 6 13:50:44 1998 +0000\n> Commit: Foo Bar (FooBar) \n> CommitDate: Sat Jun 7 13:50:44 1998 +0000\n> Logmessage: First line of logmessage override\n> Logmessage: Second line of logmessage override\n> Logmessage: Etc.\n\nI don't think that the grafts file is the right place for this kind of\ninformation. Perhaps, it would be better to have a separate file or\neven a directory with files where commit-id identifies a text file with\na new commit object, which should be placed instead of an old one. So,\nit will be easy to tell git filter-branch to use this new information.\n\nHowever, if you want more than just ability to edit commits in a text\nfile but also inspect changes using normal git commands and gitk (as it\nis possible with grafts), it will require changes to the git core, which,\nperhaps, not difficult to implement using pretend_sha1_file(), but I am\nnot sure that everyone will welcome that...\n\nDmitry\n"},{"id":"82004","messageId":"20080702173203.GA16235@cuci.nl","threadId":"14257","inReplyTo":"m3lk0kfdo1.fsf@localhost.localdomain","subject":"Re: RFC: grafts generalised","fromName":"Stephen R. van den Berg","fromEmail":"srb@cuci.nl","sentAt":"2008-07-02T17:32:03Z","receivedAt":"2008-07-02T17:32:03Z","isPatch":false,"sender":{"key":"srb@cuci.nl","avatar":"https://gravatar.com/avatar/f75389059e827634d38e9df2a9b6ecbd50028b5a454442efa1c7205b7ff29c6a?d=mp&s=160"},"body":"Jakub Narebski wrote:\n>First, if I remember correctly (from KernelTrap and now defunct Kernel\n>Traffic and one issue of Git Traffic) the 'graft' mechanizm was\n>created so it would be possible to \"graft\" (join) historical\n>conversion repository with the \"current work\" git repository (started\n>from zero when git was deemed good enough for Linux kernel\n>development).\n\nQuite. Which is exactly the spirit I'm extending here.\nI need it to stitch together history, but it needs to be more perfect\nthan mere connecting parents.\n\nAlso, the graft mechanism specifically is intended as a temporary\nsolution until one uses filter-branch to \"finalise\" the result into a\nproper repository which becomes cloneable.\n\n>The fact that git-filter-branch (and earlier cg-admin-rewrite-hist)\n>respects grafts, and rewrites history so that grafts are no-op and are\n>not needed further is a bit of side-effect.\n\nI beg to differ. It's not a side effect, it's the proper way to get\nrid of the grafts file. Grafts are temporary and ugly. In proper\nrepositories they are a sign of transition to a proper state.\nThe proper state is attained by using git filter-branch.\n\n> So I think that it would\n>be better to provide generic git-filter-branch filter which can\n>understand this \"generalized grafts\" file format, or rather\n>'description of changes' file. Put it in contrib/, and here you\n>go...\n\nThe problem is that the process of fixing history is an iterative one,\nwhich can take many months, and everytime you make a change, the\ncorrectness needs to be viewed using gitk.\n\nFor argument sake, consider the repository at hand which I'm trying to\n\"fix\", it has 33000 commits, distributed over eight branches with\nroughly 3500 merges over a timeperiod of 13 years.\nThe eight branches were eight separate CVS repositories which have\nintersecting histories, and 3500 merges between CVS repositories (i.e.\nbranches).\n\nIf I need to backpatch a certain patch into history, it is likely that\nin order to let the change ripple through, it will take 20000 commits to\nbe rewritten every time I make a slight change to history.\nIt's not really workable to ripple through 20000 commits everytime I\nmake a historical change, yet I need to view the change in gitk.\n\nUsing git filter-branch, or git sequencer basically has the same\nproblem, I need to ripple through most of history to get to a state\nwhich is viewable using gitk again. That is too long a turnaround\ncycle.\n\nUsing the proposed grafts format, I can make changes incrementally, and\nimmediately viewable (though not cloneable) on the local repository using gitk.\nThen after making all the necessary changes, one git filter-branch run\nwill \"burn\" the changes into the repository proper in one go\n(renumbering all tags, branches and merges along the way).\n-- \nSincerely,\n Stephen R. van den Berg.\n\nYou are confused; but this is your normal state.\n"},{"id":"82006","messageId":"20080702174255.GB16235@cuci.nl","threadId":"14257","inReplyTo":"g4gb7a$ket$1@ger.gmane.org","subject":"Re: RFC: grafts generalised","fromName":"Stephen R. van den Berg","fromEmail":"srb@cuci.nl","sentAt":"2008-07-02T17:42:55Z","receivedAt":"2008-07-02T17:42:55Z","isPatch":false,"sender":{"key":"srb@cuci.nl","avatar":"https://gravatar.com/avatar/f75389059e827634d38e9df2a9b6ecbd50028b5a454442efa1c7205b7ff29c6a?d=mp&s=160"},"body":"Michael J Gruber wrote:\n>Maybe the upcoming git-sequencer could be the appropriate place? It \n>tries to achieve just that: edit history by specifying a list of \n>commands. The currently planned set of commands would need to be \n\nThat's the problem. Like git filter-branch, git sequencer needs you to\nparameterise the changes, which, in my case, is hardly possible, since\nthe changes are randomlike.\nAlso, having to run the sequencer to dig 20000 commits into the past,\nthen change something, then come back up and rewrite all following\nhistory and relations (parents/tags/merges) will take a sizeable amount\nof time. I need something that can be changed at will, then viewed with\ngitk a second later.\n\nThese edits are numerous and spread over many months, so the typical \nhistory fixup-sessions involve periods where you make 30 random\nhistoricaledits per hour (which need to be viewed and checked every time\nimmediately after making the change). And say once every 4 months, you\nrun it through git filter-branch to cast everything into stone. A\ntypical git filter-branch run takes 15 minutes on a repository this\nsize.\n-- \nSincerely,\n Stephen R. van den Berg.\n\nYou are confused; but this is your normal state.\n"},{"id":"82007","messageId":"37fcd2780807021058r5ed820cfmdc98f98f36d5c8ae@mail.gmail.com","threadId":"14257","inReplyTo":"37fcd2780807021019t76008bbfq265f8bf15f59c178@mail.gmail.com","subject":"Re: RFC: grafts generalised","fromName":"Dmitry Potapov","fromEmail":"dpotapov@gmail.com","sentAt":"2008-07-02T17:58:20Z","receivedAt":"2008-07-02T17:58:20Z","isPatch":false,"sender":{"key":"dpotapov@gmail.com","avatar":"https://avatars.githubusercontent.com/u/6568595?v=4"},"body":"On Wed, Jul 2, 2008 at 9:19 PM, Dmitry Potapov wrote:\n>\n> However, if you want more than just ability to edit commits in a text\n> file but also inspect changes using normal git commands and gitk (as it\n> is possible with grafts), it will require changes to the git core, which,\n> perhaps, not difficult to implement using pretend_sha1_file(), but I am\n> not sure that everyone will welcome that...\n\nOn second thought, it may be not necessary. You can extract an old commit\nobject, edit it, put it into Git with a new SHA1, and then use the graft file to\nreplace all references from an old to a new one. And you will be able to see\nchanges immediately in gitk.\n\nDmitry\n"},{"id":"82008","messageId":"20080702175944.GC16235@cuci.nl","threadId":"14257","inReplyTo":"37fcd2780807021019t76008bbfq265f8bf15f59c178@mail.gmail.com","subject":"Re: RFC: grafts generalised","fromName":"Stephen R. van den Berg","fromEmail":"srb@cuci.nl","sentAt":"2008-07-02T17:59:44Z","receivedAt":"2008-07-02T17:59:44Z","isPatch":false,"sender":{"key":"srb@cuci.nl","avatar":"https://gravatar.com/avatar/f75389059e827634d38e9df2a9b6ecbd50028b5a454442efa1c7205b7ff29c6a?d=mp&s=160"},"body":"Dmitry Potapov wrote:\n>I don't think that the grafts file is the right place for this kind of\n>information.\n\nYet the grafts file is exactly the place where this type of\n\"overlay-information\" is being placed now. It seems to be the natural place.\n\n> Perhaps, it would be better to have a separate file or\n>even a directory with files where commit-id identifies a text file with\n>a new commit object, which should be placed instead of an old one. So,\n>it will be easy to tell git filter-branch to use this new information.\n\nNot quite sure why this makes it easier. The point is that there\nis not supposed to be a grafts file in a proper repository. Thus,\nhaving a lot of these files means a larger disruption to the core, and\nI'd like the core to be as efficient and lean as possible given an empty\ngrafts file. So I'd prefer to keep it to one file.\n\n>However, if you want more than just ability to edit commits in a text\n>file but also inspect changes using normal git commands and gitk (as it\n>is possible with grafts), it will require changes to the git core, which,\n>perhaps, not difficult to implement using pretend_sha1_file(), but I am\n>not sure that everyone will welcome that...\n\nI'd want to avoid a plethora of files, and the changes that can be\nspecified are supposed to be partial overrides, not complete rewrites.\nSo using pretend_sha1_file() is a bit overkill and more than I was\naiming for.\n\nThe point is, that the changes in grafts (as they are now) are *not*\nused when cloning. I.e. the only thing you mess up is your *own*\nrepository, not someone else's. I.e. you can't make someone remote\nthink that the repository has been altered. That would require git\nfilter-branch, which immediately changes all the historical SHA1s, and\nmakes the changes in history blatantly visible.\n-- \nSincerely,\n Stephen R. van den Berg.\n\nYou are confused; but this is your normal state.\n"},{"id":"82009","messageId":"20080702181021.GD16235@cuci.nl","threadId":"14257","inReplyTo":"37fcd2780807021058r5ed820cfmdc98f98f36d5c8ae@mail.gmail.com","subject":"Re: RFC: grafts generalised","fromName":"Stephen R. van den Berg","fromEmail":"srb@cuci.nl","sentAt":"2008-07-02T18:10:21Z","receivedAt":"2008-07-02T18:10:21Z","isPatch":false,"sender":{"key":"srb@cuci.nl","avatar":"https://gravatar.com/avatar/f75389059e827634d38e9df2a9b6ecbd50028b5a454442efa1c7205b7ff29c6a?d=mp&s=160"},"body":"Dmitry Potapov wrote:\n>On second thought, it may be not necessary. You can extract an old commit\n>object, edit it, put it into Git with a new SHA1, and then use the graft file to\n>replace all references from an old to a new one. And you will be able to see\n>changes immediately in gitk.\n\nHmmmm, interesting thought. That just might solve my problem.\nIn that case, I will stick to extending git fsck to check grafts more\nrigorously and fix git clone to *refrain* from looking at grafts.\nIf anyone still wants the extended format, I'd be willing to implement\nit, but my immediate itch for it is gone.\n-- \nSincerely,\n Stephen R. van den Berg.\n\nYou are confused; but this is your normal state.\n"},{"id":"82010","messageId":"20080702182510.GC29559@glandium.org","threadId":"14257","inReplyTo":"20080702174255.GB16235@cuci.nl","subject":"Re: RFC: grafts generalised","fromName":"Mike Hommey","fromEmail":"mh@glandium.org","sentAt":"2008-07-02T18:25:10Z","receivedAt":"2008-07-02T18:25:10Z","isPatch":false,"sender":{"key":"mh@glandium.org","avatar":"https://avatars.githubusercontent.com/u/1038527?v=4"},"body":"On Wed, Jul 02, 2008 at 07:42:55PM +0200, Stephen R. van den Berg wrote:\n> Michael J Gruber wrote:\n> >Maybe the upcoming git-sequencer could be the appropriate place? It \n> >tries to achieve just that: edit history by specifying a list of \n> >commands. The currently planned set of commands would need to be \n> \n> That's the problem. Like git filter-branch, git sequencer needs you to\n> parameterise the changes, which, in my case, is hardly possible, since\n> the changes are randomlike.\n> Also, having to run the sequencer to dig 20000 commits into the past,\n> then change something, then come back up and rewrite all following\n> history and relations (parents/tags/merges) will take a sizeable amount\n> of time. I need something that can be changed at will, then viewed with\n> gitk a second later.\n> \n> These edits are numerous and spread over many months, so the typical \n> history fixup-sessions involve periods where you make 30 random\n> historicaledits per hour (which need to be viewed and checked every time\n> immediately after making the change). And say once every 4 months, you\n> run it through git filter-branch to cast everything into stone. A\n> typical git filter-branch run takes 15 minutes on a repository this\n> size.\n\nI think the point was more about making a tool to do exactly what you\nwant, based on the new git sequencer. Note that git filter-branch could\nalso be rewritten to use the sequencer.\n\nMike\n"},{"id":"82012","messageId":"37fcd2780807021133y3a02ca80g9647b1f0854fec2b@mail.gmail.com","threadId":"14257","inReplyTo":"20080702181021.GD16235@cuci.nl","subject":"Re: RFC: grafts generalised","fromName":"Dmitry Potapov","fromEmail":"dpotapov@gmail.com","sentAt":"2008-07-02T18:33:29Z","receivedAt":"2008-07-02T18:33:29Z","isPatch":false,"sender":{"key":"dpotapov@gmail.com","avatar":"https://avatars.githubusercontent.com/u/6568595?v=4"},"body":"On Wed, Jul 2, 2008 at 10:10 PM, Stephen R. van den Berg wrote:\n> Dmitry Potapov wrote:\n>>On second thought, it may be not necessary. You can extract an old commit\n>>object, edit it, put it into Git with a new SHA1, and then use the graft file to\n>>replace all references from an old to a new one. And you will be able to see\n>>changes immediately in gitk.\n>\n> Hmmmm, interesting thought. That just might solve my problem.\n\nThis script is just a prove of the concept. It seems to work for me, but\nI don't really tested it.\n\n===========================================\n#!/bin/bash\n\nset -e\n\n# creating some silly repo\ngit init\n# creating some history\nfor ((i=0; $i<10; i++))\ndo\n\techo foo$i > foo$i\n\tgit add foo$i\n\tgit commit -m \"add foo$i\"\ndone\n\n# run gitk to see it\ngitk --all &\n\n# dump all graft info to text file\ngit rev-list --parents --all > .git/info/grafts.tmp\nmv .git/info/grafts.tmp .git/info/grafts\n\n# please choose what commit you want to edit\necho\nwhile read -p 'Edit commit: ' C\ndo\nC=$(git rev-parse \"$C\") || continue\n# edit commit C\ngit cat-file commit $C > .git/COMMIT_OBJ\nvim .git/COMMIT_OBJ\n\nC2=$(git hash-object -w -t commit .git/COMMIT_OBJ)\n\n# replace all references from C to C2\nsed -e 's/\\<'$C'\\>/'$C2'/g' < .git/info/grafts > .git/info/grafts.tmp\nmv .git/info/grafts.tmp .git/info/grafts\ndone\n===========================================\n\nDmitry\n"},{"id":"82013","messageId":"g4gho9$g42$1@ger.gmane.org","threadId":"14257","inReplyTo":"20080702182510.GC29559@glandium.org","subject":"Re: RFC: grafts generalised","fromName":"Michael J Gruber","fromEmail":"michaeljgruber+gmane@fastmail.fm","sentAt":"2008-07-02T18:34:49Z","receivedAt":"2008-07-02T18:34:49Z","isPatch":false,"sender":{"key":"git@grubix.eu","avatar":"https://avatars.githubusercontent.com/u/233215?v=4"},"body":"Mike Hommey venit, vidit, dixit 02.07.2008 20:25:\n> On Wed, Jul 02, 2008 at 07:42:55PM +0200, Stephen R. van den Berg wrote:\n>> Michael J Gruber wrote:\n>>> Maybe the upcoming git-sequencer could be the appropriate place? It \n>>> tries to achieve just that: edit history by specifying a list of \n>>> commands. The currently planned set of commands would need to be \n>> That's the problem. Like git filter-branch, git sequencer needs you to\n>> parameterise the changes, which, in my case, is hardly possible, since\n>> the changes are randomlike.\n>> Also, having to run the sequencer to dig 20000 commits into the past,\n>> then change something, then come back up and rewrite all following\n>> history and relations (parents/tags/merges) will take a sizeable amount\n>> of time. I need something that can be changed at will, then viewed with\n>> gitk a second later.\n>>\n>> These edits are numerous and spread over many months, so the typical \n>> history fixup-sessions involve periods where you make 30 random\n>> historicaledits per hour (which need to be viewed and checked every time\n>> immediately after making the change). And say once every 4 months, you\n>> run it through git filter-branch to cast everything into stone. A\n>> typical git filter-branch run takes 15 minutes on a repository this\n>> size.\n> \n> I think the point was more about making a tool to do exactly what you\n> want, based on the new git sequencer. Note that git filter-branch could\n> also be rewritten to use the sequencer.\n\nYes, that was at least my point. As I understand, git filter-branch -i \nis a candidate for that rewrite.\n\nBut I understand now that OP wants to do lots of history edits and see \nthem immediately before doing the actual (time consuming) rewrite; and \nthen do the rewrite occasionally. Rewriting is surpirsingly slow even on \ntmpfs.\n\nMichael\n"},{"id":"82014","messageId":"20080702183701.GE16235@cuci.nl","threadId":"14257","inReplyTo":"20080702182510.GC29559@glandium.org","subject":"Re: RFC: grafts generalised","fromName":"Stephen R. van den Berg","fromEmail":"srb@cuci.nl","sentAt":"2008-07-02T18:37:01Z","receivedAt":"2008-07-02T18:37:01Z","isPatch":false,"sender":{"key":"srb@cuci.nl","avatar":"https://gravatar.com/avatar/f75389059e827634d38e9df2a9b6ecbd50028b5a454442efa1c7205b7ff29c6a?d=mp&s=160"},"body":"Mike Hommey wrote:\n>> These edits are numerous and spread over many months, so the typical \n>> history fixup-sessions involve periods where you make 30 random\n>> historicaledits per hour (which need to be viewed and checked every time\n>> immediately after making the change). And say once every 4 months, you\n>> run it through git filter-branch to cast everything into stone. A\n>> typical git filter-branch run takes 15 minutes on a repository this\n>> size.\n\n>I think the point was more about making a tool to do exactly what you\n>want, based on the new git sequencer. Note that git filter-branch could\n>also be rewritten to use the sequencer.\n\nAs far as I understood it, the new git sequencer rewrites history\nproper. That is timeconsuming by definition, and thus it is *not*\npossible to make a tool based on the sequencer that supports the desired\niterative-history-rewrite workflow.\n-- \nSincerely,\n Stephen R. van den Berg.\n\nYou are confused; but this is your normal state.\n"},{"id":"82021","messageId":"20080702193157.GA21297@leksak.fem-net","threadId":"14257","inReplyTo":"20080702183701.GE16235@cuci.nl","subject":"Re: RFC: grafts generalised","fromName":"Stephan Beyer","fromEmail":"s-beyer@gmx.net","sentAt":"2008-07-02T19:31:57Z","receivedAt":"2008-07-02T19:31:57Z","isPatch":false,"sender":{"key":"s-beyer@gmx.net","avatar":"https://avatars.githubusercontent.com/u/143889?v=4"},"body":"Hi,\n\nI'm somehow quite confused about the desired workflow but I try an\nanswer.\n\nStephen R. van den Berg wrote:\n> As far as I understood it, the new git sequencer rewrites history\n> proper. That is timeconsuming by definition, and thus it is *not*\n> possible to make a tool based on the sequencer that supports the desired\n> iterative-history-rewrite workflow.\n\nIf I got the problem right, it is possible.\nBut you have to rewrite and cannot just fake history, of course.\nAnd, as Michael wrote:\n> The currently planned set of commands would need to be amended, but the\n> framework should be in place.\n\n...for example, a \"pick \" that just picks the _tree_ of the\ncommit and not the _introduced changes_. (I've never used info/grafts,\nbut if I get the principle right, such tree-picks could realize a\nlinear list of info/grafts history fakes.)\n\nStephen wrote earlier:\n> The problem I encounter is that any number of times I have to \"edit\"\n> history in a non-parameterable fashion, in any of the following ways:\n\nHm, imho sequencer is well-suited for \"non-parameterable\" stuff.\n\n> - Change parents.\n\nThe \"pick\" instruction (onto the new parent) is your friend.\n\n> - Add merges.\n\n\"merge\" instruction ;)\n\n> - Change author, committer, commitdate, authordate.\n\nsequencer doesn't allow to change committer data, but this could\nbe an easy change if you really need that.\nThe same with the author timestamp, that could only be reused from\nan old commit by using -C option on pick.\n\n> - Change the tree (because of conversion errors in the automated\n> conversion process) belonging to a single commit.\n> - Retrofit a patch which has to ripple through all of history until\n> the present.\n\n\"pause\" instruction, and then do manual changes, then\n\tgit sequencer --continue\n\nStephan has also written:\n> Also, having to run the sequencer to dig 20000 commits into the past,\n> then change something, then come back up and rewrite all following\n> history and relations (parents/tags/merges) will take a sizeable amount\n> of time.\n\nI wonder if grafts can be used in combination with sequencer in such a\nway that you rewrite foo~20000..foo~19950 and then fake the parents of\nfoo~19949 to be the rewritten once.\n\n> I need something that can be changed at will, then viewed with\n> gitk a second later.\n\nYou can run gitk whenever you did \"pause\" in the sequencer file.\n[Btw, an integration of sequencer into gitk is also on the TODO list,\n but that's OT here.]\n\nRegards,\n Stephan\n\n-- \nStephan Beyer , PGP 0x6EDDD207FCC5040F\n"},{"id":"82024","messageId":"20080702193655.GB21297@leksak.fem-net","threadId":"14257","inReplyTo":"20080702193157.GA21297@leksak.fem-net","subject":"Re: RFC: grafts generalised","fromName":"Stephan Beyer","fromEmail":"s-beyer@gmx.net","sentAt":"2008-07-02T19:36:55Z","receivedAt":"2008-07-02T19:36:55Z","isPatch":false,"sender":{"key":"s-beyer@gmx.net","avatar":"https://avatars.githubusercontent.com/u/143889?v=4"},"body":"A short typo fix:\n> I wonder if grafts can be used in combination with sequencer in such a\n> way that you rewrite foo~20000..foo~19950 and then fake the parents of\n> foo~19949 to be the rewritten once.\n\ns/once/ones/\n\nTo give it some sense. Sorry ;)\n\n-- \nStephan Beyer , PGP 0x6EDDD207FCC5040F\n"},{"id":"82027","messageId":"37fcd2780807021339u582f340dq2b2014951d5b7f63@mail.gmail.com","threadId":"14257","inReplyTo":"20080702181021.GD16235@cuci.nl","subject":"Re: RFC: grafts generalised","fromName":"Dmitry Potapov","fromEmail":"dpotapov@gmail.com","sentAt":"2008-07-02T20:39:24Z","receivedAt":"2008-07-02T20:39:24Z","isPatch":false,"sender":{"key":"dpotapov@gmail.com","avatar":"https://avatars.githubusercontent.com/u/6568595?v=4"},"body":"On Wed, Jul 2, 2008 at 10:10 PM, Stephen R. van den Berg wrote:\n>\n> In that case, I will stick to extending git fsck to check grafts more\n> rigorously and fix git clone to *refrain* from looking at grafts.\n\nLinus suggested that \"git-fsck and repacking should just consider\nit[grafts] to be an _additional_ source of parenthood rather than\na _replacement_ source.\"\n\nhttp://article.gmane.org/gmane.comp.version-control.git/84686\n\nDmitry\n"},{"id":"82028","messageId":"37fcd2780807021342j75f351a5sa525b892caedf965@mail.gmail.com","threadId":"14257","inReplyTo":"20080702193157.GA21297@leksak.fem-net","subject":"Re: RFC: grafts generalised","fromName":"Dmitry Potapov","fromEmail":"dpotapov@gmail.com","sentAt":"2008-07-02T20:42:30Z","receivedAt":"2008-07-02T20:42:30Z","isPatch":false,"sender":{"key":"dpotapov@gmail.com","avatar":"https://avatars.githubusercontent.com/u/6568595?v=4"},"body":"On Wed, Jul 2, 2008 at 11:31 PM, Stephan Beyer wrote:\n>\n> I'm somehow quite confused about the desired workflow but I try an\n> answer.\n\nI don't think we speak about any normal workflow but about importing\n\"initially disjunct CVS and SVN repositories into larger complete\nhistories inside a single git repository.\" This is one-time work, not\na regular workflow.\n\n>\n> Stephen R. van den Berg wrote:\n>> As far as I understood it, the new git sequencer rewrites history\n>> proper. That is timeconsuming by definition, and thus it is *not*\n>> possible to make a tool based on the sequencer that supports the desired\n>> iterative-history-rewrite workflow.\n>\n> If I got the problem right, it is possible.\n> But you have to rewrite and cannot just fake history, of course.\n\nUsing grafts allows you to fake history, which is very useful during\nimport, because it allows you to edit history without running any\nfilter-branch, which is very timeconsuming. Of course, at the end\nyou have to run git filter-branch to have the \"true\" history, otherwise\nanyone who clones from you will end up with a broken repo.\n\nThe purpose of rebase (and I believe the sequencer too) is rather\ndifferent -- to allow you to keep your changes as patches to the\nupstream.\n\n> I wonder if grafts can be used in combination with sequencer in such a\n> way that you rewrite foo~20000..foo~19950 and then fake the parents of\n> foo~19949 to be the rewritten once.\n\nI don't think it is a good idea. During the normal work you should never\nuse grafts. Well, you can use grafts to add old history, but using it for\nanything else is really dangerous, because its *fakes* history. git rebase\n(and AFAIK sequencer too) just re-write history of some branch. IOW, it\ncreates another branch from a different starting point using patches from\nsome existing branch and then reassign the branch name to it.\n\nDmitry\n"},{"id":"82030","messageId":"20080702211836.GF16235@cuci.nl","threadId":"14257","inReplyTo":"37fcd2780807021339u582f340dq2b2014951d5b7f63@mail.gmail.com","subject":"Re: RFC: grafts generalised","fromName":"Stephen R. van den Berg","fromEmail":"srb@cuci.nl","sentAt":"2008-07-02T21:18:36Z","receivedAt":"2008-07-02T21:18:36Z","isPatch":false,"sender":{"key":"srb@cuci.nl","avatar":"https://gravatar.com/avatar/f75389059e827634d38e9df2a9b6ecbd50028b5a454442efa1c7205b7ff29c6a?d=mp&s=160"},"body":"Dmitry Potapov wrote:\n>On Wed, Jul 2, 2008 at 10:10 PM, Stephen R. van den Berg wrote:\n>> In that case, I will stick to extending git fsck to check grafts more\n>> rigorously and fix git clone to *refrain* from looking at grafts.\n\n>Linus suggested that \"git-fsck and repacking should just consider\n>it[grafts] to be an _additional_ source of parenthood rather than\n>a _replacement_ source.\"\n\n>http://article.gmane.org/gmane.comp.version-control.git/84686\n\nYes, I know that's what he suggested, the way it should be implemented\nIMO though is by checking once without and once with regard to grafts.\nAnd still it should be such that git clone disregards grafts completely.\nI'll fix both, eventually, since I need this functionality to verify\ncorrectness for the projects I'm working on at the moment.\n\nAs for repack, it should probably ignore grafts, except for reference.\nI.e. repack/gc should consider all mentioned SHA1s in the grafts file\nto be referenced and undeletable.\n-- \nSincerely,\n Stephen R. van den Berg.\n\nYou are confused; but this is your normal state.\n"},{"id":"82031","messageId":"7vlk0k7z99.fsf@gitster.siamese.dyndns.org","threadId":"14257","inReplyTo":"37fcd2780807021339u582f340dq2b2014951d5b7f63@mail.gmail.com","subject":"Re: RFC: grafts generalised","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2008-07-02T21:27:30Z","receivedAt":"2008-07-02T21:27:30Z","isPatch":false,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"\"Dmitry Potapov\" writes:\n\n> On Wed, Jul 2, 2008 at 10:10 PM, Stephen R. van den Berg wrote:\n>>\n>> In that case, I will stick to extending git fsck to check grafts more\n>> rigorously and fix git clone to *refrain* from looking at grafts.\n>\n> Linus suggested that \"git-fsck and repacking should just consider\n> it[grafts] to be an _additional_ source of parenthood rather than\n> a _replacement_ source.\"\n>\n> http://article.gmane.org/gmane.comp.version-control.git/84686\n\nYeah, thanks for a reminder.\n\n http://thread.gmane.org/gmane.comp.version-control.git/37744/focus=37866\n\nis still on my \"things to look at\" list.\n"},{"id":"82032","messageId":"32541b130807021428o5dc0eab7p8464a8b8fc6c3a52@mail.gmail.com","threadId":"14257","inReplyTo":"20080702211836.GF16235@cuci.nl","subject":"Re: RFC: grafts generalised","fromName":"Avery Pennarun","fromEmail":"apenwarr@gmail.com","sentAt":"2008-07-02T21:28:25Z","receivedAt":"2008-07-02T21:28:25Z","isPatch":false,"sender":{"key":"apenwarr@gmail.com","avatar":"https://avatars.githubusercontent.com/u/20592?v=4"},"body":"On 7/2/08, Stephen R. van den Berg wrote:\n> Dmitry Potapov wrote:\n> >On Wed, Jul 2, 2008 at 10:10 PM, Stephen R. van den Berg wrote:\n> >> In that case, I will stick to extending git fsck to check grafts more\n> >> rigorously and fix git clone to *refrain* from looking at grafts.\n>\n> >Linus suggested that \"git-fsck and repacking should just consider\n> >it[grafts] to be an _additional_ source of parenthood rather than\n> >a _replacement_ source.\"\n>\n> >http://article.gmane.org/gmane.comp.version-control.git/84686\n>\n> Yes, I know that's what he suggested, the way it should be implemented\n> IMO though is by checking once without and once with regard to grafts.\n> And still it should be such that git clone disregards grafts completely.\n\nI could see an argument that the only modes you really need are a) use\ngrafts as replacements, and b) use grafts as additions. There is\nperhaps no need for c) ignore grafts.\n\nFor example, say I wanted to give someone a copy of my repo that\nincludes grafts (ignoring the fact that this is probably bad to do in\ngeneral). He could git-clone it and then install a copy of my grafts\nfile, as long as git-clone does (a) or (b) but not (c). On the other\nhand, if he just wants a copy of the \"real\" (graft-free) repo, then\ngit-clone needs to do (b) or (c) but not (a). git-fsck needs (b), and\nmost normal git operations want (a) (since that was the original\npurpose of grafts).\n\nBased on that, (c) is redundant, unless you're really concerned about\nnot sending redundant objects to people who clone your repo that has\ngrafts installed. But I think you probably shouldn't have people\ncloning your grafted repository anyway unless you know what you're\ndoing, and if you know what you're doing, you probably want (b). If\nyou see what I mean.\n\nHave fun,\n\nAvery\n"},{"id":"82034","messageId":"7vej6c7y8e.fsf@gitster.siamese.dyndns.org","threadId":"14257","inReplyTo":"7vlk0k7z99.fsf@gitster.siamese.dyndns.org","subject":"Re: RFC: grafts generalised","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2008-07-02T21:49:37Z","receivedAt":"2008-07-02T21:49:37Z","isPatch":false,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Junio C Hamano writes:\n\n> \"Dmitry Potapov\" writes:\n>\n>> On Wed, Jul 2, 2008 at 10:10 PM, Stephen R. van den Berg wrote:\n>>>\n>>> In that case, I will stick to extending git fsck to check grafts more\n>>> rigorously and fix git clone to *refrain* from looking at grafts.\n>>\n>> Linus suggested that \"git-fsck and repacking should just consider\n>> it[grafts] to be an _additional_ source of parenthood rather than\n>> a _replacement_ source.\"\n>>\n>> http://article.gmane.org/gmane.comp.version-control.git/84686\n>\n> Yeah, thanks for a reminder.\n>\n> http://thread.gmane.org/gmane.comp.version-control.git/37744/focus=37866\n>\n> is still on my \"things to look at\" list.\n\nThis shows how the \"object transfer ignores grafts\" side of the earlier\nsuggestion by Linus would look like to get people started. Totally\nuntested.\n\nI threw in for_each_commit_graft() in the patch so that updates to the\nreachability walker can add otherwise hidden objects, but otherwise it is\nnot used yet.\n\n builtin-pack-objects.c | 5 +++++\n builtin-send-pack.c | 3 ++-\n cache.h | 1 +\n commit.c | 10 ++++++++++\n commit.h | 2 ++\n environment.c | 1 +\n upload-pack.c | 1 +\n 7 files changed, 22 insertions(+), 1 deletions(-)\n\ndiff --git a/builtin-pack-objects.c b/builtin-pack-objects.c\nindex 28207d9..53b0b33 100644\n--- a/builtin-pack-objects.c\n+++ b/builtin-pack-objects.c\n@@ -30,6 +30,7 @@ git-pack-objects [{ -q | --progress | --all-progress }] \\n\\\n \t[--threads=N] [--non-empty] [--revs [--unpacked | --all]*] [--reflog] \\n\\\n \t[--stdout | base-name] [--include-tag] \\n\\\n \t[--keep-unreachable | --unpack-unreachable] \\n\\\n+\t[--ignore-graft] \\n\\\n \t[sha1);\n@@ -196,7 +203,10 @@ static void prepare_commit_graft(void)\n struct commit_graft *lookup_commit_graft(const unsigned char *sha1)\n {\n \tint pos;\n+\n \tprepare_commit_graft();\n+\tif (!honor_graft)\n+\t\treturn NULL;\n \tpos = commit_graft_pos(sha1);\n \tif (pos < 0)\n \t\treturn NULL;\ndiff --git a/commit.h b/commit.h\nindex 2d94d41..8f76dd9 100644\n--- a/commit.h\n+++ b/commit.h\n@@ -138,4 +138,6 @@ static inline int single_parent(struct commit *commit)\n \treturn commit->parents && !commit->parents->next;\n }\n \n+void for_each_commit_graft(void (*fn)(struct commit_graft *));\n+\n #endif /* COMMIT_H */\ndiff --git a/environment.c b/environment.c\nindex 4a88a17..eb8f36d 100644\n--- a/environment.c\n+++ b/environment.c\n@@ -41,6 +41,7 @@ enum safe_crlf safe_crlf = SAFE_CRLF_WARN;\n unsigned whitespace_rule_cfg = WS_DEFAULT_RULE;\n enum branch_track git_branch_track = BRANCH_TRACK_REMOTE;\n enum rebase_setup_type autorebase = AUTOREBASE_NEVER;\n+int honor_graft = 1;\n \n /* This is set by setup_git_dir_gently() and/or git_default_config() */\n char *git_work_tree_cfg;\ndiff --git a/upload-pack.c b/upload-pack.c\nindex b46dd36..d948d64 100644\n--- a/upload-pack.c\n+++ b/upload-pack.c\n@@ -158,6 +158,7 @@ static void create_pack_file(void)\n \t\tdie(\"git-upload-pack: unable to fork git-rev-list\");\n \n \targv[arg++] = \"pack-objects\";\n+\targv[arg++] = \"--ignore-graft\";\n \targv[arg++] = \"--stdout\";\n \tif (!no_progress)\n \t\targv[arg++] = \"--progress\";\n"},{"id":"82043","messageId":"20080702234644.GC21297@leksak.fem-net","threadId":"14257","inReplyTo":"37fcd2780807021342j75f351a5sa525b892caedf965@mail.gmail.com","subject":"Re: RFC: grafts generalised","fromName":"Stephan Beyer","fromEmail":"s-beyer@gmx.net","sentAt":"2008-07-02T23:46:44Z","receivedAt":"2008-07-02T23:46:44Z","isPatch":false,"sender":{"key":"s-beyer@gmx.net","avatar":"https://avatars.githubusercontent.com/u/143889?v=4"},"body":"Hi,\n\nOn Thu, Jul 03, 2008 at 12:42:30AM +0400, Dmitry Potapov wrote:\n> On Wed, Jul 2, 2008 at 11:31 PM, Stephan Beyer wrote:\n> > I wonder if grafts can be used in combination with sequencer in such a\n> > way that you rewrite foo~20000..foo~19950 and then fake the parents of\n> > foo~19949 to be the rewritten once.\n> \n> I don't think it is a good idea. During the normal work you should never\n> use grafts.\n\nI have written this in the context that Stephen only changes some commits\nfrom a long time ago (foo~20000) and then I showed a way how to avoid that\nsequencer rewrites the rest which takes so long.\nThis is not related to \"normal work\", but to Stephen's use case (if I\ngot it right).\n\nWhat I've meant, was:\nInstead of faking a lot of parents, changes and even merges using an\nextended grafts file, he could rewrite some patches - which can be fast -\nand then use _only one_ graft to change the parent to the changed and\nrewritten commit.\nThis can be done iteratively and seems to be a good agreement in speed\nand reliability.\n\nRegards,\n Stephan\n\n-- \nStephan Beyer , PGP 0x6EDDD207FCC5040F\n"},{"id":"82045","messageId":"7v7ic37s1r.fsf@gitster.siamese.dyndns.org","threadId":"14257","inReplyTo":"7vej6c7y8e.fsf@gitster.siamese.dyndns.org","subject":"Re: RFC: grafts generalised","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2008-07-03T00:03:12Z","receivedAt":"2008-07-03T00:03:12Z","isPatch":false,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Junio C Hamano writes:\n\n>> Yeah, thanks for a reminder.\n>>\n>> http://thread.gmane.org/gmane.comp.version-control.git/37744/focus=37866\n>>\n>> is still on my \"things to look at\" list.\n>\n> This shows how the \"object transfer ignores grafts\" side of the earlier\n> suggestion by Linus would look like to get people started. Totally\n> untested.\n>\n> I threw in for_each_commit_graft() in the patch so that updates to the\n> reachability walker can add otherwise hidden objects, but otherwise it is\n> not used yet.\n\nThis updates the earlier patch to teach the object transfer side to ignore\ngrafts, which makes things consistent between dumb commit walkers and\nnative transport. It is not meant for application as I haven't thought\nabout[*1*] nor looked into how this may interact with the \"shallow clone\"\nstuff (which is graft in disguise but implemented separately).\n\nFootnote. *1* I also suspect Linus did not think about interactions with\n\"shallow\" when he made the suggestion referenced above, as \"shallow\" was\nstill a relatively new curiosity back then.\n\nI am not sure if the addition of --ignore-graft to revision.c should be\nthere when this becomes real. I added it primarily for debugging\npurposes, as it is something the end users should never trigger in the\nnormal workflow.\n\n--\n builtin-pack-objects.c | 5 +++\n builtin-send-pack.c | 3 +-\n cache.h | 1 +\n commit.c | 10 +++++++\n commit.h | 2 +\n environment.c | 1 +\n revision.c | 4 +++\n t/t6500-graft.sh | 70 ++++++++++++++++++++++++++++++++++++++++++++++++\n upload-pack.c | 2 +\n 9 files changed, 97 insertions(+), 1 deletions(-)\n create mode 100755 t/t6500-graft.sh\n\ndiff --git a/builtin-pack-objects.c b/builtin-pack-objects.c\nindex 28207d9..53b0b33 100644\n--- a/builtin-pack-objects.c\n+++ b/builtin-pack-objects.c\n@@ -30,6 +30,7 @@ git-pack-objects [{ -q | --progress | --all-progress }] \\n\\\n \t[--threads=N] [--non-empty] [--revs [--unpacked | --all]*] [--reflog] \\n\\\n \t[--stdout | base-name] [--include-tag] \\n\\\n \t[--keep-unreachable | --unpack-unreachable] \\n\\\n+\t[--ignore-graft] \\n\\\n \t[sha1);\n@@ -196,7 +203,10 @@ static void prepare_commit_graft(void)\n struct commit_graft *lookup_commit_graft(const unsigned char *sha1)\n {\n \tint pos;\n+\n \tprepare_commit_graft();\n+\tif (!honor_graft)\n+\t\treturn NULL;\n \tpos = commit_graft_pos(sha1);\n \tif (pos < 0)\n \t\treturn NULL;\ndiff --git a/commit.h b/commit.h\nindex 2d94d41..8f76dd9 100644\n--- a/commit.h\n+++ b/commit.h\n@@ -138,4 +138,6 @@ static inline int single_parent(struct commit *commit)\n \treturn commit->parents && !commit->parents->next;\n }\n \n+void for_each_commit_graft(void (*fn)(struct commit_graft *));\n+\n #endif /* COMMIT_H */\ndiff --git a/environment.c b/environment.c\nindex 4a88a17..eb8f36d 100644\n--- a/environment.c\n+++ b/environment.c\n@@ -41,6 +41,7 @@ enum safe_crlf safe_crlf = SAFE_CRLF_WARN;\n unsigned whitespace_rule_cfg = WS_DEFAULT_RULE;\n enum branch_track git_branch_track = BRANCH_TRACK_REMOTE;\n enum rebase_setup_type autorebase = AUTOREBASE_NEVER;\n+int honor_graft = 1;\n \n /* This is set by setup_git_dir_gently() and/or git_default_config() */\n char *git_work_tree_cfg;\ndiff --git a/revision.c b/revision.c\nindex fc66755..25c96d0 100644\n--- a/revision.c\n+++ b/revision.c\n@@ -991,6 +991,10 @@ int setup_revisions(int argc, const char **argv, struct rev_info *revs, const ch\n \t\tconst char *arg = argv[i];\n \t\tif (*arg == '-') {\n \t\t\tint opts;\n+\t\t\tif (!strcmp(arg, \"--ignore-graft\")) {\n+\t\t\t\thonor_graft = 0;\n+\t\t\t\tcontinue;\n+\t\t\t}\n \t\t\tif (!prefixcmp(arg, \"--max-count=\")) {\n \t\t\t\trevs->max_count = atoi(arg + 12);\n \t\t\t\tcontinue;\ndiff --git a/t/t6500-graft.sh b/t/t6500-graft.sh\nnew file mode 100755\nindex 0000000..122ed1c\n--- /dev/null\n+++ b/t/t6500-graft.sh\n@@ -0,0 +1,70 @@\n+#!/bin/sh\n+\n+test_description='graft and ancestry traversal'\n+. test-lib.sh\n+\n+# Real history:\n+#\n+# .----D\n+# /\n+# A--B--C--E\n+#\n+# Grafted history:\n+#\n+# .----D\n+# / \\\n+# A-----C--E\n+#\n+\n+advance_one () {\n+\techo \"$1\" >file &&\n+\tgit add file &&\n+\ttest_tick &&\n+\tgit commit -m \"$1\" &&\n+\tgit rev-parse --verify HEAD >\"$1\"\n+}\n+\n+test_expect_success setup '\n+\tadvance_one A &&\n+\tadvance_one D &&\n+\n+\tgit reset --hard $(cat A) &&\n+\tadvance_one B &&\n+\tadvance_one C &&\n+\tadvance_one E &&\n+\n+\t(\n+\t\techo $(cat E) $(cat C) $(cat D)\n+\t\techo $(cat C) $(cat A)\n+\t) >.git/info/grafts &&\n+\n+\tgit log --graph --pretty=oneline --abbrev-commit --parents &&\n+\techo \" - - - - - - - - - - \" &&\n+\tgit log --graph --pretty=oneline --abbrev-commit --parents --ignore-graft\n+\n+'\n+\n+test_expect_success 'clone should lose grafts' '\n+\tgit clone --bare \"file://$(pwd)/.git\" cloned.git &&\n+\t(\n+\t\tGIT_DIR=cloned.git && export GIT_DIR &&\n+\t\tgit log --graph --pretty=oneline --abbrev-commit --parents &&\n+\t\tgit cat-file commit $(cat B) &&\n+\t\ttest_must_fail git cat-file commit $(cat D) &&\n+\t\tgit fsck --full\n+\t)\n+'\n+\n+test_expect_success 'push should lose grafts' '\n+\ttest_create_repo pushed.git &&\n+\t(\n+\t\tcd pushed.git &&\n+\t\tgit fetch ../.git master:refs/heads/master &&\n+\t\tgit log --graph --pretty=oneline --abbrev-commit --parents &&\n+\t\tgit cat-file commit $(cat ../B) &&\n+\t\ttest_must_fail git cat-file commit $(cat ../D) &&\n+\t\tgit fsck --full\n+\t)\n+'\n+\n+test_done\n\\ No newline at end of file\ndiff --git a/upload-pack.c b/upload-pack.c\nindex b46dd36..798caaa 100644\n--- a/upload-pack.c\n+++ b/upload-pack.c\n@@ -158,6 +158,7 @@ static void create_pack_file(void)\n \t\tdie(\"git-upload-pack: unable to fork git-rev-list\");\n \n \targv[arg++] = \"pack-objects\";\n+\targv[arg++] = \"--ignore-graft\";\n \targv[arg++] = \"--stdout\";\n \tif (!no_progress)\n \t\targv[arg++] = \"--progress\";\n@@ -614,6 +615,7 @@ int main(int argc, char **argv)\n \tint i;\n \tint strict = 0;\n \n+\thonor_graft = 0;\n \tfor (i = 1; i < argc; i++) {\n \t\tchar *arg = argv[i];\n \n"},{"id":"82046","messageId":"20080703001331.GF12567@machine.or.cz","threadId":"14257","inReplyTo":"20080702143519.GA8391@cuci.nl","subject":"Re: RFC: grafts generalised","fromName":"Petr Baudis","fromEmail":"pasky@suse.cz","sentAt":"2008-07-03T00:13:31Z","receivedAt":"2008-07-03T00:13:31Z","isPatch":false,"sender":{"key":"pasky@ucw.cz","avatar":"https://avatars.githubusercontent.com/u/18439?v=4"},"body":"On Wed, Jul 02, 2008 at 04:35:19PM +0200, Stephen R. van den Berg wrote:\n> - Extend the grafts file format to support something like the following syntax:\n> \n> commit eb03813cdb999f25628784bb4f07b3f4c8bfe3f6\n> Parent: 7bc72e647d54c2f713160b22e2e08c39d86c7c28\n> Merge: 3b3da24960a82a479b9ad64affab50226df02abe 13b8f53e8ccec3b08eeb6515e6a10a2a\n> Merge: ac719ed37270558f21d89676fce97eab4469b0f1\n> Tree: 32fc99814b97322174dbe97ec320cf32314959e2\n> Author: Foo Bar (FooBar) \n> AuthorDate: Sat Jun 6 13:50:44 1998 +0000\n> Commit: Foo Bar (FooBar) \n> CommitDate: Sat Jun 7 13:50:44 1998 +0000\n> Logmessage: First line of logmessage override\n> Logmessage: Second line of logmessage override\n> Logmessage: Etc.\n\n Please, don't. It adds completely unnecessary complexity and it is\n_not_ grafting anymore - look the word up in a dictionary. :-)\n\n Have a look at what you wrote above - now, Git already has a way to\nstore all this information, right? In the commit objects!\n\n So, the real solution is to take the commit objects you want to\nmodify, create new commit objects, then graft the new commit on all the\nold commit children. It fits neatly in the Git philosophy, there is no\nneed at all to tweak the current infrastructure for this and it should\nbe trivial to automate, too.\n\n-- \n\t\t\t\tPetr \"Pasky\" Baudis\nThe last good thing written in C++ was the Pachelbel Canon. -- J. Olson\n"},{"id":"82047","messageId":"20080703001614.GG12567@machine.or.cz","threadId":"14257","inReplyTo":"20080703001331.GF12567@machine.or.cz","subject":"Re: RFC: grafts generalised","fromName":"Petr Baudis","fromEmail":"pasky@suse.cz","sentAt":"2008-07-03T00:16:14Z","receivedAt":"2008-07-03T00:16:14Z","isPatch":false,"sender":{"key":"pasky@ucw.cz","avatar":"https://avatars.githubusercontent.com/u/18439?v=4"},"body":"On Thu, Jul 03, 2008 at 02:13:31AM +0200, Petr Baudis wrote:\n> So, the real solution is to take the commit objects you want to\n> modify, create new commit objects, then graft the new commit on all the\n> old commit children. It fits neatly in the Git philosophy, there is no\n> need at all to tweak the current infrastructure for this and it should\n> be trivial to automate, too.\n\n Oops, sorry; I stopped reading the branch of the thread I thought was\ngoing off on a different tangent one post too early. :-)\n\n-- \n\t\t\t\tPetr \"Pasky\" Baudis\nThe last good thing written in C++ was the Pachelbel Canon. -- J. Olson\n"},{"id":"82049","messageId":"20080703002117.GH12567@machine.or.cz","threadId":"14257","inReplyTo":"20080702173203.GA16235@cuci.nl","subject":"Re: RFC: grafts generalised","fromName":"Petr Baudis","fromEmail":"pasky@suse.cz","sentAt":"2008-07-03T00:21:17Z","receivedAt":"2008-07-03T00:21:17Z","isPatch":false,"sender":{"key":"pasky@ucw.cz","avatar":"https://avatars.githubusercontent.com/u/18439?v=4"},"body":"On Wed, Jul 02, 2008 at 07:32:03PM +0200, Stephen R. van den Berg wrote:\n> Also, the graft mechanism specifically is intended as a temporary\n> solution until one uses filter-branch to \"finalise\" the result into a\n> proper repository which becomes cloneable.\n\nGrafts are _much_ older than filter-branch and I'm not sure where did\nyou get this idea; do we claim that in any documentation?\n\n> >The fact that git-filter-branch (and earlier cg-admin-rewrite-hist)\n> >respects grafts, and rewrites history so that grafts are no-op and are\n> >not needed further is a bit of side-effect.\n> \n> I beg to differ. It's not a side effect, it's the proper way to get\n> rid of the grafts file. Grafts are temporary and ugly. In proper\n> repositories they are a sign of transition to a proper state.\n> The proper state is attained by using git filter-branch.\n\nThere's nothing ugly or necessarily temporary about grafts. One example\nof completely valid usage is adding previous history of a project to it\nlater.\n\nFirst, you don't need to carry around all the archived baggage you are\nprobably rarely going to access anyway if you don't need to; changing a\nVCS is ideal cutoff point.\n\nSecond, you don't need to worry about doing perfect conversion at the\nmoment of the switch.\n\nThird, even if you think you have done it perfectly, it will turn out\nlater that something is wrong anyway.\n\nFourth, it may not be actually _clear_ what the canonical history should\nbe. Consider linux-kernel, you can graft the BitKeeper history (or one\nof possible candidates for the ideal conversion, though one is AFAIK\nclearly favoured), or you could also graft commit-per-tarball history\neven from the times before BitKeeper; you certainly don't want either in\nthe current main history DAG.\n\n-- \n\t\t\t\tPetr \"Pasky\" Baudis\nThe last good thing written in C++ was the Pachelbel Canon. -- J. Olson\n"},{"id":"82050","messageId":"7vy74j6cal.fsf@gitster.siamese.dyndns.org","threadId":"14257","inReplyTo":"20080703001614.GG12567@machine.or.cz","subject":"Re: RFC: grafts generalised","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2008-07-03T00:28:50Z","receivedAt":"2008-07-03T00:28:50Z","isPatch":false,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Petr Baudis writes:\n\n> On Thu, Jul 03, 2008 at 02:13:31AM +0200, Petr Baudis wrote:\n>> So, the real solution is to take the commit objects you want to\n>> modify, create new commit objects, then graft the new commit on all the\n>> old commit children. It fits neatly in the Git philosophy, there is no\n>> need at all to tweak the current infrastructure for this and it should\n>> be trivial to automate, too.\n>\n> Oops, sorry; I stopped reading the branch of the thread I thought was\n> going off on a different tangent one post too early. :-)\n\nWhat you wrote was a very good summary of what Dmitry suggested earlier\n;-)\n"},{"id":"82074","messageId":"486C6B8E.5040202@viscovery.net","threadId":"14257","inReplyTo":"20080702181021.GD16235@cuci.nl","subject":"Re: RFC: grafts generalised","fromName":"Johannes Sixt","fromEmail":"j.sixt@viscovery.net","sentAt":"2008-07-03T06:02:54Z","receivedAt":"2008-07-03T06:02:54Z","isPatch":false,"sender":{"key":"j6t@kdbg.org","avatar":"https://avatars.githubusercontent.com/u/14810926?v=4"},"body":"Stephen R. van den Berg schrieb:\n> Dmitry Potapov wrote:\n>> On second thought, it may be not necessary. You can extract an old commit\n>> object, edit it, put it into Git with a new SHA1, and then use the graft file to\n>> replace all references from an old to a new one. And you will be able to see\n>> changes immediately in gitk.\n> \n> Hmmmm, interesting thought. That just might solve my problem.\n\nI don't think it would.\n\nYou want to apply a patch through a part of the history. To do that, it is\nnot sufficient to apply the patch to only one commit/tree and then fake\nparenthood of its child commits. You still need to apply the patch to all\nchildren.\n\n-- Hannes\n"},{"id":"82092","messageId":"20080703060551.GB27419@cuci.nl","threadId":"14257","inReplyTo":"20080702234644.GC21297@leksak.fem-net","subject":"Re: RFC: grafts generalised","fromName":"Stephen R. van den Berg","fromEmail":"srb@cuci.nl","sentAt":"2008-07-03T06:05:51Z","receivedAt":"2008-07-03T06:05:51Z","isPatch":false,"sender":{"key":"srb@cuci.nl","avatar":"https://gravatar.com/avatar/f75389059e827634d38e9df2a9b6ecbd50028b5a454442efa1c7205b7ff29c6a?d=mp&s=160"},"body":"Stephan Beyer wrote:\n>On Thu, Jul 03, 2008 at 12:42:30AM +0400, Dmitry Potapov wrote:\n>> On Wed, Jul 2, 2008 at 11:31 PM, Stephan Beyer wrote:\n>> > I wonder if grafts can be used in combination with sequencer in such a\n>> > way that you rewrite foo~20000..foo~19950 and then fake the parents of\n>> > foo~19949 to be the rewritten once.\n\n>> I don't think it is a good idea. During the normal work you should never\n>> use grafts.\n\n>I have written this in the context that Stephen only changes some commits\n>from a long time ago (foo~20000) and then I showed a way how to avoid that\n>sequencer rewrites the rest which takes so long.\n>This is not related to \"normal work\", but to Stephen's use case (if I\n>got it right).\n\nYou got it right.\n\n>What I've meant, was:\n>Instead of faking a lot of parents, changes and even merges using an\n>extended grafts file, he could rewrite some patches - which can be fast -\n>and then use _only one_ graft to change the parent to the changed and\n>rewritten commit.\n>This can be done iteratively and seems to be a good agreement in speed\n>and reliability.\n\nIndeed.\n-- \nSincerely,\n Stephen R. van den Berg.\n\nThis is a day for firm decisions! Or is it?\n"},{"id":"82099","messageId":"20080703071143.GC27419@cuci.nl","threadId":"14257","inReplyTo":"20080703002117.GH12567@machine.or.cz","subject":"Re: RFC: grafts generalised","fromName":"Stephen R. van den Berg","fromEmail":"srb@cuci.nl","sentAt":"2008-07-03T07:11:43Z","receivedAt":"2008-07-03T07:11:43Z","isPatch":false,"sender":{"key":"srb@cuci.nl","avatar":"https://gravatar.com/avatar/f75389059e827634d38e9df2a9b6ecbd50028b5a454442efa1c7205b7ff29c6a?d=mp&s=160"},"body":"Petr Baudis wrote:\n>On Wed, Jul 02, 2008 at 07:32:03PM +0200, Stephen R. van den Berg wrote:\n>> Also, the graft mechanism specifically is intended as a temporary\n>> solution until one uses filter-branch to \"finalise\" the result into a\n>> proper repository which becomes cloneable.\n\n>Grafts are _much_ older than filter-branch and I'm not sure where did\n>you get this idea; do we claim that in any documentation?\n\nNot in direct documentation, but it is what breaths down from posts on\nthe mailinglist like:\n\nhttp://kerneltrap.org/mailarchive/git/2008/6/10/2085624\n\nJakub Narebski:\n>Then if possible use git-filter-branch to make history recorded in\n>grafts file permanent...\n\nPetr Baudis wrote:\n>There's nothing ugly or necessarily temporary about grafts. One example\n>of completely valid usage is adding previous history of a project to it\n>later.\n\n>First, you don't need to carry around all the archived baggage you are\n>probably rarely going to access anyway if you don't need to; changing a\n>VCS is ideal cutoff point.\n\nThat depends on the project, of course, and is not a valid statement in\ngeneral. Part of the charm of full history is that git-blame and\ngit-bisect work, at arbitrary points in the past.\n\n>Second, you don't need to worry about doing perfect conversion at the\n>moment of the switch.\n\nWell, you do, if you intend to make it cloneable.\n\n>Third, even if you think you have done it perfectly, it will turn out\n>later that something is wrong anyway.\n\nNot necessarily. I have automated the checkout-verification-process which\nbasically checks out every revision from the respective old repository\nand binary-compares it with the corresponding revision in the git\nrepository. This ensures a full binary match across the board.\nWith respect to historical merges, I agree, those might not be\ncompletely correctly grafted, but the level of correctness can be\ndetermined at will, and once we achieve somewhere around 99% accuracy,\nwe consider it done (for this project).\n\n>Fourth, it may not be actually _clear_ what the canonical history should\n>be.\n\nThat depends on the project. In my project it *is* clear, so this point\ndoesn't make any difference.\n\n-- \nSincerely,\n Stephen R. van den Berg.\n\nThis is a day for firm decisions! Or is it?\n"},{"id":"82096","messageId":"20080703073041.GA28566@cuci.nl","threadId":"14257","inReplyTo":"486C6B8E.5040202@viscovery.net","subject":"Re: RFC: grafts generalised","fromName":"Stephen R. van den Berg","fromEmail":"srb@cuci.nl","sentAt":"2008-07-03T07:30:41Z","receivedAt":"2008-07-03T07:30:41Z","isPatch":false,"sender":{"key":"srb@cuci.nl","avatar":"https://gravatar.com/avatar/f75389059e827634d38e9df2a9b6ecbd50028b5a454442efa1c7205b7ff29c6a?d=mp&s=160"},"body":"Johannes Sixt wrote:\n>Stephen R. van den Berg schrieb:\n>> Dmitry Potapov wrote:\n>>> On second thought, it may be not necessary. You can extract an old commit\n>>> object, edit it, put it into Git with a new SHA1, and then use the graft file to\n>>> replace all references from an old to a new one. And you will be able to see\n>>> changes immediately in gitk.\n\n>> Hmmmm, interesting thought. That just might solve my problem.\n\n>I don't think it would.\n\n>You want to apply a patch through a part of the history. To do that, it is\n>not sufficient to apply the patch to only one commit/tree and then fake\n>parenthood of its child commits. You still need to apply the patch to all\n>children.\n\nI am aware of that.\nThere are actually two common cases:\n- Historical changes which are confined and don't ripple through. The\n above solution works just fine for that.\n- Ripple-through changes. They indeed need to be applied to every tree\n in the first-parent chain. Even though this is going to take a\n considerable amount of time, there still are certain advantages to\n doing this using the method described above:\n + You can apply the patch to every commit/tree \"interactively\" if you want.\n (Yes, I know, git-sequencer supports this one as well, but not the\n next point).\n + You can view the change at any point in time (including in relation to the\n tree that follows it), right after making the amendments (without letting\n it ripple through to the end).\n + The ripple-through does not need to be performed in topological order,\n i.e. eventually you'll have to touch everything, but you can do it\n in the order you see fit (whatever is most efficient to work on).\n + If, at some point during the ripple-through process, you find out\n that you forgot some change(s), you can abort or restart the\n ripple-through without having spent all that time waiting for a\n full-ripple-through.\n\nActually, ripple-through changes are rare. In the current project it\nseems I need exactly one, but it's buried deep in the past (sadly).\nThe reason why I need it, is to make sure that git-bisect will work for\nany revision in the past (i.e. the tree contained/contains some\ntoo-clever-for-their-own-good $Revision$-expansion dependencies)\n-- \nSincerely,\n Stephen R. van den Berg.\n\nThis is a day for firm decisions! Or is it?\n"},{"id":"82097","messageId":"486C82F5.6080405@viscovery.net","threadId":"14257","inReplyTo":"20080703073041.GA28566@cuci.nl","subject":"Re: RFC: grafts generalised","fromName":"Johannes Sixt","fromEmail":"j.sixt@viscovery.net","sentAt":"2008-07-03T07:42:45Z","receivedAt":"2008-07-03T07:42:45Z","isPatch":false,"sender":{"key":"j6t@kdbg.org","avatar":"https://avatars.githubusercontent.com/u/14810926?v=4"},"body":"Stephen R. van den Berg schrieb:\n> Actually, ripple-through changes are rare. In the current project it\n> seems I need exactly one, but it's buried deep in the past (sadly).\n> The reason why I need it, is to make sure that git-bisect will work for\n> any revision in the past (i.e. the tree contained/contains some\n> too-clever-for-their-own-good $Revision$-expansion dependencies)\n\nBut you do know that you don't need to apply the change *now*; you can\napply it at bisect-time? Unless you expect you or your mere mortal\ncoworkers are going to do dozens of bisects into that part of the history,\nI wouldn't change history *like*this*. But of course, I don't understand\nthe circumstances enough, so... just my 2 cents.\n\n-- Hannes\n"},{"id":"82121","messageId":"20080703093719.GD29838@cuci.nl","threadId":"14257","inReplyTo":"486C82F5.6080405@viscovery.net","subject":"Re: RFC: grafts generalised","fromName":"Stephen R. van den Berg","fromEmail":"srb@cuci.nl","sentAt":"2008-07-03T09:37:19Z","receivedAt":"2008-07-03T09:37:19Z","isPatch":false,"sender":{"key":"srb@cuci.nl","avatar":"https://gravatar.com/avatar/f75389059e827634d38e9df2a9b6ecbd50028b5a454442efa1c7205b7ff29c6a?d=mp&s=160"},"body":"Johannes Sixt wrote:\n>Stephen R. van den Berg schrieb:\n>> Actually, ripple-through changes are rare. In the current project it\n>> seems I need exactly one, but it's buried deep in the past (sadly).\n>> The reason why I need it, is to make sure that git-bisect will work for\n>> any revision in the past (i.e. the tree contained/contains some\n>> too-clever-for-their-own-good $Revision$-expansion dependencies)\n\n>But you do know that you don't need to apply the change *now*; you can\n>apply it at bisect-time? Unless you expect you or your mere mortal\n>coworkers are going to do dozens of bisects into that part of the history,\n>I wouldn't change history *like*this*. But of course, I don't understand\n>the circumstances enough, so... just my 2 cents.\n\nThat is exactly the case, I do expect dozens of bisects.\n-- \nSincerely,\n Stephen R. van den Berg.\n\nThis is a day for firm decisions! Or is it?\n"},{"id":"82188","messageId":"200807040243.54739.jnareb@gmail.com","threadId":"14257","inReplyTo":"20080702173203.GA16235@cuci.nl","subject":"Re: RFC: grafts generalised","fromName":"Jakub Narebski","fromEmail":"jnareb@gmail.com","sentAt":"2008-07-04T00:43:53Z","receivedAt":"2008-07-04T00:43:53Z","isPatch":false,"sender":{"key":"jnareb@gmail.com","avatar":"https://avatars.githubusercontent.com/u/2706?v=4"},"body":"On Wed, 2 July 2008, Stephen R. van den Berg wrote:\n> Jakub Narebski wrote:\n>>\n>> [...] So I think that it would\n>> be better to provide generic git-filter-branch filter which can\n>> understand this \"generalized grafts\" file format, or rather\n>> 'description of changes' file. Put it in contrib/, and here you\n>> go...\n> \n> The problem is that the process of fixing history is an iterative one,\n> which can take many months, and everytime you make a change, the\n> correctness needs to be viewed using gitk.\n[...]\n\nI wanted to propose that git-filter-branch generic \"generalized grafts\"\nfile based filter should be accompanied by extending gitk so it\nunderstand this format to...\n\n...but after reading wonderfull suggestion to create new commits with\ncorrected contents, and insert them (replace older version by them)\nusing grafts, thought and brought independently by Dmitry Potapov and\nPetr Baudis, I think that you would be best with extending gitk to\nsupport this way instead.\n\nYou would have to extend gitk to maintain reverse revision mapping\n(from revision to its children), and then you would be able to edit\nhistory interactively from within gitk, with gitk correcting its\ninternal structures to redisplay changed commits, and creating commits\nand doing grafting behind the scenes for later git-filter-branch\nrun.\n\n-- \nJakub Narebski\nPoland\n"},{"id":"82423","messageId":"4871B7A0.1090604@op5.se","threadId":"14257","inReplyTo":"20080702174255.GB16235@cuci.nl","subject":"Re: RFC: grafts generalised","fromName":"Andreas Ericsson","fromEmail":"ae@op5.se","sentAt":"2008-07-07T06:28:48Z","receivedAt":"2008-07-07T06:28:48Z","isPatch":false,"sender":{"key":"ae@op5.se","avatar":"https://gravatar.com/avatar/426e89595c75a8f5252dd0c989e5fabe5bcac616e68557427ad9aef6b0ca342a?d=mp&s=160"},"body":"Stephen R. van den Berg wrote:\n> Michael J Gruber wrote:\n>> Maybe the upcoming git-sequencer could be the appropriate place? It \n>> tries to achieve just that: edit history by specifying a list of \n>> commands. The currently planned set of commands would need to be \n> \n> That's the problem. Like git filter-branch, git sequencer needs you to\n> parameterise the changes, which, in my case, is hardly possible, since\n> the changes are randomlike.\n> Also, having to run the sequencer to dig 20000 commits into the past,\n> then change something, then come back up and rewrite all following\n> history and relations (parents/tags/merges) will take a sizeable amount\n> of time. I need something that can be changed at will, then viewed with\n> gitk a second later.\n> \n\nA second later might be too much, but for the case where you need to\nadd a patch in the middle (which I suspect is the most timeconsuming\nand tricky part at the moment), you might want to use a temporary\nbranch checked out where you need to apply the patch, apply the patch\nand then rebase the rest of the history onto that new commit. Rebase\nis fairly quick (although not a one-second thing for 20k commits), so\nyou'll get the time down quite a bit, I imagine.\n\n-- \nAndreas Ericsson andreas.ericsson@op5.se\nOP5 AB www.op5.se\nTel: +46 8-230225 Fax: +46 8-230231\n"},{"id":"82427","messageId":"20080707065933.GB25877@cuci.nl","threadId":"14257","inReplyTo":"4871B7A0.1090604@op5.se","subject":"Re: RFC: grafts generalised","fromName":"Stephen R. van den Berg","fromEmail":"srb@cuci.nl","sentAt":"2008-07-07T06:59:33Z","receivedAt":"2008-07-07T06:59:33Z","isPatch":false,"sender":{"key":"srb@cuci.nl","avatar":"https://gravatar.com/avatar/f75389059e827634d38e9df2a9b6ecbd50028b5a454442efa1c7205b7ff29c6a?d=mp&s=160"},"body":"Andreas Ericsson wrote:\n>Stephen R. van den Berg wrote:\n>>of time. I need something that can be changed at will, then viewed with\n>>gitk a second later.\n\n>A second later might be too much, but for the case where you need to\n>add a patch in the middle (which I suspect is the most timeconsuming\n>and tricky part at the moment), you might want to use a temporary\n>branch checked out where you need to apply the patch, apply the patch\n>and then rebase the rest of the history onto that new commit. Rebase\n>is fairly quick (although not a one-second thing for 20k commits), so\n>you'll get the time down quite a bit, I imagine.\n\nNot really.\nRebase does two things:\na. Apply every patch/commit again, which takes too long for 20k commits.\nb. Mess up carefully grafted parent/merge relationships.\n\nRebase is only suitable for short linear strands of commits.\nThe history I'm dealing with is neither short, nor linear.\n-- \nSincerely,\n Stephen R. van den Berg.\n\nA truly wise man never plays leapfrog with a unicorn.\n"}]}