{"thread":{"id":"16610","subject":"How to clone git repository with git-svn meta-data included?","startedAt":"2008-12-06T12:15:40Z","lastAt":"2008-12-09T20:57:06Z","messageCount":17,"participants":["Grzegorz Kossakowski","Jacob Helwig","Peter Harris","Nick Andrew","Michael J Gruber","Shawn O. Pearce","Sam Vilain"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"97251","messageId":"493A6CEC.4060601@tuffmail.com","threadId":"16610","inReplyTo":null,"subject":"How to clone git repository with git-svn meta-data included?","fromName":"Grzegorz Kossakowski","fromEmail":"grek@tuffmail.com","sentAt":"2008-12-06T12:15:40Z","receivedAt":"2008-12-06T12:15:40Z","isPatch":false,"sender":{"key":"grek@tuffmail.com","avatar":"https://gravatar.com/avatar/98a80b7992e9f598c7b24addc73338dbacf94a14e4a14f3fa50c3f00dbee7dd3?d=mp&s=160"},"body":"Hello,\n\nSome folks at Apache are experimenting with Git and we are currently seeking for the git-svn\nintegration that fits our needs and infrastructure.\n\nAfter some evaluation we decided that our setup could be described using following points:\n  a) our svn repository remains our main, official server where every committer is obligated to push\ntheir changes to at some time.\n  b) we set up clone of svn repository using git-svn. One of our members, Jukka Zitting, maintains\nsuch a service here[1]. Such repositories should be usable both for our committers (that have rights\nto push to svn) and our contributors that want to contribute random patches\n  c) we want carefully track who committed/contributed what\n\nBasically, a) implies b) and point b) looks little bit problematic right now.\nJukka has set up his hosting using method described in his e-mail[2] which basically makes use of\ngit svn. The major problem is that if one clones Jukka's repository then git svn information is not\nbeing cloned so committers have no means to push their changes to main, svn server.\n\nI've tried to play a little bit around with this issue and I tried to copy information from .git\ndirectory found on Jukka's server. This made me able to push my changes but git svn insisted on\nrebasing my repository using commits found in svn which is wrong. Basically we want such a setup\nthat uses git repository (Jukka's clone) for pulling changes and local git svn for pushing changes.\nGit svn should never try to rebase local repository because this will lead to two different trees on\ntwo different machines so we won't be able to exchange and merge changesets.\n\nIs it possible with Git right now?\n\n\nAnother point (c) which seems to be brought a couple of times but never a definitive answer has been\ngiven, AFAIK. Let's imagine we have committer C and two contributors A and B.\n\nA and B start to work on some feature and C agreed to help A and B and once their work is finished\nto merge their changes into his repository and eventually push them to main, svn repository. Now A\nand B work on implementation and from time to time their merge changes from each other. Once they\nare finished A asks C to merge their work into C's repository. Everything is fine provided we can\ntrust both A and B.\n\nWhat if A was not fair and has rewritten a few commits coming from B so they contain malicious code?\nHow we can detect something like that and how C be sure that what he merges is really work\nattributed by correct names?\n\nThanks for your answers.\n\n[1] http://jukka.zitting.name/git/\n[2] http://markmail.org/message/fzzy7nepk7olx5fl\n\n\n-- \nBest regards,\nGrzegorz Kossakowski\n"},{"id":"97295","messageId":"8c9a060812070043r472e10abu7a76152b5fe1314d@mail.gmail.com","threadId":"16610","inReplyTo":"493A6CEC.4060601@tuffmail.com","subject":"Re: How to clone git repository with git-svn meta-data included?","fromName":"Jacob Helwig","fromEmail":"jacob.helwig@gmail.com","sentAt":"2008-12-07T08:43:32Z","receivedAt":"2008-12-07T08:43:32Z","isPatch":false,"sender":{"key":"jacob.helwig@gmail.com","avatar":"https://avatars.githubusercontent.com/u/14557?v=4"},"body":"On Sat, Dec 6, 2008 at 04:15, Grzegorz Kossakowski <grek@tuffmail.com> wrote:\n> Hello,\n>\n> Some folks at Apache are experimenting with Git and we are currently seeking for the git-svn\n> integration that fits our needs and infrastructure.\n>\n> After some evaluation we decided that our setup could be described using following points:\n>  a) our svn repository remains our main, official server where every committer is obligated to push\n> their changes to at some time.\n>  b) we set up clone of svn repository using git-svn. One of our members, Jukka Zitting, maintains\n> such a service here[1]. Such repositories should be usable both for our committers (that have rights\n> to push to svn) and our contributors that want to contribute random patches\n>  c) we want carefully track who committed/contributed what\n>\n> Basically, a) implies b) and point b) looks little bit problematic right now.\n> Jukka has set up his hosting using method described in his e-mail[2] which basically makes use of\n> git svn. The major problem is that if one clones Jukka's repository then git svn information is not\n> being cloned so committers have no means to push their changes to main, svn server.\n>\n> I've tried to play a little bit around with this issue and I tried to copy information from .git\n> directory found on Jukka's server. This made me able to push my changes but git svn insisted on\n> rebasing my repository using commits found in svn which is wrong. Basically we want such a setup\n> that uses git repository (Jukka's clone) for pulling changes and local git svn for pushing changes.\n> Git svn should never try to rebase local repository because this will lead to two different trees on\n> two different machines so we won't be able to exchange and merge changesets.\n>\n> Is it possible with Git right now?\n>\n>\n> Another point (c) which seems to be brought a couple of times but never a definitive answer has been\n> given, AFAIK. Let's imagine we have committer C and two contributors A and B.\n>\n> A and B start to work on some feature and C agreed to help A and B and once their work is finished\n> to merge their changes into his repository and eventually push them to main, svn repository. Now A\n> and B work on implementation and from time to time their merge changes from each other. Once they\n> are finished A asks C to merge their work into C's repository. Everything is fine provided we can\n> trust both A and B.\n>\n> What if A was not fair and has rewritten a few commits coming from B so they contain malicious code?\n> How we can detect something like that and how C be sure that what he merges is really work\n> attributed by correct names?\n>\n> Thanks for your answers.\n>\n> [1] http://jukka.zitting.name/git/\n> [2] http://markmail.org/message/fzzy7nepk7olx5fl\n>\n>\n> --\n> Best regards,\n> Grzegorz Kossakowski\n> --\n> To unsubscribe from this list: send the line \"unsubscribe git\" in\n> the body of a message to majordomo@vger.kernel.org\n> More majordomo info at  http://vger.kernel.org/majordomo-info.html\n>\n\nGrzegorz,\n\nI use git-svn quite a bit at $work, but I haven't seen a way to clone\na git repo, and have it Just Work(TM) with git-svn in the new clone,\nunfortunately.\n\nI have been able to use clone to significantly cut down on the setup\ntime for working with git-svn, however.  I was following the\ndirections at http://subtlegradient.com/articles/2008/04/22/cloning-a-git-svn-clone\n (Clone the git-svn clone, then re-do the git svn init, and fetch to\nre-sync.)\n\nNot sure how much this will actually help you with the first part of\nyour problem.\n\n-Jacob\n"},{"id":"97305","messageId":"eaa105840812070857i27f8e920keaba3f92f5260b38@mail.gmail.com","threadId":"16610","inReplyTo":"493A6CEC.4060601@tuffmail.com","subject":"Re: How to clone git repository with git-svn meta-data included?","fromName":"Peter Harris","fromEmail":"git@peter.is-a-geek.org","sentAt":"2008-12-07T16:57:16Z","receivedAt":"2008-12-07T16:57:16Z","isPatch":false,"sender":{"key":"git@peter.is-a-geek.org","avatar":null},"body":"On Sat, Dec 6, 2008 at 7:15 AM, Grzegorz Kossakowski wrote:\n>\n> Some folks at Apache are experimenting with Git and we are currently seeking for the git-svn\n> integration that fits our needs and infrastructure.\n>\n> After some evaluation we decided that our setup could be described using following points:\n>  a) our svn repository remains our main, official server where every committer is obligated to push\n> their changes to at some time.\n>  b) we set up clone of svn repository using git-svn. One of our members, Jukka Zitting, maintains\n> such a service here[1]. Such repositories should be usable both for our committers (that have rights\n> to push to svn) and our contributors that want to contribute random patches\n>  c) we want carefully track who committed/contributed what\n>\n> Basically, a) implies b) and point b) looks little bit problematic right now.\n> Jukka has set up his hosting using method described in his e-mail[2] which basically makes use of\n> git svn. The major problem is that if one clones Jukka's repository then git svn information is not\n> being cloned so committers have no means to push their changes to main, svn server.\n\nMake sure you don't use the --no-metadata flag when setting up\ngit-svn. This will embed the metadata into commit messages, so git-svn\ncan rebuild it from scratch whenever it needs to. (You probably also\nwant git 1.6.1rc for incremental rebuild support). This also has the\nadvantage that you can see the svn revision number when looking at a\ncommit message.\n\n> I've tried to play a little bit around with this issue and I tried to copy information from .git\n> directory found on Jukka's server. This made me able to push my changes but git svn insisted on\n> rebasing my repository using commits found in svn which is wrong. Basically we want such a setup\n> that uses git repository (Jukka's clone) for pulling changes and local git svn for pushing changes.\n> Git svn should never try to rebase local repository because this will lead to two different trees on\n> two different machines so we won't be able to exchange and merge changesets.\n\nsvn doesn't really know what a merge is (not even 1.5). You MUST\nrebase in order to commit to svn. This is a limitation of svn, not\ngit.\n\nIn terms of re-pulling from the git-svn mirror, git-svn will create\nthe same commits (with the same sha1s) from svn every time, so there\nwill be no conflicts there.\n\n> Is it possible with Git right now?\n\nYes, but it might not be possible with svn, depending on which part of\nthe above \"it\" is.\n\n> What if A was not fair and has rewritten a few commits coming from B so they contain malicious code?\n> How we can detect something like that and how C be sure that what he merges is really work\n> attributed by correct names?\n\nIf C doesn't trust A, C should not pull from A. C should pull only\nfrom (trusted) B. Presumably B knows who (of A and B) did which work,\nand B's repository can be trusted?\n\nIf neither of A or B can be trusted, then you have problems that a\ncomputer cannot solve for you.\n\nYou could maybe use signed tags (\"git help tag\") - each contributor\ncould sign a certain tree state, and if you see commits attributed to\nthe other contributor after their last tag, you know something is\nfishy. But that might be more effort than either you or your\ncontributors want to go through. And while it might help with\nattribution problems, it still doesn't help with all the other\nproblems you might have with untrusted contributors.\n\nPeter Harris\n"},{"id":"97308","messageId":"493C1F36.7050504@tuffmail.com","threadId":"16610","inReplyTo":"eaa105840812070857i27f8e920keaba3f92f5260b38@mail.gmail.com","subject":"Re: How to clone git repository with git-svn meta-data included?","fromName":"Grzegorz Kossakowski","fromEmail":"grek@tuffmail.com","sentAt":"2008-12-07T19:08:38Z","receivedAt":"2008-12-07T19:08:38Z","isPatch":false,"sender":{"key":"grek@tuffmail.com","avatar":"https://gravatar.com/avatar/98a80b7992e9f598c7b24addc73338dbacf94a14e4a14f3fa50c3f00dbee7dd3?d=mp&s=160"},"body":"Peter Harris pisze:\n> Make sure you don't use the --no-metadata flag when setting up\n> git-svn. This will embed the metadata into commit messages, so git-svn\n> can rebuild it from scratch whenever it needs to. (You probably also\n> want git 1.6.1rc for incremental rebuild support). This also has the\n> advantage that you can see the svn revision number when looking at a\n> commit message.\n\nNot sure what you exactly mean here. Do you mean that if metadata is included in commit messages\nthen there is an easy way to initialize git-svn after cloning the repo?\n\nBy easy I mean:\na) it does not require to much of interactive actions to be performed\nb) it does not pull too much from svn server\n\nPoint b) is important because we usually have quite large repositories.\n\nAlso, could you point me to a place where this rebuild support is described? I would like to know\nwhat our committer has to do after cloning from Jukka's server.\n\n> svn doesn't really know what a merge is (not even 1.5). You MUST\n> rebase in order to commit to svn. This is a limitation of svn, not\n> git.\n\nYep, I'm aware of the fact that history has to be flattened but I was more worried about the point\nyou address below.\n\n> In terms of re-pulling from the git-svn mirror, git-svn will create\n> the same commits (with the same sha1s) from svn every time, so there\n> will be no conflicts there.\n\nJust to make sure: so if one person pulls from git-svn mirror and another one pulls using git svn\nrebase they result in the same tree right?\n\n>> Is it possible with Git right now?\n> \n> Yes, but it might not be possible with svn, depending on which part of\n> the above \"it\" is.\n\nIt referred mostly to cloning from git svn mirror and then being able to use git svn dcommit to push\nchanges back to svn. Since git svn data is not being cloned the question is how to recreate it.\n\n>> What if A was not fair and has rewritten a few commits coming from B so they contain malicious code?\n>> How we can detect something like that and how C be sure that what he merges is really work\n>> attributed by correct names?\n> \n> If C doesn't trust A, C should not pull from A. C should pull only\n> from (trusted) B. Presumably B knows who (of A and B) did which work,\n> and B's repository can be trusted?\n> \n> If neither of A or B can be trusted, then you have problems that a\n> computer cannot solve for you.\n\nYep, I was having in mind the case when both A and B are untrusted. I don't want my computer to\ncheck if something coming from A or B is safe or not I just want to know which bits are coming from\nA and which from B.\n\nThis is really important for us because of legal reasons.\n\n> You could maybe use signed tags (\"git help tag\") - each contributor\n> could sign a certain tree state, and if you see commits attributed to\n> the other contributor after their last tag, you know something is\n> fishy. But that might be more effort than either you or your\n> contributors want to go through. And while it might help with\n> attribution problems, it still doesn't help with all the other\n> problems you might have with untrusted contributors.\n\nThe question is why Git doesn't sign all commits by default but only tags? Creating tags all the\ntime is rather tedious process and seems to have no sense, right?\n\nDoes it mean that with current Git design it's the best to not use advanced features of Git like\ntree merging but simply go with posting e-mails with patches instead if contributors cannot be trusted?\n\nThanks for your reply.\n\n-- \nBest regards,\nGrzegorz Kossakowski\n"},{"id":"97309","messageId":"eaa105840812071230l5e8d54bcg21b36019711bc3cd@mail.gmail.com","threadId":"16610","inReplyTo":"493C1F36.7050504@tuffmail.com","subject":"Re: How to clone git repository with git-svn meta-data included?","fromName":"Peter Harris","fromEmail":"git@peter.is-a-geek.org","sentAt":"2008-12-07T20:30:28Z","receivedAt":"2008-12-07T20:30:28Z","isPatch":false,"sender":{"key":"git@peter.is-a-geek.org","avatar":null},"body":"On Sun, Dec 7, 2008 at 2:08 PM, Grzegorz Kossakowski wrote:\n> Peter Harris pisze:\n>> Make sure you don't use the --no-metadata flag when setting up\n>> git-svn. This will embed the metadata into commit messages, so git-svn\n>> can rebuild it from scratch whenever it needs to. (You probably also\n>> want git 1.6.1rc for incremental rebuild support). This also has the\n>> advantage that you can see the svn revision number when looking at a\n>> commit message.\n>\n> Not sure what you exactly mean here. Do you mean that if metadata is included in commit messages\n> then there is an easy way to initialize git-svn after cloning the repo?\n\nYes.\n\n> By easy I mean:\n> a) it does not require to much of interactive actions to be performed\n> b) it does not pull too much from svn server\n>\n> Point b) is important because we usually have quite large repositories.\n\nTo set up the remotes to mirror the remote svn-remotes, I do the clone manually:\ngit init\ngit remote add origin git://svn/mirror\ngit config --add remote.origin.fetch +refs/remotes/*:refs/remotes/*\ngit fetch\ngit reset --hard trunk\n\nAfter the git clone, I do the following:\ngit svn init -s svn://repo/sitory\ngit svn rebase\n\nNo data is transferred[1], although 'git svn rebase' does spend a\nminute or so reading the commit messages to rebuild its index.\n\nThis could all be in a common script you distribute to your users.\n\n> Also, could you point me to a place where this rebuild support is described? I would like to know\n> what our committer has to do after cloning from Jukka's server.\n\n\"git help svn\" mentions the rebuild only in passing. I'm not sure if\nit is described in better detail elsewhere.\n\n>> In terms of re-pulling from the git-svn mirror, git-svn will create\n>> the same commits (with the same sha1s) from svn every time, so there\n>> will be no conflicts there.\n>\n> Just to make sure: so if one person pulls from git-svn mirror and another one pulls using git svn\n> rebase they result in the same tree right?\n\nYes[2].\n\n>> If C doesn't trust A, C should not pull from A. C should pull only\n>> from (trusted) B. Presumably B knows who (of A and B) did which work,\n>> and B's repository can be trusted?\n>>\n>> If neither of A or B can be trusted, then you have problems that a\n>> computer cannot solve for you.\n>\n> Yep, I was having in mind the case when both A and B are untrusted. I don't want my computer to\n> check if something coming from A or B is safe or not I just want to know which bits are coming from\n> A and which from B.\n>\n> This is really important for us because of legal reasons.\n\nIf something is in A's tree, it is coming from A. Either A has\nauthority, or A has received authority from someone else, or A is\nbringing the legal problem down on himself. When A says \"Please Pull\"\n(or when A pushes) A is effectively saying \"These changes are legally\nmine to give you\".\n\nThe Developer's Certificate of Origin 1.0 was designed to address this\nissue; see also \"Signed-off-by\"\n\nOf course, if it's a legal issue, make sure you consult your own lawyer.\n\n>> You could maybe use signed tags (\"git help tag\")...\n>\n> The question is why Git doesn't sign all commits by default but only tags? Creating tags all the\n> time is rather tedious process and seems to have no sense, right?\n\nTyping in your GPG passphrase for every single little commit would be\neven more tedious, IMHO.\n\n> Does it mean that with current Git design it's the best to not use advanced features of Git like\n> tree merging but simply go with posting e-mails with patches instead if contributors cannot be trusted?\n\nThat would be my policy. At the very least, I would have a human\nreview the tree before merging it.\n\nNote that git was designed around a \"git am\" workflow, so it is very\nefficient at dealing with large numbers of patches at a time.\n\nNote also that you can do tree merging with an email-patch based\nworkflow, since git format-patch preserves parent information,\nalthough it does take a little bit more work. See also: \"git help am\"\nunder --3way.\n\nPeter Harris\n\n[1] Not strictly true. git-svn does contact the svn server to see if\nthere are any revisions newer than the latest present in the git repo,\nand will transfer those revisions (if any).\n[2] Unless (a) someone edited the svn:log (or other) revprop in\nbetween, or (b) you triggered the bug I saw reported (and fixed?) on\nthis list today. I've never personally triggered (b), but I have seen\n(a).\n"},{"id":"97313","messageId":"493C47FD.4080302@tuffmail.com","threadId":"16610","inReplyTo":"eaa105840812071230l5e8d54bcg21b36019711bc3cd@mail.gmail.com","subject":"Re: How to clone git repository with git-svn meta-data included?","fromName":"Grzegorz Kossakowski","fromEmail":"grek@tuffmail.com","sentAt":"2008-12-07T22:02:37Z","receivedAt":"2008-12-07T22:02:37Z","isPatch":false,"sender":{"key":"grek@tuffmail.com","avatar":"https://gravatar.com/avatar/98a80b7992e9f598c7b24addc73338dbacf94a14e4a14f3fa50c3f00dbee7dd3?d=mp&s=160"},"body":"Peter Harris pisze:\n> After the git clone, I do the following:\n> git svn init -s svn://repo/sitory\n> git svn rebase\n> \n> No data is transferred[1], although 'git svn rebase' does spend a\n> minute or so reading the commit messages to rebuild its index.\n\nI've tried this method with Cocoon repository\n(http://jukka.zitting.name/git/?p=cocoon.git;a=summary) and got this error:\n\ngit clone git://jukka.zitting.name/cocoon.git\ngit svn init -s https://svn.eu.apache.org/repos/asf/cocoon/\ngit svn rebase\nUnable to determine upstream SVN information from working tree history\n\ngit --version\ngit version 1.6.0.2\n\nAny idea what's wrong here?\n\n> This could all be in a common script you distribute to your users.\n\nGood suggestion.\n\n> \"git help svn\" mentions the rebuild only in passing. I'm not sure if\n> it is described in better detail elsewhere.\n\nAh, I didn't spot this earlier. Thanks.\n\n> If something is in A's tree, it is coming from A. Either A has\n> authority, or A has received authority from someone else, or A is\n> bringing the legal problem down on himself. When A says \"Please Pull\"\n> (or when A pushes) A is effectively saying \"These changes are legally\n> mine to give you\".\n> \n> The Developer's Certificate of Origin 1.0 was designed to address this\n> issue; see also \"Signed-off-by\"\n> \n> Of course, if it's a legal issue, make sure you consult your own lawyer.\n\nI see. Thanks for insightful comments.\n\n>>> You could maybe use signed tags (\"git help tag\")...\n>> The question is why Git doesn't sign all commits by default but only tags? Creating tags all the\n>> time is rather tedious process and seems to have no sense, right?\n> \n> Typing in your GPG passphrase for every single little commit would be\n> even more tedious, IMHO.\n\nYep, that's true.\n\n>> Does it mean that with current Git design it's the best to not use advanced features of Git like\n>> tree merging but simply go with posting e-mails with patches instead if contributors cannot be trusted?\n> \n> That would be my policy. At the very least, I would have a human\n> review the tree before merging it.\n\nAgreed.\n\n> Note that git was designed around a \"git am\" workflow, so it is very\n> efficient at dealing with large numbers of patches at a time.\n> \n> Note also that you can do tree merging with an email-patch based\n> workflow, since git format-patch preserves parent information,\n> although it does take a little bit more work. See also: \"git help am\"\n> under --3way.\n\nThanks for all your valuable information. As soon as I resolve problem with git svn rebase I'll\nstart reading on how git am --3way works.\n\n-- \nBest regards,\nGrzegorz Kossakowski\n"},{"id":"97316","messageId":"eaa105840812071551w2106eb72k54ec68938628d51@mail.gmail.com","threadId":"16610","inReplyTo":"493C47FD.4080302@tuffmail.com","subject":"Re: How to clone git repository with git-svn meta-data included?","fromName":"Peter Harris","fromEmail":"git@peter.is-a-geek.org","sentAt":"2008-12-07T23:51:24Z","receivedAt":"2008-12-07T23:51:24Z","isPatch":false,"sender":{"key":"git@peter.is-a-geek.org","avatar":null},"body":"On Sun, Dec 7, 2008 at 5:02 PM, Grzegorz Kossakowski wrote:\n> Peter Harris pisze:\n>> After the git clone, I do the following:\n>> git svn init -s svn://repo/sitory\n>> git svn rebase\n>>\n>> No data is transferred[1], although 'git svn rebase' does spend a\n>> minute or so reading the commit messages to rebuild its index.\n>\n> I've tried this method with Cocoon repository\n> (http://jukka.zitting.name/git/?p=cocoon.git;a=summary) and got this error:\n>\n> git clone git://jukka.zitting.name/cocoon.git\n> git svn init -s https://svn.eu.apache.org/repos/asf/cocoon/\n> git svn rebase\n> Unable to determine upstream SVN information from working tree history\n\nOdd. Usually that indicates a lack of metadata, but that repo appears\nto contain the right stuff. Beats me.\n\nPeter Harris\n"},{"id":"97317","messageId":"20081208000344.GD27057@mail.local.tull.net","threadId":"16610","inReplyTo":"8c9a060812070043r472e10abu7a76152b5fe1314d@mail.gmail.com","subject":"Re: How to clone git repository with git-svn meta-data included?","fromName":"Nick Andrew","fromEmail":"nick@nick-andrew.net","sentAt":"2008-12-08T00:03:44Z","receivedAt":"2008-12-08T00:03:44Z","isPatch":false,"sender":{"key":"nick@nick-andrew.net","avatar":"https://gravatar.com/avatar/85f25a67ca6eaa4016ed374f6d07f3cd853c886aeb7e1507eb7dbc47b00082fe?d=mp&s=160"},"body":"On Sun, Dec 07, 2008 at 12:43:32AM -0800, Jacob Helwig wrote:\n> I use git-svn quite a bit at $work, but I haven't seen a way to clone\n> a git repo, and have it Just Work(TM) with git-svn in the new clone,\n> unfortunately.\n\nAt $work I nightly publish a .tar.gz file of a pristine git repo\nsynced from svn, with the metadata. So if a developer wants to start\nusing git they untar the file into their directory and start working,\nand after that they have to do \"git svn fetch\" themselves.\n\nNick.\n"},{"id":"97343","messageId":"493D1CC2.8050407@drmicha.warpmail.net","threadId":"16610","inReplyTo":"493C47FD.4080302@tuffmail.com","subject":"Re: How to clone git repository with git-svn meta-data included?","fromName":"Michael J Gruber","fromEmail":"git@drmicha.warpmail.net","sentAt":"2008-12-08T13:10:26Z","receivedAt":"2008-12-08T13:10:26Z","isPatch":false,"sender":{"key":"git@grubix.eu","avatar":"https://avatars.githubusercontent.com/u/233215?v=4"},"body":"Grzegorz Kossakowski venit, vidit, dixit 07.12.2008 23:02:\n> Peter Harris pisze:\n>> After the git clone, I do the following:\n>> git svn init -s svn://repo/sitory\n>> git svn rebase\n>>\n>> No data is transferred[1], although 'git svn rebase' does spend a\n>> minute or so reading the commit messages to rebuild its index.\n> \n> I've tried this method with Cocoon repository\n> (http://jukka.zitting.name/git/?p=cocoon.git;a=summary) and got this error:\n> \n> git clone git://jukka.zitting.name/cocoon.git\n> git svn init -s https://svn.eu.apache.org/repos/asf/cocoon/\n> git svn rebase\n> Unable to determine upstream SVN information from working tree history\n> \n> git --version\n> git version 1.6.0.2\n\nCould it be as simple as a missing \"cd cocoon\" between git clone and git\nsvn rebase? No, you probably did that.\n\nBut note that you did not follow Peter's instructions. The point is that\nyour clone creates \"remotes/origin/trunk\" whereas Peter's instructions\nmirror the source, creating \"remotes/trunk\", which is what git svn needs\n(unless you say \"git svn init -s --prefix=origin/\" or \"git config\nsvn-remote.svn.fetch trunk:refs/trunk\" etc.). The prefix solution should\nbe the best.\n\nMichael\n\nP.S.: Peter starts off a different layout (standard svn remotes, which\nneed special instructions to be cloned). Ordinary clone + git svn init\n--prefix=origin/ should work fine for the cocoon layout.\n\nP.P.S.: We can't test cocoon unless we have an account on the apache\nserver...\n"},{"id":"97350","messageId":"20081208161049.GB31551@spearce.org","threadId":"16610","inReplyTo":"493C1F36.7050504@tuffmail.com","subject":"Re: How to clone git repository with git-svn meta-data included?","fromName":"Shawn O. Pearce","fromEmail":"spearce@spearce.org","sentAt":"2008-12-08T16:10:49Z","receivedAt":"2008-12-08T16:10:49Z","isPatch":false,"sender":{"key":"spearce@spearce.org","avatar":"https://avatars.githubusercontent.com/u/34844?v=4"},"body":"Grzegorz Kossakowski <grek@tuffmail.com> wrote:\n> Peter Harris pisze:\n> >> What if A was not fair and has rewritten a few commits coming from B so they contain malicious code?\n> >> How we can detect something like that and how C be sure that what he merges is really work\n> >> attributed by correct names?\n> > \n> > If C doesn't trust A, C should not pull from A. C should pull only\n> > from (trusted) B. Presumably B knows who (of A and B) did which work,\n> > and B's repository can be trusted?\n> > \n> > If neither of A or B can be trusted, then you have problems that a\n> > computer cannot solve for you.\n> \n> Yep, I was having in mind the case when both A and B are untrusted. I don't want my computer to\n> check if something coming from A or B is safe or not I just want to know which bits are coming from\n> A and which from B.\n> \n> This is really important for us because of legal reasons.\n\nASF probably has issues similar to that of the Android project.\n\nIn Android we built Gerrit[1] to handle this validation of identity\nfor us, and to keep track of the contributor agreements each\nindividual and corporation has signed.  Changes aren't accepted\ninto Gerrit unless the user has an accepted CLA in the data store.\n\n*1* http://review.source.android.com/\n\nGerrit 2 is actively under development and is being ported off of\nGoogle App Engine, into a pure Java webapp.  I'm running it under\nJetty, but it should work just as well under Tomcat.  :-)\n\nIf the ASF becomes more committed to supporting Git, Gerrit may be\na good way to answer some of the questions you are having about\nvalidating identity of changes.  Plus its a handy source code\nreview tool.\n \n> > You could maybe use signed tags (\"git help tag\") - each contributor\n> > could sign a certain tree state, [...]\n> \n> The question is why Git doesn't sign all commits by default but only tags? Creating tags all the\n> time is rather tedious process and seems to have no sense, right?\n\nYea, its tedious to unlock your GnuPG key every time you make a\ncommit.  Especially if you are just rebasing a series or something\nto fix a minor mistake 5 commits back before uploading somewhere.\n \n> Does it mean that with current Git design it's the best to not use advanced features of Git like\n> tree merging but simply go with posting e-mails with patches instead if contributors cannot be trusted?\n\nMost Git projects rely on patches sent to an email list, with\na single maintainer applying them to to his/her repository, and\npublishing the result.  The maintainer is thus forced to keep track\nof the CLAs (if the project uses such things) and just trust the\nFrom address of the message.\n\nCLAs in the kernel and in git itself are less enforced than say\nwhat ASF or Android requires.\n\nSome Git projects give write access to the master repository to\nmultiple trusted parties; SAMBA and X.org are good examples of this\nsort of strategy.  But I think in these cases those who have write\naccess are also very long standing members of the development\ncommunity who have known each other in person for many years,\nperhaps far longer than a DVCS concept has existed.  So trust\nbetween those with direct write access is slightly less of an\nissue for these projects.\n\nSo long story short, I think Gerrit may be worth the ASF's time,\nif Git is a serious consideration for replacing SVN.  But while a\nproject is based in SVN I think the best you can do with Git is\npublish an automatically updated git-svn mirror and permit only\nuse of \"git svn dcommit\" to upload back into the SVN repository.\n\n-- \nShawn.\n"},{"id":"97372","messageId":"493D66BB.3060907@tuffmail.com","threadId":"16610","inReplyTo":"493D1CC2.8050407@drmicha.warpmail.net","subject":"Re: How to clone git repository with git-svn meta-data included?","fromName":"Grzegorz Kossakowski","fromEmail":"grek@tuffmail.com","sentAt":"2008-12-08T18:26:03Z","receivedAt":"2008-12-08T18:26:03Z","isPatch":false,"sender":{"key":"grek@tuffmail.com","avatar":"https://gravatar.com/avatar/98a80b7992e9f598c7b24addc73338dbacf94a14e4a14f3fa50c3f00dbee7dd3?d=mp&s=160"},"body":"Michael J Gruber pisze:\n> Could it be as simple as a missing \"cd cocoon\" between git clone and git\n> svn rebase? No, you probably did that.\n\nThat would be too easy.\n\n> But note that you did not follow Peter's instructions. The point is that\n> your clone creates \"remotes/origin/trunk\" whereas Peter's instructions\n> mirror the source, creating \"remotes/trunk\", which is what git svn needs\n> (unless you say \"git svn init -s --prefix=origin/\" or \"git config\n> svn-remote.svn.fetch trunk:refs/trunk\" etc.). The prefix solution should\n> be the best.\n> \n> Michael\n> \n> P.S.: Peter starts off a different layout (standard svn remotes, which\n> need special instructions to be cloned). Ordinary clone + git svn init\n> --prefix=origin/ should work fine for the cocoon layout.\n\nThis almost worked. Actually, Cocoon repository hosted on Jukka's server does not have local head\nnamed \"trunk\" so there is no remotes/origin/trunk created during cloning process.\n\nI had to run:\n\n  git update-ref refs/remotes/origin/trunk refs/heads/master\n\nAfter doing that git svn rebase resulted in:\n[really long list of revisions]\nr707379 = f61a2d30b6ac5a5136b46fa2b9b5b91e4763feb1\nr710118 = 40997fe552e8581b75b08fed41a6b63a33d58bdf\nr720135 = a8160766ec40fd7ebf95bfa7cebfa50dfa2f9c3a\nr720180 = b094a222bab3671c8277087e7a96589ec76dd5e4\nr720182 = 736b8ed6519c64ad120de2ccf08f135062ee09db\nDone rebuilding .git/svn/origin/trunk/.rev_map.13f79535-47bb-0310-9956-ffa450edef68\nCurrent branch master is up to date.\n\nIs this expected output?\n\n> P.P.S.: We can't test cocoon unless we have an account on the apache\n> server...\n\nI guess this would be a big problem. Getting write-access access to our repository is rather formal\nprocess and I would like to avoid it.\n\nSince it looks I'm much closer to final solution (and better understanding of how our workflow\nshould look like) I hope any additional account for testing won't be needed.\n\nAnyway, thanks again for your answer!\n\n-- \nBest regards,\nGrzegorz Kossakowski\n"},{"id":"97374","messageId":"eaa105840812081040s1036b79an9914c1f74d6d7f6a@mail.gmail.com","threadId":"16610","inReplyTo":"493D66BB.3060907@tuffmail.com","subject":"Re: How to clone git repository with git-svn meta-data included?","fromName":"Peter Harris","fromEmail":"peter@peter.is-a-geek.org","sentAt":"2008-12-08T18:40:02Z","receivedAt":"2008-12-08T18:40:02Z","isPatch":false,"sender":{"key":"peter@peter.is-a-geek.org","avatar":null},"body":"On Mon, Dec 8, 2008 at 1:26 PM, Grzegorz Kossakowski wrote:\n>\n> After doing that git svn rebase resulted in:\n> [really long list of revisions]\n> r707379 = f61a2d30b6ac5a5136b46fa2b9b5b91e4763feb1\n> r710118 = 40997fe552e8581b75b08fed41a6b63a33d58bdf\n> r720135 = a8160766ec40fd7ebf95bfa7cebfa50dfa2f9c3a\n> r720180 = b094a222bab3671c8277087e7a96589ec76dd5e4\n> r720182 = 736b8ed6519c64ad120de2ccf08f135062ee09db\n> Done rebuilding .git/svn/origin/trunk/.rev_map.13f79535-47bb-0310-9956-ffa450edef68\n> Current branch master is up to date.\n>\n> Is this expected output?\n\nYes. The rfoo = sha1hash part is git-svn rebuilding its index.\n\"Current branch master is up to date\" is git-svn calling \"git rebase\n<svn-branch>\", and git saying that there is nothing to do, since there\nhave been no svn commits to that branch since the last time you ran\ngit svn rebase (or since you cloned the git mirror, or since the last\ntime the git mirror pulled from svn).\n\nPeter Harris\n"},{"id":"97375","messageId":"493D6AE9.6020504@tuffmail.com","threadId":"16610","inReplyTo":"eaa105840812081040s1036b79an9914c1f74d6d7f6a@mail.gmail.com","subject":"Re: How to clone git repository with git-svn meta-data included?","fromName":"Grzegorz Kossakowski","fromEmail":"grek@tuffmail.com","sentAt":"2008-12-08T18:43:53Z","receivedAt":"2008-12-08T18:43:53Z","isPatch":false,"sender":{"key":"grek@tuffmail.com","avatar":"https://gravatar.com/avatar/98a80b7992e9f598c7b24addc73338dbacf94a14e4a14f3fa50c3f00dbee7dd3?d=mp&s=160"},"body":"Peter Harris pisze:\n> On Mon, Dec 8, 2008 at 1:26 PM, Grzegorz Kossakowski wrote:\n>> After doing that git svn rebase resulted in:\n>> [really long list of revisions]\n>> r707379 = f61a2d30b6ac5a5136b46fa2b9b5b91e4763feb1\n>> r710118 = 40997fe552e8581b75b08fed41a6b63a33d58bdf\n>> r720135 = a8160766ec40fd7ebf95bfa7cebfa50dfa2f9c3a\n>> r720180 = b094a222bab3671c8277087e7a96589ec76dd5e4\n>> r720182 = 736b8ed6519c64ad120de2ccf08f135062ee09db\n>> Done rebuilding .git/svn/origin/trunk/.rev_map.13f79535-47bb-0310-9956-ffa450edef68\n>> Current branch master is up to date.\n>>\n>> Is this expected output?\n> \n> Yes. The rfoo = sha1hash part is git-svn rebuilding its index.\n> \"Current branch master is up to date\" is git-svn calling \"git rebase\n> <svn-branch>\", and git saying that there is nothing to do, since there\n> have been no svn commits to that branch since the last time you ran\n> git svn rebase (or since you cloned the git mirror, or since the last\n> time the git mirror pulled from svn).\n\nThanks for confirmation and explanation.\n\nThe remaining question is who should address this issue with non-existing trunk ref? Should I ask\nJukka, who maintains svn mirrors, to put instruction into his scripts that will add trunk reference?\n\nWould it be the best practice?\n\n-- \nBest regards,\nGrzegorz Kossakowski\n"},{"id":"97376","messageId":"493D6FB9.3060706@tuffmail.com","threadId":"16610","inReplyTo":"20081208161049.GB31551@spearce.org","subject":"Re: How to clone git repository with git-svn meta-data included?","fromName":"Grzegorz Kossakowski","fromEmail":"grek@tuffmail.com","sentAt":"2008-12-08T19:04:25Z","receivedAt":"2008-12-08T19:04:25Z","isPatch":false,"sender":{"key":"grek@tuffmail.com","avatar":"https://gravatar.com/avatar/98a80b7992e9f598c7b24addc73338dbacf94a14e4a14f3fa50c3f00dbee7dd3?d=mp&s=160"},"body":"Shawn O. Pearce pisze:\n> ASF probably has issues similar to that of the Android project.\n> \n> In Android we built Gerrit[1] to handle this validation of identity\n> for us, and to keep track of the contributor agreements each\n> individual and corporation has signed.  Changes aren't accepted\n> into Gerrit unless the user has an accepted CLA in the data store.\n> \n> *1* http://review.source.android.com/\n\nI'm not an expert on legal issues at Apache but in general you might be right that ASF and Android\nproject have similar issues to tackle. I've brought this point because it was previously brought in\nASF discussion on Git. so I try to act more like a bridge between two parties.\n\n> Gerrit 2 is actively under development and is being ported off of\n> Google App Engine, into a pure Java webapp.  I'm running it under\n> Jetty, but it should work just as well under Tomcat.  :-)\n\nIs there any documentation outlining Gerrit's features and status of Gerrit 2 development?\n\n> If the ASF becomes more committed to supporting Git, Gerrit may be\n> a good way to answer some of the questions you are having about\n> validating identity of changes.  Plus its a handy source code\n> review tool.\n\nI guess it's rather long run before Apache Infrastructure team *officially* starts supporting Git.\nThis is my personal view and it's not official voice of ASF by any means. SVN to Git transition\n(entirely) is complicated process for such a big organization like ASF.\n\nAnother option would be to let particular projects choose which SCM they prefer but that would mean\nthat our infra team has to support two different SCMs. This means some new folks interested in\nsupporting Git at ASF would have to join infra.\n\nAt this point, we chose to let people experiment with Git (so we are allowed to perform rather\nexpensive clone operations) in order to gain some experience and work out work-flow that suits ASF\nmodel of development. This thread is part of described process.\n\nOn the other hand, if we find Gerrit useful for us we might try to set up it somewhere unofficially\nand let people play with it.\n\n>> Does it mean that with current Git design it's the best to not use advanced features of Git like\n>> tree merging but simply go with posting e-mails with patches instead if contributors cannot be trusted?\n> \n> Most Git projects rely on patches sent to an email list, with\n> a single maintainer applying them to to his/her repository, and\n> publishing the result.  The maintainer is thus forced to keep track\n> of the CLAs (if the project uses such things) and just trust the\n> From address of the message.\n\nIt's rather important to keep in mind that at ASF we don't how practice of single maintainer so we\nneed solution that works for many committers that push to the same repo but I know Git supports this\nrather easily.\n\n> CLAs in the kernel and in git itself are less enforced than say\n> what ASF or Android requires.\n> \n> Some Git projects give write access to the master repository to\n> multiple trusted parties; SAMBA and X.org are good examples of this\n> sort of strategy.  But I think in these cases those who have write\n> access are also very long standing members of the development\n> community who have known each other in person for many years,\n> perhaps far longer than a DVCS concept has existed.  So trust\n> between those with direct write access is slightly less of an\n> issue for these projects.\n\nThat is a case for Apache to some extent. Someone is being nominated to become a committer after\nsome period of cooperation in terms of providing patch, participating in discussions etc. After such\nperiod existing committers can rather reliably judge if they trust particular person or not.\n\nYet still signed CLA is required but we check this before someone is being granted write access so\nthis can be done by hand.\n\n> So long story short, I think Gerrit may be worth the ASF's time,\n> if Git is a serious consideration for replacing SVN.  But while a\n> project is based in SVN I think the best you can do with Git is\n> publish an automatically updated git-svn mirror and permit only\n> use of \"git svn dcommit\" to upload back into the SVN repository.\n\nAs I said, I don't think ASF will switch to Git any time soon due to many different reasons\nincluding technical (IDE support), social (there are different opinions on DVCS in general) and\nother. All of them have to be addressed but in order to have any meaningful discussion we need to\ngather some experience.\n\nThanks for helping me with going through this process!\n\n-- \nBest regards,\nGrzegorz Kossakowski\n"},{"id":"97409","messageId":"493E3209.9060105@drmicha.warpmail.net","threadId":"16610","inReplyTo":"493D66BB.3060907@tuffmail.com","subject":"Re: How to clone git repository with git-svn meta-data included?","fromName":"Michael J Gruber","fromEmail":"git@drmicha.warpmail.net","sentAt":"2008-12-09T08:53:29Z","receivedAt":"2008-12-09T08:53:29Z","isPatch":false,"sender":{"key":"git@grubix.eu","avatar":"https://avatars.githubusercontent.com/u/233215?v=4"},"body":"Grzegorz Kossakowski venit, vidit, dixit 08.12.2008 19:26:\n> Michael J Gruber pisze:\n>> Could it be as simple as a missing \"cd cocoon\" between git clone and git\n>> svn rebase? No, you probably did that.\n> \n> That would be too easy.\n> \n>> But note that you did not follow Peter's instructions. The point is that\n>> your clone creates \"remotes/origin/trunk\" whereas Peter's instructions\n>> mirror the source, creating \"remotes/trunk\", which is what git svn needs\n>> (unless you say \"git svn init -s --prefix=origin/\" or \"git config\n>> svn-remote.svn.fetch trunk:refs/trunk\" etc.). The prefix solution should\n>> be the best.\n>>\n>> Michael\n>>\n>> P.S.: Peter starts off a different layout (standard svn remotes, which\n>> need special instructions to be cloned). Ordinary clone + git svn init\n>> --prefix=origin/ should work fine for the cocoon layout.\n> \n> This almost worked. Actually, Cocoon repository hosted on Jukka's server does not have local head\n> named \"trunk\" so there is no remotes/origin/trunk created during cloning process.\n> \n> I had to run:\n> \n>   git update-ref refs/remotes/origin/trunk refs/heads/master\n\nUhm, I misread gitweb output... So, cocoon really has remotes/trunk, so\nthat Peter's original suggestion would work. If the cocoon git-svn clone\nisn't going to change then Peter's way of doing it might be the best: it\nensures that future pulls from origin (cocoon) will update the correct\nrefs so that you can pull/fetch from the git-svn clone (fast) and then\ngit-svn rebase rather than git-svn rebasing directly, which would fetch\nnew commits from the svn repo first (slow).\n\nI think all in all it shows that git-svns default location for branches\nis not the best choice: they're not local, but they're no \"proper\"\nremotes either (unless you use --prefix).\n\nMichael\n"},{"id":"97452","messageId":"1228813734.28186.77.camel@maia.lan","threadId":"16610","inReplyTo":"493D6AE9.6020504@tuffmail.com","subject":"Re: How to clone git repository with git-svn meta-data included?","fromName":"Sam Vilain","fromEmail":"sam@vilain.net","sentAt":"2008-12-09T09:08:54Z","receivedAt":"2008-12-09T09:08:54Z","isPatch":false,"sender":{"key":"sam@vilain.net","avatar":"https://gravatar.com/avatar/8fc840ca854dbf6f7065b4335e3b934951c1dca3b11db688e95e471901f8f4a8?d=mp&s=160"},"body":"On Mon, 2008-12-08 at 19:43 +0100, Grzegorz Kossakowski wrote:\n> > Yes. The rfoo = sha1hash part is git-svn rebuilding its index.\n> > \"Current branch master is up to date\" is git-svn calling \"git rebase\n> > <svn-branch>\", and git saying that there is nothing to do, since there\n> > have been no svn commits to that branch since the last time you ran\n> > git svn rebase (or since you cloned the git mirror, or since the last\n> > time the git mirror pulled from svn).\n> \n> Thanks for confirmation and explanation.\n> \n> The remaining question is who should address this issue with non-existing trunk ref? Should I ask\n> Jukka, who maintains svn mirrors, to put instruction into his scripts that will add trunk reference?\n\nIt's up to the git-svn user to make sure that they prepare the refs to\nbe what git-svn expects.  This is something probably requiring more\ndocumentation and/or git-svn features to be easier.\n\n> Would it be the best practice?\n\nWell, obscure stuff should never really be best practice.  The best practice\nis to have a single git repository that is where the svn -> git migration\nhappens.  And git-svn could perhaps auto-init based on information in the\ncommit log or something.  Best practice is to enhance the tool to work the\nway it Should(tm) :)\n\nSam\n"},{"id":"97454","messageId":"493EDBA2.3070404@tuffmail.com","threadId":"16610","inReplyTo":"1228813734.28186.77.camel@maia.lan","subject":"Re: How to clone git repository with git-svn meta-data included?","fromName":"Grzegorz Kossakowski","fromEmail":"grek@tuffmail.com","sentAt":"2008-12-09T20:57:06Z","receivedAt":"2008-12-09T20:57:06Z","isPatch":false,"sender":{"key":"grek@tuffmail.com","avatar":"https://gravatar.com/avatar/98a80b7992e9f598c7b24addc73338dbacf94a14e4a14f3fa50c3f00dbee7dd3?d=mp&s=160"},"body":"Sam Vilain pisze:\n> It's up to the git-svn user to make sure that they prepare the refs to\n> be what git-svn expects.  This is something probably requiring more\n> documentation and/or git-svn features to be easier.\n\nWhat I was asking if we should add trunk ref to svn mirror so others cloning it will have\n'origin/trunk' reference created automatically during clone process so no extra steps would be needed.\n\nTo be honest, I don't understand how Git exactly handles all this refs mapping and rewriting (e.g.\nduring cloning). Having said that, I cannot foresee all possible implications of choosing particular\nmethod of solving current issues thus asking you.\n\n>> Would it be the best practice?\n> \n> Well, obscure stuff should never really be best practice.  The best practice\n> is to have a single git repository that is where the svn -> git migration\n> happens.  And git-svn could perhaps auto-init based on information in the\n> commit log or something.  Best practice is to enhance the tool to work the\n> way it Should(tm) :)\n\nCannot follow you here. We want single svn mirror but at the same time we want to our committers to\nbe able to push back to svn. What has been already proposed satisfies my need apart from the fact\nthat currently there is  small problem with our mirror setup.\n\n-- \nBest regards,\nGrzegorz Kossakowski\n"}]}