{"thread":{"id":"31267","subject":"GIT archive invocation with SHA sum","startedAt":"2012-08-17T06:25:53Z","lastAt":"2012-08-17T14:20:17Z","messageCount":3,"participants":["Manfred Rudigier","Junio C Hamano"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"197166","messageId":"95DC1AA8EC908B48939B72CF375AA5E301473171E4@alice.at.omicron.at","threadId":"31267","inReplyTo":null,"subject":"GIT archive invocation with SHA sum","fromName":"Manfred Rudigier","fromEmail":"manfred.rudigier@omicron.at","sentAt":"2012-08-17T06:25:53Z","receivedAt":"2012-08-17T06:25:53Z","isPatch":false,"sender":{"key":"manfred.rudigier@omicron.at","avatar":null},"body":"Hello,\n\nwe have recently upgraded our Ubuntu server to 12.04, which comes with a newer version of GIT. However, we have noticed that git archive -remote does not work anymore by specifying the SHA sum of a commit.\n\nBy seaching the mailing list I have found out that this was already discussed before (http://thread.gmane.org/gmane.comp.version-control.git/191815). In our company we use the git archive -remote command for a similar purpose in our build system and we must currently stick with an older version of GIT until there is a solution for this.\n\nI have read that this restriction was made for some kind of security reason, and one possibility would be to add a config option on the server repo to turn off this safety check. As we manage the git repos ourselves, this would be perfectly ok for us.\n\nWhat is the current status of this - is there already such a config option?\n\nRegards,\nManfred Rudigier\n"},{"id":"197169","messageId":"7vboiavv7r.fsf@alter.siamese.dyndns.org","threadId":"31267","inReplyTo":"95DC1AA8EC908B48939B72CF375AA5E301473171E4@alice.at.omicron.at","subject":"Re: GIT archive invocation with SHA sum","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2012-08-17T07:22:32Z","receivedAt":"2012-08-17T07:22:32Z","isPatch":false,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Manfred Rudigier <manfred.rudigier@omicron.at> writes:\n\n> we have recently upgraded our Ubuntu server to 12.04, which comes\n> with a newer version of GIT. However, we have noticed that git\n> archive -remote does not work anymore by specifying the SHA sum of\n> a commit.\n> ...\n> I have read that this restriction was made for some kind of\n> security reason, and one possibility would be to add a config\n> option on the server repo to turn off this safety check. As we\n> manage the git repos ourselves, this would be perfectly ok for us.\n>\n> What is the current status of this - is there already such a config option?\n\nNo, there isn't.\n\nI am no going to say that such a change will never happen, but you\nmay want to rethink the way you transfer the source material to your\nbuild system, especially your build system is rebuilding each and\nevery versions of the software.\n\nI can understand that the use of \"git archive\" can give you a way to\nstart from an empty directory and build afresh for every version,\nbut it is stupid to _transfer_ the full copy of the source material\n300 times over the network in order to rebuild 300 versions a day,\nall of which will be quite similar to each other (after all, the\ndifference between them will be limited by the productivity of your\nworkforce).\n\nFortunately, it is quite simple to fix.  You can maintain a single\nbare repository on the build machine (i.e. the one you currently run\n\"git archive --remote\" on), and run \"git fetch --mirror\" against the\nmachine that hosts the repository (i.e. the one you currently run\n\"git archive --remote\" against) to keep it up to date.  That way,\nyou should be able to drastically cut the amount of the network\ntraffic from the source repository and the build server.\n\nThen, you copy out from that bare repository (i.e. the one you have\non your build server that you \"git fetch --mirror\" into) using \"git\narchive\" into a fresh directory to build.  Because this invocation\nof \"git archive\" is purely local to the machine, you do not need any\nconfiguration option to bust the \"you can only see the branches\navailable to you\" security measure to copy things out.\n\nIf your build server is building only the tagged versions for\nreleases (I doubt that is the case, but I am writing this just for\ncompleteness), \"git archive --remove\" should be able to reach them,\nso you do not need such a configuration option, either.\n\nHrm?\n"},{"id":"197182","messageId":"95DC1AA8EC908B48939B72CF375AA5E30147317325@alice.at.omicron.at","threadId":"31267","inReplyTo":"7vboiavv7r.fsf@alter.siamese.dyndns.org","subject":"RE: GIT archive invocation with SHA sum","fromName":"Manfred Rudigier","fromEmail":"manfred.rudigier@omicron.at","sentAt":"2012-08-17T14:20:17Z","receivedAt":"2012-08-17T14:20:17Z","isPatch":false,"sender":{"key":"manfred.rudigier@omicron.at","avatar":null},"body":"Junio C Hamano [mailto:gitster@pobox.com] writes:\n\n>> I have read that this restriction was made for some kind of security\n>> reason, and one possibility would be to add a config option on the\n>> server repo to turn off this safety check. As we manage the git repos\n>> ourselves, this would be perfectly ok for us.\n>>\n>> What is the current status of this - is there already such a config option?\n>\n>No, there isn't.\n>\n>I am no going to say that such a change will never happen, but you may want\n>to rethink the way you transfer the source material to your build system,\n>especially your build system is rebuilding each and every versions of the\n>software.\n>\n>I can understand that the use of \"git archive\" can give you a way to start from\n>an empty directory and build afresh for every version, but it is stupid to\n>_transfer_ the full copy of the source material\n>300 times over the network in order to rebuild 300 versions a day, all of which\n>will be quite similar to each other (after all, the difference between them will\n>be limited by the productivity of your workforce).\n\nOur build system creates an embedded firmware image similar to OpenEmbedded or OpenWrt. There all components (including boot loader, kernel, busybox and many other third party components) are compiled during nightly builds. Each component has its own GIT repository on the server and our build system references those components via SHA-1 sums. Many of those components do almost NEVER change (we do not update the Linux kernel every day), so most of this code must only be fetched once using git archive.\n\n>Fortunately, it is quite simple to fix.  You can maintain a single bare repository\n>on the build machine (i.e. the one you currently run \"git archive --remote\"\n>on), and run \"git fetch --mirror\" against the machine that hosts the repository\n>(i.e. the one you currently run \"git archive --remote\" against) to keep it up to\n>date.  That way, you should be able to drastically cut the amount of the\n>network traffic from the source repository and the build server.\n\nI know that I could workaround this by mirroring all repositories locally, but it is much more convenient to use git archive right away. I agree that once the repos are mirrored, there is less traffic on the network, but traffic has not been an issue for us so far. Also, a complete mirror of all involved repositories will be much larger than just the exported tar archives (e.g. Linux kernel repo is 770MB, while the tar.gz is just 96MB).\n\nIt was a really nice feature of GIT and it is just too bad that it is no longer available.\n\nRegards,\nManfred\n"}]}