{"thread":{"id":"10109","subject":"git clone questions relating to cpio","startedAt":"2007-10-01T19:28:38Z","lastAt":"2007-10-03T16:05:46Z","messageCount":14,"participants":["Reece Dunn","Johan Herland","Johannes Schindelin","Junio C Hamano","Johannes Sixt","Chris Larson","Miklos Vajna","Shawn O. Pearce"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"54501","messageId":"3f4fd2640710011228w61ce34b5ve47ea529eed384fd@mail.gmail.com","threadId":"10109","inReplyTo":null,"subject":"git clone questions relating to cpio","fromName":"Reece Dunn","fromEmail":"msclrhd@googlemail.com","sentAt":"2007-10-01T19:28:38Z","receivedAt":"2007-10-01T19:28:38Z","isPatch":false,"sender":{"key":"msclrhd@googlemail.com","avatar":null},"body":"Hi,\n\nI am running a Linux From Scratch 6.2 system that does not have cpio\ninstalled on it. This means that I can't clone a local repository\nunless I install cpio. Is it possible to use a fallback method if cpio\nis not present, as there is no NO_CPIO option on make like there is\nfor OpenSSH, cURL and expat?\n\nAlso, I have an external USB hardrive that is mounted onto the virtual\nfilesystem. Will clones from the USB harddrive (or a USB flash drive\nthat is mounted) result in a copy being performed, not a hardlink?\n\nIdeally, the hard linking for local clones should be optional. What if\nI want to move a repository because, for example, I have imported a\nCVS repository and now want to push it to a new bare repository?\n\n- Reece\n"},{"id":"54505","messageId":"200710012342.37352.johan@herland.net","threadId":"10109","inReplyTo":"3f4fd2640710011228w61ce34b5ve47ea529eed384fd@mail.gmail.com","subject":"Re: git clone questions relating to cpio","fromName":"Johan Herland","fromEmail":"johan@herland.net","sentAt":"2007-10-01T21:42:36Z","receivedAt":"2007-10-01T21:42:36Z","isPatch":false,"sender":{"key":"johan@herland.net","avatar":"https://avatars.githubusercontent.com/u/547031?v=4"},"body":"On Monday 01 October 2007, Reece Dunn wrote:\n> Hi,\n> \n> I am running a Linux From Scratch 6.2 system that does not have cpio\n> installed on it. This means that I can't clone a local repository\n> unless I install cpio. Is it possible to use a fallback method if cpio\n> is not present, as there is no NO_CPIO option on make like there is\n> for OpenSSH, cURL and expat?\n\nUsing \"file://\" when specifying the source repo will force git-clone to use \nthe git protocol, instead of doing a copy/hardlink.\n\nI.e. change \"git clone foo bar\" to \"git clone file://foo bar\" in order to \nprevent git-clone from calling cpio.\n\nHowever, grepping for cpio in the git source tree reveals a couple of uses \nin git-merge, so you might bump into problems there...\n\n> Also, I have an external USB hardrive that is mounted onto the virtual\n> filesystem. Will clones from the USB harddrive (or a USB flash drive\n> that is mounted) result in a copy being performed, not a hardlink?\n\nHardlinks are impossible across filesystems. If you're cloning to a \ndifferent filesystem git will _have_ to make a full copy.\n\n> Ideally, the hard linking for local clones should be optional.\n\n<quote src=\"git-clone(1)\">\n--local, -l\n\n  When the repository to clone from is on a local machine, this flag \nbypasses normal \"git aware\" transport mechanism and clones the repository \nby making a copy of HEAD and everything under objects and refs directories. \nThe files under .git/objects/ directory are hardlinked to save space when \npossible. This is now the default when the source repository is specified \nwith /path/to/repo syntax, so it essentially is a no-op option. To force \ncopying instead of hardlinking (which may be desirable if you are trying to \nmake a back-up of your repository), but still avoid the usual \"git aware\" \ntransport mechanism, --no-hardlinks can be used.\n\n--no-hardlinks\n\n  Optimize the cloning process from a repository on a local filesystem by \ncopying files under .git/objects directory.\n</quote>\n\nAnd as I said above, you can use \"file://\" to force the \"git aware\" \ntransport mechanism, which bypasses the whole local copy/hardlink issue \nentirely.\n\n> What if I want to move a repository because, for example, I have imported\n> a CVS repository and now want to push it to a new bare repository?\n\nEven if you were to use hardlinks, cloning a repo followed by deleting the \noriginal will be safe (as long as you don't supply '--shared' to \ngit-clone). That's the beauty of hardlinks.\n\nI also think it's fairly safe to just 'mv' the whole repository to its new \nlocation.\n\n\nHave fun! :)\n\n...Johan\n\n-- \nJohan Herland, <johan@herland.net>\nwww.herland.net\n"},{"id":"54519","messageId":"Pine.LNX.4.64.0710020022470.28395@racer.site","threadId":"10109","inReplyTo":"3f4fd2640710011228w61ce34b5ve47ea529eed384fd@mail.gmail.com","subject":"Re: git clone questions relating to cpio","fromName":"Johannes Schindelin","fromEmail":"johannes.schindelin@gmx.de","sentAt":"2007-10-01T23:23:57Z","receivedAt":"2007-10-01T23:23:57Z","isPatch":false,"sender":{"key":"johannes.schindelin@gmx.de","avatar":"https://avatars.githubusercontent.com/u/127790?v=4"},"body":"Hi,\n\nOn Mon, 1 Oct 2007, Reece Dunn wrote:\n\n> I am running a Linux From Scratch 6.2 system that does not have cpio\n> installed on it. This means that I can't clone a local repository\n> unless I install cpio.\n\nYou might be interested in the workaround Hannes did in mingw.git; he made \na wrapper script called 'cpio' using 'tar'.\n\nCiao,\nDscho\n"},{"id":"54520","messageId":"7vwsu6pg0v.fsf@gitster.siamese.dyndns.org","threadId":"10109","inReplyTo":"Pine.LNX.4.64.0710020022470.28395@racer.site","subject":"Re: git clone questions relating to cpio","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2007-10-01T23:32:48Z","receivedAt":"2007-10-01T23:32:48Z","isPatch":false,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Johannes Schindelin <Johannes.Schindelin@gmx.de> writes:\n\n> On Mon, 1 Oct 2007, Reece Dunn wrote:\n>\n>> I am running a Linux From Scratch 6.2 system that does not have cpio\n>> installed on it. This means that I can't clone a local repository\n>> unless I install cpio.\n>\n> You might be interested in the workaround Hannes did in mingw.git; he made \n> a wrapper script called 'cpio' using 'tar'.\n\nI think that may be good enough as workaround, but I do not\nthink you would get the space saving from hardlinks that way.\n"},{"id":"54532","messageId":"4701E10C.7050405@viscovery.net","threadId":"10109","inReplyTo":"7vwsu6pg0v.fsf@gitster.siamese.dyndns.org","subject":"Re: git clone questions relating to cpio","fromName":"Johannes Sixt","fromEmail":"j.sixt@viscovery.net","sentAt":"2007-10-02T06:11:24Z","receivedAt":"2007-10-02T06:11:24Z","isPatch":false,"sender":{"key":"j6t@kdbg.org","avatar":"https://avatars.githubusercontent.com/u/14810926?v=4"},"body":"Junio C Hamano schrieb:\n> Johannes Schindelin <Johannes.Schindelin@gmx.de> writes:\n> \n>> On Mon, 1 Oct 2007, Reece Dunn wrote:\n>>\n>>> I am running a Linux From Scratch 6.2 system that does not have cpio\n>>> installed on it. This means that I can't clone a local repository\n>>> unless I install cpio.\n>> You might be interested in the workaround Hannes did in mingw.git; he made \n>> a wrapper script called 'cpio' using 'tar'.\n> \n> I think that may be good enough as workaround, but I do not\n> think you would get the space saving from hardlinks that way.\n\nFWIW, I'm thinking about changing the cpio -p (passthrough) part to use\n'cp -l --parents --target-directory=...' instead of tar; this gives us hard \nlinks, even on NTFS. But it needs GNU's cp, of course.\n\n-- Hannes\n"},{"id":"54534","messageId":"7vy7emm3v9.fsf@gitster.siamese.dyndns.org","threadId":"10109","inReplyTo":"4701E10C.7050405@viscovery.net","subject":"Re: git clone questions relating to cpio","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2007-10-02T06:23:38Z","receivedAt":"2007-10-02T06:23:38Z","isPatch":false,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Johannes Sixt <j.sixt@viscovery.net> writes:\n\n> FWIW, I'm thinking about changing the cpio -p (passthrough) part to use\n> 'cp -l --parents --target-directory=...' instead of tar; this gives us\n> hard links, even on NTFS. But it needs GNU's cp, of course.\n\nYeah, that's the reason it is not appropriate than cpio, even if\nwe forget about Windows.\n"},{"id":"54610","messageId":"3f4fd2640710021409q104a4204r522391912d5b5c8c@mail.gmail.com","threadId":"10109","inReplyTo":"200710012342.37352.johan@herland.net","subject":"Re: git clone questions relating to cpio","fromName":"Reece Dunn","fromEmail":"msclrhd@googlemail.com","sentAt":"2007-10-02T21:09:52Z","receivedAt":"2007-10-02T21:09:52Z","isPatch":false,"sender":{"key":"msclrhd@googlemail.com","avatar":null},"body":"On 01/10/2007, Johan Herland <johan@herland.net> wrote:\n> On Monday 01 October 2007, Reece Dunn wrote:\n> > Hi,\n> >\n> > I am running a Linux From Scratch 6.2 system that does not have cpio\n> > installed on it. This means that I can't clone a local repository\n> > unless I install cpio. Is it possible to use a fallback method if cpio\n> > is not present, as there is no NO_CPIO option on make like there is\n> > for OpenSSH, cURL and expat?\n>\n> Using \"file://\" when specifying the source repo will force git-clone to use\n> the git protocol, instead of doing a copy/hardlink.\n>\n> I.e. change \"git clone foo bar\" to \"git clone file://foo bar\" in order to\n> prevent git-clone from calling cpio.\n\nThanks for the tip.\n\n> However, grepping for cpio in the git source tree reveals a couple of uses\n> in git-merge, so you might bump into problems there...\n\nLooks like I'll need to install cpio, then (also allowing me to take\nadvantage of the fast local clones via hardlinks).\n\nDo you know if cpio is listed anywhere in required dependencies?\n\n> > Also, I have an external USB hardrive that is mounted onto the virtual\n> > filesystem. Will clones from the USB harddrive (or a USB flash drive\n> > that is mounted) result in a copy being performed, not a hardlink?\n>\n> Hardlinks are impossible across filesystems. If you're cloning to a\n> different filesystem git will _have_ to make a full copy.\n\nExactly. I was asking this to clarify cpio (and therefore git)\nbehaviour in this situation.\n\n> > Ideally, the hard linking for local clones should be optional.\n>\n> <quote src=\"git-clone(1)\">...</quote>\n\nIndeed, they are. Thanks for the info.\n\n> And as I said above, you can use \"file://\" to force the \"git aware\"\n> transport mechanism, which bypasses the whole local copy/hardlink issue\n> entirely.\n\nSure.\n\n> > What if I want to move a repository because, for example, I have imported\n> > a CVS repository and now want to push it to a new bare repository?\n>\n> Even if you were to use hardlinks, cloning a repo followed by deleting the\n> original will be safe (as long as you don't supply '--shared' to\n> git-clone). That's the beauty of hardlinks.\n\nThat is good to know.\n\n> I also think it's fairly safe to just 'mv' the whole repository to its new\n> location.\n\nThis also works, as long as you are not moving between a bare and\nstandard repository.\n\n> Have fun! :)\n\nWill do :)\n\nThanks,\n- Reece\n"},{"id":"54628","messageId":"200710030142.30062.johan@herland.net","threadId":"10109","inReplyTo":"3f4fd2640710021409q104a4204r522391912d5b5c8c@mail.gmail.com","subject":"[PATCH] Mention 'cpio' dependency in INSTALL","fromName":"Johan Herland","fromEmail":"johan@herland.net","sentAt":"2007-10-02T23:42:29Z","receivedAt":"2007-10-02T23:42:29Z","isPatch":true,"sender":{"key":"johan@herland.net","avatar":"https://avatars.githubusercontent.com/u/547031?v=4"},"body":"Signed-off-by: Johan Herland <johan@herland.net>\n---\n\nOn Tuesday 02 October 2007, Reece Dunn wrote:\n> Do you know if cpio is listed anywhere in required dependencies?\n\nDoing the following in the git repo:\n$ grep -r cpio *\ngit-clone.sh:           find objects -depth -print | cpio -pumd$l \"$GIT_DIR/\" || exit 1\ngit-merge.sh:   cpio -0 -o >\"$GIT_DIR/MERGE_SAVE\"\ngit-merge.sh:           cpio -iuv <\"$GIT_DIR/MERGE_SAVE\"\n\nreveals that cpio is not mentioned anywhere in the documentation,\nnor in the requirements section of the INSTALL file.\n\n\nHave fun!\n\n...Johan\n\n\n INSTALL |    3 +++\n 1 files changed, 3 insertions(+), 0 deletions(-)\n\ndiff --git a/INSTALL b/INSTALL\nindex 289b046..f1eb404 100644\n--- a/INSTALL\n+++ b/INSTALL\n@@ -79,6 +79,9 @@ Issues of note:\n \t- \"perl\" and POSIX-compliant shells are needed to use most of\n \t  the barebone Porcelainish scripts.\n \n+\t- \"cpio\" is used by git-merge for saving and restoring the index,\n+\t  and by git-clone when doing a local (possibly hardlinked) clone.\n+\n  - Some platform specific issues are dealt with Makefile rules,\n    but depending on your specific installation, you may not\n    have all the libraries/tools needed, or you may have\n-- \n1.5.3.3.1144.gf10f2\n"},{"id":"54629","messageId":"7v1wcdjbq6.fsf@gitster.siamese.dyndns.org","threadId":"10109","inReplyTo":"200710030142.30062.johan@herland.net","subject":"Re: [PATCH] Mention 'cpio' dependency in INSTALL","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2007-10-03T00:14:25Z","receivedAt":"2007-10-03T00:14:25Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Johan Herland <johan@herland.net> writes:\n\n> reveals that cpio is not mentioned anywhere in the documentation,\n> nor in the requirements section of the INSTALL file.\n\nThanks.\n\nWe use many other tools that are typically found in bog-standard\nUNIX environments, like sed, echo, cat, sort, etc. and we do not\nlist them in the INSTALL file (nor we would want to).  cpio used\nto be in the \"bog standard\" category but perhaps Linux distros\ndo not install it by default, so it is worth listing it there.\n\nAre there other commands we rely on that may not be universally\ninstalled?  I myself consider \"cut\" to be in the category, but\nother than that I do not think of anything offhand.\n"},{"id":"54664","messageId":"b6ebd0a50710022309n774afe5bpa995de6d8e4568ec@mail.gmail.com","threadId":"10109","inReplyTo":"7v1wcdjbq6.fsf@gitster.siamese.dyndns.org","subject":"Re: [PATCH] Mention 'cpio' dependency in INSTALL","fromName":"Chris Larson","fromEmail":"clarson@kergoth.com","sentAt":"2007-10-03T06:09:08Z","receivedAt":"2007-10-03T06:09:08Z","isPatch":true,"sender":{"key":"clarson@kergoth.com","avatar":"https://gravatar.com/avatar/8929179d0d33b0477876eaae4d5ca20dc4207c2eb573360adec7d91f63a3bd71?d=mp&s=160"},"body":"On 10/2/07, Junio C Hamano <gitster@pobox.com> wrote:\n> Johan Herland <johan@herland.net> writes:\n>\n> > reveals that cpio is not mentioned anywhere in the documentation,\n> > nor in the requirements section of the INSTALL file.\n>\n> Thanks.\n>\n> We use many other tools that are typically found in bog-standard\n> UNIX environments, like sed, echo, cat, sort, etc. and we do not\n> list them in the INSTALL file (nor we would want to).  cpio used\n> to be in the \"bog standard\" category but perhaps Linux distros\n> do not install it by default, so it is worth listing it there.\n>\n> Are there other commands we rely on that may not be universally\n> installed?  I myself consider \"cut\" to be in the category, but\n> other than that I do not think of anything offhand.\n\nI'd think it might be a good idea to list any utilities used that\naren't in, say, the Single Unix Specification, for example.\n-- \nChris Larson - clarson at kergoth dot com\nSoftware Engineer - MontaVista - clarson at mvista dot com\nCore Developer/Architect - TSLib, BitBake, OpenEmbedded, OpenZaurus\n"},{"id":"54672","messageId":"20071003074007.GA15339@genesis.frugalware.org","threadId":"10109","inReplyTo":"7v1wcdjbq6.fsf@gitster.siamese.dyndns.org","subject":"Re: [PATCH] Mention 'cpio' dependency in INSTALL","fromName":"Miklos Vajna","fromEmail":"vmiklos@frugalware.org","sentAt":"2007-10-03T07:40:07Z","receivedAt":"2007-10-03T07:40:07Z","isPatch":true,"sender":{"key":"vmiklos@frugalware.org","avatar":"https://gravatar.com/avatar/401c1cbbb3a5d13e650c691a2c71d6fd0b80df1a01bc74d9f1972675dd58f2bd?d=mp&s=160"},"body":"On Tue, Oct 02, 2007 at 05:14:25PM -0700, Junio C Hamano <gitster@pobox.com> wrote:\n> Are there other commands we rely on that may not be universally\n> installed?  I myself consider \"cut\" to be in the category, but\n> other than that I do not think of anything offhand.\n\nwhen using git in a chroot, i obviously had coreutils/sed/grep installed\nand the only \"extra\" package i needed (besides the curl an openssl libs)\nwas cpio\n\n- VMiklos\n"},{"id":"54676","messageId":"200710031027.48999.johan@herland.net","threadId":"10109","inReplyTo":"20071003074007.GA15339@genesis.frugalware.org","subject":"[PATCH v2] INSTALL: Update section on external dependencies","fromName":"Johan Herland","fromEmail":"johan@herland.net","sentAt":"2007-10-03T08:27:48Z","receivedAt":"2007-10-03T08:27:48Z","isPatch":true,"sender":{"key":"johan@herland.net","avatar":"https://avatars.githubusercontent.com/u/547031?v=4"},"body":"Includes:\n- Mention dependency on \"core\" utilities, including coreutils, sed, cut, grep\n- Mention dependency on cpio\n- Fix up some whitespace and linebreaking issues\n\nSigned-off-by: Johan Herland <johan@herland.net>\n---\n\nOn Wednesday 03 October 2007, Miklos Vajna wrote:\n> On Tue, Oct 02, 2007 at 05:14:25PM -0700, Junio C Hamano <gitster@pobox.com> wrote:\n> > Are there other commands we rely on that may not be universally\n> > installed?  I myself consider \"cut\" to be in the category, but\n> > other than that I do not think of anything offhand.\n> \n> when using git in a chroot, i obviously had coreutils/sed/grep installed\n> and the only \"extra\" package i needed (besides the curl an openssl libs)\n> was cpio\n\nOk, here's a more complete patch.\n\n...Johan\n\n\n INSTALL |   18 +++++++++++-------\n 1 files changed, 11 insertions(+), 7 deletions(-)\n\ndiff --git a/INSTALL b/INSTALL\nindex 289b046..244470f 100644\n--- a/INSTALL\n+++ b/INSTALL\n@@ -54,6 +54,8 @@ Issues of note:\n  - Git is reasonably self-sufficient, but does depend on a few external\n    programs and libraries:\n \n+\t- Common \"core\" utilities including coreutils, sed, cut, and grep.\n+\n \t- \"zlib\", the compression library. Git won't build without it.\n \n \t- \"openssl\".  Unless you specify otherwise, you'll get the SHA1\n@@ -63,22 +65,24 @@ Issues of note:\n \t  that come with git (git includes the one from Mozilla, and has\n \t  its own PowerPC and ARM optimized ones too - see the Makefile).\n \n-\t- \"libcurl\" and \"curl\" executable.  git-http-fetch and\n-\t  git-fetch use them.  If you do not use http\n-\t  transfer, you are probably OK if you do not have\n-\t  them.\n+\t- \"libcurl\" and \"curl\" executable.  git-http-fetch and git-fetch\n+\t  use them.  If you do not use http transfer, you are probably OK\n+\t  if you do not have them.\n \n \t- expat library; git-http-push uses it for remote lock\n \t  management over DAV.  Similar to \"curl\" above, this is optional.\n \n-        - \"wish\", the Tcl/Tk windowing shell is used in gitk to show the\n-          history graphically, and in git-gui.\n+\t- \"wish\", the Tcl/Tk windowing shell is used in gitk to show the\n+\t  history graphically, and in git-gui.\n \n-\t- \"ssh\" is used to push and pull over the net\n+\t- \"ssh\" is used to push and pull over the net.\n \n \t- \"perl\" and POSIX-compliant shells are needed to use most of\n \t  the barebone Porcelainish scripts.\n \n+\t- \"cpio\" is used by git-merge for saving and restoring the index,\n+\t  and by git-clone when doing a local (possibly hardlinked) clone.\n+\n  - Some platform specific issues are dealt with Makefile rules,\n    but depending on your specific installation, you may not\n    have all the libraries/tools needed, or you may have\n-- \n1.5.3.3.1144.gf10f2\n"},{"id":"54718","messageId":"Pine.LNX.4.64.0710031655590.28395@racer.site","threadId":"10109","inReplyTo":"200710031027.48999.johan@herland.net","subject":"Re: [PATCH v2] INSTALL: Update section on external dependencies","fromName":"Johannes Schindelin","fromEmail":"johannes.schindelin@gmx.de","sentAt":"2007-10-03T15:56:53Z","receivedAt":"2007-10-03T15:56:53Z","isPatch":true,"sender":{"key":"johannes.schindelin@gmx.de","avatar":"https://avatars.githubusercontent.com/u/127790?v=4"},"body":"Hi,\n\nOn Wed, 3 Oct 2007, Johan Herland wrote:\n\n> Includes:\n> - Mention dependency on \"core\" utilities, including coreutils, sed, cut, grep\n\nMaybe mention libc, too?  Oh, and that you need a computer? ;-)\n\n> - Fix up some whitespace and linebreaking issues\n\nThe commit message could use some of that, too... ;-)\n\nCiao,\nDscho\n"},{"id":"54719","messageId":"20071003160546.GJ2137@spearce.org","threadId":"10109","inReplyTo":"Pine.LNX.4.64.0710031655590.28395@racer.site","subject":"Re: [PATCH v2] INSTALL: Update section on external dependencies","fromName":"Shawn O. Pearce","fromEmail":"spearce@spearce.org","sentAt":"2007-10-03T16:05:46Z","receivedAt":"2007-10-03T16:05:46Z","isPatch":true,"sender":{"key":"spearce@spearce.org","avatar":"https://avatars.githubusercontent.com/u/34844?v=4"},"body":"Johannes Schindelin <Johannes.Schindelin@gmx.de> wrote:\n> On Wed, 3 Oct 2007, Johan Herland wrote:\n> \n> > Includes:\n> > - Mention dependency on \"core\" utilities, including coreutils, sed, cut, grep\n> \n> Maybe mention libc, too?  Oh, and that you need a computer? ;-)\n\nDon't forget that the computer must be plugged in and have\nelectricity as well.  I see users all too often wonder why everything\nthey type is black, on a black background...\n \n-- \nShawn.\n"}]}