git clone questions relating to cpio

Hi,

I am running a Linux From Scratch 6.2 system that does not have cpio installed on it. This means that I can't clone a local repository unless I install cpio. Is it possible to use a fallback method if cpio is not present, as there is no NO_CPIO option on make like there is for OpenSSH, cURL and expat?

Also, I have an external USB hardrive that is mounted onto the virtual filesystem. Will clones from the USB harddrive (or a USB flash drive that is mounted) result in a copy being performed, not a hardlink?

Ideally, the hard linking for local clones should be optional. What if I want to move a repository because, for example, I have imported a CVS repository and now want to push it to a new bare repository?

- Reece

On Monday 01 October 2007, Reece Dunn wrote:

Using "file://" when specifying the source repo will force git-clone to use the git protocol, instead of doing a copy/hardlink.

I.e. change "git clone foo bar" to "git clone file://foo bar" in order to prevent git-clone from calling cpio.

However, grepping for cpio in the git source tree reveals a couple of uses in git-merge, so you might bump into problems there...

Hardlinks are impossible across filesystems. If you're cloning to a different filesystem git will _have_ to make a full copy.

<quote src="git-clone(1)"> --local, -l

  When the repository to clone from is on a local machine, this flag 
bypasses normal "git aware" transport mechanism and clones the repository 
by making a copy of HEAD and everything under objects and refs directories. 
The files under .git/objects/ directory are hardlinked to save space when 
possible. This is now the default when the source repository is specified 
with /path/to/repo syntax, so it essentially is a no-op option. To force 
copying instead of hardlinking (which may be desirable if you are trying to 
make a back-up of your repository), but still avoid the usual "git aware" 
transport mechanism, --no-hardlinks can be used.

--no-hardlinks

  Optimize the cloning process from a repository on a local filesystem by 
copying files under .git/objects directory.
</quote>

And as I said above, you can use "file://" to force the "git aware" transport mechanism, which bypasses the whole local copy/hardlink issue entirely.

Even if you were to use hardlinks, cloning a repo followed by deleting the original will be safe (as long as you don't supply '--shared' to git-clone). That's the beauty of hardlinks.

I also think it's fairly safe to just 'mv' the whole repository to its new location.

Have fun! :)

...Johan

On 01/10/2007, Johan Herland <johan@herland.net> wrote:

Thanks for the tip.

Looks like I'll need to install cpio, then (also allowing me to take advantage of the fast local clones via hardlinks).

Do you know if cpio is listed anywhere in required dependencies?

Exactly. I was asking this to clarify cpio (and therefore git) behaviour in this situation.

Indeed, they are. Thanks for the info.

Sure.

That is good to know.

This also works, as long as you are not moving between a bare and standard repository.

Will do :)

Thanks,
- Reece

Signed-off-by: Johan Herland <johan@herland.net>
---

On Tuesday 02 October 2007, Reece Dunn wrote:

Doing the following in the git repo: $ grep -r cpio * git-clone.sh: find objects -depth -print | cpio -pumd$l "$GIT_DIR/" || exit 1 git-merge.sh: cpio -0 -o >"$GIT_DIR/MERGE_SAVE" git-merge.sh: cpio -iuv <"$GIT_DIR/MERGE_SAVE"

reveals that cpio is not mentioned anywhere in the documentation, nor in the requirements section of the INSTALL file.

Have fun!

...Johan

 INSTALL |    3 +++
 1 files changed, 3 insertions(+), 0 deletions(-)

Johan Herland <johan@herland.net> writes:

Thanks.

We use many other tools that are typically found in bog-standard UNIX environments, like sed, echo, cat, sort, etc. and we do not list them in the INSTALL file (nor we would want to). cpio used to be in the "bog standard" category but perhaps Linux distros do not install it by default, so it is worth listing it there.

Are there other commands we rely on that may not be universally installed? I myself consider "cut" to be in the category, but other than that I do not think of anything offhand.

On 10/2/07, Junio C Hamano <gitster@pobox.com> wrote:

I'd think it might be a good idea to list any utilities used that aren't in, say, the Single Unix Specification, for example.

On Tue, Oct 02, 2007 at 05:14:25PM -0700, Junio C Hamano <gitster@pobox.com> wrote:

when using git in a chroot, i obviously had coreutils/sed/grep installed and the only "extra" package i needed (besides the curl an openssl libs) was cpio

- VMiklos

Includes:
- Mention dependency on "core" utilities, including coreutils, sed, cut, grep
- Mention dependency on cpio
- Fix up some whitespace and linebreaking issues

Signed-off-by: Johan Herland <johan@herland.net>
---

On Wednesday 03 October 2007, Miklos Vajna wrote:

Ok, here's a more complete patch.

...Johan

 INSTALL |   18 +++++++++++-------
 1 files changed, 11 insertions(+), 7 deletions(-)

Hi,

On Wed, 3 Oct 2007, Johan Herland wrote:

Maybe mention libc, too?  Oh, and that you need a computer? ;-)

The commit message could use some of that, too... ;-)

Ciao, Dscho

Johannes Schindelin <Johannes.Schindelin@gmx.de> wrote:

Don't forget that the computer must be plugged in and have
electricity as well.  I see users all too often wonder why everything
they type is black, on a black background...
 

Hi,

On Mon, 1 Oct 2007, Reece Dunn wrote:

You might be interested in the workaround Hannes did in mingw.git; he made a wrapper script called 'cpio' using 'tar'.

Ciao, Dscho

Johannes Schindelin <Johannes.Schindelin@gmx.de> writes:

I think that may be good enough as workaround, but I do not think you would get the space saving from hardlinks that way.

Junio C Hamano schrieb:

FWIW, I'm thinking about changing the cpio -p (passthrough) part to use 'cp -l --parents --target-directory=...' instead of tar; this gives us hard links, even on NTFS. But it needs GNU's cp, of course.

-- Hannes

Johannes Sixt <j.sixt@viscovery.net> writes:

Yeah, that's the reason it is not appropriate than cpio, even if we forget about Windows.