Re: unexplained behavior/issue with git archive?

From: John Keeping <john@keeping.me.uk>
Date: Jul 23, 2015, 15:59 UTC
Message-ID: <20150723155936.GC14935@serenity.lan>
In-Reply-To: <55B10705.6090303@jvales.net>

On Thu, Jul 23, 2015 at 05:23:49PM +0200, Jan Vales wrote:

Show 35 quoted lines

> i seem to trigger behavior i do not understand with git archive.
> 
> I have this little 3 liner (vmdiff.sh):
> #!/bin/bash
> git diff --name-status "$2" "$3" > "$1.files"
> git diff --name-only "$2" "$3" |xargs -d'\n' git archive -o "$1" "$3" --
> 
> 
> For testing purpose, lets assume this call:
> # ./vmdiff.sh latest.zip HEAD^1 HEAD
> 
> # cat latest.zip.files | wc -l
> 149021
> 
> # cat latest.zip.files | egrep "^D" | wc -l
> 159
> 
> # mkdir empty; cd empty; unzip latest.zip ; find * | wc -l
> 1090
> 
> My goal is to basically diff (parts of) filesystems against each other
> and create an archive with all changed files + a file list to know what
> files were deleted. (I currently do not care about the files
> permissions+ownership, and it doesnt really matter in the current
> problem. Also dont ask, why one would store a root-filesystem in git :)
> 
> What I do not understand: why does the zip file only contains 1090
> files+dirs if the wc -l shows like 150k files and only like 159 were
> deleted?
> There should be like 149k files in that archive.
> 
> Also only the few files are all from "var" and none from etc or srv
> where definitely files changed in too! (and show up in latest.zip.files)
> 
> Is there a limit of files git archive can process?

Not explicitly, but there is a limit on the size of command lines and xargs will invoke the command multiple times if enough arguments are given.

What happens if you do:

	git diff --name-only HEAD^ HEAD | xargs -d'\n' echo | wc -l

With a small number of items, there should only be one output line, but if xargs invokes the command multiple times there will be multiple lines. For example (using -L2 to force a maximum of two arguments per invocation):

	$ printf '%s\n' a b c | xargs -d'\n' echo | wc -l
	1
	$ printf '%s\n' a b c | xargs -d'\n' -L2 echo | wc -l
	2