git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Running git-archive recursively over submodules

From
Thomas Rast <trast@student.ethz.ch>
Date
Feb 15, 2010, 20:56 UTC
Message-ID
<201002152156.17132.trast@student.ethz.ch>

I'm posting this for posterity. It came out of an IRC discussion last night. Google "git archive submodules" currently finds two scripts for the same task:

* http://kerneltrap.org/mailarchive/git/2008/3/29/1294614
  Can pack arbitrary revisions, but seems to be highly specific to the
  poster's setup.
* http://wiki.github.com/meitar/git-archive-all.sh/
  Appears to work with normal repos (I haven't really tested) but only
  packs HEAD.

Both have in common that they run git-archive in each submodule, and then merge the resulting tars.

The script below follows a different approach, based on the idea that all the objects are already there, you just need to help git find them. This of course assumes that you have the submodules checked out (or at least cloned). It builds an index that contains a flattened version of the repository and all submodules. This works for any revision you specify. The downside is that this does not respect .gitattributes.

The same trick can be used for other commands, so if people find this useful it could be added to git-submodule.

Warning: Error detection and such are left as an exercise to the user.

-- 8< -- #!/bin/sh

export revision="$1"

export GIT_INDEX_FILE=".git/tmpindex" rm -f "$GIT_INDEX_FILE"

git read-tree $revision
export up="$(pwd)"
read_one_level () {
	export GIT_ALTERNATE_OBJECT_DIRECTORIES="$GIT_ALTERNATE_OBJECT_DIRECTORIES":$(
	    git submodule foreach 'echo "$up/$path/.git/objects"' |
	    grep -E -v '^(Entering|No submodule mapping found)' |
	    tr '\n' : |
	    sed 's/:$//'
	)
	git submodule foreach '
		cd "$up"
		subcommit=$(git rev-parse :"$path")
		git rm --cached "$path"
		git read-tree -i --prefix="$path/" $subcommit
	' >/dev/null
}
while git ls-files -s | grep -q ^160000; do
    read_one_level
done
git archive --format=tar $(git write-tree)

rm -f "$GIT_INDEX_FILE" -- >8 --

-- 
Thomas Rast
trast@{inf,student}.ethz.ch
Next: Michael J Gruber
Message 1 of 4 in “Running git-archive recursively over submodules”
  1. Thomas RastFeb 15, 2010
  2. Michael J GruberFeb 16, 2010
  3. Thomas RastFeb 16, 2010
  4. Michael J GruberFeb 16, 2010

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.