git/list[1] front-page[2] threads[3] people[4] search[5] about
 

[RFC] How to pass Git config command line instructions to Submodule commands?

From
Lars Schneider <larsxschneider@gmail.com>
Date
Apr 25, 2016, 10:39 UTC
Message-ID
<60724588-B06E-47E8-9302-8709C4601826@gmail.com>
Hi,

a few folks from the Git LFS project and I try to make cloning of repositories with a lot of LFS files faster.

The core problem is that Git LFS uses a Git smudge filter to replace LFS pointers with the actual file content. Right now, a smudge filter can only be executed on an individual file which makes the operation slow for many files [1].

We solved this issue by temporarily disabling the smudge filter for the clone command via Git config (optimized in 1a8630 [2]):

    git -c filter.lfs.smudge= -c filter.lfs.required=false clone <url> <path>

Afterwards Git LFS runs a special command to download and replace all LFS content in bulk [3]. This works great for LFS repositories.

However, I noticed that git config command line instructions such as "-c filter.lfs.smudge=" are not passed to Git submodule operations. Thus this does not work as expected:

    git -c filter.lfs.smudge= -c filter.lfs.required=false clone --recursive <url> <path>

I tried to work around that by copying the relevant pieced from the Git Submodule command [4] and applying the command line Git config manually (look closely at the modified checkout command):

    git -c filter.lfs.smudge= -c filter.lfs.required=false clone $@
    if [[ -z $2 ]]; then
        CLONE_PATH=$(basename ${1%.git});
    else
        CLONE_PATH=$2;
    fi
    pushd "$CLONE_PATH"
        git submodule init
        wt_prefix=$(git rev-parse --show-prefix)
        git submodule--helper list --prefix "$wt_prefix" | {
            while read mode sha1 stage sm_path
            do
                name=$(git submodule--helper name "$sm_path") || exit
                url=$(git config submodule."$name".url)
                if ! test -d "$sm_path"/.git && ! test -f "$sm_path"/.git
                then
                    git submodule--helper clone --prefix "$wt_prefix" --path "$sm_path" --name "$name" --url "$url"
                    pushd "$sm_path"
                        git -c filter.lfs.smudge= -c filter.lfs.required=false checkout -q $sha1 || exit
                        git-lfs pull || exit
                    popd
                fi
            done
        }
    popd

Do you see an easier way to pass command line Git config instructions to the underlaying Git Submodule commands? If not, do you think a patch adding this would be worth working on?

I also started a discussion about that on the Git LFS issue page [5].

Thanks, Lars

[1] https://github.com/github/git-lfs/issues/931 [2] https://github.com/git/git/commit/1a8630dc3b1cc6f1361a4e5d94630133c24c97d9 [3] https://developer.atlassian.com/blog/2016/04/git-lfs-12-clone-faster/ [4] https://github.com/git/git/blob/6a6636270fbaf74609cd3e1bd207dd2c420d640a/git-submodule.sh#L686-L731 [5] https://github.com/github/git-lfs/issues/1172

Next: Stefan Beller
Message 1 of 36 in “[RFC] How to pass Git config command line instructions to Submodule commands?”
  1. Lars SchneiderApr 25, 2016
  2. Stefan BellerApr 25, 2016
  3. Jacob KellerApr 25, 2016
  4. Jeff KingApr 25, 2016
  5. Jeff KingApr 25, 2016
  6. Lars SchneiderApr 28, 2016
  7. Jeff KingApr 28, 2016
  8. Jeff KingApr 28, 2016
  9. Jeff KingApr 28, 2016
  10. 0/5 fixes for sanitized submodule configJeff King, Apr 28, 2016
  11. 1/5 t5550: fix typo in $HTTPD_URLJeff King, Apr 28, 2016
  12. Jacob KellerApr 28, 2016
  13. Jeff KingApr 28, 2016
  14. Jacob KellerApr 28, 2016
  15. 2/5 t5550: break submodule config test into multiple sub-testsJeff King, Apr 28, 2016
  16. Stefan BellerApr 28, 2016
  17. Jeff KingApr 28, 2016
  18. Jacob KellerApr 28, 2016
  19. 3/5 submodule: export sanitized GIT_CONFIG_PARAMETERSJeff King, Apr 28, 2016
  20. Stefan BellerApr 28, 2016
  21. Jeff KingApr 28, 2016
  22. Stefan BellerApr 28, 2016
  23. Johannes SchindelinApr 28, 2016
  24. Jacob KellerApr 28, 2016
  25. Jeff KingApr 28, 2016
  26. Jacob KellerApr 28, 2016
  27. 4/5 submodule--helper: move config-sanitizing to submodule.cJeff King, Apr 28, 2016
  28. Stefan BellerApr 28, 2016
  29. Jeff KingApr 28, 2016
  30. Lars SchneiderApr 28, 2016
  31. 5/5 submodule: use prepare_submodule_repo_env consistentlyJeff King, Apr 28, 2016
  32. Johannes SchindelinApr 28, 2016
  33. Stefan BellerApr 28, 2016
  34. Jacob KellerApr 28, 2016
  35. Lars SchneiderApr 28, 2016
  36. Jeff KingApr 28, 2016

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.