[RFC PATCH 00/10] Make submodules work if .gitmodules is not checked out

Hi,

vcsh[1] uses bare git repositories and detached work-trees to manage *distinct* sets of configuration files directly into $HOME.

In this setup multiple repositories share the same directory (namely $HOME) as their work dir, so the sets of checked out files would also need to be *disjoint* to avoid collisions (examples of collisions can be README or LICENSE files, or even .gitignore and .gitattributes).

Amongst vcsh users, a popular solution to this problem is to use sparse checkouts, representing the *intersection* of all the repositories in a common sparse-checkout file[2].

This works well but there are still limitations about the ability to use submodules because git expects the .gitmodules file to be checked out.

The user (or vcsh itself) might learn to fully populate one repository at a time when working with submodules but this is unhandy and would introduce serialization even when it's not strictly needed like in the case of _reading_ .gitmodules.

As a side note, git submodules have worked perfectly fine with detached work-trees for some time[3,4,5] so extending them to also play nice with sparse checkouts seems the next logical step to cover the vcsh use case.

This series teaches git to try and read the .gitmodules file from the index (HEAD:.gitmodules) when it's not available in the work dir.

It does so by first providing an opaque way to access the submodules configuration, and then extends the access mechanism behind the scenes.

Writing to .gitmodules still requires it to be checked out.

This series should be in line with what Stefan and Jonathan proposed; although it's not perfect yet:

  - naming of functions can be improved,
  - code can be moved around to better places,
  - maybe some notes should be added to Documentation/git-submodule.txt,
  - my git terminology may still be a little off: do "work tree" and
    "work directory" mean the same thing?

the functionality is there and we should have a decent baseline to work on.

The patchset is based on the current master (ccdcbd54c447), the test-suite passes after each commit and there are some per-patch annotations.

If anyone wanted to pick up and finish the work feel free to do so, otherwise please comment and I'll try to address issues as time permits.

Thanks,
   Antonio

[1] https://github.com/RichiH/vcsh [2] https://github.com/RichiH/vcsh/issues/120#issuecomment-387335765 [3] http://git.661346.n2.nabble.com/git-submodule-vs-GIT-WORK-TREE-td7562165.html [4] http://git.661346.n2.nabble.com/PATCH-Solve-git-submodule-issues-with-detached-work-trees-td7563377.html [5] https://github.com/git/git/commit/be8779f7ac9a3be9aa783df008d59082f4054f67

Antonio Ospite (10):
  config: make config_from_gitmodules generally useful
  submodule: factor out a config_gitmodules_set function
  t7411: be nicer to other tests and really clean things up
  submodule--helper: add a new 'config' subcommand
  submodule: use the 'submodule--helper config' command
  submodule--helper: add a '--stage' option to the 'config' sub command
  submodule: use 'submodule--helper config --stage' to stage .gitmodules
  t7506: cleanup .gitmodules properly before setting up new scenario
  submodule: support reading .gitmodules even when it's not checked out
  t7415: add new test about using HEAD:.gitmodules from the index

 builtin/fetch.c                        |   2 +-
 builtin/mv.c                           |   2 +
 builtin/rm.c                           |   7 +-
 builtin/submodule--helper.c            | 100 +++++++++++++++++++-
 cache.h                                |   1 +
 config.c                               |  26 ++++--
 config.h                               |  10 +-
 git-submodule.sh                       |  10 +-
 submodule-config.c                     |  16 +---
 submodule.c                            |  37 ++++++--
 submodule.h                            |   2 +
 t/t7411-submodule-config.sh            |  63 ++++++++++++-
 t/t7415-submodule-sparse-gitmodules.sh | 124 +++++++++++++++++++++++++
 t/t7506-status-submodule.sh            |   3 +-
 14 files changed, 357 insertions(+), 46 deletions(-)
 create mode 100755 t/t7415-submodule-sparse-gitmodules.sh