[RFC/PATCH 0/7] Rework git core for native submodules

Hi,

The purpose of this series is to convince you that we've made a lot of fundamental mistakes while designing submodules, and that we should fix them now. [1/7] argues for a new object type, and this is the core of the idea.

It's an entire beautiful design + UI/UX package.  To demo it now:

    # Switch your git to https://github.com/artagnon/git/tree/link

    $ git config --global url."git://github.com/".insteadOf gh:
    $ git config --global clone.submodulegitdir /home/<user>/bare
    $ cd /tmp
    $ git clone gh:artagnon/varlog
    $ cd varlog
    $ git clone gh:artagnon/clayoven
    # Notice how it puts clayoven.git in ~/bare
    $ git add clayoven
    # Just works!
    $ git commit -a -m "add subproject clayoven"
    $ git ls-tree HEAD
    # Try to cat-file or show the link object
    $ cd clayoven/lib
    $ git clone gh:artagnon/sandbox
    $ git add sandbox
    # Again, just works!  No cd-to-toplevel nonsense
    $ git ci -a -m "add subproject sandbox"
    $ cd ../..
    $ git rm clayoven
    $ git ci -a -m "remove subproject clayoven"
    # start hacking: read the rest of this email
    # note that git diff is broken

of compositions, essentially replacing repo, mr, gitslave, git-subtree, (the old) git-submodule, and other similar tools. All submdodules are not equal: each one will have tweakable parameters that will change how git-core treats them.

to work seamlessly with submodules without introducing any submodule-specific commands (or atleast keeping it to a minimum). The deprecation path for git-submodule is clear: first, we have to strip it down to be a very thin wrapper around existing git commands, and then announce that it's no longer necessary.

This series is two days of unedited unrebased work ([5/7] is dead code for instance). I've written in a big hurry, and it's meant to be a proof-of-concept only. I discovered lots of core bugs along the way that need to be fixed first. Off the top of my head:

1. 'git add' should not go past submodule boundaries.  I should not be
   able to 'git add clayoven/' or 'git add clayoven/LICENSE'.  In
   addition, the shell completion also needs to be fixed.

2. An empty directory containing a .git file is a perfectly valid
   worktree, but does not show up in the superproject's 'git status'
   output.  How can it be treated like an empty directory?

3. sha1_file.c:index_path() should not return paths with trailing
   slashes, I think.

4. There really must be a better way to figure out if I'm in a
   worktree than setup_git_directory_gently().

Also, I'm going to need your help to finish this. I was trying to write the 8th patch when I got stuck. I'm guessing I need to understand how wt-status extracts the differences between two trees, and filter it for added/removed link objects. Then, I have to follow the example of updating the HEAD ref to update my refs/modules/<branch>/* refs.

If you think this is all a big waste of time, and that we should focus on improving git-submodule.sh, you're probably deranged. Because it's _very_ clear to me.

Thank you for reading.

Ramkumar Ramachandra (7):
  link.c, link.h: introduce fifth object type
  sha1_file, link: write link objects to the database
  teach ce_compare_gitlink() about OBJ_LINK
  builtin/log: teach show about OBJ_LINK
  edit-link: add new builtin
  clone: introduce clone.submodulegitdir
  sha1_file: write ref_name to link object

 Makefile          |  2 ++
 alloc.c           |  3 ++
 builtin/clone.c   | 29 +++++++++++++++++++
 builtin/log.c     | 17 +++++++++++
 builtin/ls-tree.c |  4 +--
 cache.h           |  3 +-
 environment.c     | 11 -------
 git-edit-link.sh  | 87 +++++++++++++++++++++++++++++++++++++++++++++++++++++++
 link.c            | 49 +++++++++++++++++++++++++++++++
 link.h            | 26 +++++++++++++++++
 object.c          |  9 ++++++
 read-cache.c      | 33 ++++++++++++++-------
 sha1_file.c       | 48 +++++++++++++++++++++++++++++-
 13 files changed, 296 insertions(+), 25 deletions(-)
 create mode 100644 git-edit-link.sh
 create mode 100644 link.c
 create mode 100644 link.h