Re: [PATCH 0/6] git clone: Marry --recursive and --reference

From: Junio C Hamano <gitster@pobox.com>
Date: Aug 5, 2016, 19:47 UTC
Message-ID: <xmqqoa57vvzl.fsf@gitster.mtv.corp.google.com>
In-Reply-To: <20160804195159.7788-1-sbeller@google.com>

Stefan Beller <sbeller@google.com> writes:

Show 7 quoted lines

>  Currently when cloning a superproject with --recursive and --reference
>  only the superproject learns about its alternates. The submodules are
>  cloned independently, which may incur lots of network costs.
>  
>  Assume that the reference repository has the submodules at the same
>  paths as the to-be-cloned submodule and try to setup alternates from
>  there.

I am a bit fuzzy about what scenario you are trying to help. Is it something like this? [*1*]

 * The superproject is at git://frotz.xz/super.git but you locally
   have a mirror at /var/cache/super.git (bare) repository.

 * "git clone --refererence /var/cache/super.git git://frotz.xz/super.git"
   would borrow that existing local mirror to cut down the traffic.

 * The super.git project uses git://nitfol.xz/xyzzy.git as its
   submodule.  You locally have a mirror of it at
   /var/cache/xyzzy.git (bare) repository.

 * The xyzzy submodule is bound at libs/xyzzy in the superproject
   tree.

 * You want the "clone" command above with "--recursive" to do "the
   right thing".  That is, the clone of the superproject borrows
   from /var/cache/super.git local mirror, and the clone of xyzzy
   that would be made at .git/modules/xyzzy in the superproject
   would borrow from /var/cache/xyzzy.git local mirror.

What I am not sure about is how /var/cache/xyzzy.git should be automatically derived from the information given from the command line of "clone" and what the clone of the superproject contains. You'd need to make certain assumptions, like

 - The parent directory of the --reference for superproject,
   i.e. /var/cache/, is the directry that has many other mirrors;

 - The .gitmodules file in superproject says a "xyzzy" project from
   git://nitfol.xz/xyzzy.git must be checked out at "libs/xyzzy",
   and it is a reasonable assumption that we would find a local
   mirror at /var/cache/xyzzy.git (the /var/cache prefix comes from
   the above assumptino).

There may be some other assumptions you are making. I am wondering if your specific assumption is applicable generally.

[Footnote]

*1* Another common arrangement could be that you have one full clone
    of the superproject at /var/cache/super.git and its submodule
    xyzzy in /var/cache/super.git/modules/xyzzy and assuming that
    would allow you to guess where to borrow submodules from.  But
    the result of assuming this layout would already be different
    from the scenario I gave the above.  If the organization uses
    the xyzzy project only in the context of the frotz superproject,
    "borrow from submodule repository inside $GIT_DIR/modules/ of
    the superproject" may be sensible, but that makes use of "xyzzy"
    as an independent project quite awkward (also you may have
    another superproject that is unrelated to super.git that uses
    the same "xyzzy" project as its submodule).  The more "xyzzy"
    project is suitable to be used as "submodule", the more it makes
    sense to have its mirror independent from the superprojects that
    use it, by using /var/cache/{super,xyzzy}.git layout.

    IOW, both layouts are equally sensible; what layout (either one
    of the above two, or something entirely different) is your "at
    the same paths" assumption meant to serve well, and what is the
    plan to serve other layouts?