[GSoC/RFC] Ideas on git fetch --deepen

>  (upstream)
>   ---o---o---o---A---B
>
>  (you)
>                  A---B

> If you want the new history leading to the updated tip,
> you can just say:
>
>    git fetch
>
> without any --depth nor --deepen option to end up with:
>
>  (you)
>                  A---B---C---D---E---F---...---W---X---Y---Z

> Hi all,
>
> My name is Dongcan Jiang. I am studying for my Master Degree at Peking
> University
> majoring in Computer Science. I have been using Git to manage my projects
> for about half a year. It's really exciting that Git has been helping me make
> revision control much more convenient. Therefore, I am very interested in
> doing some works for Git in GSoC 2015.
>
> I have submitted my v2 patch on a microproject recently, and I am waiting
> for comments on v2 now.
>
> In the meantime, I have been scanning ideas on the git gsoc page and
> their related mails. I find that I am interested in most of them, especially
> "git fetch --deepen" idea.
>
> Here is my understanding about this idea.
>
> Although "deepen" and "depth" have different behavior, "deepen" can be
> achieved by "depth" with some extra calculation. If we know the
> distance between "my history bottom" and "your tips", we can set the sum
> of "deepen" step and this distance as "depth" step. Then we can reuse
> the logics of processing "depth" to complete it.
>
> Take graph in [1] as an example.
>
>>  (upstream)
>>   ---o---o---o---A---B
>>
>>  (you)
>>                  A---B
>
> the distance of "my history bottom"(A) and "your tips"(B) is 1, then
> "git fetch --deepen=3" can be achieved by "git fetch --depth=4"
>
> However, I am a little worried about whether this idea is enough for a
> GSoC project. I hope you can give me some comments and suggestions.
>
> [1] http://article.gmane.org/gmane.comp.version-control.git/212950
>
> Thanks,
> Dongcan
>
> --
> 江东灿（Dongcan Jiang）
> Team of Search Engine & Web Mining
> School of Electronic Engineering & Computer Science
> Peking University, Beijing, 100871, P.R.China

> Hi, all
>
> After digging into how "git fetch" works, I have found that my previous
> understanding was too rash. I'm sorry for that.
>
> I find that the current workflow of "git fetch --depth" is as follows:
>
>   1. 'fetch module' calls 'git-upload-pack service' via 'transport module'
>      to fetch ref with `depth`.
>   2. Such call is finished by pipe I/O.
>   3. git-upload-pack receives arguments such as 'depth' to fetch commits,
>      and send back to the caller.
>
> Therefore, if we want to implement "--deepen" for "git fetch", we have to
> modify the arguments protocol of git-upload-pack service by adding the
> 'shallow commit' hash.
> Then, we can fetch `depth` commits before the 'shallow commit' in
> git-upload-pack service.
> Apparently, we have to output error message when the 'shallow commit' hash
> is not in the repository.

> On Sun, Mar 8, 2015 at 3:57 PM, Dongcan Jiang <dongcan.jiang@gmail.com> wrote:
>> Hi, all
>>
>> After digging into how "git fetch" works, I have found that my previous
>> understanding was too rash. I'm sorry for that.
>>
>> I find that the current workflow of "git fetch --depth" is as follows:
>>
>>   1. 'fetch module' calls 'git-upload-pack service' via 'transport module'
>>      to fetch ref with `depth`.
>>   2. Such call is finished by pipe I/O.
>>   3. git-upload-pack receives arguments such as 'depth' to fetch commits,
>>      and send back to the caller.
>>
>> Therefore, if we want to implement "--deepen" for "git fetch", we have to
>> modify the arguments protocol of git-upload-pack service by adding the
>> 'shallow commit' hash.
>> Then, we can fetch `depth` commits before the 'shallow commit' in
>> git-upload-pack service.
>> Apparently, we have to output error message when the 'shallow commit' hash
>> is not in the repository.
>
> Close. You can't figure this shallow commit hash from client side
> (client repo is shortened). So you can't send it. What you send is
> exactly what the user gives you (e.g. --deepen=5, then you send
> "deepen 5" or similar). The server side (git-upload-pack) knows about
> the shallow boundary of the client and can walk its (i.e. server-side)
> commit graph to find out the new, deepened boundary. Then the server
> sends necessary objects plus instructions to update shallow boundary
> to the client.
>
> Also, I think  this work would include support for smart-http
> protocol. It goes a slightly different route, "git fetch" ends up at
> transport.c, but then transport-helper.c and calls git-remote-http(s)
> which is implemented by remote-curl.c. This one handles http stuff
> then passes control to git-fetch-pack.c. It's fetch-pack that talks to
> upload-pack.c
>
> Have a look at Documentation/technical/pack-protocol.txt and
> protocol-capabilities.txt (and http-protocol.txt for smart-http). The
> function that "draws" shallow boundary based on --depth is
> get_shallow_commits() in shallow.c. I suspect you need to improve it a
> bit to use with --deepen. I guess you can look at these [1]. Those
> changes touch shallow repo in a bit different way, but the main code
> path is more or less the same (smart-http not touched).
>
> [1] https://github.com/pclouds/git/commits/7edde8b83a20abb3cd404e2c5f07e3c29a2891f7
> --
> Duy