# [GSoC/RFC] Ideas on git fetch --deepen 4 messages from 2015-03-07 to 2015-03-08. Participants: Dongcan Jiang, Duy Nguyen. Thread: https://gitlist.dev/t/38739 ## Dongcan Jiang, 2015-03-07 06:32 Subject: [GSoC/RFC] Ideas on git fetch --deepen Message-ID: URL: https://gitlist.dev/e/CABwkPcoHPyid0gKjjg0NiXY4TYtCbeSJe6XK5HEDFNKR%2B-_tFQ%40mail.gmail.com ``` Hi all, My name is Dongcan Jiang. I am studying for my Master Degree at Peking University majoring in Computer Science. I have been using Git to manage my projects for about half a year. It's really exciting that Git has been helping me make revision control much more convenient. Therefore, I am very interested in doing some works for Git in GSoC 2015. I have submitted my v2 patch on a microproject recently, and I am waiting for comments on v2 now. In the meantime, I have been scanning ideas on the git gsoc page and their related mails. I find that I am interested in most of them, especially "git fetch --deepen" idea. Here is my understanding about this idea. Although "deepen" and "depth" have different behavior, "deepen" can be achieved by "depth" with some extra calculation. If we know the distance between "my history bottom" and "your tips", we can set the sum of "deepen" step and this distance as "depth" step. Then we can reuse the logics of processing "depth" to complete it. Take graph in [1] as an example. > (upstream) > ---o---o---o---A---B > > (you) > A---B the distance of "my history bottom"(A) and "your tips"(B) is 1, then "git fetch --deepen=3" can be achieved by "git fetch --depth=4" However, I am a little worried about whether this idea is enough for a GSoC project. I hope you can give me some comments and suggestions. [1] http://article.gmane.org/gmane.comp.version-control.git/212950 Thanks, Dongcan -- 江东灿（Dongcan Jiang） Team of Search Engine & Web Mining School of Electronic Engineering & Computer Science Peking University, Beijing, 100871, P.R.China ``` ## Dongcan Jiang, 2015-03-08 08:57 Subject: Re: [GSoC/RFC] Ideas on git fetch --deepen Message-ID: URL: https://gitlist.dev/e/CABwkPcoWUcK2vzqUBcEcuNzizYMtP%3DqUiew2HmDgqtivfDXbDg%40mail.gmail.com In-Reply-To: ``` Hi, all After digging into how "git fetch" works, I have found that my previous understanding was too rash. I'm sorry for that. I find that the current workflow of "git fetch --depth" is as follows: 1. 'fetch module' calls 'git-upload-pack service' via 'transport module' to fetch ref with `depth`. 2. Such call is finished by pipe I/O. 3. git-upload-pack receives arguments such as 'depth' to fetch commits, and send back to the caller. Therefore, if we want to implement "--deepen" for "git fetch", we have to modify the arguments protocol of git-upload-pack service by adding the 'shallow commit' hash. Then, we can fetch `depth` commits before the 'shallow commit' in git-upload-pack service. Apparently, we have to output error message when the 'shallow commit' hash is not in the repository. However, I still have a question for [1]. I am not quite following this: > If you want the new history leading to the updated tip, > you can just say: > > git fetch > > without any --depth nor --deepen option to end up with: > > (you) > A---B---C---D---E---F---...---W---X---Y---Z As far as I know, git fetch --depth would fetch new history to the local remotes' refs. Does it mean that we have to change the behavior of "git fetch --depth"? I hope you can give me some comments or suggestions, letting me know whether I am in the right way. Thanks. [1] http://article.gmane.org/gmane.comp.version-control.git/212950 2015-03-07 14:32 GMT+08:00 Dongcan Jiang : > Hi all, > > My name is Dongcan Jiang. I am studying for my Master Degree at Peking > University > majoring in Computer Science. I have been using Git to manage my projects > for about half a year. It's really exciting that Git has been helping me make > revision control much more convenient. Therefore, I am very interested in > doing some works for Git in GSoC 2015. > > I have submitted my v2 patch on a microproject recently, and I am waiting > for comments on v2 now. > > In the meantime, I have been scanning ideas on the git gsoc page and > their related mails. I find that I am interested in most of them, especially > "git fetch --deepen" idea. > > Here is my understanding about this idea. > > Although "deepen" and "depth" have different behavior, "deepen" can be > achieved by "depth" with some extra calculation. If we know the > distance between "my history bottom" and "your tips", we can set the sum > of "deepen" step and this distance as "depth" step. Then we can reuse > the logics of processing "depth" to complete it. > > Take graph in [1] as an example. > >> (upstream) >> ---o---o---o---A---B >> >> (you) >> A---B > > the distance of "my history bottom"(A) and "your tips"(B) is 1, then > "git fetch --deepen=3" can be achieved by "git fetch --depth=4" > > However, I am a little worried about whether this idea is enough for a > GSoC project. I hope you can give me some comments and suggestions. > > [1] http://article.gmane.org/gmane.comp.version-control.git/212950 > > Thanks, > Dongcan > > -- > 江东灿（Dongcan Jiang） > Team of Search Engine & Web Mining > School of Electronic Engineering & Computer Science > Peking University, Beijing, 100871, P.R.China -- 江东灿（Dongcan Jiang） Team of Search Engine & Web Mining School of Electronic Engineering & Computer Science Peking University, Beijing, 100871, P.R.China ``` ## Duy Nguyen, 2015-03-08 09:34 Subject: Re: [GSoC/RFC] Ideas on git fetch --deepen Message-ID: URL: https://gitlist.dev/e/CACsJy8B5HwwZei%2Bao6Z4dt1eG27Mj4z2_QDKzVPTLphm6JrVWA%40mail.gmail.com In-Reply-To: ``` On Sun, Mar 8, 2015 at 3:57 PM, Dongcan Jiang wrote: > Hi, all > > After digging into how "git fetch" works, I have found that my previous > understanding was too rash. I'm sorry for that. > > I find that the current workflow of "git fetch --depth" is as follows: > > 1. 'fetch module' calls 'git-upload-pack service' via 'transport module' > to fetch ref with `depth`. > 2. Such call is finished by pipe I/O. > 3. git-upload-pack receives arguments such as 'depth' to fetch commits, > and send back to the caller. > > Therefore, if we want to implement "--deepen" for "git fetch", we have to > modify the arguments protocol of git-upload-pack service by adding the > 'shallow commit' hash. > Then, we can fetch `depth` commits before the 'shallow commit' in > git-upload-pack service. > Apparently, we have to output error message when the 'shallow commit' hash > is not in the repository. Close. You can't figure this shallow commit hash from client side (client repo is shortened). So you can't send it. What you send is exactly what the user gives you (e.g. --deepen=5, then you send "deepen 5" or similar). The server side (git-upload-pack) knows about the shallow boundary of the client and can walk its (i.e. server-side) commit graph to find out the new, deepened boundary. Then the server sends necessary objects plus instructions to update shallow boundary to the client. Also, I think this work would include support for smart-http protocol. It goes a slightly different route, "git fetch" ends up at transport.c, but then transport-helper.c and calls git-remote-http(s) which is implemented by remote-curl.c. This one handles http stuff then passes control to git-fetch-pack.c. It's fetch-pack that talks to upload-pack.c Have a look at Documentation/technical/pack-protocol.txt and protocol-capabilities.txt (and http-protocol.txt for smart-http). The function that "draws" shallow boundary based on --depth is get_shallow_commits() in shallow.c. I suspect you need to improve it a bit to use with --deepen. I guess you can look at these [1]. Those changes touch shallow repo in a bit different way, but the main code path is more or less the same (smart-http not touched). [1] https://github.com/pclouds/git/commits/7edde8b83a20abb3cd404e2c5f07e3c29a2891f7 -- Duy ``` ## Dongcan Jiang, 2015-03-08 10:14 Subject: Re: [GSoC/RFC] Ideas on git fetch --deepen Message-ID: URL: https://gitlist.dev/e/CABwkPcoQRSFySYy%2BDvQ4TJJEDmMi%2BFPHhRumBgxdTVDdwvjAjg%40mail.gmail.com In-Reply-To: ``` It is really helpful. Thanks a lot! 2015-03-08 17:34 GMT+08:00 Duy Nguyen : > On Sun, Mar 8, 2015 at 3:57 PM, Dongcan Jiang wrote: >> Hi, all >> >> After digging into how "git fetch" works, I have found that my previous >> understanding was too rash. I'm sorry for that. >> >> I find that the current workflow of "git fetch --depth" is as follows: >> >> 1. 'fetch module' calls 'git-upload-pack service' via 'transport module' >> to fetch ref with `depth`. >> 2. Such call is finished by pipe I/O. >> 3. git-upload-pack receives arguments such as 'depth' to fetch commits, >> and send back to the caller. >> >> Therefore, if we want to implement "--deepen" for "git fetch", we have to >> modify the arguments protocol of git-upload-pack service by adding the >> 'shallow commit' hash. >> Then, we can fetch `depth` commits before the 'shallow commit' in >> git-upload-pack service. >> Apparently, we have to output error message when the 'shallow commit' hash >> is not in the repository. > > Close. You can't figure this shallow commit hash from client side > (client repo is shortened). So you can't send it. What you send is > exactly what the user gives you (e.g. --deepen=5, then you send > "deepen 5" or similar). The server side (git-upload-pack) knows about > the shallow boundary of the client and can walk its (i.e. server-side) > commit graph to find out the new, deepened boundary. Then the server > sends necessary objects plus instructions to update shallow boundary > to the client. > > Also, I think this work would include support for smart-http > protocol. It goes a slightly different route, "git fetch" ends up at > transport.c, but then transport-helper.c and calls git-remote-http(s) > which is implemented by remote-curl.c. This one handles http stuff > then passes control to git-fetch-pack.c. It's fetch-pack that talks to > upload-pack.c > > Have a look at Documentation/technical/pack-protocol.txt and > protocol-capabilities.txt (and http-protocol.txt for smart-http). The > function that "draws" shallow boundary based on --depth is > get_shallow_commits() in shallow.c. I suspect you need to improve it a > bit to use with --deepen. I guess you can look at these [1]. Those > changes touch shallow repo in a bit different way, but the main code > path is more or less the same (smart-http not touched). > > [1] https://github.com/pclouds/git/commits/7edde8b83a20abb3cd404e2c5f07e3c29a2891f7 > -- > Duy -- 江东灿（Dongcan Jiang） Team of Search Engine & Web Mining School of Electronic Engineering & Computer Science Peking University, Beijing, 100871, P.R.China ```