{"thread":{"id":"38739","subject":"[GSoC/RFC] Ideas on git fetch --deepen","startedAt":"2015-03-07T06:32:23Z","lastAt":"2015-03-08T10:14:43Z","messageCount":4,"participants":["Dongcan Jiang","Duy Nguyen"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"257227","messageId":"CABwkPcoHPyid0gKjjg0NiXY4TYtCbeSJe6XK5HEDFNKR+-_tFQ@mail.gmail.com","threadId":"38739","inReplyTo":null,"subject":"[GSoC/RFC] Ideas on git fetch --deepen","fromName":"Dongcan Jiang","fromEmail":"dongcan.jiang@gmail.com","sentAt":"2015-03-07T06:32:23Z","receivedAt":"2015-03-07T06:32:23Z","isPatch":false,"sender":{"key":"dongcan.jiang@gmail.com","avatar":"https://avatars.githubusercontent.com/u/10103762?v=4"},"body":"Hi all,\n\nMy name is Dongcan Jiang. I am studying for my Master Degree at Peking\nUniversity\nmajoring in Computer Science. I have been using Git to manage my projects\nfor about half a year. It's really exciting that Git has been helping me make\nrevision control much more convenient. Therefore, I am very interested in\ndoing some works for Git in GSoC 2015.\n\nI have submitted my v2 patch on a microproject recently, and I am waiting\nfor comments on v2 now.\n\nIn the meantime, I have been scanning ideas on the git gsoc page and\ntheir related mails. I find that I am interested in most of them, especially\n\"git fetch --deepen\" idea.\n\nHere is my understanding about this idea.\n\nAlthough \"deepen\" and \"depth\" have different behavior, \"deepen\" can be\nachieved by \"depth\" with some extra calculation. If we know the\ndistance between \"my history bottom\" and \"your tips\", we can set the sum\nof \"deepen\" step and this distance as \"depth\" step. Then we can reuse\nthe logics of processing \"depth\" to complete it.\n\nTake graph in [1] as an example.\n\n> (upstream)\n> ---o---o---o---A---B\n>\n> (you)\n> A---B\n\nthe distance of \"my history bottom\"(A) and \"your tips\"(B) is 1, then\n\"git fetch --deepen=3\" can be achieved by \"git fetch --depth=4\"\n\nHowever, I am a little worried about whether this idea is enough for a\nGSoC project. I hope you can give me some comments and suggestions.\n\n[1] http://article.gmane.org/gmane.comp.version-control.git/212950\n\nThanks,\nDongcan\n\n-- \n江东灿（Dongcan Jiang）\nTeam of Search Engine & Web Mining\nSchool of Electronic Engineering & Computer Science\nPeking University, Beijing, 100871, P.R.China\n"},{"id":"257282","messageId":"CABwkPcoWUcK2vzqUBcEcuNzizYMtP=qUiew2HmDgqtivfDXbDg@mail.gmail.com","threadId":"38739","inReplyTo":"CABwkPcoHPyid0gKjjg0NiXY4TYtCbeSJe6XK5HEDFNKR+-_tFQ@mail.gmail.com","subject":"Re: [GSoC/RFC] Ideas on git fetch --deepen","fromName":"Dongcan Jiang","fromEmail":"dongcan.jiang@gmail.com","sentAt":"2015-03-08T08:57:34Z","receivedAt":"2015-03-08T08:57:34Z","isPatch":false,"sender":{"key":"dongcan.jiang@gmail.com","avatar":"https://avatars.githubusercontent.com/u/10103762?v=4"},"body":"Hi, all\n\nAfter digging into how \"git fetch\" works, I have found that my previous\nunderstanding was too rash. I'm sorry for that.\n\nI find that the current workflow of \"git fetch --depth\" is as follows:\n\n 1. 'fetch module' calls 'git-upload-pack service' via 'transport module'\n to fetch ref with `depth`.\n 2. Such call is finished by pipe I/O.\n 3. git-upload-pack receives arguments such as 'depth' to fetch commits,\n and send back to the caller.\n\nTherefore, if we want to implement \"--deepen\" for \"git fetch\", we have to\nmodify the arguments protocol of git-upload-pack service by adding the\n'shallow commit' hash.\nThen, we can fetch `depth` commits before the 'shallow commit' in\ngit-upload-pack service.\nApparently, we have to output error message when the 'shallow commit' hash\nis not in the repository.\n\nHowever, I still have a question for [1]. I am not quite following this:\n\n> If you want the new history leading to the updated tip,\n> you can just say:\n>\n> git fetch\n>\n> without any --depth nor --deepen option to end up with:\n>\n> (you)\n> A---B---C---D---E---F---...---W---X---Y---Z\n\nAs far as I know, git fetch --depth would fetch new history to the\nlocal remotes' refs.\nDoes it mean that we have to change the behavior of \"git fetch --depth\"?\n\nI hope you can give me some comments or suggestions, letting me know\nwhether I am in the right way.\n\nThanks.\n\n[1] http://article.gmane.org/gmane.comp.version-control.git/212950\n\n2015-03-07 14:32 GMT+08:00 Dongcan Jiang :\n> Hi all,\n>\n> My name is Dongcan Jiang. I am studying for my Master Degree at Peking\n> University\n> majoring in Computer Science. I have been using Git to manage my projects\n> for about half a year. It's really exciting that Git has been helping me make\n> revision control much more convenient. Therefore, I am very interested in\n> doing some works for Git in GSoC 2015.\n>\n> I have submitted my v2 patch on a microproject recently, and I am waiting\n> for comments on v2 now.\n>\n> In the meantime, I have been scanning ideas on the git gsoc page and\n> their related mails. I find that I am interested in most of them, especially\n> \"git fetch --deepen\" idea.\n>\n> Here is my understanding about this idea.\n>\n> Although \"deepen\" and \"depth\" have different behavior, \"deepen\" can be\n> achieved by \"depth\" with some extra calculation. If we know the\n> distance between \"my history bottom\" and \"your tips\", we can set the sum\n> of \"deepen\" step and this distance as \"depth\" step. Then we can reuse\n> the logics of processing \"depth\" to complete it.\n>\n> Take graph in [1] as an example.\n>\n>> (upstream)\n>> ---o---o---o---A---B\n>>\n>> (you)\n>> A---B\n>\n> the distance of \"my history bottom\"(A) and \"your tips\"(B) is 1, then\n> \"git fetch --deepen=3\" can be achieved by \"git fetch --depth=4\"\n>\n> However, I am a little worried about whether this idea is enough for a\n> GSoC project. I hope you can give me some comments and suggestions.\n>\n> [1] http://article.gmane.org/gmane.comp.version-control.git/212950\n>\n> Thanks,\n> Dongcan\n>\n> --\n> 江东灿（Dongcan Jiang）\n> Team of Search Engine & Web Mining\n> School of Electronic Engineering & Computer Science\n> Peking University, Beijing, 100871, P.R.China\n\n\n\n-- \n江东灿（Dongcan Jiang）\nTeam of Search Engine & Web Mining\nSchool of Electronic Engineering & Computer Science\nPeking University, Beijing, 100871, P.R.China\n"},{"id":"257284","messageId":"CACsJy8B5HwwZei+ao6Z4dt1eG27Mj4z2_QDKzVPTLphm6JrVWA@mail.gmail.com","threadId":"38739","inReplyTo":"CABwkPcoWUcK2vzqUBcEcuNzizYMtP=qUiew2HmDgqtivfDXbDg@mail.gmail.com","subject":"Re: [GSoC/RFC] Ideas on git fetch --deepen","fromName":"Duy Nguyen","fromEmail":"pclouds@gmail.com","sentAt":"2015-03-08T09:34:41Z","receivedAt":"2015-03-08T09:34:41Z","isPatch":false,"sender":{"key":"pclouds@gmail.com","avatar":"https://avatars.githubusercontent.com/u/720?v=4"},"body":"On Sun, Mar 8, 2015 at 3:57 PM, Dongcan Jiang wrote:\n> Hi, all\n>\n> After digging into how \"git fetch\" works, I have found that my previous\n> understanding was too rash. I'm sorry for that.\n>\n> I find that the current workflow of \"git fetch --depth\" is as follows:\n>\n> 1. 'fetch module' calls 'git-upload-pack service' via 'transport module'\n> to fetch ref with `depth`.\n> 2. Such call is finished by pipe I/O.\n> 3. git-upload-pack receives arguments such as 'depth' to fetch commits,\n> and send back to the caller.\n>\n> Therefore, if we want to implement \"--deepen\" for \"git fetch\", we have to\n> modify the arguments protocol of git-upload-pack service by adding the\n> 'shallow commit' hash.\n> Then, we can fetch `depth` commits before the 'shallow commit' in\n> git-upload-pack service.\n> Apparently, we have to output error message when the 'shallow commit' hash\n> is not in the repository.\n\nClose. You can't figure this shallow commit hash from client side\n(client repo is shortened). So you can't send it. What you send is\nexactly what the user gives you (e.g. --deepen=5, then you send\n\"deepen 5\" or similar). The server side (git-upload-pack) knows about\nthe shallow boundary of the client and can walk its (i.e. server-side)\ncommit graph to find out the new, deepened boundary. Then the server\nsends necessary objects plus instructions to update shallow boundary\nto the client.\n\nAlso, I think this work would include support for smart-http\nprotocol. It goes a slightly different route, \"git fetch\" ends up at\ntransport.c, but then transport-helper.c and calls git-remote-http(s)\nwhich is implemented by remote-curl.c. This one handles http stuff\nthen passes control to git-fetch-pack.c. It's fetch-pack that talks to\nupload-pack.c\n\nHave a look at Documentation/technical/pack-protocol.txt and\nprotocol-capabilities.txt (and http-protocol.txt for smart-http). The\nfunction that \"draws\" shallow boundary based on --depth is\nget_shallow_commits() in shallow.c. I suspect you need to improve it a\nbit to use with --deepen. I guess you can look at these [1]. Those\nchanges touch shallow repo in a bit different way, but the main code\npath is more or less the same (smart-http not touched).\n\n[1] https://github.com/pclouds/git/commits/7edde8b83a20abb3cd404e2c5f07e3c29a2891f7\n-- \nDuy\n"},{"id":"257308","messageId":"CABwkPcoQRSFySYy+DvQ4TJJEDmMi+FPHhRumBgxdTVDdwvjAjg@mail.gmail.com","threadId":"38739","inReplyTo":"CACsJy8B5HwwZei+ao6Z4dt1eG27Mj4z2_QDKzVPTLphm6JrVWA@mail.gmail.com","subject":"Re: [GSoC/RFC] Ideas on git fetch --deepen","fromName":"Dongcan Jiang","fromEmail":"dongcan.jiang@gmail.com","sentAt":"2015-03-08T10:14:43Z","receivedAt":"2015-03-08T10:14:43Z","isPatch":false,"sender":{"key":"dongcan.jiang@gmail.com","avatar":"https://avatars.githubusercontent.com/u/10103762?v=4"},"body":"It is really helpful. Thanks a lot!\n\n2015-03-08 17:34 GMT+08:00 Duy Nguyen :\n> On Sun, Mar 8, 2015 at 3:57 PM, Dongcan Jiang wrote:\n>> Hi, all\n>>\n>> After digging into how \"git fetch\" works, I have found that my previous\n>> understanding was too rash. I'm sorry for that.\n>>\n>> I find that the current workflow of \"git fetch --depth\" is as follows:\n>>\n>> 1. 'fetch module' calls 'git-upload-pack service' via 'transport module'\n>> to fetch ref with `depth`.\n>> 2. Such call is finished by pipe I/O.\n>> 3. git-upload-pack receives arguments such as 'depth' to fetch commits,\n>> and send back to the caller.\n>>\n>> Therefore, if we want to implement \"--deepen\" for \"git fetch\", we have to\n>> modify the arguments protocol of git-upload-pack service by adding the\n>> 'shallow commit' hash.\n>> Then, we can fetch `depth` commits before the 'shallow commit' in\n>> git-upload-pack service.\n>> Apparently, we have to output error message when the 'shallow commit' hash\n>> is not in the repository.\n>\n> Close. You can't figure this shallow commit hash from client side\n> (client repo is shortened). So you can't send it. What you send is\n> exactly what the user gives you (e.g. --deepen=5, then you send\n> \"deepen 5\" or similar). The server side (git-upload-pack) knows about\n> the shallow boundary of the client and can walk its (i.e. server-side)\n> commit graph to find out the new, deepened boundary. Then the server\n> sends necessary objects plus instructions to update shallow boundary\n> to the client.\n>\n> Also, I think this work would include support for smart-http\n> protocol. It goes a slightly different route, \"git fetch\" ends up at\n> transport.c, but then transport-helper.c and calls git-remote-http(s)\n> which is implemented by remote-curl.c. This one handles http stuff\n> then passes control to git-fetch-pack.c. It's fetch-pack that talks to\n> upload-pack.c\n>\n> Have a look at Documentation/technical/pack-protocol.txt and\n> protocol-capabilities.txt (and http-protocol.txt for smart-http). The\n> function that \"draws\" shallow boundary based on --depth is\n> get_shallow_commits() in shallow.c. I suspect you need to improve it a\n> bit to use with --deepen. I guess you can look at these [1]. Those\n> changes touch shallow repo in a bit different way, but the main code\n> path is more or less the same (smart-http not touched).\n>\n> [1] https://github.com/pclouds/git/commits/7edde8b83a20abb3cd404e2c5f07e3c29a2891f7\n> --\n> Duy\n\n\n\n-- \n江东灿（Dongcan Jiang）\nTeam of Search Engine & Web Mining\nSchool of Electronic Engineering & Computer Science\nPeking University, Beijing, 100871, P.R.China\n"}]}