{"thread":{"id":"41137","subject":"Initial git clone behaviour","startedAt":"2016-01-06T22:26:05Z","lastAt":"2016-01-06T23:24:32Z","messageCount":3,"participants":["Eric Curtin","Junio C Hamano"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"275451","messageId":"CANpvso58uPtMcz_L0D=WBcCj+mSLLYZ5LpfO1i5yLo-jrb6bsA@mail.gmail.com","threadId":"41137","inReplyTo":null,"subject":"Initial git clone behaviour","fromName":"Eric Curtin","fromEmail":"ericcurtin17@gmail.com","sentAt":"2016-01-06T22:26:05Z","receivedAt":"2016-01-06T22:26:05Z","isPatch":false,"sender":{"key":"ericcurtin17@gmail.com","avatar":null},"body":"Hi Guys,\n\nI am not a git developer or a git expert but just a change I would love\nto see.\n\nWhen I clone a really large repository (like the linux kernel for\nexample) initially on a brand new machine, it can take quite a while\nbefore I can start working with the code.\n\nOften I do a standard git clone:\n\ngit clone (name of repo)\n\nFollowed by a depth=1 clone in parallel, so I can get building and\nworking with the code asap:\n\ngit clone --depth=1 (name of repo)\n\nCould we change the default behavior of git so that we initially get\nall the current files quickly so that we can start working them and\nthen getting the rest of the data? At least a user could get to work\nquicker this way. Any disadvantages of this approach? Maybe I am not\nthe first to suggest something like this.\n"},{"id":"275452","messageId":"CAPc5daXeNay1uF=qQ=G82kyu37uHhy-uEOWU6tz_bPYfFam=rA@mail.gmail.com","threadId":"41137","inReplyTo":"CANpvso58uPtMcz_L0D=WBcCj+mSLLYZ5LpfO1i5yLo-jrb6bsA@mail.gmail.com","subject":"Re: Initial git clone behaviour","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2016-01-06T23:14:45Z","receivedAt":"2016-01-06T23:14:45Z","isPatch":false,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"On Wed, Jan 6, 2016 at 2:26 PM, Eric Curtin wrote:\n>\n> Often I do a standard git clone:\n>\n> git clone (name of repo)\n>\n> Followed by a depth=1 clone in parallel, so I can get building and\n> working with the code asap:\n>\n> git clone --depth=1 (name of repo)\n>\n> Could we change the default behavior of git so that we initially get\n> all the current files quickly so that we can start working them and\n> then getting the rest of the data? At least a user could get to work\n> quicker this way. Any disadvantages of this approach?\n\nIt would put more burden on a shared and limited resource (i.e.\nthe server side).\n\nFor example, I just tried a depth=1 clone of Linus's repository from\n\n git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git\n\nwhich transferred ~150MB pack data to check out 52k files in 90 seconds.\n\nOn the other hand, a full clone transferred ~980MB pack data and it took\n170 seconds to complete. You can already see that a full clone is highly\noptimized--it does not take even twice the time of getting the most recent\ncheckout to grab 10 years worth of development (562k of commits).\n\nThis efficiency comes from some tradeoffs, and one of them is that not\nall the data necessary to check out the latest tree contents can be stored\nnear the beginning of the pack data. So \"we'll checkout the tip while the\nremainder of the data is still incoming\" would not be a workable, unless\nyou are willing to destroy the full-clone performance.\n"},{"id":"275453","messageId":"CANpvso4NZZcAYfQZPt0i2G+MWD4ppga0XQZwYDhq1syA2f1GJw@mail.gmail.com","threadId":"41137","inReplyTo":"CAPc5daXeNay1uF=qQ=G82kyu37uHhy-uEOWU6tz_bPYfFam=rA@mail.gmail.com","subject":"Re: Initial git clone behaviour","fromName":"Eric Curtin","fromEmail":"ericcurtin17@gmail.com","sentAt":"2016-01-06T23:24:32Z","receivedAt":"2016-01-06T23:24:32Z","isPatch":false,"sender":{"key":"ericcurtin17@gmail.com","avatar":null},"body":"On 6 January 2016 at 23:14, Junio C Hamano wrote:\n> On Wed, Jan 6, 2016 at 2:26 PM, Eric Curtin wrote:\n>>\n>> Often I do a standard git clone:\n>>\n>> git clone (name of repo)\n>>\n>> Followed by a depth=1 clone in parallel, so I can get building and\n>> working with the code asap:\n>>\n>> git clone --depth=1 (name of repo)\n>>\n>> Could we change the default behavior of git so that we initially get\n>> all the current files quickly so that we can start working them and\n>> then getting the rest of the data? At least a user could get to work\n>> quicker this way. Any disadvantages of this approach?\n>\n> It would put more burden on a shared and limited resource (i.e.\n> the server side).\n>\n> For example, I just tried a depth=1 clone of Linus's repository from\n>\n> git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git\n>\n> which transferred ~150MB pack data to check out 52k files in 90 seconds.\n>\n> On the other hand, a full clone transferred ~980MB pack data and it took\n> 170 seconds to complete. You can already see that a full clone is highly\n> optimized--it does not take even twice the time of getting the most recent\n> checkout to grab 10 years worth of development (562k of commits).\n>\n> This efficiency comes from some tradeoffs, and one of them is that not\n> all the data necessary to check out the latest tree contents can be stored\n> near the beginning of the pack data. So \"we'll checkout the tip while the\n> remainder of the data is still incoming\" would not be a workable, unless\n> you are willing to destroy the full-clone performance.\n\nOk, my internet connection at home is pretty terrible then! I don't get\nnowhere near these timings. It takes over an hour to do a full clone\nfrom my house. And approx 30 mins for the depth=1 (approx, did not time\nit).\n\nThat all makes sense I guess, probably not a good idea to regress the\nfull time performance for the sake of this use case. Was just a query\nreally!\n"}]}