{"thread":{"id":"53181","subject":"Git clone question","startedAt":"2020-04-07T18:21:43Z","lastAt":"2020-04-08T02:26:28Z","messageCount":3,"participants":["Sankar P","Bryan Turner"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"394990","messageId":"CAMSEaH5eZdNooNHQJAto_0dCZSFtiji8ZYvft+oZQXfzDHSyEg@mail.gmail.com","threadId":"53181","inReplyTo":"CAMSEaH6+fQuDjhY-5THoYpEjjrHU4Sofnmr-nASHaTZbHvQw1w@mail.gmail.com","subject":"Git clone question","fromName":"Sankar P","fromEmail":"sankar.curiosity@gmail.com","sentAt":"2020-04-07T18:22:40Z","receivedAt":"2020-04-07T18:21:43Z","isPatch":false,"sender":{"key":"sankar.curiosity@gmail.com","avatar":null},"body":"Hi\n\nI am trying to understand about git clone.\n\nFrom the few git videos that I have watched and using git, I\nunderstand that git stores the difference between each version, as an\nobject, with the sha of the diff as the way to address the object.\n\nHowever, what is not clear to me is, how does `git clone` then work ?\nIf a repository has a thousand commits, do we download all the\nthousand objects to the client system and then apply them one on top\nof the other ? I am sure that must not be the case because the `git\nclone` operation completes so fast, and I doubt if my disks are that\nfast.\n\nHowever, when I do a `git clone` I could see the history until the first commit.\n\nI can also partially ignore the history and clone only the last N\ncommits history too. So my question is, how does `git clone` work\nunder the hood and how is it so fast ? Does the git server save the\nexpanded git tree (with all the git patches applied) and we just\ntransfer them when we do the `git clone` ?\n\nAre there any good talks / papers / books on the internals of the\nworking of git ?\n\nThanks.\n\n\n-- \nSankar P\nhttp://psankar.blogspot.com\n"},{"id":"395001","messageId":"CAGyf7-GVYMXKpptZ8w3tcZAKp41j5_EDe9kp6YcDp7=S4LuVpQ@mail.gmail.com","threadId":"53181","inReplyTo":"CAMSEaH5eZdNooNHQJAto_0dCZSFtiji8ZYvft+oZQXfzDHSyEg@mail.gmail.com","subject":"Re: Git clone question","fromName":"Bryan Turner","fromEmail":"bturner@atlassian.com","sentAt":"2020-04-07T20:01:09Z","receivedAt":"2020-04-07T20:01:22Z","isPatch":false,"sender":{"key":"bturner@atlassian.com","avatar":"https://gravatar.com/avatar/16bcf3167981c1ef7c804e502642366d888a35b0d0b0a4ca01fdc442aa1acb1e?d=mp&s=160"},"body":"On Tue, Apr 7, 2020 at 11:21 AM Sankar P <sankar.curiosity@gmail.com> wrote:\n>\n> Hi\n>\n> I am trying to understand about git clone.\n>\n> From the few git videos that I have watched and using git, I\n> understand that git stores the difference between each version, as an\n> object, with the sha of the diff as the way to address the object.\n\nThis is not correct. Git stores full objects, not diffs/patches, and\nthe SHA is of the full object contents, not the changes to those\ncontents versus the previous contents.\n\nObjects are stored compressed (using libz). In addition to\ncompression, when objects are packed Git can use a technique called\n\"delta compression\" to allow it to build one object in terms of\nanother. This is something like a diff/patch, but it's not the delta\nthat gets hashed; it's the full object (before and after).\n\n>\n> However, what is not clear to me is, how does `git clone` then work ?\n> If a repository has a thousand commits, do we download all the\n> thousand objects to the client system and then apply them one on top\n> of the other ? I am sure that must not be the case because the `git\n> clone` operation completes so fast, and I doubt if my disks are that\n> fast.\n\nBetween libz compression and delta compression the pack file\ncontaining all of those objects tends to be substantially smaller than\nthe full set of objects.\n\n>\n> However, when I do a `git clone` I could see the history until the first commit.\n\nYes, when cloning by default you receive all the objects in the\nhistory of the repository and, as you note below, you can also perform\na \"shallow clone\" to limit what you get to some specific depth. Newer\nversions of Git are also working on introducing the concept of a\n\"partial clone\", where some types of objects (large files, for\nexample) are not downloaded up front and are instead downloaded on\nfirst access. This isn't yet widely supported, though.\n\n>\n> I can also partially ignore the history and clone only the last N\n> commits history too. So my question is, how does `git clone` work\n> under the hood and how is it so fast ? Does the git server save the\n> expanded git tree (with all the git patches applied) and we just\n> transfer them when we do the `git clone` ?\n\nAs noted above, there's no \"patching\" going on. The server does store\na packfile, though, and depending on how recently the pack was created\nit can reuse substantial portions of that pack when creating a pack to\nserve the clone.\n\n>\n> Are there any good talks / papers / books on the internals of the\n> working of git ?\n\nHave you checked https://git-scm.com/book/en/v2 ?\n\n>\n> Thanks.\n>\n>\n> --\n> Sankar P\n> http://psankar.blogspot.com\n"},{"id":"395042","messageId":"CAMSEaH6jP3xSLZ_9omJ-8nqjJVYor1TpLZWjW3+K=OZVtM3OSw@mail.gmail.com","threadId":"53181","inReplyTo":"CAGyf7-GVYMXKpptZ8w3tcZAKp41j5_EDe9kp6YcDp7=S4LuVpQ@mail.gmail.com","subject":"Re: Git clone question","fromName":"Sankar P","fromEmail":"sankar.curiosity@gmail.com","sentAt":"2020-04-08T02:27:29Z","receivedAt":"2020-04-08T02:26:28Z","isPatch":false,"sender":{"key":"sankar.curiosity@gmail.com","avatar":null},"body":"> Objects are stored compressed (using libz). In addition to\n> compression, when objects are packed Git can use a technique called\n> \"delta compression\" to allow it to build one object in terms of\n> another. This is something like a diff/patch, but it's not the delta\n> that gets hashed; it's the full object (before and after).\n\noh okay.\n\n>\n> Between libz compression and delta compression the pack file\n> containing all of those objects tends to be substantially smaller than\n> the full set of objects.\n\nFascinating.\n\n> As noted above, there's no \"patching\" going on. The server does store\n> a packfile, though, and depending on how recently the pack was created\n> it can reuse substantial portions of that pack when creating a pack to\n> serve the clone.\n\n> Have you checked https://git-scm.com/book/en/v2 ?\n\nThanks. I will check this out. The 10th chapter seems related. I\nearlier came across this but thought it was a user guide, based on the\nfirst few chapters and did not reach here.\n\nIf there are any other blog posts, talks, etc. about the delta\ncompression, packfile etc. too, please let me know. Thanks.\n\n-- \nSankar P\nhttp://psankar.blogspot.com\n"}]}