{"thread":{"id":"54131","subject":"git clone --shallow-since can result in inconsistent shallow clones","startedAt":"2020-08-25T18:38:20Z","lastAt":"2020-08-25T19:52:07Z","messageCount":2,"participants":["Nelson Elhage","Jeff King"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"404461","messageId":"CAPSG9dZV2EPpVKkOMcjv5z+NF7rUu=V-ZkZNx47rCv122HsiKg@mail.gmail.com","threadId":"54131","inReplyTo":null,"subject":"git clone --shallow-since can result in inconsistent shallow clones","fromName":"Nelson Elhage","fromEmail":"nelhage@nelhage.com","sentAt":"2020-08-25T18:38:04Z","receivedAt":"2020-08-25T18:38:20Z","isPatch":false,"sender":{"key":"nelhage@nelhage.com","avatar":null},"body":"Thank you for filling out a Git bug report!\nPlease answer the following questions to help us understand your issue.\n\nWhat did you do before the bug happened? (Steps to reproduce your issue)\n\nI ran\n\n  git clone --shallow-since=\"1548454011\" \"https://github.com/abseil/abseil-cpp\"\n\nto produce a shallow clone of abseil-cpp.git, with the aim of going\ndeep enough to grab commit `5e0dcf72c64fae912184d2e0de87195fe8f0a425`,\nwhich I know to have a commit date of `1548454011`.\n\nWhat did you expect to happen? (Expected behavior)\n\n- I expected the command to produce a valid shallow git clone.\n- I further expected the repository to include commit\n  5e0dcf72c64fae912184d2e0de87195fe8f0a425, which has a commit date <=\n  the provided `--shallow`, as do all of its descendants up to the\n  `master` branch\n\nWhat happened instead? (Actual behavior)\n\n- The clone command produced an inconsistent shallow clone. In the\nrepository I see:\n\n    $ cat .git/shallow\n    5e0dcf72c64fae912184d2e0de87195fe8f0a425\n    89ea0c5ff34aaa5855cfc7aa41f323b8a0ef0ede\n\nBut commit `5e0dcf72c64fae912184d2e0de87195fe8f0a425` is missing. An\nattempt to `git fetch --unshallow` errors out, because the server\nsends an `unshallow 5e0dcf72c64fae912184d2e0de87195fe8f0a425`, which\nwe are unable to execute since we're missing that object.\n\nThat object is also the specific one I mentioned above that I wanted.\n\nWhat's different between what you expected and what actually happened?\n\nAnything else you want to add:\n\nThe problem here is triggered by passing a `shallow-since` that lies\n*between* the first and and second parents of a merge commit that\nitself is on the first-parent spine. If we examine the relevant\nportion of `abseil-cpp.git`'s history, we find:\n\n    $ git --no-pager log --format='%h %ct' --graph\n89ea0c5ff34aaa5855cfc7aa41f323b8a0ef0ede~6..89ea0c5ff34aaa5855cfc7aa41f323b8a0ef0ede\n    *   89ea0c5 1548698816      # WANT\n    |\\\n    | * 7ec3270 1548194022      # WANT\n    * | 5e0dcf7 1548454011      # WANT\n    * | 0dffca4 1548346230      # DON'T WANT\n    * | 6b4201f 1548261751      # DON'T WANT\n    |/\n    * 0b1e6d4 1547838308        # DON'T WANT\n    * efccc50 1547753737        # DON'T WANT\n\nI've annotated the commits with WANT or DON'T WONT based on whether or\nnot their commit time is included by the `--shallow-since` filter.\n\nWhat is happening, I believe, is that we are marking 89ea0c5 as\nshallow, since its first parent is unwanted. However, marking it\nshallow causes pack generation to ignore _all_ of its parents,\nincluding 7ec3270, which we _do_ want. This results in the\ninconsistent state where we mark `5e0dcf7` as shallow (and send the\n`shallow` line), but don't send the actual object.\n\nIt's unfortunately a bit unclear to me what _should_ happen here. We\nreally want a way to mark `89ea0c5` as \"partially-shallow\", and send\nits second parent, but not its first parent, but shallowness is a\nproperty of an entire commit, not of a specific commit/parent\nrelationship. However, it'd be nice if we at least ended up with a\nconsistent state, instead of with a repository with invalid `shallow`\nmarks.\n\nPlease review the rest of the bug report below.\nYou can delete any lines you don't wish to share.\n\n\n[System Info]\ngit version:\ngit version 2.28.0.461.g40977abb40\ncpu: x86_64\nbuilt from commit: 40977abb4059c11004726852a79df64f4553944d\nsizeof-long: 8\nsizeof-size_t: 8\nshell-path: /bin/sh\nuname: Linux 5.4.0-42-generic #46-Ubuntu SMP Fri Jul 10 00:24:02 UTC 2020 x86_64\ncompiler info: gnuc: 9.3\nlibc info: glibc: 2.31\n$SHELL (typically, interactive shell): /bin/bash\n\n\n[Enabled Hooks]\n\n- Nelson Elhage\n"},{"id":"404475","messageId":"20200825195148.GC1419759@coredump.intra.peff.net","threadId":"54131","inReplyTo":"CAPSG9dZV2EPpVKkOMcjv5z+NF7rUu=V-ZkZNx47rCv122HsiKg@mail.gmail.com","subject":"Re: git clone --shallow-since can result in inconsistent shallow clones","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2020-08-25T19:51:48Z","receivedAt":"2020-08-25T19:52:07Z","isPatch":false,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Tue, Aug 25, 2020 at 11:38:04AM -0700, Nelson Elhage wrote:\n\n> It's unfortunately a bit unclear to me what _should_ happen here. We\n> really want a way to mark `89ea0c5` as \"partially-shallow\", and send\n> its second parent, but not its first parent, but shallowness is a\n> property of an entire commit, not of a specific commit/parent\n> relationship. However, it'd be nice if we at least ended up with a\n> consistent state, instead of with a repository with invalid `shallow`\n> marks.\n\nI think this is the same issue I reported recently here:\n\n  https://lore.kernel.org/git/20200721160643.GA3288097@coredump.intra.peff.net/\n\nAFAICT the shallow feature is just defective and can't accurately\nrepresent this situation. Unfortunately nobody seemed to have any bright\nideas, and the developer who implemented most of the shallow features\n(including shallow-since) is no longer active.\n\nSo I suspect it is fixable, but probably requires somebody to get pretty\nfamiliar with the shallow code, and propose a fix that involves both\ncode and a protocol change.\n\n-Peff\n"}]}