{"thread":{"id":"51532","subject":"Possible to unshallow or deepen based on local objects?","startedAt":"2019-07-26T18:43:26Z","lastAt":"2019-08-08T22:34:06Z","messageCount":3,"participants":["Mark Rushakoff","Junio C Hamano"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"379383","messageId":"CALxJwdP70PpcJKFC3FRm5t58yjxz438WwYisuFwQsJ2oaXV_9A@mail.gmail.com","threadId":"51532","inReplyTo":null,"subject":"Possible to unshallow or deepen based on local objects?","fromName":"Mark Rushakoff","fromEmail":"mark@influxdata.com","sentAt":"2019-07-26T18:43:13Z","receivedAt":"2019-07-26T18:43:26Z","isPatch":false,"sender":{"key":"mark@influxdata.com","avatar":null},"body":"I am building some CI tooling, and I am working with a large-ish repository, so\nI am trying to start with a shallow clone of the repository and deepen it on\ndemand. I am finding it very difficult to correctly switch between fetch\n--depth and fetch --deepen.\n\nI am looking for a way to \"recover\" from a fetch --depth call that made my\nhistory shallower than before. Ultimately, I may take a different approach to\njust work around this altogether, but I thought I'd ask here first.\n\nIn particular, if there is history like this:\n\nA --- B --- C --- D --- E --- F\n       \\                 \\\n         ---------------- G\nAnd my shallow history includes D..F, then I believe I am limited in my options\nfor fetching G:\n\n- If I do a plain fetch of G, or if I use --deepen, then I pull in A and all of\n  its history, which I don't strictly need or want yet.\n- If I know G's parent is B, and I fetch --depth=1, then (I think) I will see\n  B, E, F, and G, effectively losing sight of D\n\nAs a more contrived example, suppose I have a shallow clone of a repository:\n\nbash-3.2$ git clone --depth=4 -q git@github.com:git/git.git\nbash-3.2$ cd git\nbash-3.2$ git log --format='%h %d'\n3034dab  (HEAD -> master, origin/master, origin/HEAD)\n98e06de\n352253a\n4098130  (grafted)\n7b974e3\ndf63c2e\n75ce486  (grafted)\n70b39fb  (grafted)\nafc3bf6  (grafted)\nbash-3.2$ cat .git/shallow\n409813088ad55ae4a60f55412f6b5ba6a89d89e7\n70b39fbede78313656e8a6bd9b38b238ab10db2f\n75ce48674889df6a2bb493fb5d6bef0ef60ca7ae\nafc3bf6eb13d9fc489b569164819cff44db8ac17\n\nAnd then suppose I fetch with depth smaller than before:\n\nbash-3.2$ git fetch --depth=1 -q\nbash-3.2$ git log --format='%h %d'\n3034dab  (grafted, HEAD -> master, origin/master, origin/HEAD)\nbash-3.2$ cat .git/shallow\n3034dab9ed6b11970a53099a7b3ca981f1461365\n409813088ad55ae4a60f55412f6b5ba6a89d89e7\n70b39fbede78313656e8a6bd9b38b238ab10db2f\n75ce48674889df6a2bb493fb5d6bef0ef60ca7ae\nafc3bf6eb13d9fc489b569164819cff44db8ac17\n\nI can still see the commits I had before the --depth=1 fetch:\n\nbash-3.2$ git log -1 --format=oneline 75ce486\n75ce48674889df6a2bb493fb5d6bef0ef60ca7ae (grafted) Merge branch\n'di/readme-markup-fix'\n\nShort of looking through .git/shallow and removing each entry whose parents I\ncan resolve, is there a built-in command to \"unshallow based on local objects\"?\nI have tried many web search terms, I have looked through the docs for many\nlow-level git commands, I have tried variations of \"git fetch .\", and I am\nstumped.\n"},{"id":"380179","messageId":"CALxJwdM0ifKoQDMXJuuWyeXdR9-oxVpOyODBO0TXcz=d+0vq2Q@mail.gmail.com","threadId":"51532","inReplyTo":"CALxJwdP70PpcJKFC3FRm5t58yjxz438WwYisuFwQsJ2oaXV_9A@mail.gmail.com","subject":"Re: Possible to unshallow or deepen based on local objects?","fromName":"Mark Rushakoff","fromEmail":"mark@influxdata.com","sentAt":"2019-08-08T22:17:43Z","receivedAt":"2019-08-08T22:17:56Z","isPatch":false,"sender":{"key":"mark@influxdata.com","avatar":null},"body":"On Fri, Jul 26, 2019 at 11:43 AM Mark Rushakoff <mark@influxdata.com> wrote:\n>\n> I am building some CI tooling, and I am working with a large-ish repository, so\n> I am trying to start with a shallow clone of the repository and deepen it on\n> demand. I am finding it very difficult to correctly switch between fetch\n> --depth and fetch --deepen.\n>\n> I am looking for a way to \"recover\" from a fetch --depth call that made my\n> history shallower than before. Ultimately, I may take a different approach to\n> just work around this altogether, but I thought I'd ask here first.\n>\n> In particular, if there is history like this:\n>\n> A --- B --- C --- D --- E --- F\n>        \\                 \\\n>          ---------------- G\n> And my shallow history includes D..F, then I believe I am limited in my options\n> for fetching G:\n>\n> - If I do a plain fetch of G, or if I use --deepen, then I pull in A and all of\n>   its history, which I don't strictly need or want yet.\n> - If I know G's parent is B, and I fetch --depth=1, then (I think) I will see\n>   B, E, F, and G, effectively losing sight of D\n>\n> As a more contrived example, suppose I have a shallow clone of a repository:\n>\n> bash-3.2$ git clone --depth=4 -q git@github.com:git/git.git\n> bash-3.2$ cd git\n> bash-3.2$ git log --format='%h %d'\n> 3034dab  (HEAD -> master, origin/master, origin/HEAD)\n> 98e06de\n> 352253a\n> 4098130  (grafted)\n> 7b974e3\n> df63c2e\n> 75ce486  (grafted)\n> 70b39fb  (grafted)\n> afc3bf6  (grafted)\n> bash-3.2$ cat .git/shallow\n> 409813088ad55ae4a60f55412f6b5ba6a89d89e7\n> 70b39fbede78313656e8a6bd9b38b238ab10db2f\n> 75ce48674889df6a2bb493fb5d6bef0ef60ca7ae\n> afc3bf6eb13d9fc489b569164819cff44db8ac17\n>\n> And then suppose I fetch with depth smaller than before:\n>\n> bash-3.2$ git fetch --depth=1 -q\n> bash-3.2$ git log --format='%h %d'\n> 3034dab  (grafted, HEAD -> master, origin/master, origin/HEAD)\n> bash-3.2$ cat .git/shallow\n> 3034dab9ed6b11970a53099a7b3ca981f1461365\n> 409813088ad55ae4a60f55412f6b5ba6a89d89e7\n> 70b39fbede78313656e8a6bd9b38b238ab10db2f\n> 75ce48674889df6a2bb493fb5d6bef0ef60ca7ae\n> afc3bf6eb13d9fc489b569164819cff44db8ac17\n>\n> I can still see the commits I had before the --depth=1 fetch:\n>\n> bash-3.2$ git log -1 --format=oneline 75ce486\n> 75ce48674889df6a2bb493fb5d6bef0ef60ca7ae (grafted) Merge branch\n> 'di/readme-markup-fix'\n>\n> Short of looking through .git/shallow and removing each entry whose parents I\n> can resolve, is there a built-in command to \"unshallow based on local objects\"?\n> I have tried many web search terms, I have looked through the docs for many\n> low-level git commands, I have tried variations of \"git fetch .\", and I am\n> stumped.\n\nI've run into another flavor of this problem now.\n\nI have a shallow clone of a repository. Then sometime later, I obtain a bundle\ncontaining full history.\n\nI thought I could use git fetch --unshallow /path/to/repo.bundle\nmybranch:mybranch, but I've found that git does not support shallow fetches\nfrom a bundle and that unshallow is actually an alias for --depth=2147483647 in\n4dcb167fc3 (fetch: add --unshallow for turning shallow repo into complete one,\n2013-01-11).\n\nIf I do a plain fetch from the bundle, I can manually modify .git/shallow to\n\"unshallow\" my repository like the previous email in this thread. But what I\nthink I still want here is a way to tell git \"here are all the objects I have;\nupdate the shallow boundaries accordingly.\"\n"},{"id":"380180","messageId":"xmqqftmbe17t.fsf@gitster-ct.c.googlers.com","threadId":"51532","inReplyTo":"CALxJwdM0ifKoQDMXJuuWyeXdR9-oxVpOyODBO0TXcz=d+0vq2Q@mail.gmail.com","subject":"Re: Possible to unshallow or deepen based on local objects?","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2019-08-08T22:33:58Z","receivedAt":"2019-08-08T22:34:06Z","isPatch":false,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Mark Rushakoff <mark@influxdata.com> writes:\n\n> I have a shallow clone of a repository. Then sometime later, I obtain a bundle\n> containing full history.\n>\n> I thought I could use git fetch --unshallow /path/to/repo.bundle\n> mybranch:mybranch, but I've found that git does not support shallow fetches\n> from a bundle and that unshallow is actually an alias for --depth=2147483647 in\n> 4dcb167fc3 (fetch: add --unshallow for turning shallow repo into complete one,\n> 2013-01-11).\n>\n> If I do a plain fetch from the bundle, I can manually modify .git/shallow to\n> \"unshallow\" my repository like the previous email in this thread. But what I\n> think I still want here is a way to tell git \"here are all the objects I have;\n> update the shallow boundaries accordingly.\"\n\nIt sounds like a quite sensible request to me.  \n\nOffhand, I do not think of anything that would make it fundamentally\nimpossible to allow us to first add objects to the .git/objects\ndirectory by whatever means available to you (including, but not\nlimited to, exploding the object data from a full bundle) and then\nto tell Git to see if there (still) are missing commits and objects\nthat ought to exist but don't that necessitates the repository to be\nmarked \"shallow\", and adjust the boundary.\n\nI however do not think there is such a code already written X-<.\n"}]}