{"thread":{"id":"59831","subject":"\"git fetch --refetch\" and multiple (separate/orphan) branches","startedAt":"2023-06-02T21:23:09Z","lastAt":"2023-08-10T07:15:10Z","messageCount":3,"participants":["Tao Klerks","Robert Coup"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"477989","messageId":"CAPMMpoiJ4cNcAR9gO5d-749N3YW-88p1gMnX8ySGgz84Mr9coA@mail.gmail.com","threadId":"59831","inReplyTo":null,"subject":"\"git fetch --refetch\" and multiple (separate/orphan) branches","fromName":"Tao Klerks","fromEmail":"tao@klerks.biz","sentAt":"2023-06-02T21:22:53Z","receivedAt":"2023-06-02T21:23:09Z","isPatch":false,"sender":{"key":"tao@klerks.biz","avatar":"https://avatars.githubusercontent.com/u/531704?v=4"},"body":"Hi folks,\n\nI just recently noticed that \"--refetch\" was added in 2.36, and I got\npretty excited - the ability to \"fill in\" missing blobs after a\ntoo-filtered clone is something that I've wanted a number of times, as\nI mentioned in 2021 in thread\nhttps://public-inbox.org/git/CAPMMpohOuXX-0YOjV46jFZFvx7mQdj0p7s8SDR4SQxj5hEhCgg@mail.gmail.com/\n.\n\nWhen I first ran \"git fetch --refetch\" today however (git 2.38.1,\nagainst server git/2.38.4.gl1), with a configured blob filter of\n\"blob:1100M\", a much higher size than any blob in the history, it only\ngot a *relatively* small number of objects - 3GB of data rather than\nthe 18GB that a new unfiltered fetch would have retrieved.\n\nAfter some more testing I tried again, and got the expected outcome\nthat time. The relevant difference between the two attempts is that in\nthe first case, when I only got some of the objects I expected, there\nwas an updated tag as a result of the fetch. The second time, when I\ngot everything, there were no updated refs.\n\nIn this repository there are several \"independent\" sets of branches,\nand the tag updated in that first fetch belongs to one of the\nsmaller-history branches.\n\nWhat I believe is happening is that *if* there are refs to be updated\n(or new refs, presumably), *then* the objects returned to the client\nare only those required for those refs. If, on the other hand, there\nare no updated refs, then you get what is advertised in the doc: \"all\nobjects as a fresh clone would [...]\".\n\nI've tested a couple of different scenarios and the behavior seems\nconsistent with this explanation.\n\nIn a repo where all branches are derived from the same history, this\nprobably isn't very noticeable; in the repo I'm working on it makes a\nhuge difference, so the only way I can imagine getting \"correct\"\nbehavior would be to always to a \"git fetch\" right before the \"git\nfetch --refetch\".\n\nIs this a bug, or expected behavior that should be noted in the doc,\nor do we consider the multiple-independent-branches usecase to be\nedge-casey enough to be an easter egg for people like me?\n\nThanks,\nTao\n"},{"id":"478015","messageId":"CACf-nVfUotaTYeCC9XMvnYYNhjX+EW89z7fhUX0Ok9TpsVTRTw@mail.gmail.com","threadId":"59831","inReplyTo":"CAPMMpoiJ4cNcAR9gO5d-749N3YW-88p1gMnX8ySGgz84Mr9coA@mail.gmail.com","subject":"Re: \"git fetch --refetch\" and multiple (separate/orphan) branches","fromName":"Robert Coup","fromEmail":"robert@coup.net.nz","sentAt":"2023-06-03T08:18:43Z","receivedAt":"2023-06-03T08:19:00Z","isPatch":false,"sender":{"key":"robert@coup.net.nz","avatar":"https://avatars.githubusercontent.com/u/59874?v=4"},"body":"Hi Tao\n\nOn Fri, 2 Jun 2023 at 22:23, Tao Klerks wrote:\n\n> What I believe is happening is that *if* there are refs to be updated\n> (or new refs, presumably), *then* the objects returned to the client\n> are only those required for those refs. If, on the other hand, there\n> are no updated refs, then you get what is advertised in the doc: \"all\n> objects as a fresh clone would [...]\".\n>\n> I've tested a couple of different scenarios and the behavior seems\n> consistent with this explanation.\n\nDo you have a repo & steps that could reproduce this easily? Otherwise\nI can try and work up something.\n\n> Is this a bug, or expected behavior that should be noted in the doc,\n> or do we consider the multiple-independent-branches usecase to be\n> edge-casey enough to be an easter egg for people like me?\n\nAt first glance it appears to be a bug.\n\nThanks,\n\nRob :)\n"},{"id":"480428","messageId":"CAPMMpoh9xP8PomAUTHGZfYAMAgHb_BF74EVgmo1C-DQh22-cUA@mail.gmail.com","threadId":"59831","inReplyTo":"CACf-nVfUotaTYeCC9XMvnYYNhjX+EW89z7fhUX0Ok9TpsVTRTw@mail.gmail.com","subject":"Re: \"git fetch --refetch\" and multiple (separate/orphan) branches","fromName":"Tao Klerks","fromEmail":"tao@klerks.biz","sentAt":"2023-08-10T07:14:55Z","receivedAt":"2023-08-10T07:15:10Z","isPatch":false,"sender":{"key":"tao@klerks.biz","avatar":"https://avatars.githubusercontent.com/u/531704?v=4"},"body":"Hi Robert,\n\nSorry about the extended delay, I haven't had a chance to do \"git\nhacking\" in a while.\n\nOn Sat, Jun 3, 2023 at 10:18 AM Robert Coup wrote:\n>\n> On Fri, 2 Jun 2023 at 22:23, Tao Klerks wrote:\n>\n> > What I believe is happening is that *if* there are refs to be updated\n> > (or new refs, presumably), *then* the objects returned to the client\n> > are only those required for those refs. If, on the other hand, there\n> > are no updated refs, then you get what is advertised in the doc: \"all\n> > objects as a fresh clone would [...]\".\n> >\n> > I've tested a couple of different scenarios and the behavior seems\n> > consistent with this explanation.\n>\n> Do you have a repo & steps that could reproduce this easily? Otherwise\n> I can try and work up something.\n>\n\nDoes the following work? It shows that with a change to the orphan\nbranch from another client, a refetch in the original client gets\nabout half the objects (the ones for the orphan branch that was\nupdated), and in another fetch right after, with no new changes, the\nrefetch gets all 600-or-so objects.\n\n\ncreate_n_commits() {\n for i in $(seq $2); do\n echo \"another new line $RANDOM\" >> \"$1/datafile\"\n git -C \"$1\" add datafile\n git -C \"$1\" commit -m anothercommit -q\n done\n}\n\nmkdir refetch-testing\nSERVERFOLDER=refetch-testing/server\ngit init \"$SERVERFOLDER\" --bare\n\nCLIENTFOLDER=refetch-testing/client\ngit init \"$CLIENTFOLDER\"\ngit -C \"$CLIENTFOLDER\" remote add origin \"../server\"\n\ngit -C \"$CLIENTFOLDER\" checkout -b main\ncreate_n_commits \"$CLIENTFOLDER\" 100\ngit -C \"$CLIENTFOLDER\" push origin HEAD\n\ngit -C \"$CLIENTFOLDER\" checkout --orphan orphan\ncreate_n_commits \"$CLIENTFOLDER\" 100\ngit -C \"$CLIENTFOLDER\" push origin HEAD\n\necho \"---HERE IS A NORMAL FULL REFETCH---\"\ngit -C \"$CLIENTFOLDER\" fetch --refetch\necho \"---NORMAL FULL REFETCH ENDS---\"\n\nOTHERCLIENTFOLDER=refetch-testing/otherclient\ngit clone \"$SERVERFOLDER\" \"$OTHERCLIENTFOLDER\"\ngit -C \"$OTHERCLIENTFOLDER\" checkout orphan\ncreate_n_commits \"$OTHERCLIENTFOLDER\" 5\ngit -C \"$OTHERCLIENTFOLDER\" push origin HEAD\n\necho \"---HERE IS A WEIRD PARTIAL REFETCH OF ONE BRANCH ONLY---\"\ngit -C \"$CLIENTFOLDER\" fetch --refetch\necho \"---WEIRD PARTIAL REFETCH ENDS---\"\n\necho \"---HERE IS NORMAL REFETCH AGAIN---\"\ngit -C \"$CLIENTFOLDER\" fetch --refetch\necho \"---NORMAL REFETCH ENDS---\"\n\nrm -rf refetch-testing\n"}]}