{"thread":{"id":"31639","subject":"git pull transfers useless files","startedAt":"2012-09-24T17:51:20Z","lastAt":"2012-09-24T19:17:31Z","messageCount":3,"participants":["Angelo Borsotti","Junio C Hamano","Jeff King"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"199833","messageId":"CAB9Jk9BcusbERNW0ZTsuq6xd7onOuggfTxJnf7D_USNv-uhC1g@mail.gmail.com","threadId":"31639","inReplyTo":null,"subject":"git pull transfers useless files","fromName":"Angelo Borsotti","fromEmail":"angelo.borsotti@gmail.com","sentAt":"2012-09-24T17:51:20Z","receivedAt":"2012-09-24T17:51:20Z","isPatch":false,"sender":{"key":"angelo.borsotti@gmail.com","avatar":null},"body":"Hello,\n\ngit pull transfers useless files when called with the --squash option\nand merge=binary\nattribute.\nConsider the following example:\n\n#!/bin/bash\n\nset -v\ncd remote\nrm -rf * .git/\ngit init\necho '*.pdf -crlf -diff merge=binary' >.git/info/attributes\n\ntouch f1\ngit add f1\necho \"aaa\" >f1.pdf\ngit add f1.pdf\ncp .pdf f2.pdf\ngit add f2.pdf\ngit commit -m A\ncd ..\n\ncd local\nrm -rf * .git/\ngit init\necho '*.pdf -crlf -diff merge=binary' >.git/info/attributes\ngit remote add remote ../remote\n\ntouch f3\ngit add f3\ngit commit -m B\ngit checkout -b develop\n\necho \"bbb\" >f2.pdf\ngit add f2.pdf\ngit commit -m C\ngit pull -v --squash remote master\n\nls\ncat .pdf with the path of a pdf file\nthat is really large and run it.\nWhen it executes the git pull it spends on my computer some 30 seconds,\nobviously transferring the pdf file, that then it disregards because of the\nmerge=binary attribute.\nWhen a commit contains many binary files, the command spends a lot of\ntime needlessly.\n\nIs it possible to optimize it?\n\nThank you\n-Angelo Borsotti\n"},{"id":"199844","messageId":"7v3927z1pt.fsf@alter.siamese.dyndns.org","threadId":"31639","inReplyTo":"CAB9Jk9BcusbERNW0ZTsuq6xd7onOuggfTxJnf7D_USNv-uhC1g@mail.gmail.com","subject":"Re: git pull transfers useless files","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2012-09-24T18:59:26Z","receivedAt":"2012-09-24T18:59:26Z","isPatch":false,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Angelo Borsotti writes:\n\n> When it executes the git pull it spends on my computer some 30 seconds,\n> obviously transferring the pdf file, that then it disregards because of the\n> merge=binary attribute.\n> When a commit contains many binary files, the command spends a lot of\n> time needlessly.\n\nThat is hardly needless nor useless.\n\nUnless you are saying that having a large pdf file as binary is\nuseless in your history, that is. After such a merge or squash\nmerge, you still need to be able to say \"git checkout remote\" to\ncheck out their version to inspect, and you did not have that\nversion of the blob before that \"git pull\".\n"},{"id":"199847","messageId":"20120924191731.GB15858@sigill.intra.peff.net","threadId":"31639","inReplyTo":"CAB9Jk9BcusbERNW0ZTsuq6xd7onOuggfTxJnf7D_USNv-uhC1g@mail.gmail.com","subject":"Re: git pull transfers useless files","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2012-09-24T19:17:31Z","receivedAt":"2012-09-24T19:17:31Z","isPatch":false,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Mon, Sep 24, 2012 at 07:51:20PM +0200, Angelo Borsotti wrote:\n\n> #!/bin/bash\n> \n> set -v\n> cd remote\n> rm -rf * .git/\n> git init\n> echo '*.pdf -crlf -diff merge=binary' >.git/info/attributes\n> \n> touch f1\n> git add f1\n> echo \"aaa\" >f1.pdf\n> git add f1.pdf\n> cp .pdf f2.pdf\n> git add f2.pdf\n> git commit -m A\n> cd ..\n> \n> cd local\n> rm -rf * .git/\n> git init\n> echo '*.pdf -crlf -diff merge=binary' >.git/info/attributes\n> git remote add remote ../remote\n> \n> touch f3\n> git add f3\n> git commit -m B\n> git checkout -b develop\n> \n> echo \"bbb\" >f2.pdf\n> git add f2.pdf\n> git commit -m C\n> git pull -v --squash remote master\n> \n> ls\n> cat \n> set +v\n> \n> Replace .pdf with the path of a pdf file\n> that is really large and run it.\n> When it executes the git pull it spends on my computer some 30 seconds,\n> obviously transferring the pdf file, that then it disregards because of the\n> merge=binary attribute.\n\nIt does not disregard the file. The working tree is left with your\nexisting version of f2, but note that the index still marks the\nconflict. Your next step would be to resolve the conflict in some way.\nTowards that end, you can now inspect both sides:\n\n git show :2:f2.pdf ;# our side\n git show :3:f2.pdf ;# their side\n\nOr you can invoke a mergetool to start a third-party merge helper on the\nbinary files:\n\n git mergetool\n\nOr you can just resolve in favor of \"their\" side:\n\n git checkout --theirs f2.pdf\n\n>From your description, I imagine your intent is to simply resolve in\nfavor of the \"ours\", and never look at the other side. However, git does\nnot have enough information to know that.\n\nThere is no \"merge=ours\" attribute (and indeed, it would be kind of\ncrazy, since your result would depend on which direction you were\nmerging, which is something you only know at the time of merge. Hence it\nmakes sense as a command-line option for a strategy, but not something\nthat is an attribute as a file).\n\nAll that being said, we can construct a case where the contents of the\nPDF really _don't_ matter at all to the result. Like this:\n\n # new repo\n git init parent\n cd parent\n\n # make a commit with a giant file\n echo small >foo.txt\n cp .pdf big.pdf\n git add .\n git commit -m one\n\n # now get rid of the giant file\n git rm big.pdf\n git commit -m two\n\n # now merge it into another history\n git init ../child\n cd ../child\n echo unrelated >file.txt\n git add .\n git commit -m three\n git pull -v --squash ../parent master\n\nBecause we are doing a squash merge, we will throw away most of the\nhistory we fetch, and only ever look at the tip of parent/master (which\nin this case does not contain the PDF), and the shared ancestor (which\nin this case is empty, since there is no shared history).\n\nSo in theory we could get by with fetching all the commits (to do the\nhistory traversal), and the trees and blobs only from the tip commit.\nBut that is not a good idea in general for two reasons:\n\n 1. Even if that PDF is not used in the actual merge algorithm, the\n contents of the earlier commits are useful for figuring out what\n happened (e.g., when resolving another conflict, you might want to\n refer back via \"git log\").\n\n 2. It breaks git's reachability assumptions. Git always makes sure\n that if you have object X, you have all of the objects it refers\n to, the ones they refer to, and so on. This assumption underlies\n many of git's operations (e.g., what we need to send to a remote\n who claims to have commit X).\n\n In this case, since you are using --squash, you could presumably\n throw away the original history after doing the squash merge. But\n it would be quite complex to special-case this in the protocol, and\n almost certainly not worth it for this corner case.\n\n-Peff\n"}]}