{"thread":{"id":"1415","subject":"Experiences with git-clone-pack and rsync","startedAt":"2005-08-04T21:06:05Z","lastAt":"2005-08-05T06:39:44Z","messageCount":2,"participants":["Johannes Schindelin","Junio C Hamano"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"6812","messageId":"Pine.LNX.4.63.0508042230150.24657@wgmdd8.biozentrum.uni-wuerzburg.de","threadId":"1415","inReplyTo":null,"subject":"Experiences with git-clone-pack and rsync","fromName":"Johannes Schindelin","fromEmail":"johannes.schindelin@gmx.de","sentAt":"2005-08-04T21:06:05Z","receivedAt":"2005-08-04T21:06:05Z","isPatch":false,"sender":{"key":"johannes.schindelin@gmx.de","avatar":"https://avatars.githubusercontent.com/u/127790?v=4"},"body":"Hi,\n\nI just tried to clone a relatively big repository from a slow machine to a \nslow machine. I'm talking about a 1.2 gigabyte repository, packed down to \n120 megabyte, containing more than 21000 commits. When git-clone-script \ndid not show anything for over 15 minutes, I decided to find out what's \nhappening. The server sat, partly swapping, partly git-rev-list'ing, \ngit-pack-objects dozing.\n\nNow, I know some internals of git, so I went in and did an rsync, which is \nperfectly reasonable, given that I do not need the server to unpack the \nobjects, pack them again, and then - after a while - sending them as they \nwere: packed. It took 60 seconds with -- evidently -- almost no load for \nthe server.\n\nBTW, I am not quite sure why the machine started swapping. Maybe it was \nsome other process on that server, but it could have been git-rev-list \nalso, keeping those 21000 commits in memory in order to sort them. Or, it \ncould have been something worse: git holding lots and lots of objects in \nmemory.\n\nSo, I don't know if git-daemon, which basically does the same thing as \ngit-clone-pack on the server side, would not be a pretty good way to bring \ngit.kernel.org (once it exists) to a halt.\n\nMaybe there should be some kind of heuristics in git-daemon, i.e. \ngit-count-objects in reverse, to decide if it is not better to (at least \noptionally) just send the pack as is, even if it contains more objects \nthan the user actually asked for. Or, for big projects like the kernel, \njust send the pack if at least one needed object is contained in it. Hey, \ngit-http-pull already does that :-)\n\nBut maybe I just cried \"wolf\"...\n\nCiao,\nDscho\n\nP.S.: There is a serious flaw in git-fetch-pack, though, which probably \npersists when using git-daemon as server: If interrupted, it does not kill \nthe remote git-rev-list and git-pack-objects. I can bring down my poor \nserver pretty easily by issuing \"git pull\", interrupting that, \nand repeating that several times. Not sure how to fix that.\n"},{"id":"6842","messageId":"7vpsssopjj.fsf@assigned-by-dhcp.cox.net","threadId":"1415","inReplyTo":"Pine.LNX.4.63.0508042230150.24657@wgmdd8.biozentrum.uni-wuerzburg.de","subject":"Re: Experiences with git-clone-pack and rsync","fromName":"Junio C Hamano","fromEmail":"junkio@cox.net","sentAt":"2005-08-05T06:39:44Z","receivedAt":"2005-08-05T06:39:44Z","isPatch":false,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Johannes Schindelin <Johannes.Schindelin@gmx.de> writes:\n\n> But maybe I just cried \"wolf\"...\n\nI do not think you are crying wolf.  I shared the same concern\nfrom the beginning and that was partly why I was pushing for\nthe dumb server approach.\n"}]}