{"thread":{"id":"5844","subject":"Problem cloning packed-and-pruned http repository","startedAt":"2006-10-06T21:26:16Z","lastAt":"2006-10-11T14:00:28Z","messageCount":19,"participants":["Panagiotis Issaris","Sean","Junio C Hamano","Jakub Narebski","Johannes Schindelin","Petr Baudis"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"28330","messageId":"20061006212616.GA5175@lumumba.uhasselt.be","threadId":"5844","inReplyTo":null,"subject":"Problem cloning packed-and-pruned http repository","fromName":"Panagiotis Issaris","fromEmail":"takis@lumumba.uhasselt.be","sentAt":"2006-10-06T21:26:16Z","receivedAt":"2006-10-06T21:26:16Z","isPatch":false,"sender":{"key":"takis@lumumba.uhasselt.be","avatar":null},"body":"Hi\n\nI've been having trouble setting up a public repository using GIT. After\nI have pushed my repository to a directory within ~/public_html, I can\nclone it. But the repository is _big_ (261M).\n\nSo, I use \"git-repack\" on it and a \"git-prune-packed\". This makes it\nnicely fit in 14MiB. If I try to clone this pruned/packed repository\nagain both cg-clone hangs on it (as does git-clone).\n\nHere's two outputs demonstrating this. One repository was a \"cp -lr\"\nclone [1] of the other, and one was packed/pruned, the other wasn't:\ntakis@poseidon:/tmp$ cg-clone http://lumumba.uhasselt.be/takis/git/ffmpeg-h264.git\ndefaulting to local storage area\nFetching head...\nFetching objects...\nprogress: 38 objects, 159434 bytes\ncg-clone: interrupted\ntakis@poseidon:/tmp$ cg-clone http://lumumba.uhasselt.be/takis/git/ffmpeg-h264-test.git\ndefaulting to local storage area\nFetching head...\nFetching objects...\nGetting alternates list for http://lumumba.uhasselt.be/takis/git/ffmpeg-h264-test.git/\nGetting pack list for http://lumumba.uhasselt.be/takis/git/ffmpeg-h264-test.git/\nprogress: 0 objects, 0 bytes\ncg-clone: interrupted\n\nSo, I tried tracing it, to see what was going on:\ntakis@poseidon:/tmp/a$ ps x|grep git\n18386 pts/9 S+ 0:00 /bin/sh /home/takis/bin/git-clone http://lumumba.uhasselt.be/takis/git/ffmpeg-h264-test.git\n18400 pts/9 S+ 0:00 git-http-fetch -v -a -w heads/master heads/master http://lumumba.uhasselt.be/takis/git/ffmpeg-h264-test.git/\n18416 pts/10 S+ 0:00 grep git\n\ntakis@poseidon:/tmp/a$ strace -f -p 18400\nProcess 18400 attached - interrupt to quit\nselect(0, [], [], [], {0, 48000}) = 0 (Timeout)\npoll([{fd=4, events=POLLIN}], 1, 0) = 0\ngettimeofday({1160169868, 454932}, NULL) = 0\ngettimeofday({1160169868, 454989}, NULL) = 0\nselect(0, [], [], [], {0, 50000}) = 0 (Timeout)\npoll([{fd=4, events=POLLIN}], 1, 0) = 0\ngettimeofday({1160169868, 506226}, NULL) = 0\ngettimeofday({1160169868, 506277}, NULL) = 0\nselect(0, [], [], [], {0, 50000}) = 0 (Timeout)\npoll([{fd=4, events=POLLIN}], 1, 0) = 0\ngettimeofday({1160169868, 558245}, NULL) = 0\ngettimeofday({1160169868, 558296}, NULL) = 0\nselect(0, [], [], [], {0, 50000}) = 0 (Timeout)\npoll([{fd=4, events=POLLIN}], 1, 0) = 0\ngettimeofday({1160169868, 610227}, NULL) = 0\ngettimeofday({1160169868, 610277}, NULL) = 0\nselect(0, [], [], [], {0, 50000}) = 0 (Timeout)\n...\n\nAnd this keeps going on... so I can't see any data getting in :-(\n\nAny hints what I might be doing wrong?\n\nThanks in advance for any replies! :)\n\nWith friendly regards,\nTakis\n\n[1] I am aware of \"git-clone -l -s\" but wanted to make fully\nindependent copy and wasn't sure about any links between them if only\nusing \"-l\".\n-- \nOpenPGP key: http://lumumba.uhasselt.be/takis/takis_public_key.txt\nfingerprint: 6571 13A3 33D9 3726 F728 AA98 F643 B12E ECF3 E029\n"},{"id":"28332","messageId":"20061006220542.GA5890@lumumba.uhasselt.be","threadId":"5844","inReplyTo":"20061006212616.GA5175@lumumba.uhasselt.be","subject":"Re: Problem cloning packed-and-pruned http repository","fromName":"Panagiotis Issaris","fromEmail":"takis@lumumba.uhasselt.be","sentAt":"2006-10-06T22:05:42Z","receivedAt":"2006-10-06T22:05:42Z","isPatch":false,"sender":{"key":"takis@lumumba.uhasselt.be","avatar":null},"body":"Hi\n\nOn Fri, Oct 06, 2006 at 11:26:16PM +0200 or thereabouts, wrote:\n> [...]\n> I've been having trouble setting up a public repository using GIT. After\n> I have pushed my repository to a directory within ~/public_html, I can\n> clone it. But the repository is _big_ (261M).\n> \n> So, I use \"git-repack\" on it and a \"git-prune-packed\". This makes it\n> nicely fit in 14MiB. If I try to clone this pruned/packed repository\n> again both cg-clone hangs on it (as does git-clone).\n> [...] \n> takis@poseidon:/tmp$ cg-clone http://lumumba.uhasselt.be/takis/git/ffmpeg-h264-test.git\n> defaulting to local storage area\n> Fetching head...\n> Fetching objects...\n> Getting alternates list for http://lumumba.uhasselt.be/takis/git/ffmpeg-h264-test.git/\n> Getting pack list for http://lumumba.uhasselt.be/takis/git/ffmpeg-h264-test.git/\n> progress: 0 objects, 0 bytes\n> cg-clone: interrupted\n> \nApparently, it does work :-/ After a _long_ time I noticed that the\nrepository indeed got cloned... I am not sure if this is normal behavior\nor not, it seemed to take a _really_ long. I would have thought\ndownloading 14MiB should not take a long time on my ADSL line.\n\n\nWith friendly regards,\nTakis\n\n-- \nOpenPGP key: http://lumumba.uhasselt.be/takis/takis_public_key.txt\nfingerprint: 6571 13A3 33D9 3726 F728 AA98 F643 B12E ECF3 E029\n"},{"id":"28335","messageId":"BAYC1-PASMTP08A34A8FB0703E4D2ABAF9AE130@CEZ.ICE","threadId":"5844","inReplyTo":"20061006220542.GA5890@lumumba.uhasselt.be","subject":"Re: Problem cloning packed-and-pruned http repository","fromName":"Sean","fromEmail":"seanlkml@sympatico.ca","sentAt":"2006-10-06T23:49:30Z","receivedAt":"2006-10-06T23:49:30Z","isPatch":false,"sender":{"key":"seanlkml@sympatico.ca","avatar":"https://gravatar.com/avatar/f92923f54fc08c401fc59b71829d4b89e9b8087fbba45ff87c82e6a83aee02ae?d=mp&s=160"},"body":"On Sat, 7 Oct 2006 00:05:42 +0200\ntakis@lumumba.uhasselt.be (Panagiotis Issaris) wrote:\n\n> Apparently, it does work :-/ After a _long_ time I noticed that the\n> repository indeed got cloned... I am not sure if this is normal behavior\n> or not, it seemed to take a _really_ long. I would have thought\n> downloading 14MiB should not take a long time on my ADSL line.\n\nIt's not normal. There's something odd going on. I can clone your\nrepo with wget in about two minutes, while Git still hadn't downloaded\nanything after 12 minutes when I killed it.\n\nPoked around a bit, and found that if I comment out these lines from\nhttp-fetch.c:\n\n#ifndef NO_EXPAT\n if (remote_ls(repo, \"objects/pack/\", PROCESS_FILES,\n process_ls_pack, NULL) == 0)\n return 0;\n#endif\n\nThen everything downloads nice and fast. Does anyone have a guess\nwhy that would be?\n\nSean\n"},{"id":"28336","messageId":"BAYC1-PASMTP11CF83A008B0B3BA5F6B15AE100@CEZ.ICE","threadId":"5844","inReplyTo":"BAYC1-PASMTP08A34A8FB0703E4D2ABAF9AE130@CEZ.ICE","subject":"[RFC PATCH] Add WEBDAV timeout to http-fetch.","fromName":"Sean","fromEmail":"seanlkml@sympatico.ca","sentAt":"2006-10-07T02:04:01Z","receivedAt":"2006-10-07T02:04:01Z","isPatch":true,"sender":{"key":"seanlkml@sympatico.ca","avatar":"https://gravatar.com/avatar/f92923f54fc08c401fc59b71829d4b89e9b8087fbba45ff87c82e6a83aee02ae?d=mp&s=160"},"body":"\nIf a server is having problems delivering the Git repo over WEBDAV,\ntimeout after two minutes so that a regular http transfer can\nbe tried.\n\n---\n\n http-fetch.c | 1 +\n 1 files changed, 1 insertions(+), 0 deletions(-)\n\nNot sure if this is the correct fix, but it should improve the situation\nfor cloning and fetching from servers like Takis's. When connecting to\nhis server WEBDAV doesn't respond after the initial connection. Nothing\nproceeds until the OS connection times out many minutes later.\n\nThis patch sets the CURL timeout to two minutes so that things proceed\nsooner. Even with this patch it takes two extra minutes of \"dead time\"\nto complete all operations; obivously this still sucks.\n\nHowever, I don't know if the two minute timeout is long enough for\nall cases with a server where WEBDAV is functioning properly.\nHopefully someone who knows more about Curl can comment and perhaps\noffer another solution.\n\nMaybe the real solution is just to figure out and fix whatever is\ngoing on with the WEBDAV server and forget this patch.\n\nSean\n\n\ndiff --git a/http-fetch.c b/http-fetch.c\nindex bc74f30..046245a 100644\n--- a/http-fetch.c\n+++ b/http-fetch.c\n@@ -861,6 +861,7 @@ static int remote_ls(struct alt_base *re\n \tcurl_easy_setopt(slot->curl, CURLOPT_UPLOAD, 1);\n \tcurl_easy_setopt(slot->curl, CURLOPT_CUSTOMREQUEST, DAV_PROPFIND);\n \tcurl_easy_setopt(slot->curl, CURLOPT_HTTPHEADER, dav_headers);\n+\tcurl_easy_setopt(slot->curl, CURLOPT_TIMEOUT, 120);\n \n \tif (start_active_slot(slot)) {\n \t\trun_active_slot(slot);\n-- \n1.4.2.3.gabd697\n"},{"id":"28341","messageId":"20061007083046.GB12900@lumumba.uhasselt.be","threadId":"5844","inReplyTo":"20061006220401.a4485d67.seanlkml@sympatico.ca","subject":"Re: [RFC PATCH] Add WEBDAV timeout to http-fetch.","fromName":"Panagiotis Issaris","fromEmail":"takis@lumumba.uhasselt.be","sentAt":"2006-10-07T08:30:46Z","receivedAt":"2006-10-07T08:30:46Z","isPatch":true,"sender":{"key":"takis@lumumba.uhasselt.be","avatar":null},"body":"Hi,\n\nThanks for your quick reply!\n\nOn Fri, Oct 06, 2006 at 10:04:01PM -0400 or thereabouts, Sean wrote:\n> If a server is having problems delivering the Git repo over WEBDAV,\n> timeout after two minutes so that a regular http transfer can\n> be tried.\n> [...]\n> Maybe the real solution is just to figure out and fix whatever is\n> going on with the WEBDAV server and forget this patch.\nI had a quick glance at the Apache2 config on this server:\ngrep -i dav /etc/apache2/apache2.conf\n# redirects for folders with DAV methods.\nBrowserMatch \"^WebDAVFS/1.[012]\" redirect-carefully\n\nGoogle showed me some bugs occurring on other software where they said\nthe problems were happening because the redirect weren't handled\ncorrectly. I haven't got a clue about this, but thought mentioning it\nmight ring someone's bell :)\n\nWith friendly regards,\nTakis\n-- \nOpenPGP key: http://lumumba.luc.ac.be/takis/takis_public_key.txt\nfingerprint: 6571 13A3 33D9 3726 F728 AA98 F643 B12E ECF3 E029\n"},{"id":"28354","messageId":"7viriwsa75.fsf@assigned-by-dhcp.cox.net","threadId":"5844","inReplyTo":"BAYC1-PASMTP11CF83A008B0B3BA5F6B15AE100@CEZ.ICE","subject":"Re: [RFC PATCH] Add WEBDAV timeout to http-fetch.","fromName":"Junio C Hamano","fromEmail":"junkio@cox.net","sentAt":"2006-10-07T10:15:58Z","receivedAt":"2006-10-07T10:15:58Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Sean writes:\n\n> If a server is having problems delivering the Git repo over WEBDAV,\n> timeout after two minutes so that a regular http transfer can\n> be tried.\n>\n> ---\n>\n> http-fetch.c | 1 +\n> 1 files changed, 1 insertions(+), 0 deletions(-)\n>\n> Not sure if this is the correct fix, but it should improve the situation\n> for cloning and fetching from servers like Takis's. When connecting to\n> his server WEBDAV doesn't respond after the initial connection. Nothing\n> proceeds until the OS connection times out many minutes later.\n>\n> This patch sets the CURL timeout to two minutes so that things proceed\n> sooner. Even with this patch it takes two extra minutes of \"dead time\"\n> to complete all operations; obivously this still sucks.\n>\n> However, I don't know if the two minute timeout is long enough for\n> all cases with a server where WEBDAV is functioning properly.\n> Hopefully someone who knows more about Curl can comment and perhaps\n> offer another solution.\n>\n> Maybe the real solution is just to figure out and fix whatever is\n> going on with the WEBDAV server and forget this patch.\n\nI think it is prudent to protect the client from a broken server\nand it is independent from \"fixing\" the server side. It would\nperhaps make sense to make this overridable somehow but I am not\nsure how -- .git/config is too global (the problem would be per\nremote site), and having the user to set environment variable\nonly when going to specific server is too cumbersome to be\nuseful. This ideally should be a per-remote configuration\nitem.\n\n> diff --git a/http-fetch.c b/http-fetch.c\n> index bc74f30..046245a 100644\n> --- a/http-fetch.c\n> +++ b/http-fetch.c\n> @@ -861,6 +861,7 @@ static int remote_ls(struct alt_base *re\n> \tcurl_easy_setopt(slot->curl, CURLOPT_UPLOAD, 1);\n> \tcurl_easy_setopt(slot->curl, CURLOPT_CUSTOMREQUEST, DAV_PROPFIND);\n> \tcurl_easy_setopt(slot->curl, CURLOPT_HTTPHEADER, dav_headers);\n> +\tcurl_easy_setopt(slot->curl, CURLOPT_TIMEOUT, 120);\n> \n> \tif (start_active_slot(slot)) {\n> \t\trun_active_slot(slot);\n> -- \n> 1.4.2.3.gabd697\n"},{"id":"28357","messageId":"eg82tq$2uq$1@sea.gmane.org","threadId":"5844","inReplyTo":"7viriwsa75.fsf@assigned-by-dhcp.cox.net","subject":"Re: [RFC PATCH] Add WEBDAV timeout to http-fetch.","fromName":"Jakub Narebski","fromEmail":"jnareb@gmail.com","sentAt":"2006-10-07T11:27:39Z","receivedAt":"2006-10-07T11:27:39Z","isPatch":true,"sender":{"key":"jnareb@gmail.com","avatar":"https://avatars.githubusercontent.com/u/2706?v=4"},"body":"Junio C Hamano wrote:\n\n> I think it is prudent to protect the client from a broken server\n> and it is independent from \"fixing\" the server side.  It would\n> perhaps make sense to make this overridable somehow but I am not\n> sure how -- .git/config is too global (the problem would be per\n> remote site), and having the user to set environment variable\n> only when going to specific server is too cumbersome to be\n> useful.  This ideally should be a per-remote configuration\n> item.\n\nPerhaps use undocumented (hint, hint! for whoever did code this)\nper-user ~/.gitconfig?\n-- \nJakub Narebski\nWarsaw, Poland\nShadeHawk on #git\n"},{"id":"28372","messageId":"Pine.LNX.4.63.0610071930300.14200@wbgn013.biozentrum.uni-wuerzburg.de","threadId":"5844","inReplyTo":"eg82tq$2uq$1@sea.gmane.org","subject":"Re: [RFC PATCH] Add WEBDAV timeout to http-fetch.","fromName":"Johannes Schindelin","fromEmail":"johannes.schindelin@gmx.de","sentAt":"2006-10-07T17:32:25Z","receivedAt":"2006-10-07T17:32:25Z","isPatch":true,"sender":{"key":"johannes.schindelin@gmx.de","avatar":"https://avatars.githubusercontent.com/u/127790?v=4"},"body":"Hi,\n\nOn Sat, 7 Oct 2006, Jakub Narebski wrote:\n\n> Perhaps use undocumented (hint, hint! for whoever did code this)\n> per-user ~/.gitconfig?\n\nA good idea to use this (hint, hint! whoever finds out how it works can \ndocument it as well) feature.\n\nHOWEVER, Junio pointed out that he'd like a finer grain than per-repo, and \n.gitconfig is a coarser one! (BTW why do you strip Junio from your Cc: \nwhen you respond directly to his email?)\n\nCiao,\nDscho\n"},{"id":"28374","messageId":"20061007174556.GD20017@pasky.or.cz","threadId":"5844","inReplyTo":"Pine.LNX.4.63.0610071930300.14200@wbgn013.biozentrum.uni-wuerzburg.de","subject":"Re: [RFC PATCH] Add WEBDAV timeout to http-fetch.","fromName":"Petr Baudis","fromEmail":"pasky@suse.cz","sentAt":"2006-10-07T17:45:56Z","receivedAt":"2006-10-07T17:45:56Z","isPatch":true,"sender":{"key":"pasky@ucw.cz","avatar":"https://avatars.githubusercontent.com/u/18439?v=4"},"body":"(This post's gonna be really strange.)\n\nDear diary, on Sat, Oct 07, 2006 at 07:32:25PM CEST, I got a letter\nwhere Johannes Schindelin said that...\n> On Sat, 7 Oct 2006, Jakub Narebski wrote:\n> \n> > Perhaps use undocumented (hint, hint! for whoever did code this)\n> > per-user ~/.gitconfig?\n> \n> A good idea to use this (hint, hint! whoever finds out how it works can \n> document it as well) feature.\n\n(Which is no excuse for the initial implementor not documenting it,\nthough.)\n\n> HOWEVER, Junio pointed out that he'd like a finer grain than per-repo, and \n> .gitconfig is a coarser one!\n\n(I honestly don't think it's worth it at all to make this configurable.\nWhat's the point?)\n\n> (BTW why do you strip Junio from your Cc: when you respond directly to\n> his email?)\n\nJakub, what about putting a mini-FAQ to your signature? ;-)\n\nIt would be nice if gmane supported honoring mail-followup-to when\ngatewaying posts back to the mailing list (of course it would need to do\nsome smart anti-spam protection).\n\n-- \n\t\t\t\tPetr \"Pasky\" Baudis\nStuff: http://pasky.or.cz/\n#!/bin/perl -sp0777i > This patch sets the CURL timeout to two minutes so that things proceed\n> > sooner. Even with this patch it takes two extra minutes of \"dead time\"\n> > to complete all operations; obivously this still sucks.\n> >\n> > However, I don't know if the two minute timeout is long enough for\n> > all cases with a server where WEBDAV is functioning properly.\n> > Hopefully someone who knows more about Curl can comment and perhaps\n> > offer another solution.\n> >\n> > Maybe the real solution is just to figure out and fix whatever is\n> > going on with the WEBDAV server and forget this patch.\n> \n> I think it is prudent to protect the client from a broken server\n> and it is independent from \"fixing\" the server side. It would\n>[...]\nWouldn't most users ctrl-c the program before the two minute timeout occurs?\nEspecially since their appears to be nothing happening?\n\nWith friendly regards,\nTakis\n \n"},{"id":"28396","messageId":"7vvemvpxx1.fsf@assigned-by-dhcp.cox.net","threadId":"5844","inReplyTo":"20061007193559.GA27920@poseidon.issaris.org","subject":"Re: [RFC PATCH] Add WEBDAV timeout to http-fetch.","fromName":"Junio C Hamano","fromEmail":"junkio@cox.net","sentAt":"2006-10-07T22:24:10Z","receivedAt":"2006-10-07T22:24:10Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Panagiotis Issaris writes:\n\n>> > Maybe the real solution is just to figure out and fix whatever is\n>> > going on with the WEBDAV server and forget this patch.\n>> \n>> I think it is prudent to protect the client from a broken server\n>> and it is independent from \"fixing\" the server side. It would\n>>[...]\n> Wouldn't most users ctrl-c the program before the two minute timeout occurs?\n> Especially since their appears to be nothing happening?\n\nI think we are talking about the same thing -- after you kill it\nwith C-c you would want to work it around. The question is how?\n\nYou would want to re-run with timeout value of 0 second (or\nWEBDAV disabled, if we can have such an option); if it works\nthen you would want the tool to remember that you want to use\nthat particular settings but probably only when talking to that\nremote.\n"},{"id":"28397","messageId":"20061007223023.GI20017@pasky.or.cz","threadId":"5844","inReplyTo":"Pine.LNX.4.63.0610071930300.14200@wbgn013.biozentrum.uni-wuerzburg.de","subject":"Re: [RFC PATCH] Add WEBDAV timeout to http-fetch.","fromName":"Petr Baudis","fromEmail":"pasky@suse.cz","sentAt":"2006-10-07T22:30:23Z","receivedAt":"2006-10-07T22:30:23Z","isPatch":true,"sender":{"key":"pasky@ucw.cz","avatar":"https://avatars.githubusercontent.com/u/18439?v=4"},"body":"Dear diary, on Sat, Oct 07, 2006 at 07:32:25PM CEST, I got a letter\nwhere Johannes Schindelin said that...\n> Hi,\n> \n> On Sat, 7 Oct 2006, Jakub Narebski wrote:\n> \n> > Perhaps use undocumented (hint, hint! for whoever did code this)\n> > per-user ~/.gitconfig?\n> \n> A good idea to use this (hint, hint! whoever finds out how it works can \n> document it as well) feature.\n> \n> HOWEVER, Junio pointed out that he'd like a finer grain than per-repo, and \n> .gitconfig is a coarser one!\n\nActually, that doesn't matter. The point is that it is of _different_\nshape than this division. It's per remote server, even spanning several\nrepositories. So you want to be able to set it up per the server, at the\nmost global place possible for a regular user, so ~/.gitconfig might be\ngood idea.\n\n-- \n\t\t\t\tPetr \"Pasky\" Baudis\nStuff: http://pasky.or.cz/\n#!/bin/perl -sp0777i Dear diary, on Sat, Oct 07, 2006 at 07:32:25PM CEST, I got a letter\n> where Johannes Schindelin said that...\n>\n> > HOWEVER, Junio pointed out that he'd like a finer grain than per-repo, and \n> > .gitconfig is a coarser one!\n> \n> Actually, that doesn't matter. The point is that it is of _different_\n> shape than this division. It's per remote server, even spanning several\n> repositories. So you want to be able to set it up per the server, at the\n> most global place possible for a regular user, so ~/.gitconfig might be\n> good idea.\n\nActually, I do not think that anybody in her right mind would set this to \ndifferent values for different repos or servers.\n\nI _know_ that if I hit that very problem, the next thing I'd do is set the \ntimeout to 5 seconds _globally_.\n\nCiao,\nDscho\n"},{"id":"28404","messageId":"7vbqonpfyl.fsf@assigned-by-dhcp.cox.net","threadId":"5844","inReplyTo":"Pine.LNX.4.63.0610080034490.14200@wbgn013.biozentrum.uni-wuerzburg.de","subject":"Re: [RFC PATCH] Add WEBDAV timeout to http-fetch.","fromName":"Junio C Hamano","fromEmail":"junkio@cox.net","sentAt":"2006-10-08T04:52:02Z","receivedAt":"2006-10-08T04:52:02Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Johannes Schindelin writes:\n\n> Actually, I do not think that anybody in her right mind would set this to \n> different values for different repos or servers.\n>\n> I _know_ that if I hit that very problem, the next thing I'd do is set the \n> timeout to 5 seconds _globally_.\n\nLet's step back a bit.\n\nThe DAV request in question the one to remote_ls() in\nhttp-fetch.c, which tries to read directly from objects/pack/\ninstead of using objects/info/packs, and it does not matter if\nDAV request fails because it would fall back to non-DAV anyway.\n\nUsing DAV, if it works with the server, has the advantage of not\nhaving to keep objects/info/packs up-to-date from repository\nowner's point of view. But the repository owner ends up keeping\nup-to-date as a side effect of keeping info/refs up-to-date\nanyway (as I do not see a code to read that information over\nDAV), so there is no point doing this over DAV in practice.\n\nPerhaps we should remove call to remote_ls() from\nfetch_indices() unconditionally, not just protected with\nNO_EXPAT and be done with it?\n"},{"id":"28418","messageId":"Pine.LNX.4.63.0610081402430.14200@wbgn013.biozentrum.uni-wuerzburg.de","threadId":"5844","inReplyTo":"7vbqonpfyl.fsf@assigned-by-dhcp.cox.net","subject":"Re: [RFC PATCH] Add WEBDAV timeout to http-fetch.","fromName":"Johannes Schindelin","fromEmail":"johannes.schindelin@gmx.de","sentAt":"2006-10-08T12:03:17Z","receivedAt":"2006-10-08T12:03:17Z","isPatch":true,"sender":{"key":"johannes.schindelin@gmx.de","avatar":"https://avatars.githubusercontent.com/u/127790?v=4"},"body":"Hi,\n\nOn Sat, 7 Oct 2006, Junio C Hamano wrote:\n\n> [...] But the repository owner ends up keeping up-to-date as a side \n> effect of keeping info/refs up-to-date anyway (as I do not see a code to \n> read that information over DAV), so there is no point doing this over \n> DAV in practice.\n\nMakes sense to me.\n\nCiao,\nDscho\n"},{"id":"28420","messageId":"BAYC1-PASMTP053FFB92C509E9427F85B0AE110@CEZ.ICE","threadId":"5844","inReplyTo":"7vbqonpfyl.fsf@assigned-by-dhcp.cox.net","subject":"Re: [RFC PATCH] Add WEBDAV timeout to http-fetch.","fromName":"Sean","fromEmail":"seanlkml@sympatico.ca","sentAt":"2006-10-08T13:19:32Z","receivedAt":"2006-10-08T13:19:32Z","isPatch":true,"sender":{"key":"seanlkml@sympatico.ca","avatar":"https://gravatar.com/avatar/f92923f54fc08c401fc59b71829d4b89e9b8087fbba45ff87c82e6a83aee02ae?d=mp&s=160"},"body":"On Sat, 07 Oct 2006 21:52:02 -0700\nJunio C Hamano wrote:\n\n> Using DAV, if it works with the server, has the advantage of not\n> having to keep objects/info/packs up-to-date from repository\n> owner's point of view. But the repository owner ends up keeping\n> up-to-date as a side effect of keeping info/refs up-to-date\n> anyway (as I do not see a code to read that information over\n> DAV), so there is no point doing this over DAV in practice.\n> \n> Perhaps we should remove call to remote_ls() from\n> fetch_indices() unconditionally, not just protected with\n> NO_EXPAT and be done with it?\n\nThat makes a lot of sense. A server really has to always provide\na objects/info/packs anyway, just to be fetchable today by clients\nthat are compiled with NO_EXPAT.\n\n+1\n\nSean\n"},{"id":"28430","messageId":"7vfydyinto.fsf@assigned-by-dhcp.cox.net","threadId":"5844","inReplyTo":"BAYC1-PASMTP053FFB92C509E9427F85B0AE110@CEZ.ICE","subject":"Re: [RFC PATCH] Add WEBDAV timeout to http-fetch.","fromName":"Junio C Hamano","fromEmail":"junkio@cox.net","sentAt":"2006-10-08T19:56:19Z","receivedAt":"2006-10-08T19:56:19Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Sean writes:\n\n> On Sat, 07 Oct 2006 21:52:02 -0700\n> Junio C Hamano wrote:\n>\n>> Using DAV, if it works with the server, has the advantage of not\n>> having to keep objects/info/packs up-to-date from repository\n>> owner's point of view. But the repository owner ends up keeping\n>> up-to-date as a side effect of keeping info/refs up-to-date\n>> anyway (as I do not see a code to read that information over\n>> DAV), so there is no point doing this over DAV in practice.\n>> \n>> Perhaps we should remove call to remote_ls() from\n>> fetch_indices() unconditionally, not just protected with\n>> NO_EXPAT and be done with it?\n>\n> That makes a lot of sense. A server really has to always provide\n> a objects/info/packs anyway, just to be fetchable today by clients\n> that are compiled with NO_EXPAT.\n\nAnd even for an isolated group where everybody knows that\neverybody else runs DAV-enabled clients, they need info/refs\nprepared for ls-remote and git-fetch script, which means you\nwill run update-server-info to keep objects/info/packs up to\ndate.\n\nNick, do you see holes in my logic?\n\n-- >8 --\nhttp-fetch.c: drop remote_ls()\n\nWhile doing remote_ls() over DAV potentially allows the server\nside not to keep objects/info/pack up-to-date, misconfigured or\nbuggy servers can silently ignore or not to respond to DAV\nrequests and makes the client hang.\n\nThe server side (unfortunately) needs to run git-update-server-info\neven if remote_ls() removes the need to keep objects/info/pack file\nup-to-date, because the caller of git-http-fetch (git-fetch) and other\nclients that interact with the repository (e.g. git-ls-remote) need to\nread from info/refs file (there is no code to make that unnecessary by\nusing DAV yet).\n\nPerhaps the right solution in the longer-term is to make info/refs\nalso unnecessary by using DAV, and we would want to resurrect the\ncode this patch removes when we do so, but let's drop remote_ls()\nimplementation for now. It is causing problems without really\nhelping anything yet.\n\ngit will keep it for us until we need it next time.\n\nSigned-off-by: Junio C Hamano \n---\n\ndiff --git a/http-fetch.c b/http-fetch.c\nindex 8f251e7..396552d 100644\n--- a/http-fetch.c\n+++ b/http-fetch.c\n@@ -4,35 +4,6 @@ #include \"pack.h\"\n #include \"fetch.h\"\n #include \"http.h\"\n \n-#ifndef NO_EXPAT\n-#include \n-\n-/* Definitions for DAV requests */\n-#define DAV_PROPFIND \"PROPFIND\"\n-#define DAV_PROPFIND_RESP \".multistatus.response\"\n-#define DAV_PROPFIND_NAME \".multistatus.response.href\"\n-#define DAV_PROPFIND_COLLECTION \".multistatus.response.propstat.prop.resourcetype.collection\"\n-#define PROPFIND_ALL_REQUEST \"\\n\\n\\n\"\n-\n-/* Definitions for processing XML DAV responses */\n-#ifndef XML_STATUS_OK\n-enum XML_Status {\n- XML_STATUS_OK = 1,\n- XML_STATUS_ERROR = 0\n-};\n-#define XML_STATUS_OK 1\n-#define XML_STATUS_ERROR 0\n-#endif\n-\n-/* Flags that control remote_ls processing */\n-#define PROCESS_FILES (1u << 0)\n-#define PROCESS_DIRS (1u << 1)\n-#define RECURSIVE (1u << 2)\n-\n-/* Flags that remote_ls passes to callback functions */\n-#define IS_DIR (1u << 0)\n-#endif\n-\n #define PREV_BUF_SIZE 4096\n #define RANGE_HEADER_SIZE 30\n \n@@ -90,30 +61,6 @@ struct alternates_request {\n \tint http_specific;\n };\n \n-#ifndef NO_EXPAT\n-struct xml_ctx\n-{\n-\tchar *name;\n-\tint len;\n-\tchar *cdata;\n-\tvoid (*userFunc)(struct xml_ctx *ctx, int tag_closed);\n-\tvoid *userData;\n-};\n-\n-struct remote_ls_ctx\n-{\n-\tstruct alt_base *repo;\n-\tchar *path;\n-\tvoid (*userFunc)(struct remote_ls_ctx *ls);\n-\tvoid *userData;\n-\tint flags;\n-\tchar *dentry_name;\n-\tint dentry_flags;\n-\tint rc;\n-\tstruct remote_ls_ctx *parent;\n-};\n-#endif\n-\n static struct object_request *object_queue_head;\n \n static size_t fwrite_sha1_file(void *ptr, size_t eltsize, size_t nmemb,\n@@ -714,193 +661,6 @@ #endif\n \tfree(url);\n }\n \n-#ifndef NO_EXPAT\n-static void\n-xml_start_tag(void *userData, const char *name, const char **atts)\n-{\n-\tstruct xml_ctx *ctx = (struct xml_ctx *)userData;\n-\tconst char *c = strchr(name, ':');\n-\tint new_len;\n-\n-\tif (c == NULL)\n-\t\tc = name;\n-\telse\n-\t\tc++;\n-\n-\tnew_len = strlen(ctx->name) + strlen(c) + 2;\n-\n-\tif (new_len > ctx->len) {\n-\t\tctx->name = xrealloc(ctx->name, new_len);\n-\t\tctx->len = new_len;\n-\t}\n-\tstrcat(ctx->name, \".\");\n-\tstrcat(ctx->name, c);\n-\n-\tfree(ctx->cdata);\n-\tctx->cdata = NULL;\n-\n-\tctx->userFunc(ctx, 0);\n-}\n-\n-static void\n-xml_end_tag(void *userData, const char *name)\n-{\n-\tstruct xml_ctx *ctx = (struct xml_ctx *)userData;\n-\tconst char *c = strchr(name, ':');\n-\tchar *ep;\n-\n-\tctx->userFunc(ctx, 1);\n-\n-\tif (c == NULL)\n-\t\tc = name;\n-\telse\n-\t\tc++;\n-\n-\tep = ctx->name + strlen(ctx->name) - strlen(c) - 1;\n-\t*ep = 0;\n-}\n-\n-static void\n-xml_cdata(void *userData, const XML_Char *s, int len)\n-{\n-\tstruct xml_ctx *ctx = (struct xml_ctx *)userData;\n-\tfree(ctx->cdata);\n-\tctx->cdata = xmalloc(len + 1);\n-\tstrlcpy(ctx->cdata, s, len + 1);\n-}\n-\n-static int remote_ls(struct alt_base *repo, const char *path, int flags,\n-\t\t void (*userFunc)(struct remote_ls_ctx *ls),\n-\t\t void *userData);\n-\n-static void handle_remote_ls_ctx(struct xml_ctx *ctx, int tag_closed)\n-{\n-\tstruct remote_ls_ctx *ls = (struct remote_ls_ctx *)ctx->userData;\n-\n-\tif (tag_closed) {\n-\t\tif (!strcmp(ctx->name, DAV_PROPFIND_RESP) && ls->dentry_name) {\n-\t\t\tif (ls->dentry_flags & IS_DIR) {\n-\t\t\t\tif (ls->flags & PROCESS_DIRS) {\n-\t\t\t\t\tls->userFunc(ls);\n-\t\t\t\t}\n-\t\t\t\tif (strcmp(ls->dentry_name, ls->path) &&\n-\t\t\t\t ls->flags & RECURSIVE) {\n-\t\t\t\t\tls->rc = remote_ls(ls->repo,\n-\t\t\t\t\t\t\t ls->dentry_name,\n-\t\t\t\t\t\t\t ls->flags,\n-\t\t\t\t\t\t\t ls->userFunc,\n-\t\t\t\t\t\t\t ls->userData);\n-\t\t\t\t}\n-\t\t\t} else if (ls->flags & PROCESS_FILES) {\n-\t\t\t\tls->userFunc(ls);\n-\t\t\t}\n-\t\t} else if (!strcmp(ctx->name, DAV_PROPFIND_NAME) && ctx->cdata) {\n-\t\t\tls->dentry_name = xmalloc(strlen(ctx->cdata) -\n-\t\t\t\t\t\t ls->repo->path_len + 1);\n-\t\t\tstrcpy(ls->dentry_name, ctx->cdata + ls->repo->path_len);\n-\t\t} else if (!strcmp(ctx->name, DAV_PROPFIND_COLLECTION)) {\n-\t\t\tls->dentry_flags |= IS_DIR;\n-\t\t}\n-\t} else if (!strcmp(ctx->name, DAV_PROPFIND_RESP)) {\n-\t\tfree(ls->dentry_name);\n-\t\tls->dentry_name = NULL;\n-\t\tls->dentry_flags = 0;\n-\t}\n-}\n-\n-static int remote_ls(struct alt_base *repo, const char *path, int flags,\n-\t\t void (*userFunc)(struct remote_ls_ctx *ls),\n-\t\t void *userData)\n-{\n-\tchar *url = xmalloc(strlen(repo->base) + strlen(path) + 1);\n-\tstruct active_request_slot *slot;\n-\tstruct slot_results results;\n-\tstruct buffer in_buffer;\n-\tstruct buffer out_buffer;\n-\tchar *in_data;\n-\tchar *out_data;\n-\tXML_Parser parser = XML_ParserCreate(NULL);\n-\tenum XML_Status result;\n-\tstruct curl_slist *dav_headers = NULL;\n-\tstruct xml_ctx ctx;\n-\tstruct remote_ls_ctx ls;\n-\n-\tls.flags = flags;\n-\tls.repo = repo;\n-\tls.path = xstrdup(path);\n-\tls.dentry_name = NULL;\n-\tls.dentry_flags = 0;\n-\tls.userData = userData;\n-\tls.userFunc = userFunc;\n-\tls.rc = 0;\n-\n-\tsprintf(url, \"%s%s\", repo->base, path);\n-\n-\tout_buffer.size = strlen(PROPFIND_ALL_REQUEST);\n-\tout_data = xmalloc(out_buffer.size + 1);\n-\tsnprintf(out_data, out_buffer.size + 1, PROPFIND_ALL_REQUEST);\n-\tout_buffer.posn = 0;\n-\tout_buffer.buffer = out_data;\n-\n-\tin_buffer.size = 4096;\n-\tin_data = xmalloc(in_buffer.size);\n-\tin_buffer.posn = 0;\n-\tin_buffer.buffer = in_data;\n-\n-\tdav_headers = curl_slist_append(dav_headers, \"Depth: 1\");\n-\tdav_headers = curl_slist_append(dav_headers, \"Content-Type: text/xml\");\n-\n-\tslot = get_active_slot();\n-\tslot->results = &results;\n-\tcurl_easy_setopt(slot->curl, CURLOPT_INFILE, &out_buffer);\n-\tcurl_easy_setopt(slot->curl, CURLOPT_INFILESIZE, out_buffer.size);\n-\tcurl_easy_setopt(slot->curl, CURLOPT_READFUNCTION, fread_buffer);\n-\tcurl_easy_setopt(slot->curl, CURLOPT_FILE, &in_buffer);\n-\tcurl_easy_setopt(slot->curl, CURLOPT_WRITEFUNCTION, fwrite_buffer);\n-\tcurl_easy_setopt(slot->curl, CURLOPT_URL, url);\n-\tcurl_easy_setopt(slot->curl, CURLOPT_UPLOAD, 1);\n-\tcurl_easy_setopt(slot->curl, CURLOPT_CUSTOMREQUEST, DAV_PROPFIND);\n-\tcurl_easy_setopt(slot->curl, CURLOPT_HTTPHEADER, dav_headers);\n-\n-\tif (start_active_slot(slot)) {\n-\t\trun_active_slot(slot);\n-\t\tif (results.curl_result == CURLE_OK) {\n-\t\t\tctx.name = xcalloc(10, 1);\n-\t\t\tctx.len = 0;\n-\t\t\tctx.cdata = NULL;\n-\t\t\tctx.userFunc = handle_remote_ls_ctx;\n-\t\t\tctx.userData = &ls;\n-\t\t\tXML_SetUserData(parser, &ctx);\n-\t\t\tXML_SetElementHandler(parser, xml_start_tag,\n-\t\t\t\t\t xml_end_tag);\n-\t\t\tXML_SetCharacterDataHandler(parser, xml_cdata);\n-\t\t\tresult = XML_Parse(parser, in_buffer.buffer,\n-\t\t\t\t\t in_buffer.posn, 1);\n-\t\t\tfree(ctx.name);\n-\n-\t\t\tif (result != XML_STATUS_OK) {\n-\t\t\t\tls.rc = error(\"XML error: %s\",\n-\t\t\t\t\t XML_ErrorString(\n-\t\t\t\t\t\t XML_GetErrorCode(parser)));\n-\t\t\t}\n-\t\t} else {\n-\t\t\tls.rc = -1;\n-\t\t}\n-\t} else {\n-\t\tls.rc = error(\"Unable to start PROPFIND request\");\n-\t}\n-\n-\tfree(ls.path);\n-\tfree(url);\n-\tfree(out_data);\n-\tfree(in_buffer.buffer);\n-\tcurl_slist_free_all(dav_headers);\n-\n-\treturn ls.rc;\n-}\n-\n-#endif\n-\n static int fetch_indices(struct alt_base *repo)\n {\n \tunsigned char sha1[20];\n"},{"id":"28595","messageId":"loom.20061011T140959-200@post.gmane.org","threadId":"5844","inReplyTo":"7vfydyinto.fsf@assigned-by-dhcp.cox.net","subject":"Re: [RFC PATCH] Add WEBDAV timeout to http-fetch.","fromName":"Panagiotis Issaris","fromEmail":"takis.issaris@uhasselt.be","sentAt":"2006-10-11T12:13:04Z","receivedAt":"2006-10-11T12:13:04Z","isPatch":true,"sender":{"key":"takis.issaris@uhasselt.be","avatar":null},"body":"Hi,\n\nJunio C Hamano cox.net> writes:\n>[...]\n> And even for an isolated group where everybody knows that\n> everybody else runs DAV-enabled clients, they need info/refs\n> prepared for ls-remote and git-fetch script, which means you\n> will run update-server-info to keep objects/info/packs up to\n> date.\nThis patch worked excellent for me. Thanks! :) Any chance this might make it\ninto your tree? I recommended GIT to the FFmpeg developers and would like to be\nable to recommend them to use GIT without any extra patches.\n\n> Nick, do you see holes in my logic?\n\nWith friendly regards,\nTakis \n"},{"id":"28599","messageId":"Pine.LNX.4.63.0610111559060.14200@wbgn013.biozentrum.uni-wuerzburg.de","threadId":"5844","inReplyTo":"loom.20061011T140959-200@post.gmane.org","subject":"Re: [RFC PATCH] Add WEBDAV timeout to http-fetch.","fromName":"Johannes Schindelin","fromEmail":"johannes.schindelin@gmx.de","sentAt":"2006-10-11T14:00:28Z","receivedAt":"2006-10-11T14:00:28Z","isPatch":true,"sender":{"key":"johannes.schindelin@gmx.de","avatar":"https://avatars.githubusercontent.com/u/127790?v=4"},"body":"Hi,\n\nOn Wed, 11 Oct 2006, Panagiotis Issaris wrote:\n\n> Hi,\n> \n> Junio C Hamano cox.net> writes:\n> >[...]\n> > And even for an isolated group where everybody knows that\n> > everybody else runs DAV-enabled clients, they need info/refs\n> > prepared for ls-remote and git-fetch script, which means you\n> > will run update-server-info to keep objects/info/packs up to\n> > date.\n> This patch worked excellent for me. Thanks! :) Any chance this might \n> make it into your tree?\n\nIt already is: adc446fe5d5f6dc1fb5edeaa9aa016ef94e70da1 from Sun Oct 8 \n12:56:19 2006 -0700 in branch 'next'.\n\nHth,\nDscho\n"}]}