{"thread":{"id":"19403","subject":"git archive, cygwin, and --git-dir vs --remote","startedAt":"2009-05-18T14:37:14Z","lastAt":"2009-06-18T17:18:16Z","messageCount":6,"participants":["Bob Kagy","René Scharfe","Tony Finch"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"114180","messageId":"e664dae0905180737mae29811ie4cae889b3e3904f@mail.gmail.com","threadId":"19403","inReplyTo":null,"subject":"git archive, cygwin, and --git-dir vs --remote","fromName":"Bob Kagy","fromEmail":"bobkagy@gmail.com","sentAt":"2009-05-18T14:37:14Z","receivedAt":"2009-05-18T14:37:14Z","isPatch":false,"sender":{"key":"bobkagy@gmail.com","avatar":null},"body":"I'm using git on cygwin, and am confused by behavior from git archive.\n\ngit --version\ngit version 1.6.1.2\n\nThis command works as expected:\ngit --git-dir=/cygdrive/w archive --format=tar --verbose --prefix=tmp/\nHEAD | tar -xpf -\nThe latest copy is retrieved and dumped to the tmp subdirectory.\n\nHowever, I first went down the path of using the --remote option, as\ndescribed in the git-archive man page:\ngit archive --format=tar --verbose --prefix=tmp/ --remote=/cygdrive/w\nHEAD | tar -xpf -\n\nWhen I use the --remote branch it seems to create the same files.  But\nthen rather than exiting it hangs out until I kill the process, using\nabout the same CPU % as it did when creating files.\n\nI'm trying to use this for a scripted build process where all I need\nis the latest copy, not an archive.  And we're just starting with git,\nso while it works fine now I think we're supposed to be switch from\nshared drives to something more controlled, at which point I don't\nknow that the --git-dir option will work.\n\nI've tried searching the mailing list archive, and got overwhelmed by\npatches, so I apologize if this has already been discussed and I just\ncouldn't find it.\n"},{"id":"114398","messageId":"4A151A15.6040609@lsrfire.ath.cx","threadId":"19403","inReplyTo":"e664dae0905180737mae29811ie4cae889b3e3904f@mail.gmail.com","subject":"Re: git archive, cygwin, and --git-dir vs --remote","fromName":"René Scharfe","fromEmail":"rene.scharfe@lsrfire.ath.cx","sentAt":"2009-05-21T09:08:37Z","receivedAt":"2009-05-21T09:08:37Z","isPatch":false,"sender":{"key":"l.s.r@web.de","avatar":"https://avatars.githubusercontent.com/u/26122331?v=4"},"body":"Bob Kagy schrieb:\n> I'm using git on cygwin, and am confused by behavior from git archive.\n> \n> git --version\n> git version 1.6.1.2\n> \n> This command works as expected:\n> git --git-dir=/cygdrive/w archive --format=tar --verbose --prefix=tmp/\n> HEAD | tar -xpf -\n> The latest copy is retrieved and dumped to the tmp subdirectory.\n> \n> However, I first went down the path of using the --remote option, as\n> described in the git-archive man page:\n> git archive --format=tar --verbose --prefix=tmp/ --remote=/cygdrive/w\n> HEAD | tar -xpf -\n> \n> When I use the --remote branch it seems to create the same files.  But\n> then rather than exiting it hangs out until I kill the process, using\n> about the same CPU % as it did when creating files.\n\nThat's strange.  It seems that poll() reports that there is data to read\nfrom the child (which is running git-upload-archive), even though it\nalready called exit().\n\nThe following patch works around this issue by terminating the otherwise\nendless loop after read() returned nothing for the thousandth time in a\nrow.  I'm not sure that there's really no way to get a thousand empty\nreads without the child being done, though.\n\n builtin-upload-archive.c |   14 ++++++++++----\n 1 files changed, 10 insertions(+), 4 deletions(-)\n\ndiff --git a/builtin-upload-archive.c b/builtin-upload-archive.c\nindex 0206b41..3d7b11b 100644\n--- a/builtin-upload-archive.c\n+++ b/builtin-upload-archive.c\n@@ -17,6 +17,7 @@ static const char lostchild[] =\n \"git upload-archive: archiver process was lost\";\n \n #define MAX_ARGS (64)\n+#define MAX_EMPTY_READS (1000)\n \n static int run_upload_archive(int argc, const char **argv, const char *prefix)\n {\n@@ -80,10 +81,11 @@ static void error_clnt(const char *fmt, ...)\n \tdie(\"sent error to the client: %s\", buf);\n }\n \n-static void process_input(int child_fd, int band)\n+static void process_input(int child_fd, int band, int *empty_reads)\n {\n \tchar buf[16384];\n \tssize_t sz = read(child_fd, buf, sizeof(buf));\n+\t*empty_reads = sz ? 0 : *empty_reads + 1;\n \tif (sz < 0) {\n \t\tif (errno != EAGAIN && errno != EINTR)\n \t\t\terror_clnt(\"read error: %s\\n\", strerror(errno));\n@@ -96,6 +98,8 @@ int cmd_upload_archive(int argc, const char **argv, const char *prefix)\n {\n \tpid_t writer;\n \tint fd1[2], fd2[2];\n+\tint empty_reads_1 = 0;\n+\tint empty_reads_2 = 0;\n \t/*\n \t * Set up sideband subprocess.\n \t *\n@@ -147,12 +151,14 @@ int cmd_upload_archive(int argc, const char **argv, const char *prefix)\n \t\t}\n \t\tif (pfd[0].revents & POLLIN)\n \t\t\t/* Data stream ready */\n-\t\t\tprocess_input(pfd[0].fd, 1);\n+\t\t\tprocess_input(pfd[0].fd, 1, &empty_reads_1);\n \t\tif (pfd[1].revents & POLLIN)\n \t\t\t/* Status stream ready */\n-\t\t\tprocess_input(pfd[1].fd, 2);\n+\t\t\tprocess_input(pfd[1].fd, 2, &empty_reads_2);\n \t\t/* Always finish to read data when available */\n-\t\tif ((pfd[0].revents | pfd[1].revents) & POLLIN)\n+\t\tif (((pfd[0].revents | pfd[1].revents) & POLLIN) &&\n+\t\t    empty_reads_1 <= MAX_EMPTY_READS &&\n+\t\t    empty_reads_2 <= MAX_EMPTY_READS)\n \t\t\tcontinue;\n \n \t\tif (waitpid(writer, &status, 0) < 0)\n"},{"id":"114416","messageId":"alpine.LSU.2.00.0905211431060.23478@hermes-2.csi.cam.ac.uk","threadId":"19403","inReplyTo":"4A151A15.6040609@lsrfire.ath.cx","subject":"Re: git archive, cygwin, and --git-dir vs --remote","fromName":"Tony Finch","fromEmail":"dot@dotat.at","sentAt":"2009-05-21T13:32:25Z","receivedAt":"2009-05-21T13:32:25Z","isPatch":false,"sender":{"key":"dot@dotat.at","avatar":"https://avatars.githubusercontent.com/u/68429?v=4"},"body":"On Thu, 21 May 2009, René Scharfe wrote:\n>\n> That's strange.  It seems that poll() reports that there is data to read\n> from the child (which is running git-upload-archive), even though it\n> already called exit().\n\nPoll reports an FD is readable when it reaches EOF.\n\n> The following patch works around this issue by terminating the otherwise\n> endless loop after read() returned nothing for the thousandth time in a\n> row.\n\nYou should stop reading the first time read() returns 0 i.e. EOF.\n\nTony.\n-- \nf.anthony.n.finch  <dot@dotat.at>  http://dotat.at/\nGERMAN BIGHT HUMBER: SOUTHWEST 5 TO 7. MODERATE OR ROUGH. SQUALLY SHOWERS.\nMODERATE OR GOOD."},{"id":"114420","messageId":"4A156556.900@lsrfire.ath.cx","threadId":"19403","inReplyTo":"alpine.LSU.2.00.0905211431060.23478@hermes-2.csi.cam.ac.uk","subject":"Re: git archive, cygwin, and --git-dir vs --remote","fromName":"René Scharfe","fromEmail":"rene.scharfe@lsrfire.ath.cx","sentAt":"2009-05-21T14:29:42Z","receivedAt":"2009-05-21T14:29:42Z","isPatch":false,"sender":{"key":"l.s.r@web.de","avatar":"https://avatars.githubusercontent.com/u/26122331?v=4"},"body":"Tony Finch schrieb:\n> On Thu, 21 May 2009, René Scharfe wrote:\n>> That's strange.  It seems that poll() reports that there is data to read\n>> from the child (which is running git-upload-archive), even though it\n>> already called exit().\n> \n> Poll reports an FD is readable when it reaches EOF.\n\nOK, makes sense.  I still don't understand why upload-archive doesn't get\ninto an infinite loop on Linux (Fedora 10), though.\n\n>> The following patch works around this issue by terminating the otherwise\n>> endless loop after read() returned nothing for the thousandth time in a\n>> row.\n> \n> You should stop reading the first time read() returns 0 i.e. EOF.\n\nThanks.  In that case the following patch is better.\n\n\n builtin-upload-archive.c |   15 +++++++++------\n 1 files changed, 9 insertions(+), 6 deletions(-)\n\ndiff --git a/builtin-upload-archive.c b/builtin-upload-archive.c\nindex 0206b41..a3fa5b3 100644\n--- a/builtin-upload-archive.c\n+++ b/builtin-upload-archive.c\n@@ -80,16 +80,19 @@ static void error_clnt(const char *fmt, ...)\n \tdie(\"sent error to the client: %s\", buf);\n }\n \n-static void process_input(int child_fd, int band)\n+static int process_input(int child_fd, int band)\n {\n \tchar buf[16384];\n \tssize_t sz = read(child_fd, buf, sizeof(buf));\n+\tif (sz == 0)\n+\t\treturn EOF;\n \tif (sz < 0) {\n \t\tif (errno != EAGAIN && errno != EINTR)\n \t\t\terror_clnt(\"read error: %s\\n\", strerror(errno));\n-\t\treturn;\n+\t\treturn 0;\n \t}\n \tsend_sideband(1, band, buf, sz, LARGE_PACKET_MAX);\n+\treturn 0;\n }\n \n int cmd_upload_archive(int argc, const char **argv, const char *prefix)\n@@ -131,7 +134,7 @@ int cmd_upload_archive(int argc, const char **argv, const char *prefix)\n \n \twhile (1) {\n \t\tstruct pollfd pfd[2];\n-\t\tint status;\n+\t\tint status, both_at_eof = EOF;\n \n \t\tpfd[0].fd = fd1[0];\n \t\tpfd[0].events = POLLIN;\n@@ -147,12 +150,12 @@ int cmd_upload_archive(int argc, const char **argv, const char *prefix)\n \t\t}\n \t\tif (pfd[0].revents & POLLIN)\n \t\t\t/* Data stream ready */\n-\t\t\tprocess_input(pfd[0].fd, 1);\n+\t\t\tboth_at_eof &= process_input(pfd[0].fd, 1);\n \t\tif (pfd[1].revents & POLLIN)\n \t\t\t/* Status stream ready */\n-\t\t\tprocess_input(pfd[1].fd, 2);\n+\t\t\tboth_at_eof &= process_input(pfd[1].fd, 2);\n \t\t/* Always finish to read data when available */\n-\t\tif ((pfd[0].revents | pfd[1].revents) & POLLIN)\n+\t\tif (!both_at_eof)\n \t\t\tcontinue;\n \n \t\tif (waitpid(writer, &status, 0) < 0)\n"},{"id":"116461","messageId":"4A38C13E.6050800@lsrfire.ath.cx","threadId":"19403","inReplyTo":"4A156556.900@lsrfire.ath.cx","subject":"[PATCH] upload-archive: fix infinite loop on Cygwin","fromName":"René Scharfe","fromEmail":"rene.scharfe@lsrfire.ath.cx","sentAt":"2009-06-17T10:11:10Z","receivedAt":"2009-06-17T10:11:10Z","isPatch":true,"sender":{"key":"l.s.r@web.de","avatar":"https://avatars.githubusercontent.com/u/26122331?v=4"},"body":"On Cygwin, poll() reports POLLIN even for file descriptors that have\nreached their end.  This caused git upload-archive to be stuck in an\ninfinite loop, as it only looked at the POLLIN flag.\n\nIn addition to POLLIN, check if read() returned 0, which indicates\nend-of-file, and keep looping only as long as at least one of the file\ndescriptors has input.  This lets the following command finish on its\nown when run in a git repository on Cygwin, instead of it getting stuck\nafter printing all file names:\n\n\t$ git archive -v --remote . HEAD >/dev/null\n\nReported-by: Bob Kagy <bobkagy@gmail.com>\nSigned-off-by: Rene Scharfe <rene.scharfe@lsrfire.ath.cx>\n---\nThis version of the patch has been cleaned up a bit compared to the\nprevious one and is slightly shorter, but does the same.\n\n builtin-upload-archive.c |   12 +++++++-----\n 1 files changed, 7 insertions(+), 5 deletions(-)\n\ndiff --git a/builtin-upload-archive.c b/builtin-upload-archive.c\nindex 0206b41..c4cd1e1 100644\n--- a/builtin-upload-archive.c\n+++ b/builtin-upload-archive.c\n@@ -80,16 +80,17 @@ static void error_clnt(const char *fmt, ...)\n \tdie(\"sent error to the client: %s\", buf);\n }\n \n-static void process_input(int child_fd, int band)\n+static ssize_t process_input(int child_fd, int band)\n {\n \tchar buf[16384];\n \tssize_t sz = read(child_fd, buf, sizeof(buf));\n \tif (sz < 0) {\n \t\tif (errno != EAGAIN && errno != EINTR)\n \t\t\terror_clnt(\"read error: %s\\n\", strerror(errno));\n-\t\treturn;\n+\t\treturn sz;\n \t}\n \tsend_sideband(1, band, buf, sz, LARGE_PACKET_MAX);\n+\treturn sz;\n }\n \n int cmd_upload_archive(int argc, const char **argv, const char *prefix)\n@@ -131,6 +132,7 @@ int cmd_upload_archive(int argc, const char **argv, const char *prefix)\n \n \twhile (1) {\n \t\tstruct pollfd pfd[2];\n+\t\tssize_t processed[2] = { 0, 0 };\n \t\tint status;\n \n \t\tpfd[0].fd = fd1[0];\n@@ -147,12 +149,12 @@ int cmd_upload_archive(int argc, const char **argv, const char *prefix)\n \t\t}\n \t\tif (pfd[0].revents & POLLIN)\n \t\t\t/* Data stream ready */\n-\t\t\tprocess_input(pfd[0].fd, 1);\n+\t\t\tprocessed[0] = process_input(pfd[0].fd, 1);\n \t\tif (pfd[1].revents & POLLIN)\n \t\t\t/* Status stream ready */\n-\t\t\tprocess_input(pfd[1].fd, 2);\n+\t\t\tprocessed[1] = process_input(pfd[1].fd, 2);\n \t\t/* Always finish to read data when available */\n-\t\tif ((pfd[0].revents | pfd[1].revents) & POLLIN)\n+\t\tif (processed[0] || processed[1])\n \t\t\tcontinue;\n \n \t\tif (waitpid(writer, &status, 0) < 0)\n-- \n1.6.3.2.307.g21cd0\n"},{"id":"116587","messageId":"alpine.LSU.2.00.0906181817400.2474@hermes-2.csi.cam.ac.uk","threadId":"19403","inReplyTo":"4A38C13E.6050800@lsrfire.ath.cx","subject":"Re: [PATCH] upload-archive: fix infinite loop on Cygwin","fromName":"Tony Finch","fromEmail":"dot@dotat.at","sentAt":"2009-06-18T17:18:16Z","receivedAt":"2009-06-18T17:18:16Z","isPatch":true,"sender":{"key":"dot@dotat.at","avatar":"https://avatars.githubusercontent.com/u/68429?v=4"},"body":"On Wed, 17 Jun 2009, René Scharfe wrote:\n\n> On Cygwin, poll() reports POLLIN even for file descriptors that have\n> reached their end.  This caused git upload-archive to be stuck in an\n> infinite loop, as it only looked at the POLLIN flag.\n\nI think it isn't just Cygwin that does this - see\nhttp://www.greenend.org.uk/rjk/2001/06/poll.html\n\nTony.\n-- \nf.anthony.n.finch  <dot@dotat.at>  http://dotat.at/\nGERMAN BIGHT HUMBER: SOUTHWEST 5 TO 7. MODERATE OR ROUGH. SQUALLY SHOWERS.\nMODERATE OR GOOD."}]}