{"thread":{"id":"1894","subject":"StGIT cannot import properly from stdin","startedAt":"2005-09-21T07:12:07Z","lastAt":"2005-09-22T15:48:28Z","messageCount":9,"participants":["Pierre Ossman","Catalin Marinas"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"9020","messageId":"433107C7.60901@drzeus.cx","threadId":"1894","inReplyTo":null,"subject":"StGIT cannot import properly from stdin","fromName":"Pierre Ossman","fromEmail":"drzeus-list@drzeus.cx","sentAt":"2005-09-21T07:12:07Z","receivedAt":"2005-09-21T07:12:07Z","isPatch":false,"sender":{"key":"drzeus-list@drzeus.cx","avatar":null},"body":"The import command of stgit does not handle stdin in a very sane way.\nThe logic expects to be able to read the patch several times which isn't\npossible with stdin. So for it to work you have to give it the patch twice.\n\nA simple solution would be to store it to disk and then handle it the\nsame way as if a file had been given on the command line.\n\nRgds\nPierre\n"},{"id":"9022","messageId":"b0943d9e05092101033dd5d366@mail.gmail.com","threadId":"1894","inReplyTo":"433107C7.60901@drzeus.cx","subject":"Re: StGIT cannot import properly from stdin","fromName":"Catalin Marinas","fromEmail":"catalin.marinas@gmail.com","sentAt":"2005-09-21T08:03:34Z","receivedAt":"2005-09-21T08:03:34Z","isPatch":false,"sender":{"key":"catalin.marinas@gmail.com","avatar":null},"body":"On 21/09/05, Pierre Ossman wrote:\n> The import command of stgit does not handle stdin in a very sane way.\n> The logic expects to be able to read the patch several times which isn't\n> possible with stdin. So for it to work you have to give it the patch twice.\n\nIt works for me with the latest snapshot. What might happen in your\ncase is a missing \"---\" line after the patch description. The import\ncommand reads the sys.stdin file descriptor until the first \"---\"\nline. After that, the git.apply() function will read the rest of the\nlines and pass them to git-apply. You can put some prints in the\ngit.apply() function to check where it starts reading from.\n\nPatches without description cannot be read from stdin at the moment.\nMaybe I could add another option to prevent it from reading the\ndescription.\n\n-- \nCatalin\n"},{"id":"9100","messageId":"433290A5.5070008@drzeus.cx","threadId":"1894","inReplyTo":"b0943d9e05092101033dd5d366@mail.gmail.com","subject":"Re: StGIT cannot import properly from stdin","fromName":"Pierre Ossman","fromEmail":"drzeus-list@drzeus.cx","sentAt":"2005-09-22T11:08:21Z","receivedAt":"2005-09-22T11:08:21Z","isPatch":false,"sender":{"key":"drzeus-list@drzeus.cx","avatar":null},"body":"Catalin Marinas wrote:\n\n>\n>It works for me with the latest snapshot. What might happen in your\n>case is a missing \"---\" line after the patch description. The import\n>command reads the sys.stdin file descriptor until the first \"---\"\n>line. After that, the git.apply() function will read the rest of the\n>lines and pass them to git-apply. You can put some prints in the\n>git.apply() function to check where it starts reading from.\n>\n> \n>\n\nHmm... it seems you're kind of right. It doesn't need the patch twice.\nBut it needs EOF more than once. The for-loop in __parse_patch() doesn't\nstart until the EOF is recieved. And the second for-loop in _input()\nrequires a new EOF.\n\n>Patches without description cannot be read from stdin at the moment.\n>Maybe I could add another option to prevent it from reading the\n>description.\n>\n> \n>\n\nIf you would cache the input to a file (or memory) you would remove both\nof these problems.\n\nRgds\nPierre\n"},{"id":"9105","messageId":"tnxhdcd2tfn.fsf@arm.com","threadId":"1894","inReplyTo":"433290A5.5070008@drzeus.cx","subject":"Re: StGIT cannot import properly from stdin","fromName":"Catalin Marinas","fromEmail":"catalin.marinas@gmail.com","sentAt":"2005-09-22T12:11:40Z","receivedAt":"2005-09-22T12:11:40Z","isPatch":false,"sender":{"key":"catalin.marinas@gmail.com","avatar":null},"body":"Pierre Ossman wrote:\n>>It works for me with the latest snapshot. What might happen in your\n>>case is a missing \"---\" line after the patch description. The import\n>>command reads the sys.stdin file descriptor until the first \"---\"\n>>line. After that, the git.apply() function will read the rest of the\n>>lines and pass them to git-apply. You can put some prints in the\n>>git.apply() function to check where it starts reading from.\n>\n> Hmm... it seems you're kind of right. It doesn't need the patch twice.\n> But it needs EOF more than once. The for-loop in __parse_patch() doesn't\n> start until the EOF is recieved. And the second for-loop in _input()\n> requires a new EOF.\n\nThe line below in __parse_patch() ensures that the 'for' loop does not\nconsume all the input lines:\n\n if re.match('---\\s*$', line) or re.match('diff -', line):\n break\n\nUnfortunately, if the '---' line is missing, it consumes all the input\nlines. Is this the case with your patch?\n\n> If you would cache the input to a file (or memory) you would remove both\n> of these problems.\n\nTrue, but it requires more work than simply writing the patch to a\nfile and reading it twice.\n\n-- \nCatalin\n"},{"id":"9111","messageId":"4332BA3E.2080607@drzeus.cx","threadId":"1894","inReplyTo":"tnxhdcd2tfn.fsf@arm.com","subject":"Re: StGIT cannot import properly from stdin","fromName":"Pierre Ossman","fromEmail":"drzeus-list@drzeus.cx","sentAt":"2005-09-22T14:05:50Z","receivedAt":"2005-09-22T14:05:50Z","isPatch":false,"sender":{"key":"drzeus-list@drzeus.cx","avatar":null},"body":"Catalin Marinas wrote:\n\n>\n>Unfortunately, if the '---' line is missing, it consumes all the input\n>lines. Is this the case with your patch?\n>\n> \n>\n\nNope. It consumes the correct ammount of lines. The problem is that I\nhave to give it a kick (^D) to start things twice. The first for\n__parse_patch() and the second for _input(). I put some debug-prints in\nthere and the for-loops will not start executing on the first complete\nline, only EOF.\n\n>>If you would cache the input to a file (or memory) you would remove both\n>>of these problems.\n>> \n>>\n>\n>True, but it requires more work than simply writing the patch to a\n>file and reading it twice.\n>\n> \n>\n\nIf it supports everything when given a filename then shouldn't adding a\npreprocessing step to dump stdin to disk give the same code path?\n\nRgds\nPierre\n"},{"id":"9115","messageId":"tnxr7bh188q.fsf@arm.com","threadId":"1894","inReplyTo":"4332BA3E.2080607@drzeus.cx","subject":"Re: StGIT cannot import properly from stdin","fromName":"Catalin Marinas","fromEmail":"catalin.marinas@gmail.com","sentAt":"2005-09-22T14:34:45Z","receivedAt":"2005-09-22T14:34:45Z","isPatch":false,"sender":{"key":"catalin.marinas@gmail.com","avatar":null},"body":"Pierre Ossman wrote:\n>>Unfortunately, if the '---' line is missing, it consumes all the input\n>>lines. Is this the case with your patch?\n>\n> Nope. It consumes the correct ammount of lines. The problem is that I\n> have to give it a kick (^D) to start things twice. The first for\n> __parse_patch() and the second for _input(). I put some debug-prints in\n> there and the for-loops will not start executing on the first complete\n> line, only EOF.\n\nI tried it with reading from the terminal directly and you are right.\n\nUsing 'stg import -n patch < patch-file' is fine since the stdin is\nnot re-opened and the _input() loop starts automatically.\n\n'cat | stg import -n patch' also works fine since cat exits when stdin\nreceives EOF.\n\nIt looks like a problem with Python since it re-opens sys.stdin once\nit receives EOF.\n\n>>>If you would cache the input to a file (or memory) you would remove both\n>>>of these problems.\n>>\n>>True, but it requires more work than simply writing the patch to a\n>>file and reading it twice.\n>\n> If it supports everything when given a filename then shouldn't adding a\n> preprocessing step to dump stdin to disk give the same code path?\n\nI was referring to the 'cache the input to memory' variant, which is a\nbit more complicated. Caching to a file is simple but do you need to\nread patches from the terminal directly?\n\n-- \nCatalin\n"},{"id":"9118","messageId":"4332C880.1060906@drzeus.cx","threadId":"1894","inReplyTo":"tnxr7bh188q.fsf@arm.com","subject":"Re: StGIT cannot import properly from stdin","fromName":"Pierre Ossman","fromEmail":"drzeus-list@drzeus.cx","sentAt":"2005-09-22T15:06:40Z","receivedAt":"2005-09-22T15:06:40Z","isPatch":false,"sender":{"key":"drzeus-list@drzeus.cx","avatar":null},"body":"Catalin Marinas wrote:\n\n>It looks like a problem with Python since it re-opens sys.stdin once\n>it receives EOF.\n>\n> \n>\n\nBummer. I checked with the local Python guru here and he suggested\nchanging the for loop to an infinite while and f.readline().\n\n>I was referring to the 'cache the input to memory' variant, which is a\n>bit more complicated. Caching to a file is simple but do you need to\n>read patches from the terminal directly?\n>\n> \n>\n\nI don't explicitly need it, but it keeps things simple for me. I can\ncopy and paste the patch from my mailer without having to bother with\nsaving it to a temporary file.\n"},{"id":"9121","messageId":"b0943d9e050922084232f704d@mail.gmail.com","threadId":"1894","inReplyTo":"4332C880.1060906@drzeus.cx","subject":"Re: StGIT cannot import properly from stdin","fromName":"Catalin Marinas","fromEmail":"catalin.marinas@gmail.com","sentAt":"2005-09-22T15:42:28Z","receivedAt":"2005-09-22T15:42:28Z","isPatch":false,"sender":{"key":"catalin.marinas@gmail.com","avatar":null},"body":"On 22/09/05, Pierre Ossman wrote:\n> Bummer. I checked with the local Python guru here and he suggested\n> changing the for loop to an infinite while and f.readline().\n\nIt works, thanks for this. I attached a patch if you want to try.\n\n--\nCatalin\n"},{"id":"9123","messageId":"tnxmzm514tv.fsf@arm.com","threadId":"1894","inReplyTo":"b0943d9e050922084232f704d@mail.gmail.com","subject":"Re: StGIT cannot import properly from stdin","fromName":"Catalin Marinas","fromEmail":"catalin.marinas@gmail.com","sentAt":"2005-09-22T15:48:28Z","receivedAt":"2005-09-22T15:48:28Z","isPatch":false,"sender":{"key":"catalin.marinas@gmail.com","avatar":null},"body":"Catalin Marinas wrote:\n> It works, thanks for this. I attached a patch if you want to try.\n\nOops, gmail attached it as a binary. It is inserted below (as\ntext/plain):\n\n======================================================\nFix importing from stdin\n\nThe current stdin patch importing expects two EOFs since the 'for' loop\ndoesn't start before one EOF is received. As suggested, this patch changes\nthe 'for' loop with a 'while True' loop.\n\nSigned-off-by: Catalin Marinas \n---\n\n stgit/commands/imprt.py | 6 +++++-\n stgit/git.py | 5 ++++-\n 2 files changed, 9 insertions(+), 2 deletions(-)\n\ndiff --git a/stgit/commands/imprt.py b/stgit/commands/imprt.py\n--- a/stgit/commands/imprt.py\n+++ b/stgit/commands/imprt.py\n@@ -134,7 +134,11 @@ def __parse_patch(filename = None):\n authname = authemail = authdate = None\n \n descr = ''\n- for line in f:\n+ while True:\n+ line = f.readline()\n+ if not line:\n+ break\n+\n # the first 'Signed-of-by:' is the author\n if not authname and re.match('signed-off-by:\\s+', line, re.I):\n auth = re.findall('^.*?:\\s+(.*)$', line)[0]\ndiff --git a/stgit/git.py b/stgit/git.py\n--- a/stgit/git.py\n+++ b/stgit/git.py\n@@ -113,7 +113,10 @@ def get_conflicts():\n \n def _input(cmd, file_desc):\n p = popen2.Popen3(cmd)\n- for line in file_desc:\n+ while True:\n+ line = file_desc.readline()\n+ if not line:\n+ break\n p.tochild.write(line)\n p.tochild.close()\n if p.wait():\n"}]}