{"thread":{"id":"9060","subject":"Tips for debugging fast-import front-ends","startedAt":"2007-07-16T12:45:13Z","lastAt":"2007-07-17T03:27:07Z","messageCount":3,"participants":["Julian Phillips","Johannes Sixt","Shawn O. Pearce"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"47541","messageId":"Pine.LNX.4.64.0707161336570.24955@reaper.quantumfyre.co.uk","threadId":"9060","inReplyTo":null,"subject":"Tips for debugging fast-import front-ends","fromName":"Julian Phillips","fromEmail":"julian@quantumfyre.co.uk","sentAt":"2007-07-16T12:45:13Z","receivedAt":"2007-07-16T12:45:13Z","isPatch":false,"sender":{"key":"julian@quantumfyre.co.uk","avatar":"https://avatars.githubusercontent.com/u/948888?v=4"},"body":"I was wondering if anyone had some handy tips for debugging fast-import \nbased importers?\n\nI'm trying to import an SVN repository that was initially created by \ncvs2svn and, since we started using branches quite heavily _before_ \nswitching to SVN, there is some quite messy history.\n\nAt some point in the \"messy\" phase I get an error from fast-import saying:\n\nfatal: path foo not in branch\n\nThis seems to happen when I attempt to copy a file that is also changed in \nthe same commit.  What I would like to do is find out what things look \nlike just before that point - the problem is that fast-import seems to not \ncreate anything on failure.  So, apart restarting the import and stopping \njust before the broken commit, is there any way to find out what \nfast-import thinks the branch looks like?\n\nAlso, does anyone have any handy hints 'n' tips for working with \nfast-import?\n\n-- \nJulian\n\n  ---\nAir pollution is really making us pay through the nose.\n"},{"id":"47547","messageId":"469B736F.12FA24EA@eudaptics.com","threadId":"9060","inReplyTo":"Pine.LNX.4.64.0707161336570.24955@reaper.quantumfyre.co.uk","subject":"Re: Tips for debugging fast-import front-ends","fromName":"Johannes Sixt","fromEmail":"j.sixt@eudaptics.com","sentAt":"2007-07-16T13:32:31Z","receivedAt":"2007-07-16T13:32:31Z","isPatch":false,"sender":{"key":"j6t@kdbg.org","avatar":"https://avatars.githubusercontent.com/u/14810926?v=4"},"body":"Julian Phillips wrote:\n> At some point in the \"messy\" phase I get an error from fast-import saying:\n> \n> fatal: path foo not in branch\n> \n> This seems to happen when I attempt to copy a file that is also changed in\n> the same commit.  What I would like to do is find out what things look\n> like just before that point - the problem is that fast-import seems to not\n> create anything on failure.  So, apart restarting the import and stopping\n> just before the broken commit, is there any way to find out what\n> fast-import thinks the branch looks like?\n\nRedirect the input stream to a file instead of to fast-import, then feed\nfast-import from this file. This way you can reproduce the failure more\nquickly. Also, you can run fast-import from a debugger.\n\n-- Hannes\n"},{"id":"47608","messageId":"20070717032707.GJ32566@spearce.org","threadId":"9060","inReplyTo":"469B736F.12FA24EA@eudaptics.com","subject":"Re: Tips for debugging fast-import front-ends","fromName":"Shawn O. Pearce","fromEmail":"spearce@spearce.org","sentAt":"2007-07-17T03:27:07Z","receivedAt":"2007-07-17T03:27:07Z","isPatch":false,"sender":{"key":"spearce@spearce.org","avatar":"https://avatars.githubusercontent.com/u/34844?v=4"},"body":"Johannes Sixt <J.Sixt@eudaptics.com> wrote:\n> Julian Phillips wrote:\n> > At some point in the \"messy\" phase I get an error from fast-import saying:\n> > \n> > fatal: path foo not in branch\n> > \n> > This seems to happen when I attempt to copy a file that is also changed in\n> > the same commit.  What I would like to do is find out what things look\n> > like just before that point - the problem is that fast-import seems to not\n> > create anything on failure.  So, apart restarting the import and stopping\n> > just before the broken commit, is there any way to find out what\n> > fast-import thinks the branch looks like?\n> \n> Redirect the input stream to a file instead of to fast-import, then feed\n> fast-import from this file. This way you can reproduce the failure more\n> quickly. Also, you can run fast-import from a debugger.\n\nI tend to dump my test stream to a file, then process that through\nfast-import.  Running fast-import in the debugger is uh, ugly, as\nI make heavy use of FLEX_ARRAY whenever there is interesting data.\nSo simple printing of any useful data (like the name of a tree\nentry) is not so simple without doing pointer addition and type\ncasting in gdb.  Sometimes I just break down and add a few targeted\nprintf()'s to the code.  That always tells me what's up.\n\nOne easy thing that you can do is add a 'checkpoint' command\nright before the bad commit (assuming you know roughly where it is\nbreaking).  This will cause fast-import to flush all of its buffers\nback to disk, and update the Git repository, before reading the\nnext command.  The result is that when you crash you have everything\nthat fast-import processed right up to that last checkpoint.\n\nClearly doing too many checkpoints is going to remove the fast part\nof fast-import, but at least you have a -import.  ;-)\n\n\nAfter reading this thread I have decided to add some frontend\ndebugging support into fast-import, like dumping out the most\nrecently received commands and the current branch and tree state\nbefore it die()'s with a protocol error or bad path error, and maybe\nalso try to do a partial checkpoint (close out the packfile, don't\nupdate refs or marks, but instead write them to the state dump file).\nThis way the frontend developer can inspect the overall state and\nbetter understand what might have gone wrong.\n\nUnfortunately I won't get it done in the next few days, but maybe\nbefore the weekend.\n\n-- \nShawn.\n"}]}