{"thread":{"id":"33691","subject":"Re: [PATCH 4/5] git-svn: fix bottleneck in stash_placeholder_list()","startedAt":"2013-05-01T05:09:26Z","lastAt":"2013-05-28T12:57:34Z","messageCount":18,"participants":["Ilya Basin","Junio C Hamano","Eric Wong","Ray Chen"],"isPatch":true,"patchVersion":1,"patchTotal":5},"messages":[{"id":"216122","messageId":"1438528085.20130501090926@gmail.com","threadId":"33691","inReplyTo":null,"subject":"Re: [PATCH 4/5] git-svn: fix bottleneck in stash_placeholder_list()","fromName":"Ilya Basin","fromEmail":"basinilya@gmail.com","sentAt":"2013-05-01T05:09:26Z","receivedAt":"2013-05-01T05:09:26Z","isPatch":true,"sender":{"key":"basinilya@gmail.com","avatar":null},"body":"IB> + return undef if (!keys $self->{_save_ph});\nCorrect is: return undef if (!keys %{$self->{_save_ph}});\n\nIn my repo the placeholders change too often (in 1/4 commits). I'm\nthinking of using:\n'git config --unset \"svn-remote.$repo_id.added-placeholder\" path_regex'\ninstead of full rewrite.\n"},{"id":"216128","messageId":"1409591910.20130501123153@gmail.com","threadId":"33691","inReplyTo":"1438528085.20130501090926@gmail.com","subject":"Re[2]: [PATCH 4/5] git-svn: fix bottleneck in stash_placeholder_list()","fromName":"Ilya Basin","fromEmail":"basinilya@gmail.com","sentAt":"2013-05-01T08:31:53Z","receivedAt":"2013-05-01T08:31:53Z","isPatch":true,"sender":{"key":"basinilya@gmail.com","avatar":null},"body":"IB> In my repo the placeholders change too often (in 1/4 commits). I'm\nIB> thinking of using:\nIB> 'git config --unset \"svn-remote.$repo_id.added-placeholder\" path_regex'\nIB> instead of full rewrite.\n\nI need your help. There are still problems:\n\n $ grep \"define MAX_MATCHES\" ~/builds/git/git-git/config.c\n #define MAX_MATCHES 8192\n\n $ grep added-placeholder .git/config | wc -l\n 4430\n\n1/4 commits change the list of placeholders, usually 1 folder changes.\nClearing and re-adding the entries to the config takes ~1 minute.\nPressing Ctrl-C at this time makes the list incomplete.\n\nRe-adding all entries using 'config --add' is slow.\nDoes Git::config package have tools to modify multiple entries at once?\nI wonder why 'git config --get-all' is used instead of some\nGit::config routine.\n\nOtherwise, to make this atomic, I think, the modification should be made\nto a backup config file, then it should replace .git/config (or\nrewrite it with signals blocked).\n\nHow to determine GIT_DIR from Fetcher.pm?\n\nmaybe I can simply append a duplicate section\n'[svn-remote \"svn\"]'. But then I would need to escape the values\nmyself.\n\nAlso, git --unset-all leaves one empty section: '[svn-remote \"svn\"]'\nIs it a bug?\n\n-- \n"},{"id":"216154","messageId":"7vhaim8w48.fsf@alter.siamese.dyndns.org","threadId":"33691","inReplyTo":"1409591910.20130501123153@gmail.com","subject":"Re: [PATCH 4/5] git-svn: fix bottleneck in stash_placeholder_list()","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2013-05-01T17:09:59Z","receivedAt":"2013-05-01T17:09:59Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Ilya Basin writes:\n\n> IB> In my repo the placeholders change too often (in 1/4 commits). I'm\n> IB> thinking of using:\n> IB> 'git config --unset \"svn-remote.$repo_id.added-placeholder\" path_regex'\n> IB> instead of full rewrite.\n>\n> I need your help. There are still problems:\n>\n> $ grep \"define MAX_MATCHES\" ~/builds/git/git-git/config.c\n> #define MAX_MATCHES 8192\n>\n> $ grep added-placeholder .git/config | wc -l\n> 4430\n>\n> 1/4 commits change the list of placeholders, usually 1 folder changes.\n> Clearing and re-adding the entries to the config takes ~1 minute.\n\nWhile I agree both \"git config\"'s external interface and internal\nimplementation are not suited for bulk update, I have a suspicion\nthat the config mechanism is not the right place to store this\ninformation in the first place. The config is a per-Git-repository\nstate that is not versioned, which means it is applicable regardless\nof individual commits or trees (also it means it is designed not to\nbe shared across repositories). But \"You may see a file here that\notherwise should not be there only to mark that there should be an\nempty directory\" is an attribute to a particular tree, isn't it?\n\nIf you have a branch that git-svn adds a placeholder file (hence you\nwant to annotate that tree with \"This directory is there only to\nhold the placeholder file\") and you want to perform a merge on the\nGit side of that branch with another Git branch that does have real\ncontents in that directory, you would want the result to say \"This\ndirectory no longer is just for a placeholder\", but you cannot say\nthat globally by updating the config file, as the config mechanism\nis also applied to the original branch that came from git-svn, in\nwhich the directory in question is still only to hold the placeholder\nfile.\n\nA Subversion-only history does not have a reason to have .gitignore\nfile tracked in it; wouldn't a cleaner implementation to consider a\ndirectory that has .gitignore and nothing else marked with \"added\nplaceholder\", without (ab)using the config mechanism? If you are\nworried about a corner case where the Subversion side adds the file,\neven though it is not used there, probably you can add a single\ncomment line \"# added by git-svn only to keep the directory\" and\nconsider a directory that has nothing but .gitignore that consists\nof only that exact comment line an \"added placeholder\" directory to\nwork it around. Either approach would tie the information to the\ntree state, which sounds like a much more correct approach to the\n\"keep empty directory\" problem to me.\n"},{"id":"216186","messageId":"455264907.20130501235104@gmail.com","threadId":"33691","inReplyTo":"7vhaim8w48.fsf@alter.siamese.dyndns.org","subject":"Re[2]: [PATCH 4/5] git-svn: fix bottleneck in stash_placeholder_list()","fromName":"Ilya Basin","fromEmail":"basinilya@gmail.com","sentAt":"2013-05-01T19:51:04Z","receivedAt":"2013-05-01T19:51:04Z","isPatch":true,"sender":{"key":"basinilya@gmail.com","avatar":null},"body":"JCH> ...and you want to perform a merge on the\nJCH> Git side of that branch with another Git branch that does have real\nJCH> contents in that directory, you would want the result to say \"This\nJCH> directory no longer is just for a placeholder\", but you cannot say\nJCH> that globally by updating the config file\nPlaceholder files are managed when fetching from SVN. SVN doesn't\nsupport Git-like merges.\n\nJCH> comment line \"# added by git-svn only to keep the directory\" and\nJCH> consider a directory that has nothing but .gitignore that consists\nJCH> of only that exact comment line an \"added placeholder\" directory to\nJCH> work it around.\nSounds good, but it's not I who decided to use the config file.\n\n\n-- \n"},{"id":"216199","messageId":"20130501213031.GA13056@dcvr.yhbt.net","threadId":"33691","inReplyTo":"455264907.20130501235104@gmail.com","subject":"Re: [PATCH 4/5] git-svn: fix bottleneck in stash_placeholder_list()","fromName":"Eric Wong","fromEmail":"normalperson@yhbt.net","sentAt":"2013-05-01T21:30:31Z","receivedAt":"2013-05-01T21:30:31Z","isPatch":true,"sender":{"key":"e@80x24.org","avatar":null},"body":"Ilya Basin wrote:\n> JCH> comment line \"# added by git-svn only to keep the directory\" and\n> JCH> consider a directory that has nothing but .gitignore that consists\n> JCH> of only that exact comment line an \"added placeholder\" directory to\n> JCH> work it around.\n> Sounds good, but it's not I who decided to use the config file.\n\nUgh, I didn't review Ray's original commit closely enough to notice\nthis :x\n\nPerhaps we should migrate users to use YAML storage for this, instead\n(we already use YAML for Git::SVN::Memoize::YAML).\n\n\nFwiw, I've never been a fan of placeholders only accepted it since it's\noff-by-default but it worked well enough for Ray.\n\nMy personal philosophy has always been: git svn users should leave\nno trace or indication they're using a non-standard SVN client.\n"},{"id":"216202","messageId":"7v1u9q5pu5.fsf@alter.siamese.dyndns.org","threadId":"33691","inReplyTo":"20130501213031.GA13056@dcvr.yhbt.net","subject":"Re: [PATCH 4/5] git-svn: fix bottleneck in stash_placeholder_list()","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2013-05-01T21:53:54Z","receivedAt":"2013-05-01T21:53:54Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Eric Wong writes:\n\n> Ilya Basin wrote:\n>> JCH> comment line \"# added by git-svn only to keep the directory\" and\n>> JCH> consider a directory that has nothing but .gitignore that consists\n>> JCH> of only that exact comment line an \"added placeholder\" directory to\n>> JCH> work it around.\n>> Sounds good, but it's not I who decided to use the config file.\n>\n> Ugh, I didn't review Ray's original commit closely enough to notice\n> this :x\n>\n> Perhaps we should migrate users to use YAML storage for this, instead\n> (we already use YAML for Git::SVN::Memoize::YAML).\n\nBut does it solve the impedance mismatch between \"per tree\"\ninformation and \"per project\" information? Unless you key the\ninformation not just with path but also with revision or tree object\nname, use of YAML vs config would not make a difference in the\nsemantics, I am afraid.\n\nI am reading the placeholder-added flag as: \"This .gitignore file\ndoes not exist in the Subversion original; it is there only so that\nwe can keep the otherwise empty diretory in the checkout, and it\nshould not be pushed back to the Subversion side\". Am I mistaken?\n\nThat however is not a property of the directory containing it (or\nthe path to that .gitignore file) that is valid throughout the\nhistory of the project. It is a property of a specific tree object\n(or you could say it is a property of the revision). When at some\npoint in the history the upstream project adds .gitignore there\nbecause many people use git-svn to contribute to their project, it\nstops to be \"should not be pushed back\".\n\nSo it seems to me that the information this \"placeholder added\"\nthing wants to express belongs to the tree object (and .gitignore\nfile itself is a natural place to have that information).\n\n> Fwiw, I've never been a fan of placeholders only accepted it since it's\n> off-by-default but it worked well enough for Ray.\n>\n> My personal philosophy has always been: git svn users should leave\n> no trace or indication they're using a non-standard SVN client.\n"},{"id":"216229","messageId":"20130502024926.GA12172@dcvr.yhbt.net","threadId":"33691","inReplyTo":"7v1u9q5pu5.fsf@alter.siamese.dyndns.org","subject":"Re: [PATCH 4/5] git-svn: fix bottleneck in stash_placeholder_list()","fromName":"Eric Wong","fromEmail":"normalperson@yhbt.net","sentAt":"2013-05-02T02:49:26Z","receivedAt":"2013-05-02T02:49:26Z","isPatch":true,"sender":{"key":"e@80x24.org","avatar":null},"body":"Junio C Hamano wrote:\n> Eric Wong writes:\n> > Ilya Basin wrote:\n> >> JCH> comment line \"# added by git-svn only to keep the directory\" and\n> >> JCH> consider a directory that has nothing but .gitignore that consists\n> >> JCH> of only that exact comment line an \"added placeholder\" directory to\n> >> JCH> work it around.\n> >> Sounds good, but it's not I who decided to use the config file.\n> >\n> > Ugh, I didn't review Ray's original commit closely enough to notice\n> > this :x\n> >\n> > Perhaps we should migrate users to use YAML storage for this, instead\n> > (we already use YAML for Git::SVN::Memoize::YAML).\n> \n> But does it solve the impedance mismatch between \"per tree\"\n> information and \"per project\" information? Unless you key the\n> information not just with path but also with revision or tree object\n> name, use of YAML vs config would not make a difference in the\n> semantics, I am afraid.\n\nNo it doesn't solve the impedance mismatch, but the YAML project would\nbe more flexible than the git config file.\n\n> I am reading the placeholder-added flag as: \"This .gitignore file\n> does not exist in the Subversion original; it is there only so that\n> we can keep the otherwise empty diretory in the checkout, and it\n> should not be pushed back to the Subversion side\". Am I mistaken?\n\nYou're right, I had forgotten this feature completely :x\n\n> That however is not a property of the directory containing it (or\n> the path to that .gitignore file) that is valid throughout the\n> history of the project. It is a property of a specific tree object\n> (or you could say it is a property of the revision). When at some\n> point in the history the upstream project adds .gitignore there\n> because many people use git-svn to contribute to their project, it\n> stops to be \"should not be pushed back\".\n> \n> So it seems to me that the information this \"placeholder added\"\n> thing wants to express belongs to the tree object (and .gitignore\n> file itself is a natural place to have that information).\n\nPerhaps that was the better way to go...\n\nHow would (the presumably few) existing users of this feature be\naffected?\n\nCurrently with the config file, there are problems with interop between\ngit-svn users that do git <-> git repo sharing, an updated version with\nthe \"placeholder added\" .gitignore would allow git <-> git repo sharing,\nbut only between users of newer git versions. Perhaps that's fine and\nbetter than the current situation.\n"},{"id":"216230","messageId":"909519096.20130502075117@gmail.com","threadId":"33691","inReplyTo":"20130501213031.GA13056@dcvr.yhbt.net","subject":"Re[2]: [PATCH 4/5] git-svn: fix bottleneck in stash_placeholder_list()","fromName":"Ilya Basin","fromEmail":"basinilya@gmail.com","sentAt":"2013-05-02T03:51:17Z","receivedAt":"2013-05-02T03:51:17Z","isPatch":true,"sender":{"key":"basinilya@gmail.com","avatar":null},"body":"EW> My personal philosophy has always been: git svn users should leave\nEW> no trace or indication they're using a non-standard SVN client.\nPlaceholders aren't pushed back to svn.\n\n-- \n"},{"id":"216289","messageId":"12810110770.20130502213124@gmail.com","threadId":"33691","inReplyTo":"20130502024926.GA12172@dcvr.yhbt.net","subject":"Re[2]: [PATCH 4/5] git-svn: fix bottleneck in stash_placeholder_list()","fromName":"Ilya Basin","fromEmail":"basinilya@gmail.com","sentAt":"2013-05-02T17:31:24Z","receivedAt":"2013-05-02T17:31:24Z","isPatch":true,"sender":{"key":"basinilya@gmail.com","avatar":null},"body":"Hi. I won't send you updated patches until I import and test my huge\nrepo. Everything will be here:\nhttps://github.com/basinilya/git/commits/v1.8.2.2-git-svn-fixes\n\nAt the moment I've decided not to implement the Junio's proposal:\n> >> JCH> comment line \"# added by git-svn only to keep the directory\" and\n> >> JCH> consider a directory that has nothing but .gitignore that consists\n> >> JCH> of only that exact comment line an \"added placeholder\" directory to\n> >> JCH> work it around.\n\nBut the config file is not an option too: I have 400 tags, each has\n200 empty folders.\n\nInstead I decided to store the paths in a text file (see\nhttps://github.com/basinilya/git/commit/a961aedd81cb8676a52cfe71ccb6eba0f9e64b90 ).\nI'm not planning to push this change to you.\n\nThe last error I encountered is:\nr7009 = 39805bb078983e34f2fc8d2c8c02d695d00d11c0 (refs/remotes/DMC4_Basic)\nToo many open files: Can't open file '/home/il/builds/sicap/gitsvn/prd_dmc4.svn/db/revs/0/786': Too many open files at /.snapshots/persist/builds/git/git-git/perl/blib/lib/Git/SVN/Ra.pm line 282.\n\nI think It's unrelated to empty dirs.\n"},{"id":"216301","messageId":"CANtO-4OYjKpGSAxjKubvubGoXgdkkiZ9yK1unW1wrcONa3ErsQ@mail.gmail.com","threadId":"33691","inReplyTo":"20130502024926.GA12172@dcvr.yhbt.net","subject":"Re: [PATCH 4/5] git-svn: fix bottleneck in stash_placeholder_list()","fromName":"Ray Chen","fromEmail":"rchen@cs.umd.edu","sentAt":"2013-05-02T18:59:58Z","receivedAt":"2013-05-02T18:59:58Z","isPatch":true,"sender":{"key":"rchen@cs.umd.edu","avatar":"https://avatars.githubusercontent.com/u/1909064?v=4"},"body":"On Wed, May 1, 2013 at 10:49 PM, Eric Wong wrote:\n> Junio C Hamano wrote:\n>\n>> Eric Wong writes:\n>\n>> That however is not a property of the directory containing it (or\n>> the path to that .gitignore file) that is valid throughout the\n>> history of the project. It is a property of a specific tree object\n>> (or you could say it is a property of the revision). When at some\n>> point in the history the upstream project adds .gitignore there\n>> because many people use git-svn to contribute to their project, it\n>> stops to be \"should not be pushed back\".\n>>\n>> So it seems to me that the information this \"placeholder added\"\n>> thing wants to express belongs to the tree object (and .gitignore\n>> file itself is a natural place to have that information).\n>\n> Perhaps that was the better way to go...\n>\n> How would (the presumably few) existing users of this feature be\n> affected?\n>\n> Currently with the config file, there are problems with interop between\n> git-svn users that do git <-> git repo sharing, an updated version with\n> the \"placeholder added\" .gitignore would allow git <-> git repo sharing,\n> but only between users of newer git versions. Perhaps that's fine and\n> better than the current situation.\n\nThe original patch was geared towards increasing the fidelity of a\none-time svn->git migration (ie. where svn won't be used anymore). I\nrecall investigating a method to enforce this by disallowing future\ngit-svn fetches, but I can't remember if I was successful. Given this\nperspective, I'm not sure that existing users need to be supported.\n\nThen, as Junio mentions, future versions of git that store placeholder\ninfo in the tree/file object could open the possibility of proper\ngit<->git sharing and resync with the original svn repo.\n\n- Ray\n"},{"id":"216313","messageId":"20130502200947.GA26623@dcvr.yhbt.net","threadId":"33691","inReplyTo":"909519096.20130502075117@gmail.com","subject":"Re: [PATCH 4/5] git-svn: fix bottleneck in stash_placeholder_list()","fromName":"Eric Wong","fromEmail":"normalperson@yhbt.net","sentAt":"2013-05-02T20:09:47Z","receivedAt":"2013-05-02T20:09:47Z","isPatch":true,"sender":{"key":"e@80x24.org","avatar":null},"body":"Ilya Basin wrote:\n> EW> My personal philosophy has always been: git svn users should leave\n> EW> no trace or indication they're using a non-standard SVN client.\n> \n> Placeholders aren't pushed back to svn.\n\nRight, I was confused, as I often am :x\n"},{"id":"216314","messageId":"20130502204017.GB26623@dcvr.yhbt.net","threadId":"33691","inReplyTo":"12810110770.20130502213124@gmail.com","subject":"Re: [PATCH 4/5] git-svn: fix bottleneck in stash_placeholder_list()","fromName":"Eric Wong","fromEmail":"normalperson@yhbt.net","sentAt":"2013-05-02T20:40:17Z","receivedAt":"2013-05-02T20:40:17Z","isPatch":true,"sender":{"key":"e@80x24.org","avatar":null},"body":"Ilya Basin wrote:\n> Hi. I won't send you updated patches until I import and test my huge\n> repo. Everything will be here:\n> https://github.com/basinilya/git/commits/v1.8.2.2-git-svn-fixes\n> \n> At the moment I've decided not to implement the Junio's proposal:\n> > >> JCH> comment line \"# added by git-svn only to keep the directory\" and\n> > >> JCH> consider a directory that has nothing but .gitignore that consists\n> > >> JCH> of only that exact comment line an \"added placeholder\" directory to\n> > >> JCH> work it around.\n> \n> But the config file is not an option too: I have 400 tags, each has\n> 200 empty folders.\n> \n> Instead I decided to store the paths in a text file (see\n> https://github.com/basinilya/git/commit/a961aedd81cb8676a52cfe71ccb6eba0f9e64b90 ).\n> I'm not planning to push this change to you.\n> \n> The last error I encountered is:\n> r7009 = 39805bb078983e34f2fc8d2c8c02d695d00d11c0 (refs/remotes/DMC4_Basic)\n> Too many open files: Can't open file '/home/il/builds/sicap/gitsvn/prd_dmc4.svn/db/revs/0/786': Too many open files at /.snapshots/persist/builds/git/git-git/perl/blib/lib/Git/SVN/Ra.pm line 282.\n> \n> I think It's unrelated to empty dirs.\n\nCan you get an lsof on the git-svn process right before this?\nWhat's your open files limit?\n"},{"id":"216332","messageId":"77906182.20130503092642@gmail.com","threadId":"33691","inReplyTo":"20130502204017.GB26623@dcvr.yhbt.net","subject":"Re[2]: [PATCH 4/5] git-svn: fix bottleneck in stash_placeholder_list()","fromName":"Ilya Basin","fromEmail":"basinilya@gmail.com","sentAt":"2013-05-03T05:26:42Z","receivedAt":"2013-05-03T05:26:42Z","isPatch":true,"sender":{"key":"basinilya@gmail.com","avatar":null},"body":"EW> Ilya Basin wrote:\n>> Hi. I won't send you updated patches until I import and test my huge\n>> repo. Everything will be here:\n>> https://github.com/basinilya/git/commits/v1.8.2.2-git-svn-fixes\n>> \n>> At the moment I've decided not to implement the Junio's proposal:\n>> > >> JCH> comment line \"# added by git-svn only to keep the directory\" and\n>> > >> JCH> consider a directory that has nothing but .gitignore that consists\n>> > >> JCH> of only that exact comment line an \"added placeholder\" directory to\n>> > >> JCH> work it around.\n>> \n>> But the config file is not an option too: I have 400 tags, each has\n>> 200 empty folders.\n>> \n>> Instead I decided to store the paths in a text file (see\n>> https://github.com/basinilya/git/commit/a961aedd81cb8676a52cfe71ccb6eba0f9e64b90 ).\n>> I'm not planning to push this change to you.\n>> \n>> The last error I encountered is:\n>> r7009 = 39805bb078983e34f2fc8d2c8c02d695d00d11c0 (refs/remotes/DMC4_Basic)\n>> Too many open files: Can't open file '/home/il/builds/sicap/gitsvn/prd_dmc4.svn/db/revs/0/786': Too many open files at /.snapshots/persist/builds/git/git-git/perl/blib/lib/Git/SVN/Ra.pm line 282.\n>> \n>> I think It's unrelated to empty dirs.\n\nEW> Can you get an lsof on the git-svn process right before this?\n /.snapshots/persist/builds/sicap/gitsvn/aaa/.git/A4O_OTQxWc\n /.snapshots/persist/builds/sicap/gitsvn/aaa/.git/LfpcENJduN\n /.snapshots/persist/builds/sicap/gitsvn/aaa/.git/Dkk7pN4Mpz\n etc.\n\nEW> What's your open files limit?\n1024\n\n-- \n"},{"id":"216340","messageId":"1379019858.20130503104248@gmail.com","threadId":"33691","inReplyTo":"77906182.20130503092642@gmail.com","subject":"Re[3]: [PATCH 4/5] git-svn: fix bottleneck in stash_placeholder_list()","fromName":"Ilya Basin","fromEmail":"basinilya@gmail.com","sentAt":"2013-05-03T06:42:48Z","receivedAt":"2013-05-03T06:42:48Z","isPatch":true,"sender":{"key":"basinilya@gmail.com","avatar":null},"body":"EW>> Ilya Basin wrote:\n>>> Hi. I won't send you updated patches until I import and test my huge\n>>> repo. Everything will be here:\n>>> https://github.com/basinilya/git/commits/v1.8.2.2-git-svn-fixes\n>>> \n>>> At the moment I've decided not to implement the Junio's proposal:\n>>> > >> JCH> comment line \"# added by git-svn only to keep the directory\" and\n>>> > >> JCH> consider a directory that has nothing but .gitignore that consists\n>>> > >> JCH> of only that exact comment line an \"added placeholder\" directory to\n>>> > >> JCH> work it around.\n>>> \n>>> But the config file is not an option too: I have 400 tags, each has\n>>> 200 empty folders.\n>>> \n>>> Instead I decided to store the paths in a text file (see\n>>> https://github.com/basinilya/git/commit/a961aedd81cb8676a52cfe71ccb6eba0f9e64b90 ).\n>>> I'm not planning to push this change to you.\n>>> \n>>> The last error I encountered is:\n>>> r7009 = 39805bb078983e34f2fc8d2c8c02d695d00d11c0 (refs/remotes/DMC4_Basic)\n>>> Too many open files: Can't open file '/home/il/builds/sicap/gitsvn/prd_dmc4.svn/db/revs/0/786': Too many open files at /.snapshots/persist/builds/git/git-git/perl/blib/lib/Git/SVN/Ra.pm line 282.\n>>> \n>>> I think It's unrelated to empty dirs.\n\nEW>> Can you get an lsof on the git-svn process right before this?\nIB> /.snapshots/persist/builds/sicap/gitsvn/aaa/.git/A4O_OTQxWc\nIB> /.snapshots/persist/builds/sicap/gitsvn/aaa/.git/LfpcENJduN\nIB> /.snapshots/persist/builds/sicap/gitsvn/aaa/.git/Dkk7pN4Mpz\nIB> etc.\n\nEW>> What's your open files limit?\nIB> 1024\n\nWhy no call to close() from temp_release() in Git.pm?\n\n\n-- \n"},{"id":"216475","messageId":"916494719.20130506121422@gmail.com","threadId":"33691","inReplyTo":"1379019858.20130503104248@gmail.com","subject":"Re[4]: [PATCH 4/5] git-svn: fix bottleneck in stash_placeholder_list()","fromName":"Ilya Basin","fromEmail":"basinilya@gmail.com","sentAt":"2013-05-06T08:14:22Z","receivedAt":"2013-05-06T08:14:22Z","isPatch":true,"sender":{"key":"basinilya@gmail.com","avatar":null},"body":">>>> The last error I encountered is:\n>>>> r7009 = 39805bb078983e34f2fc8d2c8c02d695d00d11c0 (refs/remotes/DMC4_Basic)\n>>>> Too many open files: Can't open file '/home/il/builds/sicap/gitsvn/prd_dmc4.svn/db/revs/0/786': Too many open files at /.snapshots/persist/builds/git/git-git/perl/blib/lib/Git/SVN/Ra.pm line 282.\n>>>> \n>>>> I think It's unrelated to empty dirs.\n\nEW>>> Can you get an lsof on the git-svn process right before this?\nIB>> /.snapshots/persist/builds/sicap/gitsvn/aaa/.git/A4O_OTQxWc\nIB>> /.snapshots/persist/builds/sicap/gitsvn/aaa/.git/LfpcENJduN\nIB>> /.snapshots/persist/builds/sicap/gitsvn/aaa/.git/Dkk7pN4Mpz\nIB>> etc.\n\nEW>>> What's your open files limit?\nIB>> 1024\n\nIB> Why no call to close() from temp_release() in Git.pm?\n\nFound, fixed. It was related to empty dirs.\n\n\n\n-- \n"},{"id":"216476","messageId":"366899002.20130506125846@gmail.com","threadId":"33691","inReplyTo":"12810110770.20130502213124@gmail.com","subject":"Re[3]: [PATCH 4/5] git-svn: fix bottleneck in stash_placeholder_list()","fromName":"Ilya Basin","fromEmail":"basinilya@gmail.com","sentAt":"2013-05-06T08:58:46Z","receivedAt":"2013-05-06T08:58:46Z","isPatch":true,"sender":{"key":"basinilya@gmail.com","avatar":null},"body":"Hi Eric. I'm out of spare time and I still unable to import my repo.\nThe code of SVN.pm is too complex. Please help me.\nHere's the list of my issues:\n\n* I think git-svn doesn't handle the case, when a tag is deleted.\n I expected it to rename the ref from \"tags/tagname\" to\n \"tags/tagname@rev\", but that doesn't happen.\n If a tag is replaced, there's no way to tell what was the previous\n state of that tag: git-svn just rewrites the ref.\n On the contrary, the temporary refs (with \"@rev\" suffix), used for\n re-import subdir tags are kept after successful reimport, although\n they have no usage.\n\n* As I said already, I have 25k revisions and 200 tags created from\n subdirs in trunk. This increases the import time from 2h to 12h.\n I would bear it, if it had to be done once, but fetching a new\n revision may cause re-import of all 25k revisions too.\n You should implement some mechanism to find the parent branches of\n subdir tags. Maybe the unused refs I mentioned in the previous issue\n are good candidates for that, but I would name them somehow\n different to distinguish with deleted/replaced tags/branches.\n\n* There are mistake commits in the svn history, similar to this:\n ------------------------------------------------------------------------\n r21255 | xxx_xxxxxx_xxxxxxxxx | 2012-03-02 18:46:30 +0300 (Fri, 02 Mar 2012) | 1 line\n Changed paths:\n A /tags/dmagentenabler-4.1.31/DMAgent (from /tags:20998)\n \n Delivery 4.1.31\n ------------------------------------------------------------------------\n git-svn tries to creates a tag, containing dirs with other tags.\n Technically, behaves correctly, but it hangs, because of the size of\n the commit.\n To solve it, I had to edit the svn dump file:\n\n Node-path: tags/dmagentenabler-4.1.31/DMAgent\n Node-kind: dir\n Node-action: add\n -Node-copyfrom-rev: 20998\n -Node-copyfrom-path: tags\n +Prop-content-length: 10\n +Content-length: 10\n +\n +PROPS-END\n \n \n Revision-number: 21256\n\n It creates an empty dir, instead of copying. Since the author\n noticed the mistake, he immediately deleted the dir in the next\n revision, so it works.\n"},{"id":"216732","messageId":"20130509010512.GA25945@dcvr.yhbt.net","threadId":"33691","inReplyTo":"366899002.20130506125846@gmail.com","subject":"Re: [PATCH 4/5] git-svn: fix bottleneck in stash_placeholder_list()","fromName":"Eric Wong","fromEmail":"normalperson@yhbt.net","sentAt":"2013-05-09T01:05:12Z","receivedAt":"2013-05-09T01:05:12Z","isPatch":true,"sender":{"key":"e@80x24.org","avatar":null},"body":"Ilya Basin wrote:\n> Hi Eric. I'm out of spare time and I still unable to import my repo.\n> The code of SVN.pm is too complex. Please help me.\n\nSorry, most what I do nowadays for git-svn is ACK/NACK changes.\n\ngit-svn has made itself obsolete for most contributors, myself included;\nso it's hard for us to devote significant amounts of time on it since\nwe no longer see SVN repos in our day-to-day work.\n\nGiven the differences between branching/tagging in SVN and git, I\nsuspect some history may always be too complex/convoluted to\nautomatically import. Perhaps an interactive mode can be introduced\nto follow history...\n\nAnyways, thank you for documenting these issues and suggesting fixes.\nHopefully somebody with sufficient motivation can continue your work\ndown the line.\n"},{"id":"218652","messageId":"1421181699.20130528165734@gmail.com","threadId":"33691","inReplyTo":"366899002.20130506125846@gmail.com","subject":"Re[4]: [PATCH 4/5] git-svn: fix bottleneck in stash_placeholder_list()","fromName":"Ilya Basin","fromEmail":"basinilya@gmail.com","sentAt":"2013-05-28T12:57:34Z","receivedAt":"2013-05-28T12:57:34Z","isPatch":true,"sender":{"key":"basinilya@gmail.com","avatar":null},"body":"IB> * I think git-svn doesn't handle the case, when a tag is deleted.\nIB> I expected it to rename the ref from \"tags/tagname\" to\nIB> \"tags/tagname@rev\", but that doesn't happen.\nIB> If a tag is replaced, there's no way to tell what was the previous\nIB> state of that tag: git-svn just rewrites the ref.\n\nOK, I figured out that git-svn creates a merge commit having one of\nits parents the previous state of the tag and another parent the state\nof the new copy src folder.\n"}]}