# git svn forking an awful lot of "git config"s 5 messages from 2008-02-20 to 2008-02-21. Participants: Johannes Schindelin, Eric Wong, Jakub Narebski, Nagy Balázs. Thread: https://gitlist.dev/t/12216 ## Johannes Schindelin, 2008-02-20 15:27 Subject: git svn forking an awful lot of "git config"s Message-ID: URL: https://gitlist.dev/e/alpine.LSU.1.00.0802201520580.17164%40racer.site ``` Hi, we are approaching our first release of msysGit's installer with git-svn. However, I am experiencing a very bad performance, and an error: $ git svn fetch trace: exec: 'git-svn' 'fetch' trace: built-in: git 'config' '--bool' '--get' 'svn.fetchall' trace: built-in: git 'config' '--get' 'svn.username' trace: built-in: git 'config' '--get' 'svn.repackflags' trace: built-in: git 'config' '--bool' '--get' 'svn.quiet' trace: built-in: git 'config' '--bool' '--get' 'svn.noauthcache' trace: built-in: git 'config' '--get' 'svn.revision' trace: built-in: git 'config' '--int' '--get' 'svn.repack' trace: built-in: git 'config' '--int' '--get' 'svn.logwindowsize' trace: built-in: git 'config' '--bool' '--get' 'svn.nocheckout' trace: built-in: git 'config' '--get' 'svn.configdir' trace: built-in: git 'config' '--bool' '--get' 'svn.noMetadata' trace: built-in: git 'config' '--bool' '--get' 'svn.useSvnsyncProps' trace: built-in: git 'config' '--bool' '--get' 'svn.followparent' trace: built-in: git 'config' '--get' 'svn.authorsfile' trace: built-in: git 'config' '--bool' '--get' 'svn.useSvmProps' trace: built-in: git 'config' '--bool' '--get' 'svn.uselogauthor' trace: built-in: git 'rev-parse' '--symbolic' '--all' trace: built-in: git 'config' '-l' trace: built-in: git 'config' '-l' trace: built-in: git 'config' '-l' trace: built-in: git 'config' '--int' '--get' 'svn-remote.svn.branches-maxRev' trace: built-in: git 'config' '--int' '--get' 'svn-remote.svn.tags-maxRev' trace: built-in: git 'config' '--get' 'svn-remote.svn.url' trace: built-in: git 'config' '--get' 'svn-remote.svn.uuid' trace: built-in: git 'config' 'svn-remote.svn.branches-maxRev' '8' could not lock config file config svn-remote.svn.branches-maxRev 8: command returned error: 255 I suspect that the locking problem is due to some strange anti-virus interaction, because issuing the same command on the command line succeeds. However, did you notice the many calls to "git config"? Especially the three ones which list all values anyway? I am not really sure if that is the single reason of the slowness -- remember, Windows is mightily spawn()-challenged -- but it sure would help to have git-svn read the config once at the beginning, probably with "-z", too, and then just read from the cached values, no? Ciao, Dscho P.S.: how far is the svn:external->submodule stuff? ``` ## Eric Wong, 2008-02-20 22:39 Subject: Re: git svn forking an awful lot of "git config"s Message-ID: <20080220223953.GA32663@hand.yhbt.net> URL: https://gitlist.dev/e/20080220223953.GA32663%40hand.yhbt.net In-Reply-To: ``` Johannes Schindelin wrote: > Hi, > > we are approaching our first release of msysGit's installer with git-svn. > However, I am experiencing a very bad performance, and an error: > > $ git svn fetch > trace: exec: 'git-svn' 'fetch' > trace: built-in: git 'config' '--bool' '--get' 'svn.fetchall' > trace: built-in: git 'config' '--get' 'svn.username' > trace: built-in: git 'config' '--get' 'svn.repackflags' > trace: built-in: git 'config' '--bool' '--get' 'svn.quiet' > trace: built-in: git 'config' '--bool' '--get' 'svn.noauthcache' > trace: built-in: git 'config' '--get' 'svn.revision' > trace: built-in: git 'config' '--int' '--get' 'svn.repack' > trace: built-in: git 'config' '--int' '--get' 'svn.logwindowsize' > trace: built-in: git 'config' '--bool' '--get' 'svn.nocheckout' > trace: built-in: git 'config' '--get' 'svn.configdir' > trace: built-in: git 'config' '--bool' '--get' 'svn.noMetadata' > trace: built-in: git 'config' '--bool' '--get' 'svn.useSvnsyncProps' > trace: built-in: git 'config' '--bool' '--get' 'svn.followparent' > trace: built-in: git 'config' '--get' 'svn.authorsfile' > trace: built-in: git 'config' '--bool' '--get' 'svn.useSvmProps' > trace: built-in: git 'config' '--bool' '--get' 'svn.uselogauthor' > trace: built-in: git 'rev-parse' '--symbolic' '--all' > trace: built-in: git 'config' '-l' > trace: built-in: git 'config' '-l' > trace: built-in: git 'config' '-l' > trace: built-in: git 'config' '--int' '--get' 'svn-remote.svn.branches-maxRev' > trace: built-in: git 'config' '--int' '--get' 'svn-remote.svn.tags-maxRev' > trace: built-in: git 'config' '--get' 'svn-remote.svn.url' > trace: built-in: git 'config' '--get' 'svn-remote.svn.uuid' > trace: built-in: git 'config' 'svn-remote.svn.branches-maxRev' '8' > could not lock config file > config svn-remote.svn.branches-maxRev 8: command returned error: 255 > > I suspect that the locking problem is due to some strange anti-virus > interaction, because issuing the same command on the command line > succeeds. I believe somebody on the list also had the same problem with file locking in Windows. Unfortunately, I have little idea as to what could be wrong. Could Windows not be releasing file locks properly? It could be worth it to write a standalone Perl script that replicates the git-config calls without the rest of git-svn getting in the way... > However, did you notice the many calls to "git config"? Especially the > three ones which list all values anyway? > > I am not really sure if that is the single reason of the slowness -- > remember, Windows is mightily spawn()-challenged -- but it sure would help > to have git-svn read the config once at the beginning, probably with "-z", > too, and then just read from the cached values, no? Many months ago, I thought about implementing a transparent caching layer in Git.pm to work with git configs. Of course, that requires cooperation from all readers/writers within the process... Done correctly, it would help more than just git-svn. too. I think I had this idea around the time we made git-config output Perl hashes and arrays. > Ciao, > Dscho > > P.S.: how far is the svn:external->submodule stuff? Yikes. I've let other work pile up on my ever-growing todo-list :/ I'll see if I can dig it out and wrap it up this weekend or next... -- Eric Wong ``` ## Jakub Narebski, 2008-02-20 23:31 Subject: Re: git svn forking an awful lot of "git config"s Message-ID: URL: https://gitlist.dev/e/m3ejb7tdbo.fsf%40localhost.localdomain In-Reply-To: <20080220223953.GA32663@hand.yhbt.net> ``` Eric Wong writes: > Many months ago, I thought about implementing a transparent caching layer > in Git.pm to work with git configs. Of course, that requires > cooperation from all readers/writers within the process... Done > correctly, it would help more than just git-svn. too. Take a look how gitweb uses "git config -z -l" to read all config in one go, and save it to hash for later use, lazily. > I think I had this idea around the time we made git-config output Perl > hashes and arrays. This is an alternate solution, better for simple scripts and one-off scripts (you don't need to write "git config -z -l" parser), but I think that additional eval might be not good for performance. -- Jakub Narebski Poland ShadeHawk on #git ``` ## Nagy Balázs, 2008-02-20 23:57 Subject: Re: git svn forking an awful lot of "git config"s Message-ID: <47BCBE70.8020706@iksz.hu> URL: https://gitlist.dev/e/47BCBE70.8020706%40iksz.hu In-Reply-To: ``` Jakub Narebski wrote: > Take a look how gitweb uses "git config -z -l" to read all config > in one go, and save it to hash for later use, lazily. ... > This is an alternate solution, better for simple scripts and one-off > scripts (you don't need to write "git config -z -l" parser), but > I think that additional eval might be not good for performance. > Maybe I'll finish my git-depot proof of concept script someday. Here's the configuration reader method I use (please note git-depot doesn't use repositories -- yet): sub readConfig { my $self = shift; my ($cfg, @cfg, $cat, $key, $val); $cfg = Git::command(qw/config -z -l/); @cfg = split(/\000/m, $cfg); %{$self->{cfg}} = (); foreach (@cfg) { if (($cat, $key, $val) = (/\A(\S+)\.(\S+)[\n\r]+^(.+)\Z/ms)) { next unless $cat eq 'depot'; $self->{cfg}->{$key} = $val; } } return 1; } Works like a charm. -- -jul- ``` ## Johannes Schindelin, 2008-02-21 00:18 Subject: Re: git svn forking an awful lot of "git config"s Message-ID: URL: https://gitlist.dev/e/alpine.LSU.1.00.0802210006590.17164%40racer.site In-Reply-To: <20080220223953.GA32663@hand.yhbt.net> ``` Him On Wed, 20 Feb 2008, Eric Wong wrote: > Johannes Schindelin wrote: > > > we are approaching our first release of msysGit's installer with > > git-svn. However, I am experiencing a very bad performance, and an > > error: > > > > $ git svn fetch > > trace: exec: 'git-svn' 'fetch' > > trace: built-in: git 'config' '--bool' '--get' 'svn.fetchall' > > trace: built-in: git 'config' '--get' 'svn.username' > > trace: built-in: git 'config' '--get' 'svn.repackflags' > > trace: built-in: git 'config' '--bool' '--get' 'svn.quiet' > > trace: built-in: git 'config' '--bool' '--get' 'svn.noauthcache' > > trace: built-in: git 'config' '--get' 'svn.revision' > > trace: built-in: git 'config' '--int' '--get' 'svn.repack' > > trace: built-in: git 'config' '--int' '--get' 'svn.logwindowsize' > > trace: built-in: git 'config' '--bool' '--get' 'svn.nocheckout' > > trace: built-in: git 'config' '--get' 'svn.configdir' > > trace: built-in: git 'config' '--bool' '--get' 'svn.noMetadata' > > trace: built-in: git 'config' '--bool' '--get' 'svn.useSvnsyncProps' > > trace: built-in: git 'config' '--bool' '--get' 'svn.followparent' > > trace: built-in: git 'config' '--get' 'svn.authorsfile' > > trace: built-in: git 'config' '--bool' '--get' 'svn.useSvmProps' > > trace: built-in: git 'config' '--bool' '--get' 'svn.uselogauthor' > > trace: built-in: git 'rev-parse' '--symbolic' '--all' > > trace: built-in: git 'config' '-l' > > trace: built-in: git 'config' '-l' > > trace: built-in: git 'config' '-l' > > trace: built-in: git 'config' '--int' '--get' 'svn-remote.svn.branches-maxRev' > > trace: built-in: git 'config' '--int' '--get' 'svn-remote.svn.tags-maxRev' > > trace: built-in: git 'config' '--get' 'svn-remote.svn.url' > > trace: built-in: git 'config' '--get' 'svn-remote.svn.uuid' > > trace: built-in: git 'config' 'svn-remote.svn.branches-maxRev' '8' > > could not lock config file > > config svn-remote.svn.branches-maxRev 8: command returned error: 255 > > > > I suspect that the locking problem is due to some strange anti-virus > > interaction, because issuing the same command on the command line > > succeeds. > > I believe somebody on the list also had the same problem with file > locking in Windows. Unfortunately, I have little idea as to what could > be wrong. Could Windows not be releasing file locks properly? I am not half as concerned with locks on Windows (since they occur regularly, thanks to _two_ Antiviruses, go figure), as with unnecessary config file reading. > > However, did you notice the many calls to "git config"? Especially > > the three ones which list all values anyway? > > > > I am not really sure if that is the single reason of the slowness -- > > remember, Windows is mightily spawn()-challenged -- but it sure would > > help to have git-svn read the config once at the beginning, probably > > with "-z", too, and then just read from the cached values, no? > > Many months ago, I thought about implementing a transparent caching > layer in Git.pm to work with git configs. Of course, that requires > cooperation from all readers/writers within the process... Done > correctly, it would help more than just git-svn. too. Yes, but I am really scared there. Last time, Pasky tried to put a _lot_ into Git.pm, and it did not work here. As a consequence, I put aside a _lot_ of my time to turn the most important (to me!) perl scripts into builtins. > I think I had this idea around the time we made git-config output Perl > hashes and arrays. You mean -z? > > P.S.: how far is the svn:external->submodule stuff? > > Yikes. I've let other work pile up on my ever-growing todo-list :/ I'll > see if I can dig it out and wrap it up this weekend or next... Actually, I can understand what you're talking about. But just let me tell this: I (amongst others) are _excited_ to wait until you have that particular feature working... Ciao, Dscho ```