slow git-cherry-pick.

> On Sun, Nov 24, 2013 at 5:45 PM, Paweł Sikora <pawel.sikora@agmk.net> wrote:
>> i've recently reinstalled a fresh system (fc20-beta) on my workstation
>> and observing a big slowdown on git cherry-pick operation (git-1.8.4.2-1).
>> the previous centos installation with an old git version works faster
>> (few seconds per cherry pick). now the same operation takes >1 min.
>
> What is the git version the reinstallation? Do you cherry-pick on one

> On Sun, Nov 24, 2013 at 5:45 PM, Paweł Sikora <pawel.sikora@agmk.net> wrote:
> > i've recently reinstalled a fresh system (fc20-beta) on my workstation
> > and observing a big slowdown on git cherry-pick operation (git-1.8.4.2-1).
> > the previous centos installation with an old git version works faster
> > (few seconds per cherry pick). now the same operation takes >1 min.
> 
> What is the git version before the reinstallation?

> On Sunday 24 of November 2013 19:47:10 Duy Nguyen wrote:
>> On Sun, Nov 24, 2013 at 5:45 PM, Paweł Sikora <pawel.sikora@agmk.net> wrote:
>> > i've recently reinstalled a fresh system (fc20-beta) on my workstation
>> > and observing a big slowdown on git cherry-pick operation (git-1.8.4.2-1).
>> > the previous centos installation with an old git version works faster
>> > (few seconds per cherry pick). now the same operation takes >1 min.
>> 
>> What is the git version before the reinstallation?
>
> git-1.7.11.3-1.el5.rf.
>
> i've checked this version on another machine with centos-5.$latest
> and it does similar amout of stat/read operation quickly (~6s).
> this "fast" centos-5 machine has /home on raid-0 (2x500GB) while
> my "slow (>1min)" workstation has /home on linear lvm (250G+1T).
>
> so, i suppose that my "slow" working copy crosses disks boundary
> or spread over 1TB drive and the random git i/o impacts performance.
>
> the question still remains - does the git need to scan whole checkout
> during picking well defined set of files?

> >> > i've recently reinstalled a fresh system (fc20-beta) on my workstation
> >> > and observing a big slowdown on git cherry-pick operation
> >> > (git-1.8.4.2-1).
> >> > the previous centos installation with an old git version works faster
> >> > (few seconds per cherry pick). now the same operation takes >1 min.
> >> 
> >> What is the git version before the reinstallation?
> > 
> > git-1.7.11.3-1.el5.rf.
> > 
> > i've checked this version on another machine with centos-5.$latest
> > and it does similar amout of stat/read operation quickly (~6s).
> > this "fast" centos-5 machine has /home on raid-0 (2x500GB) while
> > my "slow (>1min)" workstation has /home on linear lvm (250G+1T).
> > 
> > so, i suppose that my "slow" working copy crosses disks boundary
> > or spread over 1TB drive and the random git i/o impacts performance.
> > 
> > the question still remains - does the git need to scan whole checkout
> > during picking well defined set of files?
> 
> We do update-index to see what local changes you have upfront in
> order to avoid stomping on them (and we do not know upfront what
> paths the cherry-picked commit would change, given that there may be
> renames involved), so the answer is unfortunately yes, we would need
> to do lstat(2) the whole thing.

> Paweł Sikora <pawel.sikora@agmk.net> writes:
>>
>
> Umm, there's a gem here that the thread missed so far:
>
>> my git repo isn't very big[1] but it's checked out on the linear lvm
>> where random i/o generally hurts and strace shows that current git version
>> performs 2x{lstat}+1x{open,read,close} [2] on whole checkout before
>            ^^^^^^^^^
>
> There's no reason why it should do the lstat() *twice* for every file.
> But Paweł is right; the code path roughly goes like this:
>
> int cmd_cherry_pick(int argc, const char **argv, const char *prefix)
> {
> [...]
>         res = sequencer_pick_revisions(&opts);
>
> int sequencer_pick_revisions(struct replay_opts *opts)
> {
> [...]
>         read_and_refresh_cache(opts);
> [...]
>         return pick_commits(todo_list, opts);
> }
>
> static int pick_commits(struct commit_list *todo_list, struct replay_opts *opts)
> {
> [...]
>         read_and_refresh_cache(opts);
>
>
> I'm too tired to dig further, but AFAICT it's just a rather obvious case
> of duplication of effort.

> On Wed, Dec 4, 2013 at 3:13 AM, Thomas Rast <tr@thomasrast.ch> wrote:
> > Paweł Sikora <pawel.sikora@agmk.net> writes:
> > 
> > Umm, there's a gem here that the thread missed so far:
> >> my git repo isn't very big[1] but it's checked out on the linear lvm
> >> where random i/o generally hurts and strace shows that current git
> >> version
> >> performs 2x{lstat}+1x{open,read,close} [2] on whole checkout before
> >> 
> >            ^^^^^^^^^
> > 
> > There's no reason why it should do the lstat() *twice* for every file.
> > But Paweł is right; the code path roughly goes like this:
> > 
> > int cmd_cherry_pick(int argc, const char **argv, const char *prefix)
> > {
> > [...]
> > 
> >         res = sequencer_pick_revisions(&opts);
> > 
> > int sequencer_pick_revisions(struct replay_opts *opts)
> > {
> > [...]
> > 
> >         read_and_refresh_cache(opts);
> > 
> > [...]
> > 
> >         return pick_commits(todo_list, opts);
> > 
> > }
> > 
> > static int pick_commits(struct commit_list *todo_list, struct replay_opts
> > *opts) {
> > [...]
> > 
> >         read_and_refresh_cache(opts);
> > 
> > I'm too tired to dig further, but AFAICT it's just a rather obvious case
> > of duplication of effort.
> 
> That's something to optimize, but it's single commit picking,
> sequencer_pick_revisions() should call single_pick() instead of
> pick_commits().
> 
> The read+close on the whole checkout looks like there's problem with
> refresh operation and git decides to read up and verify sha-1 by
> content. Pawel, if you run "strace git update-index --refresh" twice,
> does it still show 1 stat + 1 read for every entry on the second try?