git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: slow git-cherry-pick.

From
PSPaweł Sikora <pawel.sikora@agmk.net>
Date
Dec 4, 2013, 12:46 UTC
Message-ID
<1827709.rD7YulLKCL@localhost.localdomain>
In-Reply-To
<CACsJy8Be2USmGA--FLT3LERTde327Ue65CCjoLHi5SzNzUX1dw@mail.gmail.com>
On Wednesday 04 of December 2013 08:07:23 Duy Nguyen wrote:
Show 49 quoted lines
> On Wed, Dec 4, 2013 at 3:13 AM, Thomas Rast <tr@thomasrast.ch> wrote:
> > Paweł Sikora <pawel.sikora@agmk.net> writes:
> > 
> > Umm, there's a gem here that the thread missed so far:
> >> my git repo isn't very big[1] but it's checked out on the linear lvm
> >> where random i/o generally hurts and strace shows that current git
> >> version
> >> performs 2x{lstat}+1x{open,read,close} [2] on whole checkout before
> >> 
> >            ^^^^^^^^^
> > 
> > There's no reason why it should do the lstat() *twice* for every file.
> > But Paweł is right; the code path roughly goes like this:
> > 
> > int cmd_cherry_pick(int argc, const char **argv, const char *prefix)
> > {
> > [...]
> > 
> >         res = sequencer_pick_revisions(&opts);
> > 
> > int sequencer_pick_revisions(struct replay_opts *opts)
> > {
> > [...]
> > 
> >         read_and_refresh_cache(opts);
> > 
> > [...]
> > 
> >         return pick_commits(todo_list, opts);
> > 
> > }
> > 
> > static int pick_commits(struct commit_list *todo_list, struct replay_opts
> > *opts) {
> > [...]
> > 
> >         read_and_refresh_cache(opts);
> > 
> > I'm too tired to dig further, but AFAICT it's just a rather obvious case
> > of duplication of effort.
> 
> That's something to optimize, but it's single commit picking,
> sequencer_pick_revisions() should call single_pick() instead of
> pick_commits().
> 
> The read+close on the whole checkout looks like there's problem with
> refresh operation and git decides to read up and verify sha-1 by
> content. Pawel, if you run "strace git update-index --refresh" twice,
> does it still show 1 stat + 1 read for every entry on the second try?

the 'git update-index --refresh' runs quickly and strace shows only lstat() on every file. i see no massive open/read actions in this case.

$ strace -o strace-try1.log git update-index --refresh hmdb: needs update $ strace -o strace-try2.log git update-index --refresh hmdb: needs update

$ grep -c lstat strace-try1.log 33793 $ grep -c lstat strace-try2.log 33793

-- 
gpg key fingerprint = 60B4 9886 AD53 EB3E 88BB 1EB5 C52E D01B 683B 9411
Previous: Duy Nguyen
Message 9 of 9 in “slow git-cherry-pick.”
  1. Paweł SikoraNov 24, 2013
  2. Duy NguyenNov 24, 2013
  3. Duy NguyenNov 24, 2013
  4. Paweł SikoraNov 24, 2013
  5. Junio C HamanoNov 25, 2013
  6. Paweł SikoraDec 3, 2013
  7. Thomas RastDec 3, 2013
  8. Duy NguyenDec 4, 2013
  9. Paweł SikoraDec 4, 2013

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.