git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Windows performance / threading file access

From
SZStefan Zager <szager@google.com>
Date
Oct 10, 2013, 18:18 UTC
Message-ID
<CAHOQ7J_ZZ=7j-5ULd7Tdvbiqg4inhwi+fue_w6WAtNRkvZSwsg@mail.gmail.com>
Hi folks,

I don't follow the mailing list carefully, so forgive me if this has been discussed before, but:

I've noticed that when working with a very large repository using msys git, the initial checkout of a cloned repository is excruciatingly slow (80%+ of total clone time). The root cause, I think, is that git does all the file access serially, and that's really slow on Windows.

Has anyone considered threading file access to speed this up? In particular, I've got my eye on this loop in unpack-trees.c:

static struct checkout state;
static int check_updates(struct unpack_trees_options *o)
{
        unsigned cnt = 0, total = 0;
        struct progress *progress = NULL;
        struct index_state *index = &o->result;
        int i;
        int errs = 0;
        ...
        for (i = 0; i < index->cache_nr; i++) {
                struct cache_entry *ce = index->cache[i];
                if (ce->ce_flags & CE_UPDATE) {
                        display_progress(progress, ++cnt);
                        ce->ce_flags &= ~CE_UPDATE;
                        if (o->update && !o->dry_run) {
                                errs |= checkout_entry(ce, &state, NULL);
                        }
                }
        }
        stop_progress(&progress);
        if (o->update)
                git_attr_set_direction(GIT_ATTR_CHECKIN, NULL);
        return errs != 0;
}
Any thoughts on adding threading around the call to checkout_entry?
Thanks in advance,
Stefan
Next: Sebastian Schuberth
Message 1 of 12 in “Windows performance / threading file access”
  1. Stefan ZagerOct 10, 2013
  2. Sebastian SchuberthOct 10, 2013
  3. Karsten BleesOct 11, 2013
  4. Stefan ZagerOct 11, 2013
  5. Stefan ZagerOct 11, 2013
  6. Duy NguyenOct 11, 2013
  7. pro-logicOct 15, 2013
  8. Karsten BleesOct 17, 2013
  9. pro-logicOct 21, 2013
  10. Karsten BleesOct 22, 2013
  11. Sebastian SchuberthOct 22, 2013
  12. Karsten BleesOct 22, 2013

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.