git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Index files autocompletion too slow in big repositories (w / suggestion for improvement)

From
Carlos Pita <carlosjosepita@gmail.com>
Date
Apr 14, 2017, 20:06 UTC
Message-ID
<CAELgYhf1s43p62t6W14S=nDt-O247cPqsPMUDfye1OTnDND3Gg@mail.gmail.com>
Hi all,

I'm currently using git annex to manage my entire file collection (including tons of music and books) and I noticed how slow autocompletion has become for files in the index (say for git add). The main offender is a while-read-case-echo bash loop in __git_index_files that can be readily substituted with a much faster sed invocation, although I guess you didn't want the sed dependency in the first place. Anyway, here is my benchmark:

__git_index_files ()
{
    local dir="$(__gitdir)" root="${2-.}" file;
    if [ -d "$dir" ]; then
        __git_ls_files_helper "$root" "$1" | while read -r file; do
            case "$file" in
                ?*/*)
                    echo "${file%%/*}"
                ;;
                *)
                    echo "$file"
                ;;
            esac;
        done | sort | uniq;
    fi
}
time __git_index_files > /dev/null
__git_index_files ()
{
    local dir="$(__gitdir)" root="${2-.}" file;
    if [ -d "$dir" ]; then
        __git_ls_files_helper "$root" "$1" | \
            sed -r 's@^"?([^/]+)/.*$@\1@' | sort | uniq
    fi
}
time __git_index_files > /dev/null

real 0m0.830s user 0m0.597s sys 0m0.310s

real 0m0.345s user 0m0.357s sys 0m0.000s

Notice I'm also excluding the beginning double quote that appears in escaped path names.

Best regards -- Carlos

Next: Carlos Pita
Message 1 of 10 in “Index files autocompletion too slow in big repositories (w / suggestion for improvement)”
  1. Carlos PitaApr 14, 2017
  2. Carlos PitaApr 14, 2017
  3. Ævar Arnfjörð BjarmasonApr 14, 2017
  4. Jacob KellerApr 15, 2017
  5. Junio C HamanoApr 15, 2017
  6. Johannes SixtApr 15, 2017
  7. Jacob KellerApr 16, 2017
  8. Junio C HamanoApr 17, 2017
  9. Johannes SixtApr 17, 2017
  10. Johannes SixtApr 15, 2017

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.