git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: Index files autocompletion too slow in big repositories (w / suggestion for improvement)

From
Jacob Keller <jacob.keller@gmail.com>
Date
Apr 16, 2017, 00:31 UTC
Message-ID
<CA+P7+xo+wiTkByd_BrLUEyGc9z7USYT8Joo7OhjrYCUmbHjZ0Q@mail.gmail.com>
In-Reply-To
<7a07a2a8-07a6-4342-80d0-7684e6242326@kdbg.org>
On Sat, Apr 15, 2017 at 4:59 AM, Johannes Sixt <j6t@kdbg.org> wrote:
Show 49 quoted lines
> Cc Gábor.
>
> Am 15.04.2017 um 00:33 schrieb Ævar Arnfjörð Bjarmason:
>>
>> On Sat, Apr 15, 2017 at 12:08 AM, Carlos Pita <carlosjosepita@gmail.com>
>> wrote:
>>>
>>> This is much faster (below 0.1s):
>>>
>>> __git_index_files ()
>>> {
>>>     local dir="$(__gitdir)" root="${2-.}" file;
>>>     if [ -d "$dir" ]; then
>>>         __git_ls_files_helper "$root" "$1" | \
>>>             sed -r 's@/.*@@' | uniq | sort | uniq
>>>     fi
>>> }
>>>
>>> time __git_index_files
>>>
>>> real    0m0.075s
>>> user    0m0.083s
>>> sys    0m0.010s
>>>
>>> Most of the improvement is due to the simpler, non-grouping, regex.
>>> Since I expect most of the common prefixes to arrive consecutively,
>>> running uniq before sort also improves things a bit. I'm not removing
>>> leading double quotes anymore (this isn't being done by the current
>>> version, anyway) but this doesn't seem to hurt.
>>>
>>> Despite the dependence on sed this is ten times faster than the
>>> original, maybe an option to enable fast index completion or something
>>> like that might be desirable.
>>
>>
>> It's fine to depend on sed, these shell-scripts are POSIX compatible,
>> and so is sed, we use sed in a lot of the built-in shellscripts.
>
>
> This is about command line completion. We go a long way to avoid forking
> processes there. What is 10x faster on Linux despite of forking a process
> may not be so on Windows.
>
> (I'm not using bash command line completion on Windows, so I can't tell what
> the effect of your suggested change is on Windows. I hope Gábor can comment
> on it.)
>
> -- Hannes
>

In cases like this, might it be worth somehow splitting it so Linux can use the best thing, and Windows can continue using what's best for it, since it is a pretty significant advantage on Linux.

Thanks, Jake

Previous: Johannes SixtNext: Junio C Hamano
Message 7 of 10 in “Index files autocompletion too slow in big repositories (w / suggestion for improvement)”
  1. Carlos PitaApr 14, 2017
  2. Carlos PitaApr 14, 2017
  3. Ævar Arnfjörð BjarmasonApr 14, 2017
  4. Jacob KellerApr 15, 2017
  5. Junio C HamanoApr 15, 2017
  6. Johannes SixtApr 15, 2017
  7. Jacob KellerApr 16, 2017
  8. Junio C HamanoApr 17, 2017
  9. Johannes SixtApr 17, 2017
  10. Johannes SixtApr 15, 2017

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.