git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: [PATCH (GITK) v2 4/4] gitk: Implement batch lookup and caching of encoding attrs.

From
Alexander Gavrilov <angavrilov@gmail.com>
Date
Oct 10, 2008, 12:22 UTC
Message-ID
<bb6f213e0810100522v653507d6r75cc4c64b57aa459@mail.gmail.com>
In-Reply-To
<18671.16658.667581.499095@cargo.ozlabs.ibm.com>
On Fri, Oct 10, 2008 at 3:48 PM, Paul Mackerras <paulus@samba.org> wrote:
Show 11 quoted lines
> Alexander Gavrilov writes:
>
>> When the diff contains thousands of files, calling git-check-attr
>> once per file is very slow. With this patch gitk does attribute
>> lookup in batches of 30 files while reading the diff file list,
>> which leads to a very noticeable speedup.
>
> Why only 30 at a time?  The logic would be simpler if cache_gitattr
> just did all the paths in one call to git check-attr, and it should be
> able to cope with 1000 paths in one call, I would think, which is the
> most that gettreediffline will give it.

OS-enforced command-line size limit on Windows is 32K. Cramming in 1000 paths would leave only 32 characters for each path.

Show 5 quoted lines
> Also, I wonder why we now have two levels of caching of the encoding
> attribute.  Your patch 1/4 introduced path_encoding_cache, which was
> fine, but now we have path_attr_cache as well, which seems to me to
> serve exactly the same function since the encoding is the only
> attribute we ever ask about.  Surely we don't need both caches?

If the (git-gui) patch that reimplements the tcl_encoding procedure is applied, we may drop the path_encoding_cache. Current implementation is too slow for batch lookup, especially if the encoding is actually not supported, and without the cache the lookup would be done on every loading of a diff.

Show 5 quoted lines
> Even with this batching, I am a bit concerned that adding the encoding
> support might make things noticeably slower for people who don't need
> any encoding support (which would be the majority, I think).  We may
> end up needing an option to turn off the checking of the encoding
> attribute.

I hope that most diffs don't contain thousands of files at once. And actual huge diffs are likely to be relatively slow to load anyway.

Alexander
Previous: Paul MackerrasNext: Paul Mackerras
Message 7 of 12 in “Encoding support in GUI”
  1. 0/4 Encoding support in GUIAlexander Gavrilov, Sep 30, 2008
  2. 1/4 gitk: Port new encoding logic from git-gui.Alexander Gavrilov, Sep 30, 2008
  3. 2/4 gitk: Implement file contents encoding support.Alexander Gavrilov, Sep 30, 2008
  4. 3/4 gitk: Support filenames in the locale encoding.Alexander Gavrilov, Sep 30, 2008
  5. 4/4 gitk: Implement batch lookup and caching of encoding attrs.Alexander Gavrilov, Sep 30, 2008
  6. Paul MackerrasOct 10, 2008
  7. Alexander GavrilovOct 10, 2008
  8. Paul MackerrasOct 11, 2008
  9. Alexander GavrilovOct 11, 2008
  10. Paul MackerrasOct 11, 2008
  11. Paul MackerrasOct 10, 2008
  12. Paul MackerrasOct 10, 2008

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.