git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: [RFC PATCHv2 04/10] gitweb: Use Cache::Cache compatibile (get, set) output caching

From
Jakub Narebski <jnareb@gmail.com>
Date
Feb 10, 2010, 01:12 UTC
Message-ID
<201002100212.26157.jnareb@gmail.com>
In-Reply-To
<1265711427-15193-5-git-send-email-jnareb@gmail.com>
On Tue, 9 Feb 2010 at 11:30 +0100, Jakub Narebski wrote:
Show 5 quoted lines
> The cache_fetch subroutine captures output (from STDOUT only, as
> STDERR is usually logged) using either ->push_layer()/->pop_layer()
> from PerlIO::Util submodule (if it is available), or by setting and
> restoring *STDOUT.  Note that only the former could be tested reliably
> to be reliable in t9503 test!

Scratch that, I have just checked that (at least for Apache + mod_cgi, but I don't think that it matters) the latter solution, with setting and restoring *STDOUT doesn't work: I would get data in cache (so it can be restored later), but instead of output I would get Internal Server Error ("The server encountered an internal error or misconfiguration and was unable to complete your request.") without even a hint what the problem was. Sprinkling "die ...: $!" didn't help to catch this error: I suspect that the problem is with capturing.

So we either would have to live with non-core PerlIO::Util or (pure Perl) Capture::Tiny, or do the 'print -> print $out' patch...

[....]
Show 21 quoted lines
> Capturing gitweb output
> =======================
> When output (response) caching is enabled, the caching mechanism has to
> capture gitweb output (perhaps while printing it to standard output) to
> save it to cache, unless the data is available in cache and not expired.
> 
> Because die_error uses 'exit', and because it uses git_header_html and
> other printing subroutines (which output has to be captured in other
> situations), it needs to disable caching, unless we are "tee"-ing.
> Otherwise we would get no output from die_error (because it is captured),
> but also we would not get data to be saved in cache and printed, because
> 'exit' in die_error would exit capture, too.  This restricts methods and
> modules that can be used to capture output.
> 
> Below there are presented various considered and used ways of capturing the
> output, or "tee"-ing it (capturing while printing), with their advantages
> and disadvantages.
> 
> 
> Capturing output (capture)
> ~~~~~~~~~~~~~~~~~~~~~~~~~~
[...]
Show 16 quoted lines
> 5. Without 'print <sth>' to 'print $out <sth>' change to gitweb, one can try
>    manipulating *STDOUT directly, first saving *STDOUT or \*STDOUT to some
>    variable, then setting *STDOUT = $data_fh, where $data_fh is filehandle
>    opened to in-memory file.
> 
>    This seems to work, does not require large patch to gitweb, and does not
>    require extra (non-core) Perl module.  Nevertheless it seems fragile with
>    respect to restoring STDOUT, and even though basic tests (not included)
>    of this mechanism producted expected result, I wasn't able to write
>    working tests when using this method.
> 
>    We could probably examine how Capture::Tiny does it, and reuse (copy)
>    relevant parts of code, perhaps duplicating STDOUT, closing it and then
>    reopening as in-memory filehandle.
> 
>    YMMV (Your Mileage May Vary).
-- 
Jakub Narebski
Poland
Previous: Jakub NarebskiNext: Petr Baudis
Message 6 of 18 in “[RFC PATCHv2 00/10] gitweb: Simple file based output caching”
  1. Jakub NarebskiFeb 9, 2010
  2. 01/10 gitweb: href(..., -path_info => 0|1)Jakub Narebski, Feb 9, 2010
  3. 02/10 gitweb/cache.pm - Very simple file based cachingJakub Narebski, Feb 9, 2010
  4. 03/10 gitweb/cache.pm - Stat-based cache expirationJakub Narebski, Feb 9, 2010
  5. 04/10 gitweb: Use Cache::Cache compatibile (get, set) output cachingJakub Narebski, Feb 9, 2010
  6. Jakub NarebskiFeb 10, 2010
  7. Petr BaudisFeb 10, 2010
  8. Jakub NarebskiFeb 10, 2010
  9. Petr BaudisFeb 10, 2010
  10. Jakub NarebskiFeb 10, 2010
  11. Jakub NarebskiFeb 10, 2010
  12. 05/10 gitweb/cache.pm - Adaptive cache expiration timeJakub Narebski, Feb 9, 2010
  13. 06/10 gitweb: Use CHI compatibile (compute method) cachingJakub Narebski, Feb 9, 2010
  14. 07/10 gitweb/cache.pm - Use locking to avoid 'cache miss stampede' problemJakub Narebski, Feb 9, 2010
  15. 08/10 gitweb/cache.pm - Serve stale data when waiting for filling cacheJakub Narebski, Feb 9, 2010
  16. 09/10 gitweb/cache.pm - Regenerate (refresh) cache in backgroundJakub Narebski, Feb 9, 2010
  17. Jakub NarebskiFeb 9, 2010
  18. 10/10 gitweb: Show appropriate "Generating..." page when regenerating cacheJakub Narebski, Feb 9, 2010

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.