git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: [PATCH v2 2/2] convert.c: stream and fast search for binary

From
Duy Nguyen <pclouds@gmail.com>
Date
Oct 29, 2016, 12:13 UTC
Message-ID
<CACsJy8C6=m9uKc+RcL+jJWmCcKQiPc7Qm92y3zd3aUFp+nOVFA@mail.gmail.com>
In-Reply-To
<20161012134727.28365-1-tboegi@web.de>
On Wed, Oct 12, 2016 at 8:47 PM,  <tboegi@web.de> wrote:
Show 21 quoted lines
> From: Torsten Bögershausen <tboegi@web.de>
>
> When statistics are done for the autocrlf handling, the search in
> the content can be stopped, if e.g
> - a search for binary is done, and a NUL character is found
> - a search for CRLF is done, and the first CRLF is found.
>
> Similar when statistics for binary vs non-binary are gathered:
> Whenever a lone CR or NUL is found, the search can be aborted.
>
> When checking out files in "auto" mode, any file that has a "lone CR"
> or a CRLF will not be converted, so the search can be aborted early.
>
> Add the new bit, CONVERT_STAT_BITS_ANY_CR,
> which is set for either lone CR or CRLF.
>
> Many binary files have a NUL very early and it is often not necessary
> to load the whole content of a file or blob into memory.
>
> Split gather_stats() into gather_all_stats() and gather_stats_partly()
> to do a streaming handling for blobs and files in the worktree.

Maybe break this commit down a bit? the gather_all_stats and gather_stats_partly() seem independent and can standalone. So is the blob streaming, and get_convert_stats_wt.

-- 
Duy
Previous: Torsten Bögershausen
Message 11 of 11 in “convert: stream and early out”
  1. 0/2 convert: stream and early outtboegi@web.de, Oct 9, 2016
  2. Junio C HamanoOct 10, 2016
  3. 1/2 read-cache: factor out get_sha1_from_index() helpertboegi@web.de, Oct 12, 2016
  4. Junio C HamanoOct 27, 2016
  5. Duy NguyenOct 29, 2016
  6. 0/2 Stream and fast searchtboegi@web.de, Oct 12, 2016
  7. Junio C HamanoOct 27, 2016
  8. 2/2 convert.c: stream and fast search for binarytboegi@web.de, Oct 12, 2016
  9. Junio C HamanoOct 27, 2016
  10. Torsten BögershausenNov 1, 2016
  11. Duy NguyenOct 29, 2016

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.