git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: [RFC/PATCH 0/3] teach --histogram to diff

From
Tay Ray Chuan <rctay89@gmail.com>
Date
Jul 13, 2011, 16:34 UTC
Message-ID
<CALUzUxqS8y4HH44t2CuSvfQiLP-Y7hFtbi5x2rtAwPBKSJ+3Bw@mail.gmail.com>
In-Reply-To
<CAJo=hJu5ubkzUyyPM0nqP+J9CU3hBtAHfuzaLSuN214Hux4qTA@mail.gmail.com>
On Tue, Jul 12, 2011 at 10:19 PM, Shawn Pearce <spearce@spearce.org> wrote:
> On Mon, Jul 11, 2011 at 23:10, Tay Ray Chuan <rctay89@gmail.com> wrote:
>> [RFC/PATCH 3/3] xdiff/xprepare: use a smaller sample size for histogram
>
> Do we need sampling at all for histogram? Can you skip it?

Sampling is done to get a guess of lines in the file. This guess is then used to preallocated memory for the list of records. (This is just a guess; if we find more records we allocate more memory.) By doing this preallocation, we can save on malloc()'s, giving a performance boost.

But then sampling has its costs - previously, we ran up to 256 memchr('\n')s within a mmfile "block". For histogram diff, we cut the cap down to 20. (But not for the other diff algorithms - see the relevant patch text for more.) I think this gives us a good balance - time spent in guessing lines, and time gained from preallocating memory.

-- 
Cheers,
Ray Chuan
Previous: Tay Ray ChuanNext: Tay Ray Chuan
Message 10 of 24 in “teach --histogram to diff”
  1. 0/3 teach --histogram to diffTay Ray Chuan, Jul 12, 2011
  2. 1/3 teach --histogram to diffTay Ray Chuan, Jul 12, 2011
  3. 2/3 xdiff/xprepare: skip classificationTay Ray Chuan, Jul 12, 2011
  4. 3/3 xdiff/xprepare: use a smaller sample size for histogram diffTay Ray Chuan, Jul 12, 2011
  5. Junio C HamanoJul 12, 2011
  6. Tay Ray ChuanJul 13, 2011
  7. Shawn PearceJul 12, 2011
  8. Junio C HamanoJul 12, 2011
  9. Tay Ray ChuanJul 13, 2011
  10. Tay Ray ChuanJul 13, 2011
  11. 0/8 teach --histogram to diffTay Ray Chuan, Aug 1, 2011
  12. 1/8 xdiff/xprepare: use memset()Tay Ray Chuan, Aug 1, 2011
  13. 2/8 do away with xdl_mmfile_next()Tay Ray Chuan, Aug 1, 2011
  14. 3/8 xdiff/xprepare: refactor abort cleanupsTay Ray Chuan, Aug 1, 2011
  15. 4/8 xdiff/xpatience: factor out fall-back-diff functionTay Ray Chuan, Aug 1, 2011
  16. 5/8 t4033-diff-patience: factor out testsTay Ray Chuan, Aug 1, 2011
  17. 6/8 teach --histogram to diffTay Ray Chuan, Aug 1, 2011
  18. 7/8 xdiff/xprepare: skip classificationTay Ray Chuan, Aug 1, 2011
  19. 8/8 xdiff/xprepare: use a smaller sample size for histogram diffTay Ray Chuan, Aug 1, 2011
  20. 0/4 changes for rc/histogram-diff in 'next'Tay Ray Chuan, Aug 1, 2011
  21. 1/4 xdiff: do away with xdl_mmfile_next()Tay Ray Chuan, Aug 1, 2011
  22. 2/4 xdiff/xhistogram: rework handling of recursed resultsTay Ray Chuan, Aug 1, 2011
  23. 3/4 xdiff/xhistogram: rely on xdl_trim_ends()Tay Ray Chuan, Aug 1, 2011
  24. 4/4 xdiff/xhistogram: drop need for additional variableTay Ray Chuan, Aug 1, 2011

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.