git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: [PATCH] sha1_file: don't malloc the whole compressed result when writing out objects

From
Nicolas Pitre <nico@fluxnic.net>
Date
Feb 22, 2010, 17:36 UTC
Message-ID
<alpine.LFD.2.00.1002221233000.1946@xanadu.home>
In-Reply-To
<7v4ol9vl0l.fsf@alter.siamese.dyndns.org>
On Sun, 21 Feb 2010, Junio C Hamano wrote:
Show 25 quoted lines
> Junio C Hamano <gitster@pobox.com> writes:
> 
> > Nicolas Pitre <nico@fluxnic.net> writes:
> >
> >> And what real life case would trigger this?  Given the size of the 
> >> window for this to happen, what are your chances?
> >
> >> Of course the odds for me to be struck by lightning also exist.  And if 
> >> I work really really hard at it then I might be able to trigger that 
> >> pathological case above even before the next thunderstorm.  But in 
> >> practice I'm hardly concerned by either of those possibilities.
> >
> > The real life case for any of this triggers for me is zero, as I won't be
> > mistreating git as a continuous & asynchronous back-up tool.
> >
> > But then that would make the whole discussion moot.  There are people who
> > file "bug reports" with an artificial reproduction recipe built around a
> > loop that runs dd continuously overwriting a file while "git add" is asked
> > to add it.
> 
> Having said all that, I like your approach better.  It is not worth paying
> the price of unnecessary memcpy(3) that would _only_ help catching the
> insanely artificial test case, but your patch strikes a good balance of
> small overhead to catch the easier-to-trigger (either by stupidity, malice
> or mistake) cases.

I think it also catches the bad RAM case which is probably more common too.

Show 13 quoted lines
> So I am tempted to discard the "paranoia" patch, and replace with your two
> patches, with the following caveats in the log message.
> 
> --- /var/tmp/2	2010-02-21 22:23:30.000000000 -0800
> +++ /var/tmp/1	2010-02-21 22:23:22.000000000 -0800
> @@ -21,7 +21,9 @@
>      deflate operation has consumed that data, and make sure it matches
>      with the expected SHA1.  This way we can rely on the CRC32 checked by
>      the inflate operation to provide a good indication that the data is still
> -    coherent with its SHA1 hash.
> +    coherent with its SHA1 hash.  One pathological case we ignore is when
> +    the data is modified before (or during) deflate call, but changed back
> +    before it is hashed.
ACK.
Nicolas
Previous: Junio C HamanoNext: Junio C Hamano
Message 11 of 13 in “sha1_file: don't malloc the whole compressed result when writing out objects”
  1. sha1_file: don't malloc the whole compressed result when writing out objectsNicolas Pitre, Feb 21, 2010
  2. Junio C HamanoFeb 21, 2010
  3. Nicolas PitreFeb 21, 2010
  4. Junio C HamanoFeb 21, 2010
  5. Junio C HamanoFeb 21, 2010
  6. Nicolas PitreFeb 22, 2010
  7. Junio C HamanoFeb 22, 2010
  8. Nicolas PitreFeb 22, 2010
  9. Junio C HamanoFeb 22, 2010
  10. Junio C HamanoFeb 22, 2010
  11. Nicolas PitreFeb 22, 2010
  12. Junio C HamanoFeb 22, 2010
  13. Dmitry PotapovFeb 22, 2010

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.