git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: [PATCH] drop support for "experimental" loose objects

From
Junio C Hamano <gitster@pobox.com>
Date
Nov 27, 2013, 18:57 UTC
Message-ID
<xmqqeh61u0z9.fsf@gitster.dls.corp.google.com>
In-Reply-To
<20131127093043.GA23429@sigill.intra.peff.net>
Jeff King <peff@peff.net> writes:
Show 8 quoted lines
> The vast majority of blobs in git.git will be stored as packed deltas.
> That means the streaming code will fall back to doing the regular
> in-core access. We _could_ therefore use that in-core copy to do our
> sha1 check rather than streaming; but of course we never get access to
> it outside of stream_blob_to_fd, and it is discarded. However, we do
> keep a copy in the delta base cache. When we immediately ask to unpack
> the exact same entry for check_sha1_signature, we can pull the copy
> straight out of the cache without having to re-inflate the object.

OK, that explains the overhead of 20% that is lower than one would naïvely expect. Thanks.

Show 7 quoted lines
> Yes, I think it is a reasonable addition to the streaming API. However,
> I do not think there are any callsites which would currently want it.
> All of the current users of stream_blob_to_fd use read_sha1_file as
> their alternative, and not parse_object. So we are not verifying the
> sha1 in either case (we may want to change that, of course, but that is
> a bigger decision than just trying to bring streaming and non-streaming
> code-paths into parity).

True. I am not offhand sure if we want to make read_sha1_file() to rehash, but I agree that it is a question different from what we are asking in this discussion.

Show 7 quoted lines
> I also wondered if parse_object itself had problems with double-reading
> or failing to verify. But its use goes the opposite direction; it wants
> to verify the sha1 of the blob object, but it knows that it does not
> actually need the data. So it streams (as of 090ea12) to check the
> signature, but then discards each buffer-full after hashing it.
>
> -Peff
Previous: Jeff KingNext: Jeff King
Message 27 of 28 in “corrupt object memory allocation error”
  1. Joey HessNov 20, 2013
  2. Jeff KingNov 20, 2013
  3. Joey HessNov 20, 2013
  4. drop support for "experimental" loose objectsJeff King, Nov 21, 2013
  5. Jeff KingNov 21, 2013
  6. Duy NguyenNov 21, 2013
  7. Keshav KiniNov 21, 2013
  8. Jeff KingNov 21, 2013
  9. Junio C HamanoNov 21, 2013
  10. Jonathan NiederNov 23, 2013
  11. Jeff KingNov 23, 2013
  12. Jonathan NiederNov 23, 2013
  13. Joey HessNov 21, 2013
  14. Christian CouderNov 21, 2013
  15. Jeff KingNov 22, 2013
  16. Christian CouderNov 22, 2013
  17. Jeff KingNov 22, 2013
  18. Christian CouderNov 22, 2013
  19. Jeff KingNov 22, 2013
  20. Junio C HamanoNov 22, 2013
  21. Jeff KingNov 22, 2013
  22. Joey HessNov 22, 2013
  23. Jeff KingNov 24, 2013
  24. Jeff KingNov 24, 2013
  25. Junio C HamanoNov 25, 2013
  26. Jeff KingNov 27, 2013
  27. Junio C HamanoNov 27, 2013
  28. Jeff KingNov 27, 2013

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.