git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: [PATCH v2 01/24] builtin/pack-objects: make hash agnostic

From
brian m. carlson <sandals@crustytoothpaste.net>
Date
Feb 24, 2020, 03:42 UTC
Message-ID
<20200224034259.GG6462@camp.crustytoothpaste.net>
In-Reply-To
<20200223215759.GC564691@coredump.intra.peff.net>
On 2020-02-23 at 21:57:59, Jeff King wrote:
Show 9 quoted lines
> On Sat, Feb 22, 2020 at 08:17:26PM +0000, brian m. carlson wrote:
> 
> > Avoid hard-coding a hash size, instead preferring to use the_hash_algo.
> > [...]
> > -			hashwrite(out, base_sha1, 20);
> > +			hashwrite(out, base_sha1, the_hash_algo->rawsz);
> 
> Yeah, looks obviously correct (and I think this is new from the
> pack-reuse patches of mine that Christian sent recently).
I believe it is, which is why I CC'd you on it.
Show 5 quoted lines
> The name "base_sha1" is clearly not great either. It could be changed
> trivially, but the real sin is that it comes from
> nth_packed_object_sha1(). It could be replaced with
> nth_packed_object_oid() at the cost of an extra hash copy, which isn't
> too bad.

I probably should send a series getting rid of the rest of the "sha1" places in our code, because there are a lot of them, but I just haven't gotten around to it yet. And yeah, as you mentioned, there are still a lot of places using nth_packed_object_sha1.

Show 9 quoted lines
> It would be nice to get rid of that function entirely. In most cases,
> it's either free to do so (we end up copying the result into an oid
> anyway) or we pay for one extra hashcpy (out of the mmap into a local
> struct). But the one in pack-check.c:verify_packfile() is tricky; we
> store a pointer per object into the index mmap, and we'd have to switch
> to storing an oid per object. Given that the code isn't commonly run
> (and other operations like _generating_ the index in the first place are
> clearly OK with the extra memory hit), I think I'd be OK with that in
> the name of cleanliness.
Yeah, that's why I hadn't switched it out earlier.
Show 5 quoted lines
> I think it would also be correct to cast the mmap'd bytes to a "struct
> object_id" given that the struct contains the hash bytes as the first
> member. I worry a little that we'd get no compiler warning of the
> breakage if that assumption changes, but it also seems unlikely to do
> so.

In the future, struct object_id will get a new member (an algorithm value), but I think it's fine to make the assumption that the hash bytes are first.

-- 
brian m. carlson: Houston, Texas, US
OpenPGP: https://keybase.io/bk2204
Previous: Jeff KingNext: Jeff King
Message 5 of 51 in “SHA-256 stage 4 implementation, part 1/3”
  1. 00/24 SHA-256 stage 4 implementation, part 1/3brian m. carlson, Feb 22, 2020
  2. 01/24 builtin/pack-objects: make hash agnosticbrian m. carlson, Feb 22, 2020
  3. Jeff KingFeb 23, 2020
  4. Jeff KingFeb 24, 2020
  5. brian m. carlsonFeb 24, 2020
  6. Jeff KingFeb 24, 2020
  7. 03/24 hex: introduce parsing variants taking hash algorithmsbrian m. carlson, Feb 22, 2020
  8. 07/24 t6300: abstract away SHA-1-specific constantsbrian m. carlson, Feb 22, 2020
  9. Junio C HamanoFeb 24, 2020
  10. Jeff KingFeb 24, 2020
  11. Junio C HamanoFeb 24, 2020
  12. 12/24 setup: allow check_repository_format to read repository formatbrian m. carlson, Feb 22, 2020
  13. 11/24 t/helper: make repository tests hash independentbrian m. carlson, Feb 22, 2020
  14. 14/24 builtin/init-db: add environment variable for new repo hashbrian m. carlson, Feb 22, 2020
  15. 06/24 t: use hash-specific lookup tables to define test constantsbrian m. carlson, Feb 22, 2020
  16. 10/24 t/helper: initialize repository if necessarybrian m. carlson, Feb 22, 2020
  17. Junio C HamanoFeb 24, 2020
  18. brian m. carlsonFeb 25, 2020
  19. 22/24 fast-import: make find_marks work on any mark setbrian m. carlson, Feb 22, 2020
  20. 19/24 tag: store SHA-256 signatures in a headerbrian m. carlson, Feb 22, 2020
  21. Junio C HamanoFeb 24, 2020
  22. 23/24 fast-import: add a generic function to iterate over marksbrian m. carlson, Feb 22, 2020
  23. 13/24 builtin/init-db: allow specifying hash algorithm on command linebrian m. carlson, Feb 22, 2020
  24. Junio C HamanoFeb 24, 2020
  25. brian m. carlsonFeb 25, 2020
  26. 17/24 commit: use expected signature header for SHA-256brian m. carlson, Feb 22, 2020
  27. Junio C HamanoFeb 24, 2020
  28. 18/24 gpg-interface: improve interface for parsing tagsbrian m. carlson, Feb 22, 2020
  29. Junio C HamanoFeb 24, 2020
  30. Johannes SchindelinFeb 25, 2020
  31. Junio C HamanoFeb 25, 2020
  32. brian m. carlsonFeb 26, 2020
  33. Junio C HamanoFeb 26, 2020
  34. brian m. carlsonFeb 26, 2020
  35. Johannes SchindelinFeb 27, 2020
  36. Junio C HamanoFeb 27, 2020
  37. 20/24 fast-import: permit reading multiple marks filesbrian m. carlson, Feb 22, 2020
  38. Junio C HamanoJun 5, 2020
  39. Junio C HamanoJun 5, 2020
  40. brian m. carlsonJun 5, 2020
  41. Junio C HamanoJun 5, 2020
  42. 24/24 fast-import: add options for rewriting submodulesbrian m. carlson, Feb 22, 2020
  43. 21/24 fast-import: add helper function for inserting mark object entriesbrian m. carlson, Feb 22, 2020
  44. 04/24 hex: add functions to parse hex object IDs in any algorithmbrian m. carlson, Feb 22, 2020
  45. 16/24 worktree: allow repository version 1brian m. carlson, Feb 22, 2020
  46. 05/24 repository: require a build flag to use SHA-256brian m. carlson, Feb 22, 2020
  47. 02/24 hash: implement and use a context cloning functionbrian m. carlson, Feb 22, 2020
  48. 15/24 init-db: move writing repo version into a functionbrian m. carlson, Feb 22, 2020
  49. 08/24 t6300: make hash algorithm independentbrian m. carlson, Feb 22, 2020
  50. 09/24 t/helper/test-dump-split-index: initialize git repositorybrian m. carlson, Feb 22, 2020
  51. Junio C HamanoFeb 24, 2020

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.