git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: [PATCH 8/9] sha1-file: use loose object cache for quick existence check

From
Jeff King <peff@peff.net>
Date
Dec 5, 2018, 20:17 UTC
Message-ID
<20181205201728.GA19936@sigill.intra.peff.net>
In-Reply-To
<48ab861b-46c8-2bd5-0508-fa106efd8e56@web.de>
On Wed, Dec 05, 2018 at 07:41:44PM +0100, René Scharfe wrote:
Show 6 quoted lines
> > If we start to convert those, there's a
> > little bit of a rabbit hole, but it's actually not too bad.
> 
> You don't need to crawl in just for quick_has_loose(), but eventually
> everything has to be converted.  It seems a bit much for one patch, but
> perhaps that's just my ever-decreasing attention span speaking.

Yeah, my normal process here is to dig to the bottom of the rabbit hole, and then break it into actual patches. I just shared the middle state here. ;)

I suspect the http bits could be split off into their own thing. The bits in sha1-file.c I'd plan to mostly do all together, as they are a family of related functions.

Mostly I wasn't sure how to wrap this up with the other changes. It's obviously pretty invasive, and I don't want it to get in the way of actual functional changes we've been discussing.

Show 11 quoted lines
> > @@ -879,15 +879,15 @@ int git_open_cloexec(const char *name, int flags)
> >   * Note that it may point to static storage and is only valid until another
> >   * call to stat_sha1_file().
> >   */
> > -static int stat_sha1_file(struct repository *r, const unsigned char *sha1,
> > -			  struct stat *st, const char **path)
> > +static int stat_loose_object(struct repository *r, const struct object_id *oid,
> > +			     struct stat *st, const char **path)
> 
> Hmm, read_sha1_file() was renamed to read_object_file() earlier this
> year, and I'd have expected this to become stat_object_file().  Names..

read_object_file() is about reading an object from _any_ source. These are specifically about loose objects, and I think that distinction is important (both here and for map_loose_object, etc).

I'd actually argue that read_object_file() should just be read_object(), but that already exists. Sigh. (I think it's fixable, but obviously orthogonal to this topic).

> Anyway, the comment above and one a few lines below should be updated
> as well.
Thanks, fixed.
Show 5 quoted lines
> >  static int open_sha1_file(struct repository *r,
> > -			  const unsigned char *sha1, const char **path)
> > +			  const struct object_id *oid, const char **path)
> 
> That function should lose the "sha1" in its name as well.
Yep, fixed.
Show 6 quoted lines
> > -static void *unpack_sha1_rest(git_zstream *stream, void *buffer, unsigned long size, const unsigned char *sha1)
> > +static void *unpack_loose_rest(git_zstream *stream,
> > +			       void *buffer, unsigned long size,
> > +			       const struct object_id *oid)
> 
> Hmm, both old and new name here look weird to me at this point.

It makes more sense in the pairing of unpack_sha1_header() and unpack_sha1_rest(). Maybe "body" would be better than "rest".

At any rate, it probably makes sense to rename them together (but I didn't touch the "header" one here). Maybe the name changes should come as a separate patch. I was mostly changing them here because I was changing the signatures anyway, and had to touch all of the callers.

-Peff
Previous: René ScharfeNext: Geert Jansen
Message 73 of 99 in “index-pack: improve performance on NFS”
  1. index-pack: improve performance on NFSJansen, Geert, Oct 25, 2018
  2. Junio C HamanoOct 26, 2018
  3. Ævar Arnfjörð BjarmasonOct 26, 2018
  4. Junio C HamanoOct 27, 2018
  5. Jeff KingOct 27, 2018
  6. Ævar Arnfjörð BjarmasonOct 27, 2018
  7. 0/4 index-pack: optionally turn off SHA-1 collision checkingÆvar Arnfjörð Bjarmason, Oct 28, 2018
  8. Geert JansenOct 30, 2018
  9. Junio C HamanoOct 30, 2018
  10. 0/3 index-pack: test updatesÆvar Arnfjörð Bjarmason, Oct 30, 2018
  11. index-pack: add ability to disable SHA-1 collision checkÆvar Arnfjörð Bjarmason, Nov 13, 2018
  12. Junio C HamanoNov 14, 2018
  13. Ævar Arnfjörð BjarmasonNov 14, 2018
  14. 1/3 pack-objects test: modernize styleÆvar Arnfjörð Bjarmason, Oct 30, 2018
  15. 2/3 pack-objects tests: don't leave test .git corrupt at endÆvar Arnfjörð Bjarmason, Oct 30, 2018
  16. 3/3 index-pack tests: don't leave test repo dirty at endÆvar Arnfjörð Bjarmason, Oct 30, 2018
  17. 1/4 pack-objects test: modernize styleÆvar Arnfjörð Bjarmason, Oct 28, 2018
  18. 2/4 pack-objects tests: don't leave test .git corrupt at endÆvar Arnfjörð Bjarmason, Oct 28, 2018
  19. 3/4 index-pack tests: don't leave test repo dirty at endÆvar Arnfjörð Bjarmason, Oct 28, 2018
  20. 4/4 index-pack: add ability to disable SHA-1 collision checkÆvar Arnfjörð Bjarmason, Oct 28, 2018
  21. Jeff KingOct 29, 2018
  22. Jeff KingOct 29, 2018
  23. Ævar Arnfjörð BjarmasonOct 29, 2018
  24. Jeff KingOct 29, 2018
  25. Geert JansenNov 7, 2018
  26. Jeff KingNov 8, 2018
  27. Geert JansenNov 8, 2018
  28. Jeff KingNov 8, 2018
  29. Geert JansenNov 8, 2018
  30. Ævar Arnfjörð BjarmasonNov 8, 2018
  31. Ævar Arnfjörð BjarmasonNov 9, 2018
  32. Jeff KingNov 12, 2018
  33. 0/9 caching loose objectsJeff King, Nov 12, 2018
  34. 1/9 fsck: do not reuse child_process structsJeff King, Nov 12, 2018
  35. Derrick StoleeNov 12, 2018
  36. 2/9 submodule--helper: prefer strip_suffix() to ends_with()Jeff King, Nov 12, 2018
  37. Stefan BellerNov 12, 2018
  38. 3/9 rename "alternate_object_database" to "object_directory"Jeff King, Nov 12, 2018
  39. Derrick StoleeNov 12, 2018
  40. Jeff KingNov 12, 2018
  41. Ramsay JonesNov 12, 2018
  42. 4/9 sha1_file_name(): overwrite buffer instead of appendingJeff King, Nov 12, 2018
  43. Derrick StoleeNov 12, 2018
  44. 5/9 handle alternates paths the same as the main object dirJeff King, Nov 12, 2018
  45. Derrick StoleeNov 12, 2018
  46. Jeff KingNov 12, 2018
  47. Derrick StoleeNov 12, 2018
  48. 6/9 sha1-file: use an object_directory for the main object dirJeff King, Nov 12, 2018
  49. Derrick StoleeNov 12, 2018
  50. Jeff KingNov 12, 2018
  51. Stefan BellerNov 12, 2018
  52. Jeff KingNov 22, 2018
  53. Stefan BellerNov 12, 2018
  54. 7/9 object-store: provide helpers for loose_objects_cacheJeff King, Nov 12, 2018
  55. René ScharfeNov 12, 2018
  56. Jeff KingNov 12, 2018
  57. 8/9 sha1-file: use loose object cache for quick existence checkJeff King, Nov 12, 2018
  58. Derrick StoleeNov 12, 2018
  59. Ævar Arnfjörð BjarmasonNov 12, 2018
  60. Jeff KingNov 12, 2018
  61. Ævar Arnfjörð BjarmasonNov 12, 2018
  62. Ævar Arnfjörð BjarmasonNov 12, 2018
  63. Ævar Arnfjörð BjarmasonNov 13, 2018
  64. René ScharfeNov 14, 2018
  65. René ScharfeDec 2, 2018
  66. Jeff KingDec 3, 2018
  67. René ScharfeDec 4, 2018
  68. Jeff KingDec 5, 2018
  69. René ScharfeDec 5, 2018
  70. Jeff KingDec 5, 2018
  71. Jeff KingDec 5, 2018
  72. René ScharfeDec 5, 2018
  73. Jeff KingDec 5, 2018
  74. Geert JansenNov 12, 2018
  75. René ScharfeNov 27, 2018
  76. Jeff KingDec 1, 2018
  77. 9/9 fetch-pack: drop custom loose object cacheJeff King, Nov 12, 2018
  78. René ScharfeNov 12, 2018
  79. Ævar Arnfjörð BjarmasonNov 12, 2018
  80. Jeff KingNov 12, 2018
  81. René ScharfeNov 12, 2018
  82. Derrick StoleeNov 12, 2018
  83. Stefan BellerNov 12, 2018
  84. Ævar Arnfjörð BjarmasonNov 9, 2018
  85. Duy NguyenNov 9, 2018
  86. Ævar Arnfjörð BjarmasonNov 10, 2018
  87. Jeff KingNov 12, 2018
  88. Geert JansenNov 12, 2018
  89. Duy NguyenOct 27, 2018
  90. Jeff KingOct 29, 2018
  91. Junio C HamanoOct 29, 2018
  92. Jeff KingOct 29, 2018
  93. Ævar Arnfjörð BjarmasonOct 29, 2018
  94. Geert JansenOct 29, 2018
  95. Jeff KingOct 29, 2018
  96. Geert JansenOct 29, 2018
  97. Jeff KingOct 29, 2018
  98. Stefan BellerOct 29, 2018
  99. Jeff KingOct 29, 2018

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.