git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: [RFC PATCH 2/4] sha1_file: extract type and size from object_info

From
JTJonathan Tan <jonathantanmy@google.com>
Date
Jun 12, 2017, 19:52 UTC
Message-ID
<20170612125254.33ba633a@twelve2.svl.corp.google.com>
In-Reply-To
<20170610070133.boicdvkudifz7b4c@sigill.intra.peff.net>

On Sat, 10 Jun 2017 03:01:33 -0400 Jeff King <peff@peff.net> wrote:

Show 20 quoted lines
> On Fri, Jun 09, 2017 at 12:23:24PM -0700, Jonathan Tan wrote:
> 
> > Looking at the 3 primary functions (sha1_object_info_extended,
> > read_object, has_sha1_file_with_flags), they independently implement
> > mechanisms such as object replacement, retrying the packed store after
> > failing to find the object in the packed store then the loose store, and
> > being able to mark a packed object as bad and then retrying the whole
> > process. Consolidating these mechanisms would be a great help to
> > maintainability.
> > 
> > Such a consolidated function would need to handle the read_object() case
> > (which returns the object data, type, and size) and the
> > sha1_object_info_extended() case (which returns the object type, size,
> > and some additional information, not all of which can be "turned off" by
> > passing NULL in "struct object_info").
> 
> I like the idea of consolidating the logic. But I can't help but feel
> that pulling these fields out of object_info is a step backwards. The
> point of that struct is to let the caller specify which aspects of the
> object they're interested in

My issue was that there are some parts that cannot be turned off (in particular, the object_info.u.packed part). Having said that, reading the packed object itself should give us enough information to populate that, so I'll take a look and see if this is possible.

Show 10 quoted lines
> Another approach to this whole mess is to have a single function for
> acquiring a "handle" to an object, along with functions to query aspects
> of a handle. That would let callers iteratively ask for the parts they
> care about, and we could lazily fill the handle info (so information we
> pick up while servicing one property of the object gets cached and
> returned for free if the caller asks for it later).
> 
> That's a much bigger change, though it may have other benefits (e.g., we
> could be passing around handles instead of object buffers, which would
> make it more natural to stream object content in many cases).

There are a few safeguards that, I think, only work with the current get-everything-then-forget-about-it approach (the packed-loose-packed retry mechanism, and the desperate retry-if-corrupt-packed-object one). If we have a handle with a cache, then, for example, we would lose the ability to retry packed after loose if the handle has already declared that the object is loose.

Previous: Jeff KingNext: Jeff King
Message 6 of 65 in “Improvements to sha1_file”
  1. 0/4 Improvements to sha1_fileJonathan Tan, Jun 9, 2017
  2. 1/4 sha1_file: teach packed_object_info about typenameJonathan Tan, Jun 9, 2017
  3. Junio C HamanoJun 12, 2017
  4. 2/4 sha1_file: extract type and size from object_infoJonathan Tan, Jun 9, 2017
  5. Jeff KingJun 10, 2017
  6. Jonathan TanJun 12, 2017
  7. Jeff KingJun 12, 2017
  8. 4/4 sha1_file, fsck: add missing blob supportJonathan Tan, Jun 9, 2017
  9. 3/4 sha1_file: consolidate storage-agnostic object fnsJonathan Tan, Jun 9, 2017
  10. 0/4 Improvements to sha1_fileJonathan Tan, Jun 13, 2017
  11. 1/4 sha1_file: teach packed_object_info about typenameJonathan Tan, Jun 13, 2017
  12. 2/4 sha1_file: move delta base cache code upJonathan Tan, Jun 13, 2017
  13. Junio C HamanoJun 15, 2017
  14. 3/4 sha1_file: consolidate storage-agnostic object fnsJonathan Tan, Jun 13, 2017
  15. Junio C HamanoJun 15, 2017
  16. Jonathan TanJun 15, 2017
  17. Jeff KingJun 17, 2017
  18. Junio C HamanoJun 19, 2017
  19. 4/4 sha1_file, fsck: add missing blob supportJonathan Tan, Jun 13, 2017
  20. Junio C HamanoJun 15, 2017
  21. Jonathan TanJun 15, 2017
  22. Junio C HamanoJun 15, 2017
  23. 0/8 Improvements to sha1_fileJonathan Tan, Jun 20, 2017
  24. 1/8 sha1_file: teach packed_object_info about typenameJonathan Tan, Jun 20, 2017
  25. 2/8 sha1_file: rename LOOKUP_UNKNOWN_OBJECTJonathan Tan, Jun 20, 2017
  26. Junio C HamanoJun 21, 2017
  27. Jonathan TanJun 21, 2017
  28. 3/8 sha1_file: rename LOOKUP_REPLACE_OBJECTJonathan Tan, Jun 20, 2017
  29. Junio C HamanoJun 21, 2017
  30. 4/8 sha1_file: move delta base cache code upJonathan Tan, Jun 20, 2017
  31. 5/8 sha1_file: refactor read_objectJonathan Tan, Jun 20, 2017
  32. Junio C HamanoJun 21, 2017
  33. 6/8 sha1_file: improve sha1_object_info_extendedJonathan Tan, Jun 20, 2017
  34. Jeff KingJun 24, 2017
  35. Jonathan TanJun 26, 2017
  36. Junio C HamanoJun 26, 2017
  37. Jonathan TanJun 26, 2017
  38. Junio C HamanoJun 26, 2017
  39. 8/8 sha1_file: refactor has_sha1_file_with_flagsJonathan Tan, Jun 20, 2017
  40. 7/8 sha1_file: do not access pack if unneededJonathan Tan, Jun 20, 2017
  41. Junio C HamanoJun 21, 2017
  42. Jeff KingJun 24, 2017
  43. Junio C HamanoJun 24, 2017
  44. Jeff KingJun 24, 2017
  45. Jonathan TanJun 26, 2017
  46. Junio C HamanoJun 21, 2017
  47. Jeff KingJun 24, 2017
  48. 0/8 Improvements to sha1_fileJonathan Tan, Jun 22, 2017
  49. 1/8 sha1_file: teach packed_object_info about typenameJonathan Tan, Jun 22, 2017
  50. 2/8 sha1_file: rename LOOKUP_UNKNOWN_OBJECTJonathan Tan, Jun 22, 2017
  51. 3/8 sha1_file: rename LOOKUP_REPLACE_OBJECTJonathan Tan, Jun 22, 2017
  52. 8/8 sha1_file: refactor has_sha1_file_with_flagsJonathan Tan, Jun 22, 2017
  53. Christian CouderJul 18, 2017
  54. Jonathan TanJul 18, 2017
  55. Johannes SchindelinJul 19, 2017
  56. sha1_file: use access(), not lstat(), if possibleJonathan Tan, Jul 19, 2017
  57. Junio C HamanoJul 20, 2017
  58. Johannes SchindelinJul 22, 2017
  59. Junio C HamanoJul 22, 2017
  60. Johannes SchindelinJul 25, 2017
  61. 6/8 sha1_file: improve sha1_object_info_extendedJonathan Tan, Jun 22, 2017
  62. 5/8 sha1_file: refactor read_objectJonathan Tan, Jun 22, 2017
  63. 4/8 sha1_file: move delta base cache code upJonathan Tan, Jun 22, 2017
  64. 7/8 sha1_file: do not access pack if unneededJonathan Tan, Jun 22, 2017
  65. Junio C HamanoJun 22, 2017

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.