git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: [PATCH v2 02/13] pack-objects: add --path-walk option

From
Derrick Stolee <stolee@gmail.com>
Date
May 16, 2025, 15:27 UTC
Message-ID
<6087094c-2639-4262-8026-89e4d8aa8c8f@gmail.com>
In-Reply-To
<052c534a-c2a4-4ff6-9130-4c1e96e1ef72@gmail.com>
On 5/6/25 3:39 PM, Derrick Stolee wrote:
> On 5/2/25 7:21 PM, Taylor Blau wrote:
>> On Mon, Mar 24, 2025 at 03:22:38PM +0000, Derrick Stolee via GitGitGadget wrote:
Show 16 quoted lines
>>> +static int add_objects_by_path(const char *path,
>>> +                   struct oid_array *oids,
>>> +                   enum object_type type,
>>> +                   void *data)
>>> +{
>>> +    struct object_entry **delta_list;
>>> +    size_t oe_start = to_pack.nr_objects;
>>> +    size_t oe_end;
>>> +    unsigned int sub_list_size;
>>
>> Could you help me understand why sub_list_size is an unsigned int and
>> not a size_t here?
> 
> This is based on a similar "list_size" used in ll_find_deltas(), but
> since we are starting from scratch here it would be better to use
> size_t.

Sorry for the delay, but the find_deltas() method called in both places expects unsigned ints, so we are stuck with this until we decide to upgrade it in both places.

Show 39 quoted lines
>>> +    unsigned int *processed = data;
>>> +
>>> +    /*
>>> +     * First, add all objects to the packing data, including the ones
>>> +     * marked UNINTERESTING (translated to 'exclude') as they can be
>>> +     * used as delta bases.
>>> +     */
>>> +    for (size_t i = 0; i < oids->nr; i++) {
>>> +        int exclude;
>>> +        struct object_info oi = OBJECT_INFO_INIT;
>>> +        struct object_id *oid = &oids->oid[i];
>>> +
>>> +        /* Skip objects that do not exist locally. */
>>> +        if (exclude_promisor_objects &&
>>> +            oid_object_info_extended(the_repository, oid, &oi,
>>> +                         OBJECT_INFO_FOR_PREFETCH) < 0)
>>> +            continue;
>>> +
>>> +        exclude = !is_oid_interesting(the_repository, oid);
>>> +
>>> +        if (exclude && !thin)
>>> +            continue;
>>
>> ...here we only skip calling add_object_entry() if the object is
>> excluded (which lookup_object() returning NULL would cause above) *and*
>> we're not using thin packs. That makes sense, since if we have a missing
>> object, but we're using thin packs, then it's OK to skip.
>>
>> So I think all of this looks good. However, I find the interface here a
>> little awkward. The function is called "is_oid_interesting()", but it
>> determines that fact by checking whether or not the object has the
>> UNINTERSTING bit set on its flag, and then negating the result. But then
>> we negate it again here at the caller to obtain whether or not we should
>> "exclude" the object.
>>
>> I was wondering if there were other callers of is_oid_interesting() that
>> don't negate its result. But it doesn't look like there are (at least in
>> this patch). I wonder if it would be cleaner just to inline this instead
>> of having the result negated at multiple levels.

Double-checked now and there seem to be no other callers, so this will be renamed to the opposite case.

Show 16 quoted lines
>>> +    return 0;
>>> +}
>>> +
>>> +static void get_object_list_path_walk(struct rev_info *revs)
>>> +{
>>> +    struct path_walk_info info = PATH_WALK_INFO_INIT;
>>> +    unsigned int processed = 0;
>>> +
>>> +    info.revs = revs;
>>> +    info.path_fn = add_objects_by_path;
>>> +    info.path_fn_data = &processed;
>>> +    revs->tag_objects = 1;
>>
>> What is the purpose of setting revs->tag_objects to 1 here? Do we need
>> to restore it back to its original value before returning? A comment
>> here would be useful, I think.
I think this is not necessary and should be removed.

Thanks, -Stolee

Previous: Derrick StoleeNext: Derrick Stolee via GitGitGadget
Message 31 of 75 in “PATH WALK II: Add --path-walk option to 'git pack-objects'”
  1. 00/13 PATH WALK II: Add --path-walk option to 'git pack-objects'Derrick Stolee via GitGitGadget, Mar 10, 2025
  2. 01/13 pack-objects: extract should_attempt_deltas()Derrick Stolee via GitGitGadget, Mar 10, 2025
  3. Taylor BlauMar 12, 2025
  4. Derrick StoleeMar 20, 2025
  5. 03/13 pack-objects: update usage to match docsDerrick Stolee via GitGitGadget, Mar 10, 2025
  6. Taylor BlauMar 12, 2025
  7. 02/13 pack-objects: add --path-walk optionDerrick Stolee via GitGitGadget, Mar 10, 2025
  8. Taylor BlauMar 12, 2025
  9. Derrick StoleeMar 20, 2025
  10. 04/13 p5313: add performance tests for --path-walkDerrick Stolee via GitGitGadget, Mar 10, 2025
  11. 05/13 pack-objects: introduce GIT_TEST_PACK_PATH_WALKDerrick Stolee via GitGitGadget, Mar 10, 2025
  12. 06/13 t5538: add tests to confirm deltas in shallow pushesDerrick Stolee via GitGitGadget, Mar 10, 2025
  13. 07/13 repack: add --path-walk optionDerrick Stolee via GitGitGadget, Mar 10, 2025
  14. 08/13 pack-objects: enable --path-walk via configDerrick Stolee via GitGitGadget, Mar 10, 2025
  15. 09/13 scalar: enable path-walk during push via configDerrick Stolee via GitGitGadget, Mar 10, 2025
  16. 10/13 pack-objects: refactor path-walk delta phaseDerrick Stolee via GitGitGadget, Mar 10, 2025
  17. Taylor BlauMar 12, 2025
  18. Derrick StoleeMar 20, 2025
  19. 11/13 pack-objects: thread the path-based compressionDerrick Stolee via GitGitGadget, Mar 10, 2025
  20. 12/13 path-walk: add new 'edge_aggressive' optionDerrick Stolee via GitGitGadget, Mar 10, 2025
  21. 13/13 pack-objects: allow --shallow and --path-walkDerrick Stolee via GitGitGadget, Mar 10, 2025
  22. Junio C HamanoMar 10, 2025
  23. Taylor BlauMar 12, 2025
  24. Derrick StoleeMar 20, 2025
  25. 00/13 PATH WALK II: Add --path-walk option to 'git pack-objects'Derrick Stolee via GitGitGadget, Mar 24, 2025
  26. 01/13 pack-objects: extract should_attempt_deltas()Derrick Stolee via GitGitGadget, Mar 24, 2025
  27. Taylor BlauMay 2, 2025
  28. 02/13 pack-objects: add --path-walk optionDerrick Stolee via GitGitGadget, Mar 24, 2025
  29. Taylor BlauMay 2, 2025
  30. Derrick StoleeMay 6, 2025
  31. Derrick StoleeMay 16, 2025
  32. 03/13 pack-objects: update usage to match docsDerrick Stolee via GitGitGadget, Mar 24, 2025
  33. 04/13 p5313: add performance tests for --path-walkDerrick Stolee via GitGitGadget, Mar 24, 2025
  34. Taylor BlauMay 2, 2025
  35. 06/13 t5538: add tests to confirm deltas in shallow pushesDerrick Stolee via GitGitGadget, Mar 24, 2025
  36. Taylor BlauMay 2, 2025
  37. Derrick StoleeMay 16, 2025
  38. 05/13 pack-objects: introduce GIT_TEST_PACK_PATH_WALKDerrick Stolee via GitGitGadget, Mar 24, 2025
  39. Taylor BlauMay 2, 2025
  40. Derrick StoleeMay 6, 2025
  41. 07/13 repack: add --path-walk optionDerrick Stolee via GitGitGadget, Mar 24, 2025
  42. Taylor BlauMay 2, 2025
  43. 08/13 pack-objects: enable --path-walk via configDerrick Stolee via GitGitGadget, Mar 24, 2025
  44. Taylor BlauMay 2, 2025
  45. Derrick StoleeMay 6, 2025
  46. Derrick StoleeMay 16, 2025
  47. 09/13 scalar: enable path-walk during push via configDerrick Stolee via GitGitGadget, Mar 24, 2025
  48. Taylor BlauMay 7, 2025
  49. 10/13 pack-objects: refactor path-walk delta phaseDerrick Stolee via GitGitGadget, Mar 24, 2025
  50. Taylor BlauMay 7, 2025
  51. Derrick StoleeMay 16, 2025
  52. Taylor BlauMay 29, 2025
  53. 12/13 path-walk: add new 'edge_aggressive' optionDerrick Stolee via GitGitGadget, Mar 24, 2025
  54. 11/13 pack-objects: thread the path-based compressionDerrick Stolee via GitGitGadget, Mar 24, 2025
  55. Taylor BlauMay 7, 2025
  56. 13/13 pack-objects: allow --shallow and --path-walkDerrick Stolee via GitGitGadget, Mar 24, 2025
  57. Junio C HamanoMay 2, 2025
  58. Taylor BlauMay 2, 2025
  59. Taylor BlauMay 2, 2025
  60. Taylor BlauMay 7, 2025
  61. 00/13 PATH WALK II: Add --path-walk option to 'git pack-objects'Derrick Stolee via GitGitGadget, May 16, 2025
  62. 01/13 pack-objects: extract should_attempt_deltas()Derrick Stolee via GitGitGadget, May 16, 2025
  63. 02/13 pack-objects: add --path-walk optionDerrick Stolee via GitGitGadget, May 16, 2025
  64. 03/13 pack-objects: update usage to match docsDerrick Stolee via GitGitGadget, May 16, 2025
  65. 04/13 p5313: add performance tests for --path-walkDerrick Stolee via GitGitGadget, May 16, 2025
  66. 05/13 pack-objects: introduce GIT_TEST_PACK_PATH_WALKDerrick Stolee via GitGitGadget, May 16, 2025
  67. 06/13 t5538: add tests to confirm deltas in shallow pushesDerrick Stolee via GitGitGadget, May 16, 2025
  68. 07/13 repack: add --path-walk optionDerrick Stolee via GitGitGadget, May 16, 2025
  69. 08/13 pack-objects: enable --path-walk via configDerrick Stolee via GitGitGadget, May 16, 2025
  70. 09/13 scalar: enable path-walk during push via configDerrick Stolee via GitGitGadget, May 16, 2025
  71. 10/13 pack-objects: refactor path-walk delta phaseDerrick Stolee via GitGitGadget, May 16, 2025
  72. 11/13 pack-objects: thread the path-based compressionDerrick Stolee via GitGitGadget, May 16, 2025
  73. 12/13 path-walk: add new 'edge_aggressive' optionDerrick Stolee via GitGitGadget, May 16, 2025
  74. 13/13 pack-objects: allow --shallow and --path-walkDerrick Stolee via GitGitGadget, May 16, 2025
  75. Taylor BlauMay 29, 2025

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.