git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: [PATCH 3/3] connected: search promisor objects generically

From
Christian Couder <christian.couder@gmail.com>
Date
Jun 23, 2026, 07:45 UTC
Message-ID
<CAP8UFD1tqBBRiJV18xBMcDDT4Q7xCkqOLrtJGAO7o4oA=-Vr=w@mail.gmail.com>
In-Reply-To
<20260622-pks-connected-generic-promisor-checks-v1-3-25eba2698202@pks.im>
On Mon, Jun 22, 2026 at 10:50 AM Patrick Steinhardt <ps@pks.im> wrote:
Show 59 quoted lines
>
> When performing connectivity checks we have to figure out whether any of
> the new objects are promisor objects, as we cannot assume full
> connectivity if so.
>
> This check is performed by iterating through all packfiles in the
> repository and searching each of them for the given object. Of course,
> this mechanism is quite specific to implementation details of the object
> database, as we assume that it uses packfiles in the first place.
>
> Refactor the logic so that we instead use `odb_for_each_object_ext()`
> with an object prefix filter and the `ODB_FOR_EACH_OBJECT_PROMISOR_ONLY`
> flag. This will yield all objects that have the exact object name and
> that are part of a promisor pack in a generic way.
>
> Signed-off-by: Patrick Steinhardt <ps@pks.im>
> ---
>  connected.c | 39 +++++++++++++++++++++++++--------------
>  1 file changed, 25 insertions(+), 14 deletions(-)
>
> diff --git a/connected.c b/connected.c
> index 7e26976832..9a666f0cdf 100644
> --- a/connected.c
> +++ b/connected.c
> @@ -11,6 +11,13 @@
>  #include "packfile.h"
>  #include "promisor-remote.h"
>
> +static int promised_object_cb(const struct object_id *oid UNUSED,
> +                             struct object_info *oi UNUSED,
> +                             void *payload UNUSED)
> +{
> +       return 1;
> +}
> +
>  /*
>   * If we feed all the commits we want to verify to this command
>   *
> @@ -46,6 +53,11 @@ int check_connected(oid_iterate_fn fn, void *cb_data,
>         }
>
>         if (repo_has_promisor_remote(the_repository)) {
> +               struct odb_for_each_object_options opts = {
> +                       .flags = ODB_FOR_EACH_OBJECT_PROMISOR_ONLY,
> +                       .prefix_hex_len = the_repository->hash_algo->hexsz,
> +               };
> +
>                 /*
>                  * For partial clones, we don't want to have to do a regular
>                  * connectivity check because we have to enumerate and exclude
> @@ -54,31 +66,30 @@ int check_connected(oid_iterate_fn fn, void *cb_data,
>                  * object is a promisor object. Instead, just make sure we
>                  * received, in a promisor packfile, the objects pointed to by
>                  * each wanted ref.
> -                *
> -                * Before checking for promisor packs, be sure we have the
> -                * latest pack-files loaded into memory.
>                  */
> -               odb_reprepare(the_repository->objects);

Like Junio, I am not sure it's correct to remove the `odb_reprepare(the_repository->objects)` call.

I think it was added for good reasons in b739d971 (connected.c: reprepare packs for corner cases, 2020-03-13) and I am not sure odb_for_each_object_ext() is performing something similar.

At least the commit message should mention this change and explain a bit why the reasons the call was added are not valid anymore.

Show 36 quoted lines
>                 do {
> -                       struct packed_git *p;
> -
> -                       repo_for_each_pack(the_repository, p) {
> -                               if (!p->pack_promisor)
> -                                       continue;
> -                               if (find_pack_entry_one(oid, p))
> -                                       goto promisor_pack_found;
> +                       opts.prefix = oid;
> +
> +                       err = odb_for_each_object_ext(the_repository->objects,
> +                                                     NULL, promised_object_cb,
> +                                                     NULL, &opts);
> +                       if (err < 0)
> +                               break;
> +                       if (err > 0) {
> +                               err = 0;
> +                               continue;
>                         }
> +
>                         /*
>                          * Fallback to rev-list with oid and the rest of the
>                          * object IDs provided by fn.
>                          */
>                         goto no_promisor_pack_found;
> -promisor_pack_found:
> -                       ;
>                 } while ((oid = fn(cb_data)) != NULL);
> +
>                 if (opt->err_fd)
>                         close(opt->err_fd);
> -               return 0;
> +               return err;
>         }
>
>  no_promisor_pack_found:

These changes are difficult to understand as there are a number of `goto`, `break`, `return`, etc involved.

I think it comes in the first place from check_connected() doing too many things, and adding a preparatory commit to refactor it would help.

For example the preparatory commit could move a lot of code from check_connected() to the following new functions:

/*
 * Returns:
 *   1  = all wanted OIDs found in promisor packs: connected, done.
 *   0  = at least one OID not found: caller must fall back to rev-list.
 *  <0  = error.
 * On the fallback (0) return, *oid is left pointing at the first
 * not-found OID so the rev-list path can resume the iteration.
 */
static int check_connected_promisor(oid_iterate_fn fn, void *cb_data,
                                     const struct object_id **oid);
/*
 * In a non-promisor repo, pass the first OID as `oid`.
 * Otherwise pass the first not-found OID resumed from
 * check_connected_promisor() as `oid`.
 */
static int check_connected_rev_list(oid_iterate_fn fn, void *cb_data,
                                     struct check_connected_options *opt,
                                     const struct object_id *oid);
Previous: Patrick SteinhardtNext: Patrick Steinhardt
Message 8 of 25 in “connected: search promisor objects generically”
  1. 0/3 connected: search promisor objects genericallyPatrick Steinhardt, Jun 22, 2026
  2. 1/3 odb/source-packed: extract logic to skip certain packsPatrick Steinhardt, Jun 22, 2026
  3. Junio C HamanoJun 22, 2026
  4. 2/3 odb/source-packed: support flags when iterating an object prefixPatrick Steinhardt, Jun 22, 2026
  5. 3/3 connected: search promisor objects genericallyPatrick Steinhardt, Jun 22, 2026
  6. Junio C HamanoJun 22, 2026
  7. Patrick SteinhardtJun 24, 2026
  8. Christian CouderJun 23, 2026
  9. Patrick SteinhardtJun 24, 2026
  10. 0/4 connected: search promisor objects genericallyPatrick Steinhardt, Jun 24, 2026
  11. 1/4 odb/source-packed: extract logic to skip certain packsPatrick Steinhardt, Jun 24, 2026
  12. 2/4 odb/source-packed: support flags when iterating an object prefixPatrick Steinhardt, Jun 24, 2026
  13. Christian CouderJun 24, 2026
  14. Patrick SteinhardtJun 25, 2026
  15. 3/4 connected: split out promisor-based connectivity checkPatrick Steinhardt, Jun 24, 2026
  16. 4/4 connected: search promisor objects genericallyPatrick Steinhardt, Jun 24, 2026
  17. Junio C HamanoJun 24, 2026
  18. Patrick SteinhardtJun 25, 2026
  19. 0/4 connected: search promisor objects genericallyPatrick Steinhardt, Jun 25, 2026
  20. 1/4 odb/source-packed: extract logic to skip certain packsPatrick Steinhardt, Jun 25, 2026
  21. 2/4 odb/source-packed: support flags when iterating an object prefixPatrick Steinhardt, Jun 25, 2026
  22. 3/4 connected: split out promisor-based connectivity checkPatrick Steinhardt, Jun 25, 2026
  23. 4/4 connected: search promisor objects genericallyPatrick Steinhardt, Jun 25, 2026
  24. Junio C HamanoJun 25, 2026
  25. Christian CouderJun 26, 2026

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.