git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: [PATCH 1/2] submodule--helper: initial clone learns retry logic

From
Stefan Beller <sbeller@google.com>
Date
Jun 9, 2016, 23:38 UTC
Message-ID
<CAGZ79ka1=nH5iYB_jbNm8gsVYcMiZHVnKg49kEUuEMiL9PaqXQ@mail.gmail.com>
In-Reply-To
<xmqqlh2egko4.fsf@gitster.mtv.corp.google.com>
On Thu, Jun 9, 2016 at 1:40 PM, Junio C Hamano <gitster@pobox.com> wrote:
Show 60 quoted lines
> Junio C Hamano <gitster@pobox.com> writes:
>
>> Stefan Beller <sbeller@google.com> writes:
>>
>>> instead. But that is still unspecified, so we rather go with
>>>
>>> static int compare_ce(const void *one, const void *two, void *cb_data)
>>> {
>>>     const struct cache_entry *ce_one = one, *ce_two = two;
>>>     return strcmp(ce_one->name, ce_two->name);
>>> }
>>
>> As I said below, I do not think it is worth addressing by making the
>> code's behaviour on real systems worse.  As long as what you have as
>> the key into priority queue is a pointer to cache_entry, you cannot
>> make it better from that point of view.
>
> ... because having to strcmp() their names would be much more
> expensive than the pointer comparison.
>
> However, I think you could use a pointer into a single array as
> an element of prio_queue.  I notice here:
>
>         for (; suc->current < suc->list.nr; suc->current++) {
> -               const struct cache_entry *ce = suc->list.entries[suc->current];
> +               ce = suc->list.entries[suc->current];
>                 if (prepare_to_clone_next_submodule(ce, child, suc, err)) {
> +                       *ce_task_cb = (struct cache_entry *) ce;
>                         suc->current++;
>                         return 1;
>                 }
>         }
>
> that list.entries[] can serve as such an array.  If you pass around
> the pointer to its element instead, i.e.
>
> -               ce = suc->list.entries[suc->current];
> +               ceP = &suc->list.entries[suc->current];
> -               if (prepare_to_clone_next_submodule(ce, child, suc, err)) {
> +               if (prepare_to_clone_next_submodule(*ceP, child, suc, err)) {
> +                       *ce_task_cb = (struct cache_entry *) ce;
> -                       *ce_task_cb = ceP;
>                 ...
>         }
>         /*
>          * The loop above tried cloning each submodule once,
>          * now try the stragglers again.
>          */
> -       ce = (struct cache_entry *) prio_queue_get(&suc->failed_queue);
> +       ceP = (struct cache_entry **) prio_queue_get(&suc->failed_queue);
>
> then the elements you are pushing into prio-queue would not be ce
> (pointer to a cache entry), but would be a pointer of an array that
> holds many pointers to cache entries, so it becomes kosher to
> compare them for ordering.
>
> That would probably not add too much overhead at runtime; it may
> have to involve a bit of code restructuring, so I do not know if it
> is worth it, though.
>

I realized the patch above was bogus for another reason: In update_clone_task_finished we never distinguish between first-failing submodules and repeated fails, such that we may end up in an infinite loop.

So I think the easiest way forward would be if we would pass around the index and then we can properly determine if we tried that already. (Though casting pointer to int is not possible, so we'll pass around a pointer to an int)

Previous: Junio C HamanoNext: Stefan Beller
Message 7 of 8 in “Dealing with a lot of submodules”
  1. 0/2 Dealing with a lot of submodulesStefan Beller, Jun 9, 2016
  2. 1/2 submodule--helper: initial clone learns retry logicStefan Beller, Jun 9, 2016
  3. Junio C HamanoJun 9, 2016
  4. Stefan BellerJun 9, 2016
  5. Junio C HamanoJun 9, 2016
  6. Junio C HamanoJun 9, 2016
  7. Stefan BellerJun 9, 2016
  8. 2/2 submodule update: continue when a clone failsStefan Beller, Jun 9, 2016

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.