Re: [PATCH] Add fetch.recurseSubmoduleParallelism config option

From: Junio C Hamano <gitster@pobox.com>
Date: Oct 12, 2015, 23:50 UTC
Message-ID: <xmqqa8rnbq9k.fsf@gitster.mtv.corp.google.com>
In-Reply-To: <CAGZ79kZuZZivs8czV2P6uHWaU6ay1hG21k-_G9tgN5KbV6jW8w@mail.gmail.com>

Stefan Beller <sbeller@google.com> writes:

> There is core.preloadIndex to enable parallel index preload, but
> that is boolean and not giving fine control to the user. We want to give
> fine control to the user here I'd assume.

I'd approach this as "fetching multiple submodules at a time", if I were deciding its name.

> ... We could also make it a
> submodule specifc thing (submodule.jobs), but that would collide
> with submodule.<name>.<foo> maybe?

I do not think so.  You can have

	area.attr1
	area.attr3
        area."userthing1".attr1
        area."userthing2".attr1

and the parser can differenciate them just fine.

So if you want

    [submodule]
        fetchParallel = 16
	updateParallel = 4

I do not think that would interfere with any

    [submodule "name"]
    	var = val

You can choose to even allow an attribute that is fundamentally per "userthing" (e.g. the branch, the remote, the submodule) defined with area."userthing".attr, but make area.attr to be the fallback value for unspecified area."userthing9".attr (I think http.*.* hierarchy takes that approach), but I do not think the parallelism of fetching is something that should be specified per submodule.

Show 7 quoted lines

>> The parallel_process API could learn a new "verbose" feature that it
>> by itself shows some messages like
>>
>>     "processing the 'frotz' job with N tasks"
>>     "M tasks finished (N still running)"
>
> I know what to fill in for M and N, 'frotz' is a bit unclear to me.

The caller would pass the label to pp_init(); in this codepath perhaps it will say 'submodule fetch' or something.