git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: [PATCH 28/28] attr: convert to new threadsafe API

From
Stefan Beller <sbeller@google.com>
Date
Oct 11, 2016, 17:56 UTC
Message-ID
<CAGZ79kaQKDdJfMOjDKEK_dZJhgj+R7rByQS++B3OOBy6uO1x2w@mail.gmail.com>
In-Reply-To
<xmqqmviaaina.fsf@gitster.mtv.corp.google.com>
On Tue, Oct 11, 2016 at 10:40 AM, Junio C Hamano <gitster@pobox.com> wrote:
Show 51 quoted lines
> Stefan Beller <sbeller@google.com> writes:
>
>> This revamps the API of the attr subsystem to be thread safe.
>> Before we had the question and its results in one struct type.
>> The typical usage of the API was
>>
>>     static struct git_attr_check;
>
> I think you meant "*check" at the end, perhaps?
>
>>
>>     if (!check)
>>         check = git_attr_check_initl("text", NULL);
>>
>>     git_check_attr(path, check);
>>     act_on(check->value[0]);
>>
>> * While the check for attributes to be questioned only need to
>>   be initalized once as that part will be read only after its
>>   initialisation, the answer may be different for each path.
>>   Because of that we need to decouple the check and the answer,
>>   such that each thread can obtain an answer for the path it
>>   is currently processing.
>
> Yes, it is good to separate questions and answers.  I think answers
> should be owned by the caller, though.  I do not think of a good
> reason why you want to make it impossible to do something like this:
>
>         struct git_attr_check_result *result_1 = ...allocate...;
>         struct git_attr_check_result *result_2 = ...allocate...;
>
>         loop {
>                 struct strbuf path = next_path();
>                 git_check_attr(result_1, path.buf, check_1);
>                 if (strbuf_strip_suffix(&path, ".c")) {
>                         strbuf_addstr(&path, ".h");
>                         git_check_attr(result_2, path.buf, check_2);
>                         do something using result_1[] and result_2[];
>                 } else {
>                         do something using result_1[];
>                 }
>         }
>
> Do we already have a design of the "result" thing that is concrete
> enough to require us to declare that the result is owned by the
> implementation and asking another question has to destroy the answer
> to the previous question?  Otherwise, I'd rather not to see us make
> the API unnecessarily hard to use.  While I do want us to avoid
> overengineering for excessive flexibility, I somehow feel "you
> cannot control the lifetime of the result, it is owned by the
> subsystem" falls the other side of the line.

True, we had that issue for other APIs (IIRC path related things, with 4 static buffers that round robin). I did not like that design decision, but I felt it was okay, as the above did not occur to me.

In the case above, we could just copy the result_1->values and then re-use the result_1, but I agree that this may be somewhat error prone if you're not familiar with the decisions in this series.

So in case of the caller owning the result, we could pull the static trick for now and only use a different approach when we use it in actual threaded code, i.e. the code in convert.c could become:

    static struct git_attr_check *check;
    static struct git_attr_result *result;
    if (!check) {
        check = git_attr_check_initl("crlf", "ident",
            "filter", "eol", "text", NULL);
        result = git_attr_result_alloc(5);
        user_convert_tail = &user_convert;
        git_config(read_convert_config, NULL);
    }
    if (!git_check_attr(path, check, result)) {
        ...
>> +  empty `struct git_attr_check` as alloced by git_attr_check_alloc()
>
> "allocated", not "alloced".
ok.
Show 7 quoted lines
>
>> +  can be prepared by calling `git_attr_check_alloc()` function and
>> +  then attributes you want to ask about can be added to it with
>> +  `git_attr_check_append()` function.
>> +  git_attr_check_initl is thread safe, i.e. you can call it
>
> Spell it `git_attr_check_initl()` for consistency.
ok.
Show 33 quoted lines
>
>> +  from different threads at the same time; internally however only one
>> +  call at a time is processed. If the calls from different threads have
>> +  the same arguments, the returned `git_attr_check` may be the same.
>
> I find this description a bit confusing.  At least the way I
> envisioned was that when this piece of code is run by multiple
> people at the same time,
>
>         static struct git_attr_check *check = NULL;
>         git_attr_check_initl(&check, ...);
>
> we won't waste the "check" by allocated by the first one by
> overwriting it with another one allocated by the second one.  So
> "the same arguments" does not come into the picture.  A single
> variable is either
>
>  * already allocated and initialised by the an earlier call to
>    initl() by somebody else, or
>
>  * originally NULL when you call initl(), and the implementation
>    makes sure that other people wait while you allocate, initialise
>    and assign it to the variable, or
>
>  * originally NULL when you call initl(), but the implementation
>    notices that somebody else is doing the second bullet point
>    above, and you wait until that somebody else finishes and then
>    you return without doing anything (because by that time, "check"
>    is filled by that other party doing the second bullet point
>    above).
>
> There is no need to check for "the same arguments".
>

I see. So we assume that there are no different arguments at the same time, i.e. all threads run the same code when it comes to attrs.

Brandon wrote:
Show 5 quoted lines
> On 10/10, Stefan Beller wrote:
>>   be initalized once as that part will be read only after its
>>       initialized
>>   initialisation, the answer may be different for each path.
> should this be the US spelling 'initialization'?

Yes, we'd want to be consistent, indeed. Sometimes the British spelling slips through as that's what I learned in high school.

Specifically for initialise:
$ git grep initialise
contrib/examples/git-notes.sh:                  die "Will not
initialise with empty tree"
object.h: * it can return "yes we have, and here is a half-initialised object"
object.h: * half-initialised objects, the caller is expected to initialize them
revision.c:static struct treesame_state *initialise_treesame(struct
rev_info *revs, struct commit *commit)
revision.c:                             ts = initialise_treesame(revs, commit);
+ a lot of french translations.
The American spelling is found a lot more.

Thanks, Stefan

Previous: Junio C HamanoNext: Junio C Hamano
Message 35 of 44 in “Revamping the attr subsystem!”
  1. 00/28 Revamping the attr subsystem!Stefan Beller, Oct 11, 2016
  2. 01/28 commit.c: use strchrnul() to scan for one lineStefan Beller, Oct 11, 2016
  3. 02/28 attr.c: use strchrnul() to scan for one lineStefan Beller, Oct 11, 2016
  4. 03/28 attr.c: update a stale comment on "struct match_attr"Stefan Beller, Oct 11, 2016
  5. 07/28 attr.c: simplify macroexpand_one()Stefan Beller, Oct 11, 2016
  6. 05/28 attr.c: complete a sentence in a commentStefan Beller, Oct 11, 2016
  7. 04/28 attr.c: explain the lack of attr-name syntax check in parse_attr()Stefan Beller, Oct 11, 2016
  8. 06/28 attr.c: mark where #if DEBUG ends more clearlyStefan Beller, Oct 11, 2016
  9. 09/28 attr.c: plug small leak in parse_attr_line()Stefan Beller, Oct 11, 2016
  10. 08/28 attr.c: tighten constness around "git_attr" structureStefan Beller, Oct 11, 2016
  11. 10/28 attr: rename function and struct related to checking attributesStefan Beller, Oct 11, 2016
  12. 14/28 attr: retire git_check_attrs() APIStefan Beller, Oct 11, 2016
  13. 13/28 attr: convert git_check_attrs() callers to use the new APIStefan Beller, Oct 11, 2016
  14. 16/28 attr: add counted string version of git_attr()Stefan Beller, Oct 11, 2016
  15. 17/28 attr: expose validity check for attribute namesStefan Beller, Oct 11, 2016
  16. Brandon WilliamsOct 11, 2016
  17. Stefan BellerOct 11, 2016
  18. Brandon WilliamsOct 11, 2016
  19. Stefan BellerOct 11, 2016
  20. Brandon WilliamsOct 11, 2016
  21. 15/28 attr: add counted string version of git_check_attr()Stefan Beller, Oct 11, 2016
  22. 19/28 attr.c: add push_stack() helperStefan Beller, Oct 11, 2016
  23. 18/28 attr: support quoting pathname patterns in C styleStefan Beller, Oct 11, 2016
  24. 21/28 attr.c: rename a local variable checkStefan Beller, Oct 11, 2016
  25. 25/28 attr.c: outline the future plans by heavily commentingStefan Beller, Oct 11, 2016
  26. 20/28 attr.c: pass struct git_attr_check down the callchainStefan Beller, Oct 11, 2016
  27. 26/28 attr: make git_attr_counted staticStefan Beller, Oct 11, 2016
  28. Brandon WilliamsOct 11, 2016
  29. Stefan BellerOct 11, 2016
  30. 27/28 attr: make git_check_attr_counted staticStefan Beller, Oct 11, 2016
  31. 23/28 attr.c: introduce empty_attr_check_elems()Stefan Beller, Oct 11, 2016
  32. 28/28 attr: convert to new threadsafe APIStefan Beller, Oct 11, 2016
  33. Brandon WilliamsOct 11, 2016
  34. Junio C HamanoOct 11, 2016
  35. Stefan BellerOct 11, 2016
  36. Junio C HamanoOct 11, 2016
  37. Stefan BellerOct 11, 2016
  38. Junio C HamanoOct 11, 2016
  39. 24/28 attr.c: always pass check[] to collect_some_attrs()Stefan Beller, Oct 11, 2016
  40. 22/28 attr.c: correct ugly hack for git_all_attrs()Stefan Beller, Oct 11, 2016
  41. 11/28 attr: (re)introduce git_check_attr() and struct git_attr_checkStefan Beller, Oct 11, 2016
  42. Brandon WilliamsOct 11, 2016
  43. Junio C HamanoOct 11, 2016
  44. 12/28 attr: convert git_all_attrs() to use "struct git_attr_check"Stefan Beller, Oct 11, 2016

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.