git/list[1] front-page[2] threads[3] people[4] search[5] about
 

[RFC PATCH 0/5] bypass config.c global state with configset

From
Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Date
Mar 17, 2023, 05:01 UTC
Message-ID
<RFC-cover-0.5-00000000000-20230317T042408Z-avarab@gmail.com>
In-Reply-To
<pull.1463.v2.git.git.1678925506.gitgitgadget@gmail.com>
On Thu, Mar 16 2023, Glen Choo via GitGitGadget wrote:
Show 6 quoted lines
> After reflecting on Ævar's responses on v1, I'm fairly convinced that
> "struct config_reader" shouldn't exist in the long term. I've written my
> thoughts on a good long term direction in the "Leftover bits" section. Based
> on that, I've also updated my WIP libification patches [1] to remove "struct
> config_reader" from the library interface, and think it looks a lot better
> as a result.

That libification url (https://github.com/git/git/compare/master...chooglen:git:config-lib-parsing) doesn't work for me, and I didn't find a branch with that name in your published repo. So maybe you've already done all this post-libification work...

Show 19 quoted lines
> = Leftover bits
>
> We still need a global "the_reader" because config callbacks are reading
> auxiliary information about the config (e.g. line number, file name) via
> global functions (e.g. current_config_line(), current_config_name()). This
> is either because the callback uses this info directly (like
> builtin/config.c printing the filename and scope of the value) or for error
> reporting (like git_parse_int() reporting the filename of the value it
> failed to parse).
>
> If we had a way to plumb the state from "struct config_reader" to the config
> callback functions, we could initialize "struct config_reader" in the config
> machinery whenever we read config (instead of asking the caller to
> initialize "struct config_reader" themselves), and config reading could
> become a thread-safe operation. There isn't an obvious way to plumb this
> state to config callbacks without adding an additional arg to config_fn_t
> and incurring a lot of churn, but if we start replacing "config_fn_t" with
> the configset API (which we've independently wanted for some time), this may
> become feasible.

...in any case. This RFC expands a bit on my comments on the v1 (at [1] and upthread). It doesn't get all the way there, but with the small change in 5/5 we've gotten rid of current_config_line(), the 1-4/5 are trivial pre-refactorings to make that diff smaller (e.g. moving the "struct key_value_info" around in config.h).

Maybe it still makes sense to go for this "the_reader" intermediate step, but I can't help but think that we could just go for it all in one leap, and that you've just got stuck on thinking that you needed to change "config_fn_t" for all its callers.

As the 5/5 here shows we have various orthagonal uses of the "config_fn_t" in config.c, and can just implement a new callback type for the edge cases where we need the file & line info.

This still leave the current_config_name() etc, which e.g. builtin/config.c still uses. In your series you've needed to add the new "reader" parameter for everything from do_config_from(), but if we're doing that can't we instead just go straight to passing a "struct key_value_info *" (perhaps with an added "name" field) all the way down, replacing "cf->linenr" etc?

Instead you end up extending "the_reader" everywhere, including to e.g. configset_iter, which I think as the 5/5 here shows isn't needed, but maybe I've missed something.

Similarly, you mention git_parse_int() wanting to report a filename and/or line number. I'm aware that it can do that, but it doesn't do so in the common case, e.g.:

	git -c format.filenameMaxLength=abc log
	fatal: bad numeric config value 'abc' for 'format.filenamemaxlength': invalid unit

And the same goes for writing it to e.g. ~/.gitconfig. It's only if you use "git config --file" or similar that we'll report a filename.

So just as with the current_config_line() I wonder if you just grepped for e.g. git_config_int() and thought because we have a lot of users of it that all of them would require this data, but for e.g. this log.c caller (and most or all of the others) we'll be reading the normal config, and aren't getting any useful info from die_bad_number() that we wouldn't get from an error function that didn't need the "linenr" etc.

Show 9 quoted lines
> And if we do this, "struct config_reader" itself will probably become
> obsolete, because we'd be able to plumb only the relevant state for the
> current operation, e.g. if we are parsing a config file, we'd pass only the
> config file parsing state, instead of "struct config_reader", which also
> contains config set iterating state. In such a scenario, we'd probably want
> to pass "struct key_value_info" to the config callback, since that's all the
> callback should be interested in anyway. Interestingly, this was proposed by
> Junio back in [4], and we didn't do this back then out of concern for the
> churn (just like in v1).

I think we can make it even simpler than that, and from playing around with builtin/config.c a bit after the 5/5 here I got a POC working (but am not posting it here, didn't have time to clean it up).

We can just make config_set_callback() and configset_iter() non-static, so e.g. the builtin/config.c caller that implements "--show-origin" can keep its config_with_options(...) call, but instead of "streaming" the config, it'll buffer it up into a configset.

The advantage of that is that with the configset API we'll get a "struct key_value_info *" for free on the other end. I.e. we'll configset_iter() with a fn=NULL, but with a defined "config_kvi_fn_t", which 5/5 is adding.

But I haven't done that work (and am not planning to finish this), but maybe this helps.

We'll also need to track the equivalent of "cf->linenr" etc. while we do the actual initial parse. I think it might be simpler to start by converting those "linenr" to a "struct key_value_info" that's placed in the "config_source" right away.

I.e. when we pass it to the error handlers we'll need to give them access to the "linenr", but we don't want to provide e.g. "eof" (which is internal-only state).

I wonder how much else you're converting here is actually dead code in the end (or can trivially be made dead). E.g. the current_config_line() change you make in 1/8 is never going to use the "cf_global" if combined with the 5/5 change here.

1. https://lore.kernel.org/git/230308.867cvrziac.gmgdl@evledraar.gmail.com/
Ævar Arnfjörð Bjarmason (5):
  config.h: move up "struct key_value_info"
  config.c: use "enum config_origin_type", not "int"
  config API: add a config_origin_type_name() helper
  config.c: refactor configset_iter()
  config API: add and use a repo_config_kvi()
 builtin/remote.c       | 11 +++----
 config.c               | 67 ++++++++++++++++++++++++++----------------
 config.h               | 29 +++++++++++++-----
 t/helper/test-config.c | 13 ++++----
 t/t5505-remote.sh      |  7 +++--
 5 files changed, 80 insertions(+), 47 deletions(-)
-- 
2.40.0.rc1.1034.g5867a1b10c5
Previous: Jonathan TanNext: Ævar Arnfjörð Bjarmason
Message 44 of 72 in “[RFC] config.c: use struct for config reading state”
  1. 0/6 [RFC] config.c: use struct for config reading stateGlen Choo via GitGitGadget, Mar 1, 2023
  2. 2/6 config.c: don't assign to "cf" directlyGlen Choo via GitGitGadget, Mar 1, 2023
  3. 1/6 config.c: plumb config_source through static fnsGlen Choo via GitGitGadget, Mar 1, 2023
  4. Junio C HamanoMar 3, 2023
  5. 3/6 config.c: create config_reader and the_readerGlen Choo via GitGitGadget, Mar 1, 2023
  6. Junio C HamanoMar 3, 2023
  7. 5/6 config.c: remove current_config_kviGlen Choo via GitGitGadget, Mar 1, 2023
  8. Calvin WanMar 6, 2023
  9. 4/6 config.c: plumb the_reader through callbacksGlen Choo via GitGitGadget, Mar 1, 2023
  10. Ævar Arnfjörð BjarmasonMar 8, 2023
  11. Glen ChooMar 8, 2023
  12. Junio C HamanoMar 8, 2023
  13. 6/6 config.c: remove current_parsing_scopeGlen Choo via GitGitGadget, Mar 1, 2023
  14. Jonathan TanMar 6, 2023
  15. Glen ChooMar 6, 2023
  16. Jonathan TanMar 6, 2023
  17. Ævar Arnfjörð BjarmasonMar 8, 2023
  18. Glen ChooMar 8, 2023
  19. Ævar Arnfjörð BjarmasonMar 7, 2023
  20. Glen ChooMar 7, 2023
  21. Ævar Arnfjörð BjarmasonMar 7, 2023
  22. Junio C HamanoMar 7, 2023
  23. Glen ChooMar 7, 2023
  24. Ævar Arnfjörð BjarmasonMar 8, 2023
  25. Glen ChooMar 8, 2023
  26. 0/8 config.c: use struct for config reading stateGlen Choo via GitGitGadget, Mar 16, 2023
  27. 2/8 config.c: don't assign to "cf_global" directlyGlen Choo via GitGitGadget, Mar 16, 2023
  28. Jonathan TanMar 16, 2023
  29. Junio C HamanoMar 16, 2023
  30. Glen ChooMar 16, 2023
  31. 1/8 config.c: plumb config_source through static fnsGlen Choo via GitGitGadget, Mar 16, 2023
  32. Jonathan TanMar 16, 2023
  33. 3/8 config.c: create config_reader and the_readerGlen Choo via GitGitGadget, Mar 16, 2023
  34. Jonathan TanMar 16, 2023
  35. 4/8 config.c: plumb the_reader through callbacksGlen Choo via GitGitGadget, Mar 16, 2023
  36. 6/8 config.c: remove current_parsing_scopeGlen Choo via GitGitGadget, Mar 16, 2023
  37. 5/8 config.c: remove current_config_kviGlen Choo via GitGitGadget, Mar 16, 2023
  38. 7/8 config: report cached filenames in die_bad_number()Glen Choo via GitGitGadget, Mar 16, 2023
  39. Jonathan TanMar 16, 2023
  40. Glen ChooMar 16, 2023
  41. 8/8 config.c: rename "struct config_source cf"Glen Choo via GitGitGadget, Mar 16, 2023
  42. Glen ChooMar 16, 2023
  43. Jonathan TanMar 16, 2023
  44. 0/5 bypass config.c global state with configsetÆvar Arnfjörð Bjarmason, Mar 17, 2023
  45. 1/5 config.h: move up "struct key_value_info"Ævar Arnfjörð Bjarmason, Mar 17, 2023
  46. 2/5 config.c: use "enum config_origin_type", not "int"Ævar Arnfjörð Bjarmason, Mar 17, 2023
  47. 3/5 config API: add a config_origin_type_name() helperÆvar Arnfjörð Bjarmason, Mar 17, 2023
  48. 4/5 config.c: refactor configset_iter()Ævar Arnfjörð Bjarmason, Mar 17, 2023
  49. 5/5 config API: add and use a repo_config_kvi()Ævar Arnfjörð Bjarmason, Mar 17, 2023
  50. Junio C HamanoMar 17, 2023
  51. Jonathan TanMar 17, 2023
  52. Junio C HamanoMar 17, 2023
  53. Glen ChooMar 17, 2023
  54. Glen ChooMar 17, 2023
  55. Glen ChooMar 17, 2023
  56. Ævar Arnfjörð BjarmasonMar 29, 2023
  57. 0/8 config.c: use struct for config reading stateGlen Choo via GitGitGadget, Mar 28, 2023
  58. 1/8 config.c: plumb config_source through static fnsGlen Choo via GitGitGadget, Mar 28, 2023
  59. 2/8 config.c: don't assign to "cf_global" directlyGlen Choo via GitGitGadget, Mar 28, 2023
  60. 3/8 config.c: create config_reader and the_readerGlen Choo via GitGitGadget, Mar 28, 2023
  61. Ævar Arnfjörð BjarmasonMar 29, 2023
  62. Junio C HamanoMar 29, 2023
  63. Glen ChooMar 29, 2023
  64. Glen ChooMar 30, 2023
  65. 4/8 config.c: plumb the_reader through callbacksGlen Choo via GitGitGadget, Mar 28, 2023
  66. 5/8 config.c: remove current_config_kviGlen Choo via GitGitGadget, Mar 28, 2023
  67. 6/8 config.c: remove current_parsing_scopeGlen Choo via GitGitGadget, Mar 28, 2023
  68. 7/8 config: report cached filenames in die_bad_number()Glen Choo via GitGitGadget, Mar 28, 2023
  69. 8/8 config.c: rename "struct config_source cf"Glen Choo via GitGitGadget, Mar 28, 2023
  70. Glen ChooMar 28, 2023
  71. Junio C HamanoMar 28, 2023
  72. Glen ChooMar 28, 2023

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.