git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: [PATCH v3 2/3] config: add hashtable for config parsing & retrieval

From
Ramsay Jones <ramsay@ramsay1.demon.co.uk>
Date
Jun 24, 2014, 15:32 UTC
Message-ID
<53A99A28.1090302@ramsay1.demon.co.uk>
In-Reply-To
<53A853E9.8060801@gmail.com>
On 23/06/14 17:20, Tanay Abhra wrote:
> On 06/23/2014 07:57 AM, Ramsay Jones wrote:
>> On 23/06/14 11:11, Tanay Abhra wrote:
[snip]
Show 40 quoted lines
>>> +static struct hashmap *get_config_cache(void)
>>> +{
>>> +	static struct hashmap config_cache;
>>> +	if (!hashmap_initialized) {
>>> +		config_cache_init(&config_cache);
>>> +		hashmap_initialized = 1;
>>> +		git_config(config_cache_callback, NULL);
>>> +	}
>>> +	return &config_cache;
>>> +}
>>
>> [I have not been following this series at all (sorry I haven't had
>> the time to spare), so take these comments with a very big pinch of
>> salt! ie just ignore me if it's already been discussed etc. ;-) ]
>>
>> The 'git config' command can be used to read arbitrary files (so long
>> as they conform to the config syntax). For example, see the --file and
>> --blob options to git-config. At present, I think only scripted commands
>> use this facility (eg git-submodule). Noting the singleton config_cache,
>> what happens when git-submodule becomes a C builtin, or indeed any other
>> C builtin wants to take advantage of the new code when processing a non-
>> standard config file?
>>
> 
> This series was mainly to replace git_config() invocations around the codebase.
> There are currently 111 git_config() invocations, each of which causes a file
> reread whenever called. git_config() only feeds values from the standard config
> files(i.e repo, user and global config).
> 
> For reading config values from specific files or blobs, there are three functions
> git_config_with_options, git_config_from_file & git_config_from_blob which can be
> easily used inside a C builtin or anywhere in the code.
> 
> The bulk of git_config_api calls are only for git_config(). For example,
> git_config_from_file() has three hits only in entire codebase,
> git_config_with_options() has 5 hits, so I concentrated on generating a cache
> for the usual config files only. For other files, the callers can fall back on older
> API functions like I had mentioned above.
> 
> Forgive me if I inferred your question incorrectly. More below.

Hmm, maybe. The "... take advantage of the new code" refers to the possibility (or otherwise) of re-using your work to update these "older API" functions to the new API style. (also, see Junio's response). [In order to do this, I would have expected to see one hash table for each file/blob, so the singleton object took me by surprise.]

An "out of scope for this project" is a perfectly acceptable response (*particularly* since it is very late in the day to be bringing this up!).

Show 50 quoted lines
>>> +static struct config_cache_entry *config_cache_find_entry(const char *key)
>>> +{
>>> +	struct hashmap *config_cache;
>>> +	struct config_cache_entry k;
>>> +	struct config_cache_entry *found_entry;
>>> +	char *normalized_key;
>>> +	int ret;
>>> +	config_cache = get_config_cache();
>>> +	ret = git_config_parse_key(key, &normalized_key, NULL);
>>> +
>>> +	if (ret)
>>> +		return NULL;
>>> +
>>> +	hashmap_entry_init(&k, strhash(normalized_key));
>>> +	k.key = normalized_key;
>>> +	found_entry = hashmap_get(config_cache, &k, NULL);
>>> +	free(normalized_key);
>>> +	return found_entry;
>>> +}
>>> +
>>> +static struct string_list *config_cache_get_value(const char *key)
>>> +{
>>> +	struct config_cache_entry *e = config_cache_find_entry(key);
>>> +	return e ? &e->value_list : NULL;
>>> +}
>>> +
>>> +static int config_cache_add_value(const char *key, const char *value)
>>> +{
>>> +	struct hashmap *config_cache;
>>> +	struct config_cache_entry *e;
>>> +	struct string_list_item *item;
>>> +	int *boolean_null_flag;
>>> +
>>> +	config_cache = get_config_cache();
>>> +	e = config_cache_find_entry(key);
>>> +
>>> +	boolean_null_flag = xcalloc(1, sizeof(*boolean_null_flag));
>>> +
>>> +	if (!e) {
>>> +		e = xmalloc(sizeof(*e));
>>> +		hashmap_entry_init(e, strhash(key));
>>> +		e->key = xstrdup(key);
>>
>> config_cache_find_entry() searches for (and hashes the) normalized_key.
>> Should you not be entering the normalized key here?
>>
> 
> config_cache_add_value() is fed key-values pairs through the git_config()
> callback mechanism, which normalises the key beforehand, so no need for
> renormalising.

Ah, yes, I forgot that the parsing code does a tolower() at various places while accumulating the key string. So the (potentially) non- normalized keys come from the user via the new API functions and, rather than putting code to normalize the key in each of those, just do it once in config_cache_find_entry(). (Although, you could possibly do that in config_cache_get_value()). OK.

Hmm, maybe add a short comment to that effect? dunno.

ATB, Ramsay Jones

Previous: Tanay AbhraNext: Matthieu Moy
Message 11 of 35 in “git config cache & special querying api utilizing the cache”
  1. 0/3 git config cache & special querying api utilizing the cacheTanay Abhra, Jun 23, 2014
  2. 1/3 string-list: add string_list initialiser helper functionsTanay Abhra, Jun 23, 2014
  3. Torsten BögershausenJun 23, 2014
  4. Tanay AbhraJun 23, 2014
  5. 2/3 config: add hashtable for config parsing & retrievalTanay Abhra, Jun 23, 2014
  6. Matthieu MoyJun 23, 2014
  7. Tanay AbhraJun 24, 2014
  8. Karsten BleesJun 25, 2014
  9. Ramsay JonesJun 23, 2014
  10. Tanay AbhraJun 23, 2014
  11. Ramsay JonesJun 24, 2014
  12. Matthieu MoyJun 26, 2014
  13. Junio C HamanoJun 23, 2014
  14. Tanay AbhraJun 24, 2014
  15. Junio C HamanoJun 25, 2014
  16. Tanay AbhraJun 24, 2014
  17. Ramsay JonesJun 24, 2014
  18. Junio C HamanoJun 25, 2014
  19. Karsten BleesJun 25, 2014
  20. Junio C HamanoJun 25, 2014
  21. Matthieu MoyJun 26, 2014
  22. Junio C HamanoJun 26, 2014
  23. Karsten BleesJun 26, 2014
  24. Junio C HamanoJun 26, 2014
  25. Karsten BleesJun 27, 2014
  26. Matthieu MoyJun 27, 2014
  27. Junio C HamanoJun 27, 2014
  28. Junio C HamanoJun 23, 2014
  29. Tanay AbhraJun 24, 2014
  30. Matthieu MoyJun 26, 2014
  31. Karsten BleesJun 25, 2014
  32. Matthieu MoyJun 26, 2014
  33. 3/3 test-config: add usage examples for non-callback query functionsTanay Abhra, Jun 23, 2014
  34. Eric SunshineJun 25, 2014
  35. Tanay AbhraJun 26, 2014

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.