git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: [PATCH 05/22] read-cache: add index reading api

From
Thomas Gummerer <t.gummerer@gmail.com>
Date
Jul 8, 2013, 11:20 UTC
Message-ID
<87pputqowr.fsf@gmail.com>
In-Reply-To
<CACsJy8C0F+v3g+gPon6Y8+z7ObN1Jv8Ln==RNrVsRp7aya74hw@mail.gmail.com>
Duy Nguyen <pclouds@gmail.com> writes:
Show 72 quoted lines
> On Sun, Jul 7, 2013 at 3:11 PM, Thomas Gummerer <t.gummerer@gmail.com> wrote:
>> +/*
>> + * Options by which the index should be filtered when read partially.
>> + *
>> + * pathspec: The pathspec which the index entries have to match
>> + * seen: Used to return the seen parameter from match_pathspec()
>> + * max_prefix, max_prefix_len: These variables are set to the longest
>> + *     common prefix, the length of the longest common prefix of the
>> + *     given pathspec
>> + *
>> + * read_staged: used to indicate if the conflicted entries (entries
>> + *     with a stage) should be included
>> + * read_cache_tree: used to indicate if the cache-tree should be read
>> + * read_resolve_undo: used to indicate if the resolve undo data should
>> + *     be read
>> + */
>> +struct filter_opts {
>> +       const char **pathspec;
>> +       char *seen;
>> +       char *max_prefix;
>> +       int max_prefix_len;
>> +
>> +       int read_staged;
>> +       int read_cache_tree;
>> +       int read_resolve_undo;
>> +};
>> +
>>  struct index_state {
>>         struct cache_entry **cache;
>>         unsigned int version;
>> @@ -270,6 +297,8 @@ struct index_state {
>>         struct hash_table name_hash;
>>         struct hash_table dir_hash;
>>         struct index_ops *ops;
>> +       struct internal_ops *internal_ops;
>> +       struct filter_opts *filter_opts;
>>  };
>
> ...
>
>> -/* remember to discard_cache() before reading a different cache! */
>> -int read_index_from(struct index_state *istate, const char *path)
>> +
>> +int read_index_filtered_from(struct index_state *istate, const char *path,
>> +                            struct filter_opts *opts)
>>  {
>>         int fd, err, i;
>>         struct stat st_old, st_new;
>> @@ -1337,7 +1425,7 @@ int read_index_from(struct index_state *istate, const char *path)
>>                 if (istate->ops->verify_hdr(mmap, mmap_size) < 0)
>>                         err = 1;
>>
>> -               if (istate->ops->read_index(istate, mmap, mmap_size) < 0)
>> +               if (istate->ops->read_index(istate, mmap, mmap_size, opts) < 0)
>>                         err = 1;
>>                 istate->timestamp.sec = st_old.st_mtime;
>>                 istate->timestamp.nsec = ST_MTIME_NSEC(st_old);
>> @@ -1345,6 +1433,7 @@ int read_index_from(struct index_state *istate, const char *path)
>>                         die_errno("cannot stat the open index");
>>
>>                 munmap(mmap, mmap_size);
>> +               istate->filter_opts = opts;
>>                 if (!index_changed(&st_old, &st_new) && !err)
>>                         return istate->cache_nr;
>>         }
>
> Putting filter_opts in index_state feels like a bad design. Iterator
> information should be separated from the iterated object, so that two
> callers can walk through the same index without stepping on each other
> (I'm not talking about multithreading, a caller may walk a bit, then
> the other caller starts walking, then the former caller resumes
> walking again in a call chain).

Yes, you're right. We need the filter_opts to see what part of the index has been loaded [1] and which part has been skipped, but it shouldn't be used for filtering in the for_each_index_entry function.

I think there should be two versions of the for_each_index_entry function then, where the for_each_index_entry function would behave the same way as the for_each_index_entry_filtered function with the filter_opts parameter set to NULL: for_each_index_entry_filtered(struct index_state *, each_cache_entry_fn, void *cb_data, struct filter_opts *) for_each_index_entry(struct index_state *, each_cache_entry_fn, void *cb_data)

Both of them then should call index_change_filter_opts to make sure all the entries that are needed are loaded in the in-memory format.

Does that make sense?
[1] That is only important for the new index-v5 file format, which can
    be loaded partially.  The filter_opts could always be set to NULL,
    as the whole index is always loaded anyway.
Previous: Duy NguyenNext: Duy Nguyen
Message 10 of 51 in “Index v5”
  1. 00/22 Index v5Thomas Gummerer, Jul 7, 2013
  2. 01/22 t2104: Don't fail for index versions other than [23]Thomas Gummerer, Jul 7, 2013
  3. 02/22 read-cache: split index file version specific functionalityThomas Gummerer, Jul 7, 2013
  4. 03/22 read-cache: move index v2 specific functions to their own fileThomas Gummerer, Jul 7, 2013
  5. 04/22 read-cache: Re-read index if index file changedThomas Gummerer, Jul 7, 2013
  6. 05/22 read-cache: add index reading apiThomas Gummerer, Jul 7, 2013
  7. Duy NguyenJul 8, 2013
  8. Thomas GummererJul 8, 2013
  9. Duy NguyenJul 8, 2013
  10. Thomas GummererJul 8, 2013
  11. Duy NguyenJul 8, 2013
  12. Thomas GummererJul 8, 2013
  13. 5/22 Add documentation for the index apiThomas Gummerer, Jul 8, 2013
  14. Duy NguyenJul 9, 2013
  15. Thomas GummererJul 9, 2013
  16. Duy NguyenJul 10, 2013
  17. Thomas GummererJul 11, 2013
  18. Duy NguyenJul 11, 2013
  19. Duy NguyenJul 11, 2013
  20. Junio C HamanoJul 8, 2013
  21. Thomas GummererJul 8, 2013
  22. Junio C HamanoJul 8, 2013
  23. Thomas GummererJul 9, 2013
  24. 06/22 make sure partially read index is not changedThomas Gummerer, Jul 7, 2013
  25. Junio C HamanoJul 8, 2013
  26. Thomas GummererJul 8, 2013
  27. 07/22 dir.c: use index apiThomas Gummerer, Jul 7, 2013
  28. 08/22 tree.c: use index apiThomas Gummerer, Jul 7, 2013
  29. 09/22 name-hash.c: use index apiThomas Gummerer, Jul 7, 2013
  30. 10/22 grep.c: Use index apiThomas Gummerer, Jul 7, 2013
  31. 11/22 ls-files.c: use the index apiThomas Gummerer, Jul 7, 2013
  32. 12/22 read-cache: make read_blob_data_from_index use index apiThomas Gummerer, Jul 7, 2013
  33. 13/22 documentation: add documentation of the index-v5 file formatThomas Gummerer, Jul 7, 2013
  34. Duy NguyenJul 11, 2013
  35. Thomas GummererJul 11, 2013
  36. Duy NguyenJul 11, 2013
  37. Thomas GummererJul 11, 2013
  38. Duy NguyenJul 11, 2013
  39. 14/22 read-cache: make in-memory format aware of stat_crcThomas Gummerer, Jul 7, 2013
  40. 15/22 read-cache: read index-v5Thomas Gummerer, Jul 7, 2013
  41. Eric SunshineJul 7, 2013
  42. Thomas GummererJul 8, 2013
  43. 16/22 read-cache: read resolve-undo dataThomas Gummerer, Jul 7, 2013
  44. 17/22 read-cache: read cache-tree in index-v5Thomas Gummerer, Jul 7, 2013
  45. Eric SunshineJul 7, 2013
  46. 18/22 read-cache: write index-v5Thomas Gummerer, Jul 7, 2013
  47. Eric SunshineJul 7, 2013
  48. 19/22 read-cache: write index-v5 cache-tree dataThomas Gummerer, Jul 7, 2013
  49. 20/22 read-cache: write resolve-undo data for index-v5Thomas Gummerer, Jul 7, 2013
  50. 21/22 update-index.c: rewrite index when index-version is givenThomas Gummerer, Jul 7, 2013
  51. 22/22 p0003-index.sh: add perf test for the index formatsThomas Gummerer, Jul 7, 2013

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.