git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: [PATCH/RFC v2 0/16] Introduce index file format version 5

From
Nguyen Thai Ngoc Duy <pclouds@gmail.com>
Date
Aug 9, 2012, 02:28 UTC
Message-ID
<CACsJy8DqwgzrbXXU+ea2QWJ_oujcJe4_ZVmR_-Sy02Gnk1Hetg@mail.gmail.com>
In-Reply-To
<7vvcgttkeu.fsf@alter.siamese.dyndns.org>
On Wed, Aug 8, 2012 at 11:31 PM, Junio C Hamano <gitster@pobox.com> wrote:
Show 19 quoted lines
> The current code that access nth entry from the index->cache[nth]
> would need to be updated to use an accessor function, whether the
> "nth" comes from index_name_pos() or from the for-loop that iterates
> over the entire index.  For the latter, you would need to give the
> users a function that returns a cursor into the in-core index to
> allow iterating over it.
>
> When you use an in-core representation that is not a flat array, the
> type of "nth", which is essentially a cursor, may have to change to
> something that is richer than a simple integer, in order to give the
> implementation of the in-core index a more efficient way to access
> the entry than traversing the leaves of the tree depth first, and
> you would need to update index_name_pos() to return such a "cursor".
> That design and development cost is part of updating the in-core
> data structure. In the end result, the runtime cost to manipulate an
> index entry that the cursor refers to should be minimum, as that
> would be the cost paid by all the users of the API anyway, even if
> we _were_ starting from an ideal world where there weren't any flat
> in-core index in the first place.

Interesting. So you hide the entire tree walk behind the cursor concept. And we can make pathspec filter as part of cursor initialization. Index iteration code this way looks really neat (compared to how we do traverse sha-1 trees nowadays). The hard part is updating the index while iterating (or avodiing running into such a situation). Maybe C++ STL has done it already with std::map::iterator. I fear that by hiding the trees, we might miss some optimization opportunities. But I haven't figured it all out yet so I may be wrong.

-- 
Duy
Previous: Junio C HamanoNext: Thomas Rast
Message 54 of 59 in “Introduce index file format version 5”
  1. 0/16 Introduce index file format version 5Thomas Gummerer, Aug 5, 2012
  2. 01/16 Modify cache_header to prepare for other index formatsThomas Gummerer, Aug 5, 2012
  3. Junio C HamanoAug 6, 2012
  4. Thomas GummererAug 7, 2012
  5. Junio C HamanoAug 7, 2012
  6. 02/16 Modify read functions to prepare for other index formatsThomas Gummerer, Aug 5, 2012
  7. 03/16 Modify match_stat_basic to prepare for other index formatsThomas Gummerer, Aug 5, 2012
  8. 04/16 Modify write functions to prepare for other index formatsThomas Gummerer, Aug 5, 2012
  9. Junio C HamanoAug 6, 2012
  10. Thomas GummererAug 7, 2012
  11. 05/16 t2104: Don't fail for index versions other than [23]Thomas Gummerer, Aug 5, 2012
  12. Junio C HamanoAug 6, 2012
  13. 06/16 t3700: sleep for 1 second, to avoid interfering with the racy codeThomas Gummerer, Aug 5, 2012
  14. Junio C HamanoAug 6, 2012
  15. Thomas GummererAug 7, 2012
  16. Junio C HamanoAug 8, 2012
  17. Junio C HamanoAug 8, 2012
  18. Thomas GummererAug 9, 2012
  19. Junio C HamanoAug 9, 2012
  20. Thomas GummererAug 9, 2012
  21. 07/16 Add documentation of the index-v5 file formatThomas Gummerer, Aug 5, 2012
  22. 08/16 Make in-memory format aware of stat_crcThomas Gummerer, Aug 5, 2012
  23. Junio C HamanoAug 6, 2012
  24. Thomas GummererAug 7, 2012
  25. 09/16 Read index-v5Thomas Gummerer, Aug 5, 2012
  26. Junio C HamanoAug 6, 2012
  27. Thomas GummererAug 8, 2012
  28. Junio C HamanoAug 8, 2012
  29. Thomas GummererAug 8, 2012
  30. Junio C HamanoAug 8, 2012
  31. 10/16 Read resolve-undo dataThomas Gummerer, Aug 5, 2012
  32. Junio C HamanoAug 6, 2012
  33. Thomas GummererAug 7, 2012
  34. 11/16 Read cache-tree in index-v5Thomas Gummerer, Aug 5, 2012
  35. 12/16 Write index-v5Thomas Gummerer, Aug 5, 2012
  36. 13/16 Write index-v5 cache-tree dataThomas Gummerer, Aug 5, 2012
  37. 14/16 Write resolve-undo data for index-v5Thomas Gummerer, Aug 5, 2012
  38. 15/16 update-index.c: add a force-rewrite optionThomas Gummerer, Aug 5, 2012
  39. Junio C HamanoAug 6, 2012
  40. Thomas GummererAug 8, 2012
  41. 16/16 p0002-index.sh: add perf test for the index formatsThomas Gummerer, Aug 5, 2012
  42. Nguyễn Thái Ngọc DuyAug 6, 2012
  43. 1/2 Move index v2 specific code out of read-cacheNguyễn Thái Ngọc Duy, Aug 6, 2012
  44. 2/2 Add index-v5Nguyễn Thái Ngọc Duy, Aug 6, 2012
  45. Robin RosenbergAug 7, 2012
  46. Thomas GummererAug 8, 2012
  47. Junio C HamanoAug 6, 2012
  48. Thomas GummererAug 6, 2012
  49. Junio C HamanoAug 6, 2012
  50. Nguyen Thai Ngoc DuyAug 7, 2012
  51. Junio C HamanoAug 8, 2012
  52. Nguyen Thai Ngoc DuyAug 8, 2012
  53. Junio C HamanoAug 8, 2012
  54. Nguyen Thai Ngoc DuyAug 9, 2012
  55. Thomas RastAug 7, 2012
  56. Junio C HamanoAug 7, 2012
  57. Thomas RastAug 8, 2012
  58. Junio C HamanoAug 8, 2012
  59. Nguyen Thai Ngoc DuyAug 8, 2012

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.