git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: [PATCH v2 1/3] repository: fix repo_read_index with submodules

From
BWBrandon Williams <bmwill@google.com>
Date
Dec 18, 2017, 23:05 UTC
Message-ID
<20171218230541.GC147973@google.com>
In-Reply-To
<20171218230511.GB2641@hank>
On 12/18, Thomas Gummerer wrote:
Show 113 quoted lines
> On 12/18, Brandon Williams wrote:
> > On 12/17, Thomas Gummerer wrote:
> > > repo_read_index calls read_index_from, which takes an path argument for
> > > the location of the index file.  For the split index however it relies
> > 
> > > on the current working directory to construct the path using git_path.
> > 
> > This line isn't actually true and should probably be fixed.  git_path
> > doesn't rely on the CWD but rather it relies on the gitdir of the main
> > repository (the_repository).
> 
> Right, let me fix that.  Thanks!
> 
> > > 
> > > repo_read_index calls read_index_from with the full path for the index
> > > file, however it doesn't change the cwd, so when split index mode is
> > > turned on, read_index_from can't find the file for the split index.
> > > 
> > > For example t3007-ls-files-recurse-submodules.sh was broken with
> > > GIT_TEST_SPLIT_INDEX set in 188dce131f ("ls-files: use repository
> > > object", 2017-06-22), and t7814-grep-recurse-submodules.sh was also
> > > broken in a similar manner, probably by introducing struct repository
> > > there, although I didn't track down the exact commit for that.
> > > 
> > > Fix this by introducing a new read_index_for_repo function, which knows
> > > about the correct paths for the submodules.
> > > 
> > > The alternative would have been to make the callers pass in the base
> > > path for the split index, however that ended up being more complicated,
> > > and I think we want to converge towards using struct repository for
> > > things like these anyway.
> > > 
> > > Signed-off-by: Thomas Gummerer <t.gummerer@gmail.com>
> > > ---
> > >  cache.h      |  1 +
> > >  read-cache.c | 16 ++++++++++++++--
> > >  repository.c |  2 +-
> > >  3 files changed, 16 insertions(+), 3 deletions(-)
> > > 
> > > diff --git a/cache.h b/cache.h
> > > index cb5db7bf83..d42bea1ef7 100644
> > > --- a/cache.h
> > > +++ b/cache.h
> > > @@ -614,6 +614,7 @@ extern int read_index_preload(struct index_state *, const struct pathspec *paths
> > >  extern int do_read_index(struct index_state *istate, const char *path,
> > >  			 int must_exist); /* for testting only! */
> > >  extern int read_index_from(struct index_state *, const char *path);
> > > +extern int read_index_for_repo(const struct repository *);
> > >  extern int is_index_unborn(struct index_state *);
> > >  extern int read_index_unmerged(struct index_state *);
> > >  
> > > diff --git a/read-cache.c b/read-cache.c
> > > index 2eb81a66b9..70357febdc 100644
> > > --- a/read-cache.c
> > > +++ b/read-cache.c
> > > @@ -20,6 +20,7 @@
> > >  #include "split-index.h"
> > >  #include "utf8.h"
> > >  #include "fsmonitor.h"
> > > +#include "repository.h"
> > >  
> > >  /* Mask for the name length in ce_flags in the on-disk index */
> > >  
> > > @@ -1871,7 +1872,8 @@ static void freshen_shared_index(char *base_sha1_hex, int warn)
> > >  	free(shared_index);
> > >  }
> > >  
> > > -int read_index_from(struct index_state *istate, const char *path)
> > > +static int do_read_index_from(struct index_state *istate, const char *path,
> > > +			      const struct repository *repo)
> > >  {
> > >  	struct split_index *split_index;
> > >  	int ret;
> > > @@ -1896,7 +1898,7 @@ int read_index_from(struct index_state *istate, const char *path)
> > >  		split_index->base = xcalloc(1, sizeof(*split_index->base));
> > >  
> > >  	base_sha1_hex = sha1_to_hex(split_index->base_sha1);
> > > -	base_path = git_path("sharedindex.%s", base_sha1_hex);
> > > +	base_path = repo_git_path(repo, "sharedindex.%s", base_sha1_hex);
> > >  	ret = do_read_index(split_index->base, base_path, 1);
> > >  	if (hashcmp(split_index->base_sha1, split_index->base->sha1))
> > >  		die("broken index, expect %s in %s, got %s",
> > > @@ -1909,6 +1911,16 @@ int read_index_from(struct index_state *istate, const char *path)
> > >  	return ret;
> > >  }
> > >  
> > > +int read_index_for_repo(const struct repository *repo)
> > > +{
> > > +	return do_read_index_from(repo->index, repo->index_file, repo);
> > > +}
> > 
> > > +
> > > +int read_index_from(struct index_state *istate, const char *path)
> > > +{
> > > +	return do_read_index_from(istate, path, the_repository);
> > > +}
> > 
> > This looks fine, though I wonder what the point of passing in the index
> > file even was since we end just ended up reading the 'sharedindex' file based
> > on the git path. I'm just curious about how this function evolved.
> 
> There are some callsites that are using an index different form
> $gitdir/index, or even GIT_INDEX_FILE.  e.g. see builtin/am.c [*1*],
> which uses it's own 'patch-merge-index' in the am state directory for
> its internal operations.
> 
> The split index mode was later bolted on, and the sharedindex.xxxx
> would always go in $gitdir for the repository.  Others probably know
> quite a bit more about this, while I'm always interested in index
> related things as that's how I got started with the git project, I
> couldn't follow all the conversations that were going on there.
> 
> *1*: https://github.com/gitster/git/blob/52015aaf9d19c97b52c47c7046058e6d029ff856/builtin/am.c#L1844
Thanks for the explanation and pointer! :) 
Show 22 quoted lines
> 
> > > +
> > >  int is_index_unborn(struct index_state *istate)
> > >  {
> > >  	return (!istate->cache_nr && !istate->timestamp.sec);
> > > diff --git a/repository.c b/repository.c
> > > index bb2fae5446..928b1f553d 100644
> > > --- a/repository.c
> > > +++ b/repository.c
> > > @@ -229,5 +229,5 @@ int repo_read_index(struct repository *repo)
> > >  	if (!repo->index)
> > >  		repo->index = xcalloc(1, sizeof(*repo->index));
> > >  
> > > -	return read_index_from(repo->index, repo->index_file);
> > > +	return read_index_for_repo(repo);
> > >  }
> > > -- 
> > > 2.15.1.620.gb9897f4670
> > > 
> > 
> > -- 
> > Brandon Williams
-- 
Brandon Williams
Previous: Thomas GummererNext: Thomas Gummerer
Message 29 of 95 in “fixes for split index mode”
  1. 0/3 fixes for split index modeThomas Gummerer, Dec 10, 2017
  2. 1/3 repository: fix repo_read_index with submodulesThomas Gummerer, Dec 10, 2017
  3. Brandon WilliamsDec 11, 2017
  4. Thomas GummererDec 11, 2017
  5. 2/3 prune: fix pruning with multiple worktrees and split indexThomas Gummerer, Dec 10, 2017
  6. Brandon WilliamsDec 11, 2017
  7. Thomas GummererDec 11, 2017
  8. 3/3 travis: run tests with GIT_TEST_SPLIT_INDEXThomas Gummerer, Dec 10, 2017
  9. Eric SunshineDec 10, 2017
  10. SZEDER GáborDec 11, 2017
  11. Thomas GummererDec 11, 2017
  12. Lars SchneiderDec 12, 2017
  13. Junio C HamanoDec 12, 2017
  14. Thomas GummererDec 12, 2017
  15. Junio C HamanoDec 12, 2017
  16. Thomas GummererDec 13, 2017
  17. Lars SchneiderDec 13, 2017
  18. Junio C HamanoDec 13, 2017
  19. Lars SchneiderDec 13, 2017
  20. Thomas GummererDec 13, 2017
  21. 0/3 fixes for split index modeThomas Gummerer, Dec 17, 2017
  22. 2/3 prune: fix pruning with multiple worktrees and split indexThomas Gummerer, Dec 17, 2017
  23. Brandon WilliamsDec 18, 2017
  24. Thomas GummererJan 3, 2018
  25. Junio C HamanoJan 4, 2018
  26. 1/3 repository: fix repo_read_index with submodulesThomas Gummerer, Dec 17, 2017
  27. Brandon WilliamsDec 18, 2017
  28. Thomas GummererDec 18, 2017
  29. Brandon WilliamsDec 18, 2017
  30. 3/3 travis: run tests with GIT_TEST_SPLIT_INDEXThomas Gummerer, Dec 17, 2017
  31. Lars SchneiderDec 18, 2017
  32. Thomas GummererJan 4, 2018
  33. Lars SchneiderJan 5, 2018
  34. Thomas GummererJan 7, 2018
  35. 0/3 fixes for split index modeThomas Gummerer, Jan 7, 2018
  36. 1/3 read-cache: fix reading the shared index for other reposThomas Gummerer, Jan 7, 2018
  37. Duy NguyenJan 8, 2018
  38. Thomas GummererJan 8, 2018
  39. Thomas GummererJan 13, 2018
  40. Brandon WilliamsJan 8, 2018
  41. Duy NguyenJan 9, 2018
  42. Brandon WilliamsJan 16, 2018
  43. Duy NguyenJan 17, 2018
  44. Brandon WilliamsJan 17, 2018
  45. Jonathan NiederJan 17, 2018
  46. Duy NguyenJan 18, 2018
  47. Junio C HamanoJan 19, 2018
  48. Thomas GummererJan 20, 2018
  49. Junio C HamanoJan 22, 2018
  50. Thomas GummererJan 27, 2018
  51. Junio C HamanoFeb 7, 2018
  52. 2/3 split-index: don't write cache tree with null oid entriesThomas Gummerer, Jan 7, 2018
  53. 3/3 travis: run tests with GIT_TEST_SPLIT_INDEXThomas Gummerer, Jan 7, 2018
  54. 4/3 read-cache: don't try to write index if we can't write shared indexThomas Gummerer, Jan 13, 2018
  55. Duy NguyenJan 14, 2018
  56. 1/3 read-cache.c: change type of "temp" in write_shared_index()Nguyễn Thái Ngọc Duy, Jan 14, 2018
  57. 2/3 read-cache.c: move tempfile creation/cleanup out of write_shared_indexNguyễn Thái Ngọc Duy, Jan 14, 2018
  58. 3/3 read-cache: don't write index twice if we can't write shared indexNguyễn Thái Ngọc Duy, Jan 14, 2018
  59. SZEDER GáborJan 18, 2018
  60. Duy NguyenJan 18, 2018
  61. Jeff KingJan 18, 2018
  62. Duy NguyenJan 18, 2018
  63. Duy NguyenJan 18, 2018
  64. Jeff KingJan 18, 2018
  65. SZEDER GáborJan 18, 2018
  66. Duy NguyenJan 19, 2018
  67. 0/5 Travis CI: don't run the test suite as root in the 32 bit Linux buildSZEDER Gábor, Jan 22, 2018
  68. 2/5 travis-ci: use 'set -e' in the 32 bit Linux build jobSZEDER Gábor, Jan 22, 2018
  69. Jeff KingJan 23, 2018
  70. Jeff KingJan 23, 2018
  71. SZEDER GáborJan 24, 2018
  72. Jeff KingJan 24, 2018
  73. 5/5 travis-ci: don't fail if user already exists on 32 bit Linux build jobSZEDER Gábor, Jan 22, 2018
  74. Jeff KingJan 23, 2018
  75. Duy NguyenJan 24, 2018
  76. SZEDER GáborJan 24, 2018
  77. 1/5 travis-ci: use 'set -x' for the commands under 'su' in the 32 bit Linux buildSZEDER Gábor, Jan 22, 2018
  78. 3/5 travis-ci: don't repeat the path of the cache directorySZEDER Gábor, Jan 22, 2018
  79. Jeff KingJan 23, 2018
  80. SZEDER GáborJan 24, 2018
  81. 4/5 travis-ci: don't run the test suite as root in the 32 bit Linux buildSZEDER Gábor, Jan 22, 2018
  82. Jeff KingJan 23, 2018
  83. SZEDER GáborJan 24, 2018
  84. Jeff KingJan 24, 2018
  85. Jeff KingJan 24, 2018
  86. Jeff KingJan 24, 2018
  87. SZEDER GáborJan 22, 2018
  88. Eric SunshineJan 22, 2018
  89. SZEDER GáborJan 22, 2018
  90. Duy NguyenJan 24, 2018
  91. Lars SchneiderJan 26, 2018
  92. Thomas GummererJan 14, 2018
  93. Thomas GummererJan 18, 2018
  94. Junio C HamanoJan 19, 2018
  95. Thomas GummererJan 19, 2018

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.