git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: [PATCH 1/8] sparse-index: create expand_to_pattern_list()

From
Victoria Dye <vdye@github.com>
Date
May 16, 2022, 20:36 UTC
Message-ID
<12df1777-9f02-ca6f-7d17-57aebe25d06c@github.com>
In-Reply-To
<f1194d56d331611446c285a1d070509d73bd5f43.1652724693.git.gitgitgadget@gmail.com>
Derrick Stolee via GitGitGadget wrote:
Show 42 quoted lines
> From: Derrick Stolee <dstolee@microsoft.com>
> 
> This is the first change in a series to allow modifying the
> sparse-checkout pattern set without expanding a sparse index to a full
> one in the process. Here, we focus on the problem of expanding the
> pattern set through a command like 'git sparse-checkout add <path>'
> which needs to create new index entries for the paths now being written
> to the worktree.
> 
> To achieve this, we need to be able to replace sparse directory entries
> with their contained files and subdirectories. Once this is complete,
> other code paths can discover those cache entries and write the
> corresponding files to disk before committing the index.
> 
> We already have logic in ensure_full_index() that expands the index
> entries, so we will use that as our base. Create a new method,
> expand_to_pattern_list(), which takes a pattern list, but for now mostly
> ignores it. The current implementation is only correct when the pattern
> list is NULL as that does the same as ensure_full_index(). In fact,
> ensure_full_index() is converted to a shim over
> expand_to_pattern_list().
> 
> A future update will actually implement expand_to_pattern_list() to its
> full capabilities. For now, it is created and documented.
> 
> Signed-off-by: Derrick Stolee <derrickstolee@github.com>
> ---
>  sparse-index.c | 35 ++++++++++++++++++++++++++++++++---
>  sparse-index.h | 14 ++++++++++++++
>  2 files changed, 46 insertions(+), 3 deletions(-)
> 
> diff --git a/sparse-index.c b/sparse-index.c
> index 8636af72de5..37c7df877a6 100644
> --- a/sparse-index.c
> +++ b/sparse-index.c
> @@ -248,19 +248,41 @@ static int add_path_to_index(const struct object_id *oid,
>  	return 0;
>  }
>  
> -void ensure_full_index(struct index_state *istate)
> +void expand_to_pattern_list(struct index_state *istate,
> +			      struct pattern_list *pl)
Hyper-nit: I don't think this is aligned (it's probably not worth fixing
unless you end up rerolling for something else).
Show 76 quoted lines
>  {
>  	int i;
>  	struct index_state *full;
>  	struct strbuf base = STRBUF_INIT;
>  
> +	/*
> +	 * If the index is already full, then keep it full. We will convert
> +	 * it to a sparse index on write, if possible.
> +	 */
>  	if (!istate || !istate->sparse_index)
>  		return;
>  
> +	/*
> +	 * If our index is sparse, but our new pattern set does not use
> +	 * cone mode patterns, then we need to expand the index before we
> +	 * continue. A NULL pattern set indicates a full expansion to a
> +	 * full index.
> +	 */
> +	if (pl && !pl->use_cone_patterns)
> +		pl = NULL;
> +
>  	if (!istate->repo)
>  		istate->repo = the_repository;
>  
> -	trace2_region_enter("index", "ensure_full_index", istate->repo);
> +	/*
> +	 * A NULL pattern set indicates we are expanding a full index, so
> +	 * we use a special region name that indicates the full expansion.
> +	 * This is used by test cases, but also helps to differentiate the
> +	 * two cases.
> +	 */
> +	trace2_region_enter("index",
> +			    pl ? "expand_to_pattern_list" : "ensure_full_index",
> +			    istate->repo);
>  
>  	/* initialize basics of new index */
>  	full = xcalloc(1, sizeof(struct index_state));
> @@ -322,7 +344,14 @@ void ensure_full_index(struct index_state *istate)
>  	cache_tree_free(&istate->cache_tree);
>  	cache_tree_update(istate, 0);
>  
> -	trace2_region_leave("index", "ensure_full_index", istate->repo);
> +	trace2_region_leave("index",
> +			    pl ? "expand_to_pattern_list" : "ensure_full_index",
> +			    istate->repo);
> +}
> +
> +void ensure_full_index(struct index_state *istate)
> +{
> +	expand_to_pattern_list(istate, NULL);
>  }
>  
>  void ensure_correct_sparsity(struct index_state *istate)
> diff --git a/sparse-index.h b/sparse-index.h
> index 633d4fb7e31..037b541f49d 100644
> --- a/sparse-index.h
> +++ b/sparse-index.h
> @@ -23,4 +23,18 @@ void expand_to_path(struct index_state *istate,
>  struct repository;
>  int set_sparse_index_config(struct repository *repo, int enable);
>  
> +struct pattern_list;
> +
> +/**
> + * Scan the given index and compare its entries to the given pattern list.
> + * If the index is sparse and the pattern list uses cone mode patterns,
> + * then modify the index to contain the all of the file entries within that
> + * new pattern list. This expands sparse directories only as far as needed.
> + *
> + * If the pattern list is NULL or does not use cone mode patterns, then the
> + * index is expanded to a full index.
> + */
> +void expand_to_pattern_list(struct index_state *istate,
> +			      struct pattern_list *pl);
> +
>  #endif
Previous: Derrick Stolee via GitGitGadgetNext: Derrick Stolee
Message 3 of 55 in “Sparse index: integrate with sparse-checkout”
  1. 0/8 Sparse index: integrate with sparse-checkoutDerrick Stolee via GitGitGadget, May 16, 2022
  2. 1/8 sparse-index: create expand_to_pattern_list()Derrick Stolee via GitGitGadget, May 16, 2022
  3. Victoria DyeMay 16, 2022
  4. Derrick StoleeMay 16, 2022
  5. 4/8 sparse-checkout: --no-sparse-index needs a full indexDerrick Stolee via GitGitGadget, May 16, 2022
  6. 2/8 sparse-index: introduce partially-sparse indexesDerrick Stolee via GitGitGadget, May 16, 2022
  7. 6/8 sparse-index: complete partial expansionDerrick Stolee via GitGitGadget, May 16, 2022
  8. Victoria DyeMay 16, 2022
  9. Derrick StoleeMay 17, 2022
  10. 7/8 p2000: add test for 'git sparse-checkout [add|set]'Derrick Stolee via GitGitGadget, May 16, 2022
  11. 5/8 sparse-index: partially expand directoriesDerrick Stolee via GitGitGadget, May 16, 2022
  12. Victoria DyeMay 16, 2022
  13. 3/8 cache-tree: implement cache_tree_find_path()Derrick Stolee via GitGitGadget, May 16, 2022
  14. 8/8 sparse-checkout: integrate with sparse indexDerrick Stolee via GitGitGadget, May 16, 2022
  15. Victoria DyeMay 16, 2022
  16. Derrick StoleeMay 17, 2022
  17. 00/10 Sparse index: integrate with sparse-checkoutDerrick Stolee via GitGitGadget, May 19, 2022
  18. 02/10 t1092: stress test 'git sparse-checkout set'Derrick Stolee via GitGitGadget, May 19, 2022
  19. 03/10 sparse-index: create expand_to_pattern_list()Derrick Stolee via GitGitGadget, May 19, 2022
  20. Junio C HamanoMay 19, 2022
  21. Derrick StoleeMay 20, 2022
  22. 04/10 sparse-index: introduce partially-sparse indexesDerrick Stolee via GitGitGadget, May 19, 2022
  23. Junio C HamanoMay 19, 2022
  24. Derrick StoleeMay 20, 2022
  25. Junio C HamanoMay 20, 2022
  26. 01/10 t1092: refactor 'sparse-index contents' testDerrick Stolee via GitGitGadget, May 19, 2022
  27. 05/10 cache-tree: implement cache_tree_find_path()Derrick Stolee via GitGitGadget, May 19, 2022
  28. Junio C HamanoMay 19, 2022
  29. Derrick StoleeMay 20, 2022
  30. 07/10 sparse-index: partially expand directoriesDerrick Stolee via GitGitGadget, May 19, 2022
  31. Junio C HamanoMay 20, 2022
  32. Derrick StoleeMay 20, 2022
  33. 06/10 sparse-checkout: --no-sparse-index needs a full indexDerrick Stolee via GitGitGadget, May 19, 2022
  34. Junio C HamanoMay 19, 2022
  35. 09/10 p2000: add test for 'git sparse-checkout [add|set]'Derrick Stolee via GitGitGadget, May 19, 2022
  36. 10/10 sparse-checkout: integrate with sparse indexDerrick Stolee via GitGitGadget, May 19, 2022
  37. 08/10 sparse-index: complete partial expansionDerrick Stolee via GitGitGadget, May 19, 2022
  38. Junio C HamanoMay 21, 2022
  39. Derrick StoleeMay 23, 2022
  40. Derrick StoleeMay 23, 2022
  41. Junio C HamanoMay 23, 2022
  42. Junio C HamanoMay 23, 2022
  43. Derrick StoleeMay 25, 2022
  44. Junio C HamanoMay 25, 2022
  45. 00/10 Sparse index: integrate with sparse-checkoutDerrick Stolee via GitGitGadget, May 23, 2022
  46. 01/10 t1092: refactor 'sparse-index contents' testDerrick Stolee via GitGitGadget, May 23, 2022
  47. 04/10 sparse-index: introduce partially-sparse indexesDerrick Stolee via GitGitGadget, May 23, 2022
  48. 03/10 sparse-index: create expand_index()Derrick Stolee via GitGitGadget, May 23, 2022
  49. 02/10 t1092: stress test 'git sparse-checkout set'Derrick Stolee via GitGitGadget, May 23, 2022
  50. 05/10 cache-tree: implement cache_tree_find_path()Derrick Stolee via GitGitGadget, May 23, 2022
  51. 07/10 sparse-index: partially expand directoriesDerrick Stolee via GitGitGadget, May 23, 2022
  52. 06/10 sparse-checkout: --no-sparse-index needs a full indexDerrick Stolee via GitGitGadget, May 23, 2022
  53. 09/10 p2000: add test for 'git sparse-checkout [add|set]'Derrick Stolee via GitGitGadget, May 23, 2022
  54. 08/10 sparse-index: complete partial expansionDerrick Stolee via GitGitGadget, May 23, 2022
  55. 10/10 sparse-checkout: integrate with sparse indexDerrick Stolee via GitGitGadget, May 23, 2022

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.