git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: [PATCH 09/13] sparse-checkout: refactor temporary sparse_checkout_patterns

From
Jeff King <peff@peff.net>
Date
Jun 4, 2024, 09:53 UTC
Message-ID
<20240604095348.GB1298378@coredump.intra.peff.net>
In-Reply-To
<Zl7Fe6132gy1JclD@framework>
On Tue, Jun 04, 2024 at 09:42:51AM +0200, Patrick Steinhardt wrote:
Show 8 quoted lines
> > But there are two possible leaks here:
> > 
> >   1. If the index already had a populated sparse_checkout_patterns,
> >      we've obliterated it. We can fix this by saving and restoring it,
> >      rather than always setting it back to NULL.
> 
> So this isn't only a leak, but also a potential bug because we did not
> restore the old pattern list?

That's a good question, but I think in practice, it's not possible to trigger the bug. Within a run that calls this function, we only care about index->sparse_checkout_patterns in the context of this function itself (that is, we do not otherwise try to read the sparse index). And in this function, we always want to use the pattern_list that the caller hands us.

So nobody actually cares about the "restored" pattern list at all (except that in the lazy-load case, we leak it). Every call will overwrite it anyway.

Show 18 quoted lines
> > @@ -241,7 +243,12 @@ static int update_working_directory(struct pattern_list *pl)
> >  
> >  	clean_tracked_sparse_directories(r);
> >  
> > -	r->index->sparse_checkout_patterns = NULL;
> > +	if (r->index->sparse_checkout_patterns != pl) {
> > +		clear_pattern_list(r->index->sparse_checkout_patterns);
> > +		FREE_AND_NULL(r->index->sparse_checkout_patterns);
> > +	} else {
> > +		r->index->sparse_checkout_patterns = old_pl;
> > +	}
> >  	return result;
> >  }
> 
> What I find weird is that we sometimes restore the old value, and
> sometimes we don't. I get that it makes sense to conditionally free only
> the lazy-loaded list. But shouldn't we then unconditionally assign
> `old_pl`?

Yes, I think it would be more correct to do so (otherwise we risk leaking old_pl). In practice, as I mentioned above, this is the only function that actually assigns to the index list. So after this hunk, I think we'd always come in to the function with a NULL value in the index (it starts NULL, we restore NULL after using a passed-in value, and we restore free-and-NULL after lazy-loading).

So we _could_ actually drop the "save old_pl" hunk from the beginning of the function entirely. I only hit that case after trying something like:

  /* only throw away patterns if they were the ones we own */
  if (r->index->sparse_checkout_patterns == pl)
	r->index->sparse_checkout_patterns = NULL;

at the end of the function to fix the leak. And with that code, we _do_ see the lazy-loaded entries in subsequent calls. If we instead do:

  /* throw away lazy-loaded ones */
  if (r->index->sparse_checkout_patterns != pl) {
	clear_pattern_list(r->index->sparse_checkout_patterns);
	FREE_AND_NULL(r->index->sparse_checkout_patterns);
  }

Then I think you don't really need the else clause at all. But I was trying to be defensive and not make too many assumptions. I think in that spirit, just restoring old_pl always is the right thing, even though I don't think we'd ever trigger that.

-Peff
Previous: Patrick SteinhardtNext: Jeff King
Message 15 of 43 in “leak fixes for sparse-checkout code”
  1. 0/13 leak fixes for sparse-checkout codeJeff King, May 31, 2024
  2. 01/13 sparse-checkout: free string list in write_cone_to_file()Jeff King, May 31, 2024
  3. 02/13 sparse-checkout: pass string literals directly to add_pattern()Jeff King, May 31, 2024
  4. Junio C HamanoMay 31, 2024
  5. Jeff KingJun 4, 2024
  6. 03/13 dir.c: free strings in sparse cone pattern hashmapsJeff King, May 31, 2024
  7. 04/13 sparse-checkout: clear patterns when init() sees existing sparse fileJeff King, May 31, 2024
  8. 05/13 dir.c: free removed sparse-pattern hashmap entriesJeff King, May 31, 2024
  9. 06/13 dir.c: always copy input to add_pattern()Jeff King, May 31, 2024
  10. Junio C HamanoMay 31, 2024
  11. 07/13 sparse-checkout: reuse --stdin buffer when reading patternsJeff King, May 31, 2024
  12. 08/13 sparse-checkout: always free "line" strbuf after reading inputJeff King, May 31, 2024
  13. 09/13 sparse-checkout: refactor temporary sparse_checkout_patternsJeff King, May 31, 2024
  14. Patrick SteinhardtJun 4, 2024
  15. Jeff KingJun 4, 2024
  16. 10/13 sparse-checkout: free sparse_filename after useJeff King, May 31, 2024
  17. Patrick SteinhardtJun 4, 2024
  18. Jeff KingJun 4, 2024
  19. Patrick SteinhardtJun 4, 2024
  20. 11/13 sparse-checkout: free pattern list in sparse_checkout_list()Jeff King, May 31, 2024
  21. 12/13 sparse-checkout: free string list after displayingJeff King, May 31, 2024
  22. 13/13 sparse-checkout: free duplicate hashmap entriesJeff King, May 31, 2024
  23. Patrick SteinhardtJun 4, 2024
  24. Jeff KingMay 31, 2024
  25. Junio C HamanoMay 31, 2024
  26. 0/13 leak fixes for sparse-checkout codeJeff King, Jun 4, 2024
  27. 01/13 sparse-checkout: free string list in write_cone_to_file()Jeff King, Jun 4, 2024
  28. 02/13 sparse-checkout: pass string literals directly to add_pattern()Jeff King, Jun 4, 2024
  29. 03/13 dir.c: free strings in sparse cone pattern hashmapsJeff King, Jun 4, 2024
  30. 04/13 sparse-checkout: clear patterns when init() sees existing sparse fileJeff King, Jun 4, 2024
  31. 05/13 dir.c: free removed sparse-pattern hashmap entriesJeff King, Jun 4, 2024
  32. 06/13 dir.c: always copy input to add_pattern()Jeff King, Jun 4, 2024
  33. René ScharfeJun 5, 2024
  34. Jeff KingJun 5, 2024
  35. Junio C HamanoJun 5, 2024
  36. 07/13 sparse-checkout: reuse --stdin buffer when reading patternsJeff King, Jun 4, 2024
  37. 08/13 sparse-checkout: always free "line" strbuf after reading inputJeff King, Jun 4, 2024
  38. 09/13 sparse-checkout: refactor temporary sparse_checkout_patternsJeff King, Jun 4, 2024
  39. 10/13 sparse-checkout: free sparse_filename after useJeff King, Jun 4, 2024
  40. 11/13 sparse-checkout: free pattern list in sparse_checkout_list()Jeff King, Jun 4, 2024
  41. 12/13 sparse-checkout: free string list after displayingJeff King, Jun 4, 2024
  42. 13/13 sparse-checkout: free duplicate hashmap entriesJeff King, Jun 4, 2024
  43. Patrick SteinhardtJun 4, 2024

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.