git/list[1] front-page[2] threads[3] people[4] search[5] about
 

[PATCH 4/3] list-objects-filter: use empty string instead of NULL for sparse "base"

From
Jeff King <peff@peff.net>
Date
Sep 15, 2019, 16:51 UTC
Message-ID
<20190915165156.GA28436@sigill.intra.peff.net>
In-Reply-To
<20190915010942.GA19787@sigill.intra.peff.net>
On Sat, Sep 14, 2019 at 09:09:42PM -0400, Jeff King wrote:
Show 14 quoted lines
> On Mon, Sep 09, 2019 at 01:08:24PM -0400, Jeff King wrote:
> 
> > I'll work up what I sent earlier into a real patch, and include some of
> > this discussion.
> 
> Here it is. I pulled Jon's tests out into their own patch (mostly
> because it makes it easier to give credit). Then patch 2 is my fix, and
> patch 3 is the message fixups he had done.
> 
> This replaces what's queued in js/partial-clone-sparse-blob.
> 
>   [1/3]: t5616: test cloning/fetching with sparse:oid=<oid> filter
>   [2/3]: list-objects-filter: delay parsing of sparse oid
>   [3/3]: list-objects-filter: give a more specific error sparse parsing error

And here's a bonus patch that I found while running under ASan/UBSan (since I wanted to double-check the memory handling of patch 2 when merged with 'next').

-- >8 --
Subject: list-objects-filter: use empty string instead of NULL for sparse "base"

We use add_excludes_from_blob_to_list() to parse a sparse blob. Since we don't have a base path, we pass NULL and 0 for the base and baselen, respectively. But the rest of the exclude code passes a literal empty string instead of NULL for this case. And indeed, we eventually end up with match_pathname() calling fspathncmp(), which then calls the system strncmp(path, base, baselen).

This works on many platforms, which notice that baselen is 0 and do not look at the bytes of "base" at all. But it does violate the C standard, and building with SANITIZE=undefined will complain. You can also see it by instrumenting fspathncmp like this:

	diff --git a/dir.c b/dir.c
	index d021c908e5..4bb3d3ec96 100644
	--- a/dir.c
	+++ b/dir.c
	@@ -71,6 +71,8 @@ int fspathcmp(const char *a, const char *b)
	 int fspathncmp(const char *a, const char *b, size_t count)
	 {
	+	if (!a || !b)
	+		BUG("null fspathncmp arguments");
	 	return ignore_case ? strncasecmp(a, b, count) : strncmp(a, b, count);
	 }

We could perhaps be more defensive in match_pathname(), but even if we did so, it makes sense for this code to match the rest of the exclude callers.

Signed-off-by: Jeff King <peff@peff.net>
---
 list-objects-filter.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/list-objects-filter.c b/list-objects-filter.c
index 50f0c6d07b..83c788e8b5 100644
--- a/list-objects-filter.c
+++ b/list-objects-filter.c
@@ -472,7 +472,7 @@ static void *filter_sparse_oid__init(
 		die(_("unable to access sparse blob in '%s'"),
 		    filter_options->sparse_oid_name);
 	d->omits = omitted;
-	if (add_excludes_from_blob_to_list(&sparse_oid, NULL, 0, &d->el) < 0)
+	if (add_excludes_from_blob_to_list(&sparse_oid, "", 0, &d->el) < 0)
 		die(_("unable to parse sparse filter data in %s"),
 		    oid_to_hex(&sparse_oid));
 
-- 
2.23.0.667.gcccf1fbb03
Previous: Jeff KingNext: Jeff Hostetler
Message 15 of 19 in “partial-clone: fix two issues with sparse filter handling”
  1. 0/2 partial-clone: fix two issues with sparse filter handlingJon Simons, Aug 29, 2019
  2. 1/2 list-objects-filter: only parse sparse OID when 'have_git_dir'Jon Simons, Aug 29, 2019
  3. Junio C HamanoAug 30, 2019
  4. Jeff KingSep 4, 2019
  5. Junio C HamanoSep 5, 2019
  6. Jeff HostetlerSep 9, 2019
  7. Jeff KingSep 9, 2019
  8. Jeff HostetlerSep 9, 2019
  9. 0/3 clone --filter=sparse:oid bugsJeff King, Sep 15, 2019
  10. 1/3 t5616: test cloning/fetching with sparse:oid=<oid> filterJeff King, Sep 15, 2019
  11. 2/3 list-objects-filter: delay parsing of sparse oidJeff King, Sep 15, 2019
  12. Jeff KingSep 15, 2019
  13. Junio C HamanoSep 17, 2019
  14. 3/3 list-objects-filter: give a more specific error sparse parsing errorJeff King, Sep 15, 2019
  15. 4/3 list-objects-filter: use empty string instead of NULL for sparse "base"Jeff King, Sep 15, 2019
  16. Jeff HostetlerSep 16, 2019
  17. Junio C HamanoSep 9, 2019
  18. Jeff HostetlerSep 9, 2019
  19. 2/2 list-objects-filter: handle unresolved sparse filter OIDJon Simons, Aug 29, 2019

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.