# [PATCH 0/2] upload-pack.c: fix partial clone allowed filter regression 4 messages from 2020-12-03 to 2020-12-04. Participants: Taylor Blau, Jeff King. Thread: https://gitlist.dev/t/54754 ## Taylor Blau, 2020-12-03 18:55 Subject: [PATCH 0/2] upload-pack.c: fix partial clone allowed filter regression Message-ID: URL: https://gitlist.dev/e/cover.1607021483.git.me%40ttaylorr.com ``` Hi, Here's a regression fix that I noticed this morning while deploying 2.28 out to GitHub. The gist is that we accidentally call 'string_list_clear(..., 1)' when freeing the list of allowed/banned object filters, but the ->util pointer either contains '0', or '1'. So, the free() of the util pointer is bogus, and we end up crashing. The fix in the second patch is straightforward, but it doesn't work without the first patch, which teaches 'git clone' to match 'git fetch' and not ignore an error from 'transport_fetch_refs()'. Unfortunately, this regression has been in Git since 2.28. Fortunately, nobody has seemed to notice, so I doubt that it's bitten anybody out in the wild. But, it's still worth fixing, since it is so obviously broken. Thanks, Taylor Taylor Blau (2): builtin/clone.c: don't ignore transport_fetch_refs() errors upload-pack.c: don't free allowed_filters util pointers builtin/clone.c | 15 +++++++++++---- t/t5616-partial-clone.sh | 10 +++++++++- upload-pack.c | 2 +- 3 files changed, 21 insertions(+), 6 deletions(-) -- 2.29.2.533.g07db1f5344 ``` ## Taylor Blau, 2020-12-03 18:55 Subject: [PATCH 2/2] upload-pack.c: don't free allowed_filters util pointers Message-ID: URL: https://gitlist.dev/e/a3c721054f337eac2b3a038e51693f44b68dd619.1607021483.git.me%40ttaylorr.com In-Reply-To: ``` To keep track of which object filters are allowed or not, 'git upload-pack' stores the name of each filter in a string_list, and sets it ->util pointer to be either 0 or 1, indicating whether it is banned or allowed. Later on, we attempt to clear that list, but we incorrectly ask for the util pointers to be free()'d, too. This behavior (introduced back in 6dd3456a8c (upload-pack.c: allow banning certain object filter(s), 2020-08-03)) leads to an invalid free, and causes us to crash. In order to trigger this, one needs to fetch from a server that (a) has at least one object filter allowed, and (b) issue a fetch that contains a subset of the allowed filters (i.e., we cannot ask for a banned filter, since this causes us to die() before we hit the bogus string_list_clear()). In that case, whatever banned filters exist will cause a noop free() (since those ->util pointers are set to 0), but the first allowed filter we try to free will crash us. We never noticed this in the tests because we didn't have an example of setting 'uploadPackFilter' configuration variables and then following up with a valid fetch. The first new 'git clone' prevents further regression here. For good measure on top, add a test which checks the same behavior at a tree depth greater than 0. Signed-off-by: Taylor Blau --- t/t5616-partial-clone.sh | 10 +++++++++- upload-pack.c | 2 +- 2 files changed, 10 insertions(+), 2 deletions(-) diff --git a/t/t5616-partial-clone.sh b/t/t5616-partial-clone.sh index f4d49d8335..5da945cf15 100755 --- a/t/t5616-partial-clone.sh +++ b/t/t5616-partial-clone.sh @@ -281,7 +281,15 @@ test_expect_success 'upload-pack limits tree depth filters' ' test_config -C srv.bare uploadpackfilter.tree.maxDepth 0 && test_must_fail ok=sigpipe git clone --no-checkout --filter=tree:1 \ "file://$(pwd)/srv.bare" pc3 2>err && - test_i18ngrep "tree filter allows max depth 0, but got 1" err + test_i18ngrep "tree filter allows max depth 0, but got 1" err && + + git clone --no-checkout --filter=tree:0 "file://$(pwd)/srv.bare" pc4 && + + test_config -C srv.bare uploadpackfilter.tree.maxDepth 5 && + git clone --no-checkout --filter=tree:5 "file://$(pwd)/srv.bare" pc5 && + test_must_fail ok=sigpipe git clone --no-checkout --filter=tree:6 \ + "file://$(pwd)/srv.bare" pc6 2>err && + test_i18ngrep "tree filter allows max depth 5, but got 6" err ' test_expect_success 'partial clone fetches blobs pointed to by refs even if normally filtered out' ' diff --git a/upload-pack.c b/upload-pack.c index 5dc8e1f844..d89c7e4a02 100644 --- a/upload-pack.c +++ b/upload-pack.c @@ -154,7 +154,7 @@ static void upload_pack_data_clear(struct upload_pack_data *data) string_list_clear(&data->deepen_not, 0); object_array_clear(&data->extra_edge_obj); list_objects_filter_release(&data->filter_options); - string_list_clear(&data->allowed_filters, 1); + string_list_clear(&data->allowed_filters, 0); free((char *)data->pack_objects_hook); } -- 2.29.2.533.g07db1f5344 ``` ## Taylor Blau, 2020-12-03 18:55 Subject: [PATCH 1/2] builtin/clone.c: don't ignore transport_fetch_refs() errors Message-ID: <27a817597850a2e6a13efe1d5ea105d70e1e4040.1607021483.git.me@ttaylorr.com> URL: https://gitlist.dev/e/27a817597850a2e6a13efe1d5ea105d70e1e4040.1607021483.git.me%40ttaylorr.com In-Reply-To: ``` If 'git clone' couldn't execute 'transport_fetch_refs()' (e.g., because of an error on the remote's side in 'git upload-pack'), then it will silently ignore it. Even though this has been the case at least since clone was ported to C (way back in 8434c2f1af (Build in clone, 2008-04-27)), 'git fetch' doesn't ignore these and reports any failures it sees. That suggests that ignoring the return value in 'git clone' is simply an oversight that should be corrected. That's exactly what this patch does. (Noticing and fixing this is no coincidence, we'll want it in the next patch in order to demonstrate a regression in 'git upload-pack' via a 'git clone'.) There's no additional logging here, but that matches how 'git fetch' handles the same case. An assumption there is that whichever part of transport_fetch_refs() fails will complain loudly, so any additional logging here is redundant. Co-authored-by: Jeff King Signed-off-by: Jeff King Signed-off-by: Taylor Blau --- builtin/clone.c | 15 +++++++++++---- 1 file changed, 11 insertions(+), 4 deletions(-) diff --git a/builtin/clone.c b/builtin/clone.c index a0841923cf..a5630337e4 100644 --- a/builtin/clone.c +++ b/builtin/clone.c @@ -1293,8 +1293,11 @@ int cmd_clone(int argc, const char **argv, const char *prefix) break; } - if (!is_local && !complete_refs_before_fetch) - transport_fetch_refs(transport, mapped_refs); + if (!is_local && !complete_refs_before_fetch) { + err = transport_fetch_refs(transport, mapped_refs); + if (err) + goto cleanup; + } remote_head = find_ref_by_name(refs, "HEAD"); remote_head_points_at = @@ -1339,8 +1342,11 @@ int cmd_clone(int argc, const char **argv, const char *prefix) if (is_local) clone_local(path, git_dir); - else if (refs && complete_refs_before_fetch) - transport_fetch_refs(transport, mapped_refs); + else if (refs && complete_refs_before_fetch) { + err = transport_fetch_refs(transport, mapped_refs); + if (err) + goto cleanup; + } update_remote_refs(refs, mapped_refs, remote_head_points_at, branch_top.buf, reflog_msg.buf, transport, @@ -1367,6 +1373,7 @@ int cmd_clone(int argc, const char **argv, const char *prefix) junk_mode = JUNK_LEAVE_REPO; err = checkout(submodule_progress); +cleanup: free(remote_name); strbuf_release(&reflog_msg); strbuf_release(&branch_top); -- 2.29.2.533.g07db1f5344 ``` ## Jeff King, 2020-12-04 21:04 Subject: Re: [PATCH 2/2] upload-pack.c: don't free allowed_filters util pointers Message-ID: URL: https://gitlist.dev/e/X8qkRRrWCbDn1MAu%40coredump.intra.peff.net In-Reply-To: ``` On Thu, Dec 03, 2020 at 01:55:18PM -0500, Taylor Blau wrote: > We never noticed this in the tests because we didn't have an example of > setting 'uploadPackFilter' configuration variables and then following up > with a valid fetch. The first new 'git clone' prevents further > regression here. For good measure on top, add a test which checks the > same behavior at a tree depth greater than 0. Very nice. The fix is obviously the right thing, but I like the improvements to the tests here especially. -Peff ```