{"thread":{"id":"66848","subject":"[PATCH 0/2] repack: write filtered-out objects as loose objects","startedAt":"2026-11-13T22:11:49Z","lastAt":"2026-11-14T20:08:57Z","messageCount":4,"participants":["Adithya Chakilam via GitGitGadget","Junio C Hamano"],"isPatch":true,"patchVersion":1,"patchTotal":2},"messages":[{"id":"558429","messageId":"pull.2314.git.1794607911.gitgitgadget@gmail.com","threadId":"66848","inReplyTo":null,"subject":"[PATCH 0/2] repack: write filtered-out objects as loose objects","fromName":"Adithya Chakilam via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2026-11-13T22:11:49Z","receivedAt":"2026-11-13T22:11:49Z","isPatch":true,"sender":{"key":"achakila@codeaurora.org","avatar":null},"body":"Some key-value object stores manage and retrieve Git objects individually.\nThe existing --filter-to option writes filtered-out objects into packs, so\nusing it with such stores requires unpacking the resulting packs before\nimporting the objects.\n\nThis series adds --filter-to-loose=dir to git repack, allowing filtered-out\nobjects to be written directly as loose objects into another object\ndirectory.\n\nWith --filter-to-loose, repack streams the filtered pack into\nunpack-objects, avoiding that extra step and the intermediate pack file. For\nexample:\n\ngit repack -a --filter=blob=1m --filter-to-loose=/path/to/objects\n\nThe destination is treated as a Git object directory.\n\nThe first patch adds the option for objects already stored in packs. The\nsecond extends it to include loose objects excluded by the filter, so the\ndestination also receives those objects. Objects already present in the\ndestination are not rewritten. With -d, the source copies of those loose\nobjects are removed after all objects have been written successfully.\n\nAdithya Chakilam (2):\n  repack: add --filter-to-loose to write filtered-out objects as loose\n    objects\n  repack: also copy loose objects excluded by --filter-to-loose\n\n Documentation/git-repack.adoc |  14 ++-\n builtin/repack.c              |  32 +++++-\n repack-filtered.c             | 152 ++++++++++++++++++++++++++-\n repack.h                      |  14 +++\n t/t7700-repack.sh             | 189 ++++++++++++++++++++++++++++++++++\n 5 files changed, 394 insertions(+), 7 deletions(-)\n\n\nbase-commit: 1891de46f79dc444a47db1c87bd0c8c76b5463f2\nPublished-As: https://github.com/gitgitgadget/git/releases/tag/pr-2314%2Fachakilam-nv%2Ffilter-to-loose-v1\nFetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-2314/achakilam-nv/filter-to-loose-v1\nPull-Request: https://github.com/gitgitgadget/git/pull/2314\n-- \ngitgitgadget\n\n"},{"id":"558430","messageId":"4488a1534ebe7def72055ab5530d4f03177fbe52.1794607911.git.gitgitgadget@gmail.com","threadId":"66848","inReplyTo":"pull.2314.git.1794607911.gitgitgadget@gmail.com","subject":"[PATCH 1/2] repack: add --filter-to-loose to write filtered-out objects as loose objects","fromName":"Adithya Chakilam via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2026-11-13T22:11:50Z","receivedAt":"2026-11-13T22:11:50Z","isPatch":true,"sender":{"key":"achakila@codeaurora.org","avatar":null},"body":"From: Adithya Chakilam <achakilam@nvidia.com>\n\nSome external object stores (like key-value) manage and retrieve git\nobjects individually. Although repack's --filter-to option can separate\nobjects from the main object store, it writes them into packs. Using\nthose objects individually then requires an additional unpacking step.\n\nAdd --filter-to-loose=<dir> to write filtered-out objects directly as\nloose objects into <dir>, treating it as an object directory. Stream\nthe filtered pack into unpack-objects instead of writing an intermediate\npack file to disk.\n\nThe filtered pack is built with --stdin-packs, so this initially handles\nonly objects already stored in packs. Loose objects in the source\nrepository are left in place.\n\nSigned-off-by: Adithya Chakilam <achakilam@nvidia.com>\n---\n Documentation/git-repack.adoc |  10 ++-\n builtin/repack.c              |  21 ++++--\n repack-filtered.c             |  44 +++++++++++-\n repack.h                      |   1 +\n t/t7700-repack.sh             | 130 ++++++++++++++++++++++++++++++++++\n 5 files changed, 199 insertions(+), 7 deletions(-)\n\ndiff --git a/Documentation/git-repack.adoc b/Documentation/git-repack.adoc\nindex a1f9e64f66..df1390154f 100644\n--- a/Documentation/git-repack.adoc\n+++ b/Documentation/git-repack.adoc\n@@ -12,7 +12,7 @@ SYNOPSIS\n 'git repack' [-a] [-A] [-d] [-f] [-F] [-l] [-n] [-q] [-b] [-m]\n \t[--window=<n>] [--depth=<n>] [--threads=<n>] [--keep-pack=<pack-name>]\n \t[--write-midx[=<mode>]] [--name-hash-version=<n>] [--path-walk]\n-\t[--filter=<filter-spec>] [--drop-filtered [--dry-run]]\n+\t[--filter=<filter-spec> [--filter-to-loose=<dir>]] [--drop-filtered [--dry-run]]\n \n DESCRIPTION\n -----------\n@@ -183,6 +183,14 @@ depth is 4095.\n \t`objects` and `objects/info/alternates` sections of\n \tlinkgit:gitrepository-layout[5].\n \n+--filter-to-loose=<dir>::\n+\tLike `--filter-to`, but treat `<dir>` as an object directory\n+\tand write the filtered out objects there as loose objects\n+\tinstead of as a pack. Cannot be combined with `--filter-to`\n+\tor `--max-pack-size`. Objects already present in `<dir>` are\n+\tnot rewritten. All objects are written before any existing\n+\tpack is removed.\n+\n --drop-filtered::\n \tDelete the local objects that match the `--filter` specification\n \tinstead of keeping them in a separate packfile, reclaiming the\ndiff --git a/builtin/repack.c b/builtin/repack.c\nindex 21835453c6..140a992f1b 100644\n--- a/builtin/repack.c\n+++ b/builtin/repack.c\n@@ -41,7 +41,7 @@ static const char *const git_repack_usage[] = {\n \tN_(\"git repack [-a] [-A] [-d] [-f] [-F] [-l] [-n] [-q] [-b] [-m]\\n\"\n \t   \"[--window=<n>] [--depth=<n>] [--threads=<n>] [--keep-pack=<pack-name>]\\n\"\n \t   \"[--write-midx[=<mode>]] [--name-hash-version=<n>] [--path-walk]\\n\"\n-\t   \"[--filter=<filter-spec>] [--drop-filtered [--dry-run]]\"),\n+\t   \"[--filter=<filter-spec> [--filter-to-loose=<dir>]] [--drop-filtered [--dry-run]]\"),\n \tNULL\n };\n \n@@ -182,6 +182,7 @@ int cmd_repack(int argc,\n \tconst char *cruft_expiration = NULL;\n \tconst char *expire_to = NULL;\n \tconst char *filter_to = NULL;\n+\tconst char *filter_to_loose = NULL;\n \tconst char *opt_window = NULL;\n \tconst char *opt_window_memory = NULL;\n \tconst char *opt_depth = NULL;\n@@ -257,6 +258,8 @@ int cmd_repack(int argc,\n \t\t\t   N_(\"pack prefix to store a pack containing pruned objects\")),\n \t\tOPT_STRING(0, \"filter-to\", &filter_to, N_(\"dir\"),\n \t\t\t   N_(\"pack prefix to store a pack containing filtered out objects\")),\n+\t\tOPT_STRING(0, \"filter-to-loose\", &filter_to_loose, N_(\"dir\"),\n+\t\t\t   N_(\"object directory to write filtered out objects into as loose objects\")),\n \t\tOPT_BOOL(0, \"drop-filtered\", &drop_filtered,\n \t\t\t\tN_(\"delete filtered out objects (requires --filter)\")),\n \t\tOPT_BOOL(0, \"dry-run\", &dry_run,\n@@ -282,8 +285,16 @@ int cmd_repack(int argc,\n \tpo_args.depth = xstrdup_or_null(opt_depth);\n \tpo_args.threads = xstrdup_or_null(opt_threads);\n \n-\tdie_for_incompatible_opt2(drop_filtered, \"--drop-filtered\",\n-\t\t!!filter_to, \"--filter-to\");\n+\tdie_for_incompatible_opt3(drop_filtered, \"--drop-filtered\",\n+\t\t\t!!filter_to, \"--filter-to\",\n+\t\t\t!!filter_to_loose, \"--filter-to-loose\");\n+\tdie_for_incompatible_opt2(!!filter_to, \"--filter-to\",\n+\t\t\t!!filter_to_loose, \"--filter-to-loose\");\n+\tdie_for_incompatible_opt2(!!filter_to_loose, \"--filter-to-loose\",\n+\t\t\t!!po_args.max_pack_size, \"--max-pack-size\");\n+\n+\tif (filter_to_loose)\n+\t\tfilter_to = filter_to_loose;\n \n \tif (dry_run && !drop_filtered)\n \t\tdie(_(\"--dry-run only takes effect with --drop-filtered\"));\n@@ -553,7 +564,8 @@ int cmd_repack(int argc,\n \t\tstrvec_pushf(&cmd.args, \"--filter=%s\",\n \t\t\t     expand_list_objects_filter_spec(&po_args.filter_options));\n \telse if (filter_to)\n-\t\tdie(_(\"option '%s' can only be used along with '%s'\"), \"--filter-to\", \"--filter\");\n+\t\tdie(_(\"option '%s' can only be used along with '%s'\"),\n+\t\t    filter_to_loose ? \"--filter-to-loose\" : \"--filter-to\", \"--filter\");\n \n \tif (geometry.split_factor)\n \t\tcmd.in = -1;\n@@ -698,6 +710,7 @@ int cmd_repack(int argc,\n \t\tstruct write_pack_opts opts = {\n \t\t\t.po_args = &po_args,\n \t\t\t.destination = filter_to,\n+\t\t\t.loose = !!filter_to_loose,\n \t\t\t.packdir = packdir,\n \t\t\t.packtmp = packtmp,\n \t\t};\ndiff --git a/repack-filtered.c b/repack-filtered.c\nindex 869b9fc6e3..51c9dfeb13 100644\n--- a/repack-filtered.c\n+++ b/repack-filtered.c\n@@ -9,19 +9,40 @@\n #include \"list-objects-filter.h\"\n #include \"odb.h\"\n #include \"promisor-remote.h\"\n+#include \"sigchain.h\"\n+\n+static void start_unpack_objects(struct child_process *unpack,\n+\t\t\t\t const char *objdir)\n+{\n+\tstrvec_push(&unpack->args, \"unpack-objects\");\n+\tstrvec_push(&unpack->args, \"-q\");\n+\tstrvec_pushf(&unpack->env, \"GIT_OBJECT_DIRECTORY=%s\", objdir);\n+\tstrvec_push(&unpack->env, \"GIT_ALTERNATE_OBJECT_DIRECTORIES\");\n+\tunpack->git_cmd = 1;\n+\tunpack->in = -1;\n+\tif (start_command(unpack))\n+\t\tdie(_(\"could not start unpack-objects for '%s'\"), objdir);\n+}\n \n int write_filtered_pack(const struct write_pack_opts *opts,\n \t\t\tstruct existing_packs *existing,\n \t\t\tstruct string_list *names)\n {\n \tstruct child_process cmd = CHILD_PROCESS_INIT;\n+\tstruct child_process unpack = CHILD_PROCESS_INIT;\n \tstruct string_list_item *item;\n \tFILE *in;\n \tint ret;\n \tconst char *caret;\n \tconst char *pack_prefix = write_pack_opts_pack_prefix(opts);\n \n-\tprepare_pack_objects(&cmd, opts->po_args, opts->destination);\n+\tif (opts->loose) {\n+\t\tsigchain_push(SIGPIPE, SIG_IGN);\n+\t\tstart_unpack_objects(&unpack, opts->destination);\n+\t\tprepare_pack_objects(&cmd, opts->po_args, \"--stdout\");\n+\t} else {\n+\t\tprepare_pack_objects(&cmd, opts->po_args, opts->destination);\n+\t}\n \n \tstrvec_push(&cmd.args, \"--stdin-packs\");\n \n@@ -29,10 +50,18 @@ int write_filtered_pack(const struct write_pack_opts *opts,\n \t\tstrvec_pushf(&cmd.args, \"--keep-pack=%s\", item->string);\n \n \tcmd.in = -1;\n+\tif (opts->loose)\n+\t\tcmd.out = unpack.in;\n \n \tret = start_command(&cmd);\n-\tif (ret)\n+\tif (ret) {\n+\t\tif (opts->loose) {\n+\t\t\tclose(unpack.in);\n+\t\t\tfinish_command(&unpack);\n+\t\t\tsigchain_pop(SIGPIPE);\n+\t\t}\n \t\treturn ret;\n+\t}\n \n \t/*\n \t * Here 'names' contains only the pack(s) that were just\n@@ -52,6 +81,17 @@ int write_filtered_pack(const struct write_pack_opts *opts,\n \t\tfprintf(in, \"%s%s.pack\\n\", caret, item->string);\n \tfclose(in);\n \n+\tif (opts->loose) {\n+\t\tret = finish_command(&cmd);\n+\t\tif (finish_command(&unpack))\n+\t\t\tret = -1;\n+\t\tsigchain_pop(SIGPIPE);\n+\t\tif (ret)\n+\t\t\tret = error(_(\"failed to write filtered out objects to '%s'\"),\n+\t\t\t\t    opts->destination);\n+\t\treturn ret;\n+\t}\n+\n \treturn finish_pack_objects_cmd(existing->repo->hash_algo, opts, &cmd,\n \t\t\t\t       names);\n }\ndiff --git a/repack.h b/repack.h\nindex 61e554e4ed..d5528745ec 100644\n--- a/repack.h\n+++ b/repack.h\n@@ -41,6 +41,7 @@ void repack_remove_redundant_pack(struct repository *repo, const char *dir_name,\n struct write_pack_opts {\n \tstruct pack_objects_args *po_args;\n \tconst char *destination;\n+\tbool loose;\n \tconst char *packdir;\n \tconst char *packtmp;\n };\ndiff --git a/t/t7700-repack.sh b/t/t7700-repack.sh\nindex f0a390e3c6..5aa4b698f1 100755\n--- a/t/t7700-repack.sh\n+++ b/t/t7700-repack.sh\n@@ -489,6 +489,136 @@ test_expect_success '--filter-to stores filtered out objects' '\n \ttest \"$blob_content\" = \"content1\"\n '\n \n+test_expect_success '--filter-to-loose writes filtered out objects loose' '\n+\trm -rf loose-work loose-src.git loose-dst.git &&\n+\tgit init --bare loose-dst.git &&\n+\tgit init loose-work &&\n+\ttest_commit -C loose-work base file1 content1 &&\n+\tgit clone --bare loose-work loose-src.git &&\n+\tgit -C loose-src.git repack -a -d &&\n+\n+\tblob=$(git -C loose-src.git rev-parse HEAD:file1) &&\n+\tgit -C loose-src.git -c repack.writebitmaps=false repack -a -d \\\n+\t\t--filter=blob:none --filter-to-loose=../loose-dst.git/objects &&\n+\n+\ttest_path_is_file loose-dst.git/objects/$(test_oid_to_path $blob) &&\n+\tfind loose-dst.git/objects/pack -name \"*.pack\" >actual-packs &&\n+\ttest_must_be_empty actual-packs &&\n+\ttest_must_fail git -C loose-src.git cat-file -e $blob &&\n+\n+\techo $(pwd)/loose-dst.git/objects >loose-src.git/objects/info/alternates &&\n+\techo content1 >expect &&\n+\tgit -C loose-src.git cat-file blob $blob >actual &&\n+\ttest_cmp expect actual\n+'\n+\n+test_expect_success '--filter-to-loose writes objects that also exist in the source' '\n+\trm -rf loose-work loose-src.git loose-dst.git &&\n+\tgit init --bare loose-dst.git &&\n+\tgit init loose-work &&\n+\ttest_commit -C loose-work base file1 content1 &&\n+\tgit clone --bare loose-work loose-src.git &&\n+\tgit -C loose-src.git repack -a -d &&\n+\n+\tblob=$(git -C loose-src.git rev-parse HEAD:file1) &&\n+\tgit -C loose-src.git -c repack.writebitmaps=false repack -a -d \\\n+\t\t--filter=blob:none --filter-to-loose=../loose-dst.git/objects &&\n+\n+\ttest_path_is_file loose-dst.git/objects/$(test_oid_to_path $blob)\n+'\n+\n+# Create loose-work/loose-src.git/loose-dst.git; $big is a 2k blob on\n+# refs/heads/big in loose-src.git, packed unless $1 is \"loose\".\n+setup_loose_src () {\n+\trm -rf loose-work loose-src.git loose-dst.git &&\n+\tgit init --bare loose-dst.git &&\n+\tgit init loose-work &&\n+\ttest_commit -C loose-work base small small &&\n+\tgit clone --bare loose-work loose-src.git &&\n+\n+\tbig=$(test-tool genrandom big 2k | git -C loose-src.git hash-object -w --stdin) &&\n+\ttree=$(printf \"100644 blob %s\\tbig\\n\" $big | git -C loose-src.git mktree) &&\n+\tcommit=$(git -C loose-src.git commit-tree -m big $tree) &&\n+\tgit -C loose-src.git update-ref refs/heads/big $commit &&\n+\n+\tif test \"$1\" != loose\n+\tthen\n+\t\tgit -C loose-src.git repack -a -d\n+\tfi\n+}\n+\n+test_expect_success '--filter-to-loose failure to write leaves the source intact' '\n+\tsetup_loose_src &&\n+\t>not-a-dir &&\n+\n+\tls loose-src.git/objects/pack/*.pack >packs-before &&\n+\ttest_must_fail git -C loose-src.git -c repack.writebitmaps=false \\\n+\t\trepack -a -d --filter=blob:limit=1k \\\n+\t\t--filter-to-loose=../not-a-dir &&\n+\n+\tls loose-src.git/objects/pack/*.pack >packs-after &&\n+\ttest_cmp packs-before packs-after &&\n+\n+\tfind loose-src.git/objects/pack -name \".tmp-*\" >tmp-leftovers &&\n+\ttest_must_be_empty tmp-leftovers &&\n+\tgit -C loose-src.git cat-file -e $big\n+'\n+\n+test_expect_success '--filter-to-loose does not move objects from kept packs' '\n+\tsetup_loose_src &&\n+\n+\tfor p in loose-src.git/objects/pack/pack-*.pack\n+\tdo\n+\t\t>${p%.pack}.keep || return 1\n+\tdone &&\n+\n+\tgit -C loose-src.git -c repack.writebitmaps=false repack -a -d \\\n+\t\t--filter=blob:limit=1k --filter-to-loose=../loose-dst.git/objects &&\n+\n+\ttest_path_is_missing loose-dst.git/objects/$(test_oid_to_path $big) &&\n+\tgit -C loose-src.git cat-file -e $big\n+'\n+\n+test_expect_success '--filter-to-loose leaves unreachable objects in cruft packs alone' '\n+\tsetup_loose_src loose &&\n+\tgit -C loose-src.git update-ref -d refs/heads/big &&\n+\tgit -C loose-src.git repack --cruft -d &&\n+\n+\tgit -C loose-src.git -c repack.writebitmaps=false repack -a -d --cruft \\\n+\t\t--filter=blob:limit=1k --filter-to-loose=../loose-dst.git/objects &&\n+\n+\ttest_path_is_missing loose-dst.git/objects/$(test_oid_to_path $big) &&\n+\tgit -C loose-src.git cat-file -e $big\n+'\n+\n+test_expect_success '--filter-to-loose output survives gc and prune of the source' '\n+\tsetup_loose_src &&\n+\n+\tgit -C loose-src.git -c repack.writebitmaps=false repack -a -d \\\n+\t\t--filter=blob:limit=1k --filter-to-loose=../loose-dst.git/objects &&\n+\n+\techo $(pwd)/loose-dst.git/objects >loose-src.git/objects/info/alternates &&\n+\tgit -C loose-src.git gc --prune=now &&\n+\tgit -C loose-src.git prune --expire=now &&\n+\n+\tgit -C loose-src.git fsck &&\n+\tgit -C loose-src.git cat-file -e $big &&\n+\ttest_path_is_file loose-dst.git/objects/$(test_oid_to_path $big)\n+'\n+\n+test_expect_success '--filter-to-loose requires --filter, rejects --filter-to and --max-pack-size' '\n+\ttest_must_fail git -C bare.git repack -a --filter-to-loose=../x 2>err &&\n+\ttest_grep \"can only be used along with\" err &&\n+\n+\ttest_must_fail git -C bare.git repack -a --filter=blob:none \\\n+\t\t--filter-to=../y --filter-to-loose=../x 2>err &&\n+\ttest_grep \"cannot be used together\" err &&\n+\n+\ttest_must_fail git -C bare.git repack -a --filter=blob:none \\\n+\t\t--filter-to-loose=../x --max-pack-size=1M 2>err &&\n+\ttest_grep \"cannot be used together\" err\n+'\n+\n test_expect_success '--filter works with --max-pack-size' '\n \trm -rf filtered.git &&\n \tgit init --bare filtered.git &&\n-- \ngitgitgadget\n\n\n"},{"id":"558431","messageId":"ba2534e3bc411352271f085725287c468860ae85.1794607911.git.gitgitgadget@gmail.com","threadId":"66848","inReplyTo":"pull.2314.git.1794607911.gitgitgadget@gmail.com","subject":"[PATCH 2/2] repack: also copy loose objects excluded by --filter-to-loose","fromName":"Adithya Chakilam via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2026-11-13T22:11:51Z","receivedAt":"2026-11-13T22:11:51Z","isPatch":true,"sender":{"key":"achakila@codeaurora.org","avatar":null},"body":"From: Adithya Chakilam <achakilam@nvidia.com>\n\n--filter-to-loose only writes objects that were in an existing pack,\nbecause the filtered pack is built with --stdin-packs. Loose objects\nthat the filter excludes stay behind in the source repository, so the\ndestination does not get the complete set of filtered-out objects.\n\nCopy such loose objects to the destination as well. An object already\npresent in the destination is not rewritten. With -d, the source copies\nare removed once everything is written.\n\nSigned-off-by: Adithya Chakilam <achakilam@nvidia.com>\n---\n Documentation/git-repack.adoc |   4 ++\n builtin/repack.c              |  11 ++++\n repack-filtered.c             | 108 ++++++++++++++++++++++++++++++++++\n repack.h                      |  13 ++++\n t/t7700-repack.sh             |  59 +++++++++++++++++++\n 5 files changed, 195 insertions(+)\n\ndiff --git a/Documentation/git-repack.adoc b/Documentation/git-repack.adoc\nindex df1390154f..e2025c29ee 100644\n--- a/Documentation/git-repack.adoc\n+++ b/Documentation/git-repack.adoc\n@@ -191,6 +191,10 @@ depth is 4095.\n \tnot rewritten. All objects are written before any existing\n \tpack is removed.\n \n+\tLoose objects in this repository that the filter excludes are\n+\tcopied to `<dir>` as well, and with `-d` the source copies are\n+\tremoved once everything has been written.\n+\n --drop-filtered::\n \tDelete the local objects that match the `--filter` specification\n \tinstead of keeping them in a separate packfile, reclaiming the\ndiff --git a/builtin/repack.c b/builtin/repack.c\nindex 140a992f1b..a946ad28a3 100644\n--- a/builtin/repack.c\n+++ b/builtin/repack.c\n@@ -183,6 +183,7 @@ int cmd_repack(int argc,\n \tconst char *expire_to = NULL;\n \tconst char *filter_to = NULL;\n \tconst char *filter_to_loose = NULL;\n+\tstruct oidset moved_loose = OIDSET_INIT;\n \tconst char *opt_window = NULL;\n \tconst char *opt_window_memory = NULL;\n \tconst char *opt_depth = NULL;\n@@ -721,6 +722,14 @@ int cmd_repack(int argc,\n \t\tret = write_filtered_pack(&opts, &existing, &names);\n \t\tif (ret)\n \t\t\tgoto cleanup;\n+\n+\t\tif (filter_to_loose) {\n+\t\t\tret = move_loose_filtered_objects(repo,\n+\t\t\t\t\t&po_args.filter_options, filter_to,\n+\t\t\t\t\t&moved_loose);\n+\t\t\tif (ret)\n+\t\t\t\tgoto cleanup;\n+\t\t}\n \t}\n \n \tstring_list_sort(&names);\n@@ -777,6 +786,7 @@ int cmd_repack(int argc,\n \t\tif (show_progress)\n \t\t\topts |= PRUNE_PACKED_VERBOSE;\n \t\tprune_packed_objects(repo, opts);\n+\t\tremove_moved_loose_objects(repo, &moved_loose);\n \n \t\tif (!keep_unreachable &&\n \t\t    (!(pack_everything & LOOSEN_UNREACHABLE) ||\n@@ -801,6 +811,7 @@ cleanup:\n \tstring_list_clear(&keep_pack_list, 0);\n \tstring_list_clear(&names, 1);\n \toidset_clear(&drop_oids);\n+\toidset_clear(&moved_loose);\n \texisting_packs_release(&existing);\n \tpack_geometry_release(&geometry);\n \tpack_objects_args_release(&po_args);\ndiff --git a/repack-filtered.c b/repack-filtered.c\nindex 51c9dfeb13..cb1f034f09 100644\n--- a/repack-filtered.c\n+++ b/repack-filtered.c\n@@ -9,6 +9,11 @@\n #include \"list-objects-filter.h\"\n #include \"odb.h\"\n #include \"promisor-remote.h\"\n+#include \"copy.h\"\n+#include \"object-file.h\"\n+#include \"odb/source-files.h\"\n+#include \"oidset.h\"\n+#include \"path.h\"\n #include \"sigchain.h\"\n \n static void start_unpack_objects(struct child_process *unpack,\n@@ -171,3 +176,106 @@ cleanup:\n \toidset_clear(&all_promisor_blobs);\n \treturn ret;\n }\n+\n+static int collect_loose_oid(const struct object_id *oid,\n+\t\t\t     const char *path UNUSED, void *cb_data)\n+{\n+\toidset_insert(cb_data, oid);\n+\treturn 0;\n+}\n+\n+static void dest_loose_path(struct strbuf *buf, const char *objdir,\n+\t\t\t    const struct object_id *oid)\n+{\n+\tconst char *hex = oid_to_hex(oid);\n+\n+\tstrbuf_reset(buf);\n+\tstrbuf_addf(buf, \"%s/%.2s/%s\", objdir, hex, hex + 2);\n+}\n+\n+int move_loose_filtered_objects(struct repository *repo,\n+\t\t\t\tconst struct list_objects_filter_options *filter,\n+\t\t\t\tconst char *dest_objdir,\n+\t\t\t\tstruct oidset *moved)\n+{\n+\tstruct odb_source_files *files =\n+\t\todb_source_files_downcast(repo->objects->source);\n+\tstruct oidset loose = OIDSET_INIT;\n+\tstruct oidset omitted = OIDSET_INIT;\n+\tstruct oidset_iter iter;\n+\tconst struct object_id *oid;\n+\tstruct strbuf src = STRBUF_INIT;\n+\tstruct strbuf dst = STRBUF_INIT;\n+\tstruct strbuf tmp = STRBUF_INIT;\n+\tint ret = 0;\n+\n+\tif (filter->choice != LOFC_BLOB_LIMIT)\n+\t\treturn 0;\n+\n+\tret = for_each_loose_file_in_source(&files->dirs->loose->base,\n+\t\t\t\t\t    collect_loose_oid, NULL, NULL,\n+\t\t\t\t\t    &loose);\n+\tif (ret)\n+\t\tgoto out;\n+\tret = list_objects_filter__filter_oidset(repo, filter, &loose,\n+\t\t\t\t\t\t &omitted);\n+\tif (ret)\n+\t\tgoto out;\n+\n+\toidset_iter_init(&omitted, &iter);\n+\twhile ((oid = oidset_iter_next(&iter))) {\n+\t\todb_loose_path(files->dirs->loose, &src, oid);\n+\t\tdest_loose_path(&dst, dest_objdir, oid);\n+\n+\t\t/*\n+\t\t * Only the destination decides whether the copy is needed;\n+\t\t * the object being present in the source says nothing.\n+\t\t */\n+\t\tif (!access(dst.buf, F_OK)) {\n+\t\t\toidset_insert(moved, oid);\n+\t\t\tcontinue;\n+\t\t}\n+\n+\t\tif (safe_create_leading_directories(repo, dst.buf) < 0) {\n+\t\t\tret = error_errno(_(\"unable to create leading directories of '%s'\"),\n+\t\t\t\t\t  dst.buf);\n+\t\t\tgoto out;\n+\t\t}\n+\t\tstrbuf_reset(&tmp);\n+\t\tstrbuf_addf(&tmp, \"%s.tmp-%\"PRIuMAX, dst.buf,\n+\t\t\t    (uintmax_t)getpid());\n+\t\tif (copy_file(repo, tmp.buf, src.buf, 0444) ||\n+\t\t    finalize_object_file(repo, tmp.buf, dst.buf)) {\n+\t\t\tunlink(tmp.buf);\n+\t\t\tret = error(_(\"unable to copy loose object '%s' to '%s'\"),\n+\t\t\t\t    src.buf, dst.buf);\n+\t\t\tgoto out;\n+\t\t}\n+\t\toidset_insert(moved, oid);\n+\t}\n+\n+out:\n+\toidset_clear(&loose);\n+\toidset_clear(&omitted);\n+\tstrbuf_release(&src);\n+\tstrbuf_release(&dst);\n+\tstrbuf_release(&tmp);\n+\treturn ret;\n+}\n+\n+void remove_moved_loose_objects(struct repository *repo,\n+\t\t\t\tconst struct oidset *moved)\n+{\n+\tstruct odb_source_files *files =\n+\t\todb_source_files_downcast(repo->objects->source);\n+\tstruct strbuf path = STRBUF_INIT;\n+\tstruct oidset_iter iter;\n+\tconst struct object_id *oid;\n+\n+\toidset_iter_init(moved, &iter);\n+\twhile ((oid = oidset_iter_next(&iter))) {\n+\t\todb_loose_path(files->dirs->loose, &path, oid);\n+\t\tunlink(path.buf);\n+\t}\n+\tstrbuf_release(&path);\n+}\ndiff --git a/repack.h b/repack.h\nindex d5528745ec..18a0dc6314 100644\n--- a/repack.h\n+++ b/repack.h\n@@ -168,6 +168,19 @@ int write_filtered_pack(const struct write_pack_opts *opts,\n \t\t\tstruct existing_packs *existing,\n \t\t\tstruct string_list *names);\n \n+/*\n+ * Copy loose objects that the filter excludes into the loose-object store\n+ * at `dest_objdir`, recording them in `moved`.\n+ */\n+int move_loose_filtered_objects(struct repository *repo,\n+\t\t\t\tconst struct list_objects_filter_options *filter,\n+\t\t\t\tconst char *dest_objdir,\n+\t\t\t\tstruct oidset *moved);\n+\n+/* Delete the source copies of objects copied by move_loose_filtered_objects. */\n+void remove_moved_loose_objects(struct repository *repo,\n+\t\t\t\tconst struct oidset *moved);\n+\n int enumerate_promisor_blobs(struct repository *repo,\n \t\t\t     const struct list_objects_filter_options *filter,\n \t\t\t     struct oidset *to_drop);\ndiff --git a/t/t7700-repack.sh b/t/t7700-repack.sh\nindex 5aa4b698f1..37dc5fdb13 100755\n--- a/t/t7700-repack.sh\n+++ b/t/t7700-repack.sh\n@@ -547,6 +547,44 @@ setup_loose_src () {\n \tfi\n }\n \n+test_expect_success '--filter-to-loose copies loose filtered out blobs, and -d removes the source copy' '\n+\tsetup_loose_src loose &&\n+\ttest_path_is_file loose-src.git/objects/$(test_oid_to_path $big) &&\n+\n+\t# without -d the source copy is left alone\n+\tgit -C loose-src.git -c repack.writebitmaps=false repack -a \\\n+\t\t--filter=blob:limit=1k --filter-to-loose=../loose-dst.git/objects &&\n+\ttest_path_is_file loose-dst.git/objects/$(test_oid_to_path $big) &&\n+\ttest_path_is_file loose-src.git/objects/$(test_oid_to_path $big) &&\n+\n+\t# with -d it is removed from the source after being copied\n+\trm -rf loose-dst.git/objects/$(dirname $(test_oid_to_path $big)) &&\n+\tgit -C loose-src.git -c repack.writebitmaps=false repack -a -d \\\n+\t\t--filter=blob:limit=1k --filter-to-loose=../loose-dst.git/objects &&\n+\ttest_path_is_file loose-dst.git/objects/$(test_oid_to_path $big) &&\n+\ttest_path_is_missing loose-src.git/objects/$(test_oid_to_path $big) &&\n+\techo $(pwd)/loose-dst.git/objects >loose-src.git/objects/info/alternates &&\n+\tgit -C loose-src.git cat-file -e $big\n+'\n+\n+test_expect_success '--filter-to-loose leaves an identical existing destination object alone' '\n+\tsetup_loose_src &&\n+\n+\ttest-tool genrandom big 2k |\n+\tGIT_OBJECT_DIRECTORY=loose-dst.git/objects git hash-object -w --stdin &&\n+\n+\tpath=loose-dst.git/objects/$(test_oid_to_path $big) &&\n+\ttest_path_is_file $path &&\n+\n+\tls -i $path >before &&\n+\tgit -C loose-src.git -c repack.writebitmaps=false repack -a -d \\\n+\t\t--filter=blob:limit=1k --filter-to-loose=../loose-dst.git/objects &&\n+\n+\tls -i $path >after &&\n+\ttest_cmp before after &&\n+\tgit -C loose-dst.git fsck\n+'\n+\n test_expect_success '--filter-to-loose failure to write leaves the source intact' '\n \tsetup_loose_src &&\n \t>not-a-dir &&\n@@ -564,6 +602,27 @@ test_expect_success '--filter-to-loose failure to write leaves the source intact\n \tgit -C loose-src.git cat-file -e $big\n '\n \n+test_expect_success '--filter-to-loose failure to copy a loose object leaves the source intact' '\n+\tsetup_loose_src loose &&\n+\tpath=$(test_oid_to_path $big) &&\n+\t>loose-dst.git/objects/$(dirname $path) &&\n+\n+\tfind loose-src.git/objects/pack -name \"*.pack\" | sort >packs-before &&\n+\ttest_must_fail git -C loose-src.git -c repack.writebitmaps=false \\\n+\t\trepack -a -d --filter=blob:limit=1k \\\n+\t\t--filter-to-loose=../loose-dst.git/objects 2>err &&\n+\n+\ttest_grep \"unable to create leading directories\" err &&\n+\tfind loose-src.git/objects/pack -name \"*.pack\" | sort >packs-after &&\n+\ttest_cmp packs-before packs-after &&\n+\n+\tfind loose-src.git/objects/pack -name \".tmp-*\" >tmp-leftovers &&\n+\ttest_must_be_empty tmp-leftovers &&\n+\n+\ttest_path_is_file loose-src.git/objects/$path &&\n+\tgit -C loose-src.git cat-file -e $big\n+'\n+\n test_expect_success '--filter-to-loose does not move objects from kept packs' '\n \tsetup_loose_src &&\n \n-- \ngitgitgadget\n\n"},{"id":"558483","messageId":"xmqqmrrbnpk6.fsf@gitster.g","threadId":"66848","inReplyTo":"4488a1534ebe7def72055ab5530d4f03177fbe52.1794607911.git.gitgitgadget@gmail.com","subject":"Re: [PATCH 1/2] repack: add --filter-to-loose to write filtered-out objects as loose objects","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2026-11-14T20:08:57Z","receivedAt":"2026-11-14T20:08:57Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"\"Adithya Chakilam via GitGitGadget\" <gitgitgadget@gmail.com> writes:\n\n> From: Adithya Chakilam <achakilam@nvidia.com>\n>\n> Some external object stores (like key-value) manage and retrieve git\n> objects individually. Although repack's --filter-to option can separate\n> objects from the main object store, it writes them into packs. Using\n> those objects individually then requires an additional unpacking step.\n>\n> Add --filter-to-loose=<dir> to write filtered-out objects directly as\n> loose objects into <dir>, treating it as an object directory. Stream\n> the filtered pack into unpack-objects instead of writing an intermediate\n> pack file to disk.\n>\n> The filtered pack is built with --stdin-packs, so this initially handles\n> only objects already stored in packs. Loose objects in the source\n> repository are left in place.\n\nI haven't decided whether this new feature makes sense or not, so I\nwon't comment on that aspect.  Take the following review comments\nwith that in mind (i.e., pretend as if I said \"if we assume doing\nthis makes sense, then...\" before every sentence).\n\n> +--filter-to-loose=<dir>::\n> +\tLike `--filter-to`, but treat `<dir>` as an object directory\n> +\tand write the filtered out objects there as loose objects\n> +\tinstead of as a pack. Cannot be combined with `--filter-to`\n> +\tor `--max-pack-size`. Objects already present in `<dir>` are\n> +\tnot rewritten. All objects are written before any existing\n> +\tpack is removed.\n\nWe talk about \"already present in ...\", so presumably we require the\ndirectory to exist before we run.  Have a test that ensures that\npassing a non-existing directory will make the command error out\nbefore doing any real work.\n\n> diff --git a/builtin/repack.c b/builtin/repack.c\n> index 21835453c6..140a992f1b 100644\n> --- a/builtin/repack.c\n> +++ b/builtin/repack.c\n> @@ -41,7 +41,7 @@ static const char *const git_repack_usage[] = {\n>  \tN_(\"git repack [-a] [-A] [-d] [-f] [-F] [-l] [-n] [-q] [-b] [-m]\\n\"\n>  \t   \"[--window=<n>] [--depth=<n>] [--threads=<n>] [--keep-pack=<pack-name>]\\n\"\n>  \t   \"[--write-midx[=<mode>]] [--name-hash-version=<n>] [--path-walk]\\n\"\n> -\t   \"[--filter=<filter-spec>] [--drop-filtered [--dry-run]]\"),\n> +\t   \"[--filter=<filter-spec> [--filter-to-loose=<dir>]] [--drop-filtered [--dry-run]]\"),\n\nThe above documentation and help text consistently tell us that the\nvalue passed is a directory (do we use the abbreviated <dir> so\ncommonly?  If not, it would make sense to spell it out, i.e.,\n<directory>, for clarity).\n\n> +\t\tOPT_STRING(0, \"filter-to-loose\", &filter_to_loose, N_(\"dir\"),\n> +\t\t\t   N_(\"object directory to write filtered out objects into as loose objects\")),\n\nHere, too.  \n\nThis should use OPT_FILENAME so that a \"git repack\" started from a\nsubdirectory with a relative path to the external loose object store\nwill find the directory at the correct place, I think.  Have a test\nto validate the combination to see that such a repack that was given\na relative path to a directory find ...\n\n\tmkdir extra-loose-objects &&\n\tmkdir sub/dir &&\n\t(\n\t\tcd sub/dir &&\n\t\tgit repack --filter-to-loose=../../extra-loose-objects ...\n\t)\n\nthe directory correct at the right place.\n\n> diff --git a/repack-filtered.c b/repack-filtered.c\n> index 869b9fc6e3..51c9dfeb13 100644\n> --- a/repack-filtered.c\n> +++ b/repack-filtered.c\n> @@ -29,10 +50,18 @@ int write_filtered_pack(const struct write_pack_opts *opts,\n>  \t\tstrvec_pushf(&cmd.args, \"--keep-pack=%s\", item->string);\n>  \n>  \tcmd.in = -1;\n> +\tif (opts->loose)\n> +\t\tcmd.out = unpack.in;\n>  \n>  \tret = start_command(&cmd);\n> -\tif (ret)\n> +\tif (ret) {\n> +\t\tif (opts->loose) {\n> +\t\t\tclose(unpack.in);\n\nPlease double check that this is not closing the same thing twice.\nIIRC, a failing start_command() would close all file descriptors\nassociated with cmd so you may have already closed unpack.in when\nstart_command() gave us a failure, which presumably closed cmd.out.\n\n\n"}]}