{"thread":{"id":"56447","subject":"[PATCH 0/4] Compile-time extensions for list-object-filter","startedAt":"2021-09-05T23:52:36Z","lastAt":"2021-09-08T14:23:28Z","messageCount":11,"participants":["Andrew Olsen via GitGitGadget","Ævar Arnfjörð Bjarmason","Bagas Sanjaya","Andrew Olsen","Robert Coup"],"isPatch":true,"patchVersion":1,"patchTotal":4},"messages":[{"id":"434740","messageId":"pull.1031.git.1630885899.gitgitgadget@gmail.com","threadId":"56447","inReplyTo":null,"subject":"[PATCH 0/4] Compile-time extensions for list-object-filter","fromName":"Andrew Olsen via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2021-09-05T23:51:35Z","receivedAt":"2021-09-05T23:52:36Z","isPatch":true,"sender":{"key":"andrew.olsen@koordinates.com","avatar":"https://gravatar.com/avatar/f316140b761c57e8a68f9f93ea80a2aceac0c1d37df1b4aec24997025d441a34?d=mp&s=160"},"body":"Adds an extension: option to list-object-filters, these are implemented by\nstatic libraries that must be compiled into Git. The Makefile argument\nFILTER_EXTENSIONS makes it easier to compile these extensions into a custom\nbuild of Git. When no custom filter-extensions are supplied, Git works as\nnormal.\n\nAndrew Olsen (4):\n  Compile-time extensions for list-object-filter\n  Makefile for list-object-filter extensions\n  Sample list-object-filter extensions\n  Documentation for list-object-filter extensions\n\n .gitignore                                    |   1 +\n Documentation/config/uploadpack.txt           |   7 +-\n Documentation/rev-list-options.txt            |   4 +\n Makefile                                      |  35 +++-\n compat/vcbuild/README                         |   5 +-\n config.mak.uname                              |   6 +-\n contrib/buildsystems/CMakeLists.txt           |   7 +\n contrib/filter-extensions/README.txt          | 153 ++++++++++++++++++\n contrib/filter-extensions/rand/.gitignore     |   2 +\n contrib/filter-extensions/rand/Makefile       |  28 ++++\n contrib/filter-extensions/rand/rand.c         | 103 ++++++++++++\n contrib/filter-extensions/rand_cpp/.gitignore |   2 +\n contrib/filter-extensions/rand_cpp/Makefile   |  34 ++++\n .../rand_cpp/adapter_functions.c              |   6 +\n .../rand_cpp/adapter_functions.h              |  10 ++\n contrib/filter-extensions/rand_cpp/rand.cpp   | 103 ++++++++++++\n generate-list-objects-filter-extensions.sh    |  53 ++++++\n list-objects-filter-extensions.h              | 107 ++++++++++++\n list-objects-filter-options.c                 |  47 ++++++\n list-objects-filter-options.h                 |   6 +\n list-objects-filter.c                         |  84 ++++++++++\n 21 files changed, 793 insertions(+), 10 deletions(-)\n create mode 100644 contrib/filter-extensions/README.txt\n create mode 100644 contrib/filter-extensions/rand/.gitignore\n create mode 100644 contrib/filter-extensions/rand/Makefile\n create mode 100644 contrib/filter-extensions/rand/rand.c\n create mode 100644 contrib/filter-extensions/rand_cpp/.gitignore\n create mode 100644 contrib/filter-extensions/rand_cpp/Makefile\n create mode 100644 contrib/filter-extensions/rand_cpp/adapter_functions.c\n create mode 100644 contrib/filter-extensions/rand_cpp/adapter_functions.h\n create mode 100644 contrib/filter-extensions/rand_cpp/rand.cpp\n create mode 100755 generate-list-objects-filter-extensions.sh\n create mode 100644 list-objects-filter-extensions.h\n\n\nbase-commit: e0a2f5cbc585657e757385ad918f167f519cfb96\nPublished-As: https://github.com/gitgitgadget/git/releases/tag/pr-1031%2Fkoordinates%2Flist-objects-filter-extensions-v1\nFetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-1031/koordinates/list-objects-filter-extensions-v1\nPull-Request: https://github.com/gitgitgadget/git/pull/1031\n-- \ngitgitgadget\n"},{"id":"434741","messageId":"a5383a68cc024eca9bb95fc3c24dce499efaaa19.1630885899.git.gitgitgadget@gmail.com","threadId":"56447","inReplyTo":"pull.1031.git.1630885899.gitgitgadget@gmail.com","subject":"[PATCH 1/4] Compile-time extensions for list-object-filter","fromName":"Andrew Olsen via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2021-09-05T23:51:36Z","receivedAt":"2021-09-05T23:52:36Z","isPatch":true,"sender":{"key":"andrew.olsen@koordinates.com","avatar":"https://gravatar.com/avatar/f316140b761c57e8a68f9f93ea80a2aceac0c1d37df1b4aec24997025d441a34?d=mp&s=160"},"body":"From: Andrew Olsen <andrew.olsen@koordinates.com>\n\nAdds an extension:<custom-filter> option to list-object-filters,\nthese are implemented by static libraries that must be compiled into\nGit. C code changes only - Makefile changes follow.\n\nSigned-off-by: Andrew Olsen <andrew.olsen@koordinates.com>\n---\n .gitignore                                 |   1 +\n generate-list-objects-filter-extensions.sh |  53 ++++++++++\n list-objects-filter-extensions.h           | 107 +++++++++++++++++++++\n list-objects-filter-options.c              |  47 +++++++++\n list-objects-filter-options.h              |   6 ++\n list-objects-filter.c                      |  84 ++++++++++++++++\n 6 files changed, 298 insertions(+)\n create mode 100755 generate-list-objects-filter-extensions.sh\n create mode 100644 list-objects-filter-extensions.h\n\ndiff --git a/.gitignore b/.gitignore\nindex 311841f9bed..3564cb01ad7 100644\n--- a/.gitignore\n+++ b/.gitignore\n@@ -190,6 +190,7 @@\n /gitweb/static/gitweb.min.*\n /config-list.h\n /command-list.h\n+/list-objects-filter-extensions.c\n *.tar.gz\n *.dsc\n *.deb\ndiff --git a/generate-list-objects-filter-extensions.sh b/generate-list-objects-filter-extensions.sh\nnew file mode 100755\nindex 00000000000..422b1ce837f\n--- /dev/null\n+++ b/generate-list-objects-filter-extensions.sh\n@@ -0,0 +1,53 @@\n+#!/bin/sh\n+\n+if [ $# -gt 0 ]; then\n+\n+\t# ARGS has one argument per line\n+\tARGS=$(echo \"$@\" | xargs printf '%s\\n')\n+\n+\t# Every argument should be path to a filter extension library.\n+\tINVALID_ARGS=$(echo \"$ARGS\" | grep -v '\\.a$')\n+\tif [ -n \"$INVALID_ARGS\" ] ; then\n+\t\tprintf \"Error: all arguments must be paths to .a files: \\n%s\\n\" \\\n+\t\t\t\"${INVALID_ARGS}\" >&2\n+\t\texit 1\n+\tfi\n+\n+\t# qux/foo.a -> foo\n+\tNAMES=$(echo \"$ARGS\" | sed -e 's!.*/!!' -e 's!.a$!!')\n+\n+\t# Filter extension names must be valid C symbols so they can be linked by name.\n+\tINVALID_NAMES=$(echo \"$NAMES\" | grep -v '^[A-Za-z0-9_]\\+$')\n+\tif [ -n \"$INVALID_NAMES\" ] ; then\n+\t\tprintf \"Error: all library names must also be valid C symbols: \\n%s\\n\" \\\n+\t\t\t\"${INVALID_NAMES}\" >&2\n+\t\texit 1\n+\tfi\n+\n+\t# foo -> filter_extension_foo\n+\tEXTS=$(echo \"$NAMES\" | sed -e 's!^!filter_extension_!')\n+\n+\t# filter_extension_foo -> [\\t]filter_extension_foo,\n+\tDECLARATIONS=$(echo \"$EXTS\" | sed -e 's!^!\\t!' -e 's!$!,!')\n+\n+\t# filter_extension_foo -> [\\t]&filter_extension_foo,\n+\tARRAY=$(echo \"$EXTS\" | sed -e 's!^!\\t\\&!' -e 's!$!,!')\n+fi\n+\n+echo '/* Automatically generated by generate-list-objects-filter-extensions.sh */'\n+echo\n+echo '#include \"git-compat-util.h\"'\n+echo '#include \"list-objects-filter-extensions.h\"'\n+echo\n+\n+if [ $# -gt 0 ]; then\n+\techo 'extern const struct filter_extension'\n+\techo \"${DECLARATIONS%?}\"\n+\techo ';'\n+\techo\n+fi\n+\n+echo 'const struct filter_extension *filter_extensions[] = {'\n+echo \"${ARRAY}\"\n+echo '\tNULL,'\n+echo '};'\n\\ No newline at end of file\ndiff --git a/list-objects-filter-extensions.h b/list-objects-filter-extensions.h\nnew file mode 100644\nindex 00000000000..35ebe1ead31\n--- /dev/null\n+++ b/list-objects-filter-extensions.h\n@@ -0,0 +1,107 @@\n+#ifndef GIT_LIST_OBJECTS_FILTER_EXTENSIONS_H\n+#define GIT_LIST_OBJECTS_FILTER_EXTENSIONS_H\n+\n+/**\n+ * The List-Objects-Filter Extensions API can be used to develop filter\n+ * extensions for git-upload-pack/git-rev-list/etc.\n+ *\n+ * See contrib/filter-extensions/README.md for more details and examples.\n+ *\n+ * The API defines three functions to implement a filter operation. Note that\n+ * each filter implementing this API must compiled into Git as a static library.\n+ * There is some plumbing in the Makefile to help with this via\n+ * FILTER_EXTENSIONS.\n+ *\n+ * 1. You write a filter and compile it into your custom build of git.\n+ *    See list_objects_filter_ext_filter_fn.\n+ * 2. A filter request is received that specifically names the filter extension\n+ *    that you have written, ie: \"--filter=extension:<name>[=<arg>]\"\n+ * 3. Your list_objects_filter_ext_init_fn() is called.\n+ * 4. Your list_objects_filter_ext_filter_fn() is called for each object\n+ *    at least once.\n+ * 5. Your list_objects_filter_ext_free_fn() is called.\n+ */\n+\n+#include \"list-objects-filter.h\"\n+\n+\n+/* Whether to add or remove a specific object from any current omitset. */\n+enum list_objects_filter_omit {\n+       LOFO_KEEP = -1,\n+       LOFO_IGNORE = 0,\n+       LOFO_OMIT = 1,\n+};\n+\n+/*\n+ * This is a corollary to `list_objects_filter__init()` and constructs the\n+ * filter, parsing and validating any user-provided `filter_arg` (via\n+ * `--filter=extension:<name>=<arg>`). Use `context` for any filter-allocated\n+ * context data.\n+ *\n+ * Return 0 on success and non-zero on error.\n+ */\n+typedef\n+int list_objects_filter_ext_init_fn(\n+    const struct repository *r,\n+    const char* filter_arg,\n+    void **context\n+);\n+\n+/*\n+ * This is a corollary to `list_objects_filter__free()`, destroying the filter\n+ * and any filter-allocated context data.\n+ */\n+typedef\n+void list_objects_filter_ext_free_fn(\n+    const struct repository *r,\n+    void *context\n+);\n+\n+/*\n+ * This is a corollary to `list_objects_filter__filter_object()`, and\n+ * decides how to handle the object `obj`.\n+ *\n+ * omit provides a flag determining whether to explicitly add or remove\n+ * the object from any current omitset.\n+ */\n+typedef\n+enum list_objects_filter_result list_objects_filter_ext_filter_fn(\n+\tconst struct repository *r,\n+\tconst enum list_objects_filter_situation filter_situation,\n+\tstruct object *obj,\n+\tconst char *pathname,\n+\tconst char *filename,\n+\tenum list_objects_filter_omit *omit,\n+\tvoid *context\n+);\n+\n+/*\n+ * To implement a filter extension called \"mine\", you should define\n+ * a const struct filter_extension called filter_extension_mine,\n+ * in the following manner:\n+ *\n+ * const struct filter_extension filter_extension_mine = {\n+ *     \"mine\",\n+ *     &my_init_fn,\n+ *     &my_filter_object_fn,\n+ *     &my_free_fn\n+ * };\n+ *\n+ * See contrib/filter-extensions/README.md for more details and examples.\n+ */\n+\n+struct filter_extension {\n+    const char *name;\n+    list_objects_filter_ext_init_fn* init_fn;\n+    list_objects_filter_ext_filter_fn* filter_object_fn;\n+    list_objects_filter_ext_free_fn* free_fn;\n+};\n+\n+/*\n+ * The filter_extensions array is defined in list_objects_filter_extensions.c\n+ * which is generated at compile time from the FILTER_EXTENSIONS variable.\n+ */\n+extern const struct filter_extension *filter_extensions[];\n+\n+\n+#endif /* GIT_LIST_OBJECTS_FILTER_EXTENSIONS_H */\ndiff --git a/list-objects-filter-options.c b/list-objects-filter-options.c\nindex fd8d59f653a..e92499f29c2 100644\n--- a/list-objects-filter-options.c\n+++ b/list-objects-filter-options.c\n@@ -15,6 +15,11 @@ static int parse_combine_filter(\n \tconst char *arg,\n \tstruct strbuf *errbuf);\n \n+static int parse_extension_filter(\n+\tstruct list_objects_filter_options *filter_options,\n+\tconst char *arg,\n+\tstruct strbuf *errbuf);\n+\n const char *list_object_filter_config_name(enum list_objects_filter_choice c)\n {\n \tswitch (c) {\n@@ -31,6 +36,8 @@ const char *list_object_filter_config_name(enum list_objects_filter_choice c)\n \t\treturn \"sparse:oid\";\n \tcase LOFC_OBJECT_TYPE:\n \t\treturn \"object:type\";\n+\tcase LOFC_EXTENSION:\n+\t\treturn \"extension\";\n \tcase LOFC_COMBINE:\n \t\treturn \"combine\";\n \tcase LOFC__COUNT:\n@@ -91,6 +98,9 @@ static int gently_parse_list_objects_filter(\n \t\tfilter_options->choice = LOFC_SPARSE_OID;\n \t\treturn 0;\n \n+\t} else if (skip_prefix(arg, \"extension:\", &v0)) {\n+\t\treturn parse_extension_filter(filter_options, v0, errbuf);\n+\n \t} else if (skip_prefix(arg, \"sparse:path=\", &v0)) {\n \t\tif (errbuf) {\n \t\t\tstrbuf_addstr(\n@@ -209,6 +219,41 @@ cleanup:\n \treturn result;\n }\n \n+static int parse_extension_filter(\n+\tstruct list_objects_filter_options *filter_options,\n+\tconst char *arg,\n+\tstruct strbuf *errbuf)\n+{\n+\tint result = 0;\n+\tstruct strbuf **params = strbuf_split_str(arg, '=', 2);\n+\n+\tif (!params[0]) {\n+\t\tstrbuf_addstr(errbuf, _(\"expected 'extension:<name>[=<parameter>]'\"));\n+\t\tresult = 1;\n+\t\tgoto cleanup;\n+\t}\n+\n+\tif (params[1]) {\n+\t\t// This extension has a parameter. Remove trailing \"=\" from the name.\n+\t\tsize_t last = params[0]->len - 1;\n+\t\tassert(params[0]->buf[last] == '=');\n+\t\tstrbuf_remove(params[0], last, 1);\n+\n+\t\tfilter_options->extension_value = xstrdup(params[1]->buf);\n+\t}\n+\n+\tfilter_options->extension_name = xstrdup(params[0]->buf);\n+\tfilter_options->choice = LOFC_EXTENSION;\n+\n+cleanup:\n+\tstrbuf_list_free(params);\n+\tif (result) {\n+\t\tlist_objects_filter_release(filter_options);\n+\t\tmemset(filter_options, 0, sizeof(*filter_options));\n+\t}\n+\treturn result;\n+}\n+\n static int allow_unencoded(char ch)\n {\n \tif (ch <= ' ' || ch == '%' || ch == '+')\n@@ -349,6 +394,8 @@ void list_objects_filter_release(\n \t\treturn;\n \tstring_list_clear(&filter_options->filter_spec, /*free_util=*/0);\n \tfree(filter_options->sparse_oid_name);\n+\tfree(filter_options->extension_name);\n+\tfree(filter_options->extension_value);\n \tfor (sub = 0; sub < filter_options->sub_nr; sub++)\n \t\tlist_objects_filter_release(&filter_options->sub[sub]);\n \tfree(filter_options->sub);\ndiff --git a/list-objects-filter-options.h b/list-objects-filter-options.h\nindex da5b6737e27..df3e360324e 100644\n--- a/list-objects-filter-options.h\n+++ b/list-objects-filter-options.h\n@@ -15,6 +15,7 @@ enum list_objects_filter_choice {\n \tLOFC_TREE_DEPTH,\n \tLOFC_SPARSE_OID,\n \tLOFC_OBJECT_TYPE,\n+\tLOFC_EXTENSION,\n \tLOFC_COMBINE,\n \tLOFC__COUNT /* must be last */\n };\n@@ -58,6 +59,11 @@ struct list_objects_filter_options {\n \tunsigned long tree_exclude_depth;\n \tenum object_type object_type;\n \n+\t/* LOFC_EXTENSION values */\n+\n+\tchar *extension_name;\n+\tchar *extension_value;\n+\n \t/* LOFC_COMBINE values */\n \n \t/* This array contains all the subfilters which this filter combines. */\ndiff --git a/list-objects-filter.c b/list-objects-filter.c\nindex 1c1ee3d1bb1..037c674b1c3 100644\n--- a/list-objects-filter.c\n+++ b/list-objects-filter.c\n@@ -10,6 +10,7 @@\n #include \"list-objects.h\"\n #include \"list-objects-filter.h\"\n #include \"list-objects-filter-options.h\"\n+#include \"list-objects-filter-extensions.h\"\n #include \"oidmap.h\"\n #include \"oidset.h\"\n #include \"object-store.h\"\n@@ -620,6 +621,88 @@ static void filter_object_type__init(\n \tfilter->free_fn = free;\n }\n \n+/*\n+ * A filter which passes the objects to a compile-time extension.\n+ * The extension needs to implement the filter_extension interface\n+ * defined in list-objects-filter-extension.h.\n+ * See contrib/filter-extensions/README.md\n+ */\n+\n+struct filter_extension_data {\n+\tconst struct filter_extension *extension;\n+\tvoid *context;\n+};\n+\n+static enum list_objects_filter_result filter_extension_filter_object(\n+\tstruct repository *r,\n+\tenum list_objects_filter_situation filter_situation,\n+\tstruct object *obj,\n+\tconst char *pathname,\n+\tconst char *filename,\n+\tstruct oidset *omits,\n+\tvoid *filter_data)\n+{\n+\tstruct filter_extension_data *d = filter_data;\n+\n+\tenum list_objects_filter_omit omit_it = LOFO_IGNORE;\n+\n+\tenum list_objects_filter_result ret =\n+\t\td->extension->filter_object_fn(\n+\t\t\tr,\n+\t\t\tfilter_situation,\n+\t\t\tobj,\n+\t\t\tpathname,\n+\t\t\tfilename,\n+\t\t\t&omit_it,\n+\t\t\td->context);\n+\n+\tif (omits) {\n+\t\tif (omit_it == LOFO_KEEP)\n+\t\t\toidset_remove(omits, &obj->oid);\n+\t\telse if (omit_it == LOFO_OMIT)\n+\t\t\toidset_insert(omits, &obj->oid);\n+\t}\n+\treturn ret;\n+}\n+\n+static void filter_extension_free(void *filter_data)\n+{\n+\tstruct filter_extension_data *d = filter_data;\n+\td->extension->free_fn(the_repository, d->context);\n+\tfree(d);\n+}\n+\n+static void filter_extension__init(\n+\tstruct list_objects_filter_options *filter_options,\n+\tstruct filter *filter)\n+{\n+\tstruct filter_extension_data *d = xcalloc(1, sizeof(*d));\n+\tint i, r;\n+\n+\tfor (i = 0; filter_extensions[i] != NULL; i++) {\n+\t\tif (!strcmp(\n+\t\t\tfilter_options->extension_name,\n+\t\t\tfilter_extensions[i]->name))\n+\t\t\tbreak;\n+\t}\n+\tif (filter_extensions[i] == NULL) {\n+\t\tdie(_(\"No filter extension found with name %s\"),\n+\t\t\tfilter_options->extension_name);\n+\t}\n+\td->extension = filter_extensions[i];\n+\n+\tr = d->extension->init_fn(\n+\t\tthe_repository, filter_options->extension_value, &d->context);\n+\tif (r) {\n+\t\tdie(_(\"Error initialising filter extension %s: %d\"),\n+\t\t\tfilter_options->extension_name, r);\n+\t}\n+\n+\tfilter->filter_data = d;\n+\tfilter->filter_object_fn = &filter_extension_filter_object;\n+\tfilter->free_fn = &filter_extension_free;\n+}\n+\n /* A filter which only shows objects shown by all sub-filters. */\n struct combine_filter_data {\n \tstruct subfilter *sub;\n@@ -767,6 +850,7 @@ static filter_init_fn s_filters[] = {\n \tfilter_trees_depth__init,\n \tfilter_sparse_oid__init,\n \tfilter_object_type__init,\n+\tfilter_extension__init,\n \tfilter_combine__init,\n };\n \n-- \ngitgitgadget\n\n"},{"id":"434742","messageId":"22475475cae2d32dfa664709f283e4c6e8f60e83.1630885899.git.gitgitgadget@gmail.com","threadId":"56447","inReplyTo":"pull.1031.git.1630885899.gitgitgadget@gmail.com","subject":"[PATCH 4/4] Documentation for list-object-filter extensions","fromName":"Andrew Olsen via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2021-09-05T23:51:39Z","receivedAt":"2021-09-05T23:52:36Z","isPatch":true,"sender":{"key":"andrew.olsen@koordinates.com","avatar":"https://gravatar.com/avatar/f316140b761c57e8a68f9f93ea80a2aceac0c1d37df1b4aec24997025d441a34?d=mp&s=160"},"body":"From: Andrew Olsen <andrew.olsen@koordinates.com>\n\nExplains how to develop a custom extension for list-objects-filter\nbehavior, and how to compile it into a custom build of Git using the\nFILTER_EXTENSIONS Makefile argument.\n\nSigned-off-by: Andrew Olsen <andrew.olsen@koordinates.com>\n---\n Documentation/config/uploadpack.txt  |   7 +-\n Documentation/rev-list-options.txt   |   4 +\n contrib/filter-extensions/README.txt | 153 +++++++++++++++++++++++++++\n 3 files changed, 161 insertions(+), 3 deletions(-)\n create mode 100644 contrib/filter-extensions/README.txt\n\ndiff --git a/Documentation/config/uploadpack.txt b/Documentation/config/uploadpack.txt\nindex 32fad5bbe81..b2ef2421a6d 100644\n--- a/Documentation/config/uploadpack.txt\n+++ b/Documentation/config/uploadpack.txt\n@@ -66,9 +66,10 @@ uploadpackfilter.allow::\n uploadpackfilter.<filter>.allow::\n \tExplicitly allow or ban the object filter corresponding to\n \t`<filter>`, where `<filter>` may be one of: `blob:none`,\n-\t`blob:limit`, `object:type`, `tree`, `sparse:oid`, or `combine`.\n-\tIf using combined filters, both `combine` and all of the nested\n-\tfilter kinds must be allowed. Defaults to `uploadpackfilter.allow`.\n+\t`blob:limit`, `tree`, `sparse:oid`, `combine`, or a named filter extension\n+\t`extension:<name>`. If using combined filters, both `combine` and all of\n+\tthe nested filter kinds must be allowed. Defaults to\n+\t`uploadpackfilter.allow`.\n \n uploadpackfilter.tree.maxDepth::\n \tOnly allow `--filter=tree:<n>` when `<n>` is no more than the value of\ndiff --git a/Documentation/rev-list-options.txt b/Documentation/rev-list-options.txt\nindex b7bd27e1713..d7a317f0aa1 100644\n--- a/Documentation/rev-list-options.txt\n+++ b/Documentation/rev-list-options.txt\n@@ -914,6 +914,10 @@ Note that the form '--filter=sparse:path=<path>' that wants to read\n from an arbitrary path on the filesystem has been dropped for security\n reasons.\n +\n+The form '--filter=extension:<name>[=<arg>]' uses a compile-time extension\n+to implement a named filter. Filter extensions may take an argument string\n+which is passed via `<arg>`.\n++\n Multiple '--filter=' flags can be specified to combine filters. Only\n objects which are accepted by every filter are included.\n +\ndiff --git a/contrib/filter-extensions/README.txt b/contrib/filter-extensions/README.txt\nnew file mode 100644\nindex 00000000000..3d5921cda9b\n--- /dev/null\n+++ b/contrib/filter-extensions/README.txt\n@@ -0,0 +1,153 @@\n+= List-Objects-Filter Extensions API\n+:pp: {plus}{plus}\n+\n+This API can be used to develop filter extensions used for custom filtering\n+behaviour with `git-upload-pack` and `git-rev-list`. The API is defined in\n+link:../../list-objects-filter-extensions.h[list-objects-filter-extensions.h]\n+and defines three functions to implement a filter operation.\n+\n+NOTE: Each filter implementing this API must compiled into Git as a\n+static library. There is some plumbing in the Makefile to help with this\n+via `FILTER_EXTENSIONS`.\n+\n+== Overview\n+\n+. You write a filter and compile it into your custom build of git.\n+. A filter request is received that specifically names the filter extension\n+that you have written, ie: `--filter=extension:<name>[=<arg>]`\n+. The `init_fn` function of your filter is called.\n+. The `filter_object_fn` function of your filter is called for each object\n+at least once.\n+. The `free_fn` function of your filter is called.\n+\n+== Examples\n+\n+*link:./rand/[`rand`]* is a filter that matches all trees and a random\n+percentage of blobs, where the percentage is parsed from the filter arg. It\n+imports and uses the `oid_to_hex()` and `trace_key_printf()` functions from the\n+Git API.\n+\n+Build via:\n+\n+[,console]\n+----\n+$ make FILTER_EXTENSIONS=contrib/filter-extensions/rand/rand.a\n+    ...\n+    SUBDIR contrib/filter-extensions/rand\n+    ...\n+----\n+\n+We can run against git's own repo:\n+\n+[,console]\n+----\n+$ ./git rev-list refs/heads/master --objects --max-count 1 --filter=extension:rand=3 --filter-print-omitted | grep -c '^~'\n+filter-rand: matching 3%\n+filter-rand: done: count=4068 (blob=3866 tree=202) matched=117 elapsed=0.005017s rate=810843.1/s average=1.2us\n+3749  # number of omitted blobs = 3866 - 117\n+----\n+\n+== Development\n+\n+See the examples for a basic implementation. The comments in\n+link:../../list-objects-filter.h[`list-objects-filter.h`] and the built-in\n+filter implementations in\n+link:../../list-objects-filter.c[`list-objects-filter.c`] are important to\n+understand how filters are implemented - `filter_blobs_limit()` provides a\n+simple example, and `filter_sparse()` is more complex.\n+\n+The API differences between the built-in filters and the filter extensions:\n+\n+. Filter extensions don't handle ``omitset``s directly, instead setting `omit`.\n+. Filter extensions receive a void pointer they can use for context.\n+\n+== Building\n+\n+There is some plumbing in the Git Makefile to help with this via\n+`FILTER_EXTENSIONS`, setting it to space-separated paths of the filter extension\n+static libraries indicates that these filters should be compiled into git.\n+For example:\n+\n+[,console]\n+----\n+make FILTER_EXTENSIONS=contrib/filter-extensions/rand/rand.a\n+----\n+\n+Filter extensions don't need to be within the Git source tree. A filter\n+extension static library should either exist at the given path - ie, `rand.a`\n+should exist - or there should be a Makefile in that directory which will create\n+it when `make rand.a` is run. (Such a Makefile should also have a `clean` target\n+which deletes all object files and brings the directory back to its initial\n+state).\n+\n+The static library should define a struct of type `filter_extension` called\n+`filter_extension_NAME` where `NAME` is the name of your extension (ie `rand`\n+for `rand.a`). See\n+link:../../list-objects-filter-extensions.h[list-objects-filter-extensions.h]\n+\n+This definition should follow the following pattern:\n+\n+[,C]\n+----\n+#include \"list-objects-filter-extensions.h\"\n+\n+/* Definitions of rand_init, rand_filter_object, rand_free ... */\n+\n+const struct filter_extension filter_extension_rand = {\n+    \"rand\",\n+    &rand_init,\n+    &rand_filter_object,\n+    &rand_free,\n+};\n+----\n+\n+(The names of your `init_fn`, `filter_object_fn` and `free_fn` are not\n+important, but the string literal should again be the the name of your extension\n+- `\"rand\"` for the filter extension in `rand.a`.)\n+\n+You may use library functions from Git if you include the relevant Git headers,\n+since the filter extensions and Git itself will be linked together into a single\n+binary.\n+\n+You may depend on other libraries if you indicate that they are to be linked\n+into the Git binary using `LDFLAGS`. See the C{pp} example below.\n+\n+== Developing in C{pp} (and other languages)\n+\n+You can develop filter extensions with C{pp}, but many Git header files are not\n+compatible with modern C{pp}, so you won't be able to directly use Git library\n+functions. However, you can use them if you create wrapper functions in C that\n+delegates to the Git library functions you need, but which are also C{pp}\n+compatible. See link:./rand_cpp/[`rand_cpp`] for a simple example. A similar\n+solution would be to implement the extension itself in C, and have the\n+extension do any operations that require Git library functions, but have it\n+delegate to a C wrapper API that you add to a C{pp} library that already\n+contains the domain-specific operations that you need. In either case, remember\n+to wrap any functions that must be C-compatible with `extern C` when declaring\n+or defining them from within C{pp}.\n+\n+To build the C{pp} example:\n+\n+[,console]\n+----\n+make FILTER_EXTENSIONS=contrib/filter-extensions/rand_cpp/rand_cpp.a \\\n+     LDFLAGS=-lstdc++\n+----\n+\n+For other languages you'll either need to port definitions of some internal Git\n+structs (at a minimum, `object`, `object_id`, `repository`, and `hash_algo`) -\n+or again, you could write the extension in C but have it delegate to a domain\n+specific library in the language of your choice that has a C-compatible API.\n+Extra libraries can be required using `LDFLAGS`.\n+\n+== Linking more than one filter extension\n+\n+To link in more than one extension, set `FILTER_EXTENSIONS` to the\n+space-separated paths of all the extensions you want linked. For example, to\n+link in both example filters at once:\n+\n+[,console]\n+----\n+make FILTER_EXTENSIONS=\"contrib/filter-extensions/rand/rand.a contrib/filter-extensions/rand_cpp/rand_cpp.a\" \\\n+     LDFLAGS=-lstdc++\n+----\n-- \ngitgitgadget\n"},{"id":"434743","messageId":"8e0d11ea53a080e8212768f370fb8f05eaded312.1630885899.git.gitgitgadget@gmail.com","threadId":"56447","inReplyTo":"pull.1031.git.1630885899.gitgitgadget@gmail.com","subject":"[PATCH 3/4] Sample list-object-filter extensions","fromName":"Andrew Olsen via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2021-09-05T23:51:38Z","receivedAt":"2021-09-05T23:52:36Z","isPatch":true,"sender":{"key":"andrew.olsen@koordinates.com","avatar":"https://gravatar.com/avatar/f316140b761c57e8a68f9f93ea80a2aceac0c1d37df1b4aec24997025d441a34?d=mp&s=160"},"body":"From: Andrew Olsen <andrew.olsen@koordinates.com>\n\nBasic filter extension example which filters to a random subset of\nblobs, and another example which shows how to do the same in C++ and\nhow to link in another library required by a filter extension.\nDocumentation changes follow.\n\nSigned-off-by: Andrew Olsen <andrew.olsen@koordinates.com>\n---\n contrib/filter-extensions/rand/.gitignore     |   2 +\n contrib/filter-extensions/rand/Makefile       |  28 +++++\n contrib/filter-extensions/rand/rand.c         | 103 ++++++++++++++++++\n contrib/filter-extensions/rand_cpp/.gitignore |   2 +\n contrib/filter-extensions/rand_cpp/Makefile   |  34 ++++++\n .../rand_cpp/adapter_functions.c              |   6 +\n .../rand_cpp/adapter_functions.h              |  10 ++\n contrib/filter-extensions/rand_cpp/rand.cpp   | 103 ++++++++++++++++++\n 8 files changed, 288 insertions(+)\n create mode 100644 contrib/filter-extensions/rand/.gitignore\n create mode 100644 contrib/filter-extensions/rand/Makefile\n create mode 100644 contrib/filter-extensions/rand/rand.c\n create mode 100644 contrib/filter-extensions/rand_cpp/.gitignore\n create mode 100644 contrib/filter-extensions/rand_cpp/Makefile\n create mode 100644 contrib/filter-extensions/rand_cpp/adapter_functions.c\n create mode 100644 contrib/filter-extensions/rand_cpp/adapter_functions.h\n create mode 100644 contrib/filter-extensions/rand_cpp/rand.cpp\n\ndiff --git a/contrib/filter-extensions/rand/.gitignore b/contrib/filter-extensions/rand/.gitignore\nnew file mode 100644\nindex 00000000000..9eca6c88cf2\n--- /dev/null\n+++ b/contrib/filter-extensions/rand/.gitignore\n@@ -0,0 +1,2 @@\n+*.a\n+*.o\ndiff --git a/contrib/filter-extensions/rand/Makefile b/contrib/filter-extensions/rand/Makefile\nnew file mode 100644\nindex 00000000000..267221ee952\n--- /dev/null\n+++ b/contrib/filter-extensions/rand/Makefile\n@@ -0,0 +1,28 @@\n+# Run this via `FILTER_EXTENSIONS=contrib/filter-extensions/rand/rand.a make`\n+# from the main git directory. That way we inherit useful variables.\n+\n+ifneq ($(findstring s,$(MAKEFLAGS)),s)\n+ifndef V\n+\tQUIET_CC       = @echo '   ' CC $@;\n+\tQUIET_AR       = @echo '   ' AR $@;\n+endif\n+endif\n+\n+FILTER_STATIC_LIB = rand.a\n+\n+all: $(FILTER_STATIC_LIB)\n+ifeq ($(MAKELEVEL),0)\n+\t$(error \"Run via parent git make\")\n+endif\n+\t@:\n+\n+$(FILTER_STATIC_LIB): rand.o\n+\t$(QUIET_AR)$(AR) $(ARFLAGS) $@ $^\n+\n+rand.o: rand.c\n+\t$(QUIET_CC)$(CC) -c $(ALL_CFLAGS) $<\n+\n+clean:\n+\t$(RM) $(FILTER_STATIC_LIB) rand.o\n+\n+.PHONY: all clean\ndiff --git a/contrib/filter-extensions/rand/rand.c b/contrib/filter-extensions/rand/rand.c\nnew file mode 100644\nindex 00000000000..af153709345\n--- /dev/null\n+++ b/contrib/filter-extensions/rand/rand.c\n@@ -0,0 +1,103 @@\n+#include \"../../../git-compat-util.h\"\n+#include \"../../../list-objects-filter-extensions.h\"\n+#include \"../../../object.h\"\n+#include \"../../../hash.h\"\n+#include \"../../../trace.h\"\n+\n+\n+static struct trace_key trace_filter = TRACE_KEY_INIT(FILTER);\n+\n+struct rand_context {\n+\tint percentageMatch;\n+\tint matchCount;\n+\tint blobCount;\n+\tint treeCount;\n+\tuint64_t started_at;\n+};\n+\n+static int rand_init(\n+\tconst struct repository *r,\n+\tconst char *filter_arg,\n+\tvoid **context)\n+{\n+\tstruct rand_context *ctx = calloc(1, sizeof(struct rand_context));\n+\n+\tctx->percentageMatch = atoi(filter_arg);\n+\tif (ctx->percentageMatch > 100 || ctx->percentageMatch < 0) {\n+\tfprintf(stderr, \"filter-rand: warning: invalid match %%: %s\\n\",\n+\t\tfilter_arg);\n+\tctx->percentageMatch = 1;  // default 1%\n+\t}\n+\tfprintf(stderr, \"filter-rand: matching %d%%\\n\", ctx->percentageMatch);\n+\tctx->started_at = getnanotime();\n+\t(*context) = ctx;\n+\n+\treturn 0;\n+}\n+\n+static enum list_objects_filter_result rand_filter_object(\n+\tconst struct repository *r,\n+\tconst enum list_objects_filter_situation filter_situation,\n+\tstruct object *obj,\n+\tconst char *pathname,\n+\tconst char *filename,\n+\tenum list_objects_filter_omit *omit,\n+\tvoid *context)\n+{\n+\tstruct rand_context *ctx = (struct rand_context*)(context);\n+\n+\tif ((ctx->blobCount + ctx->treeCount + 1) % 100000 == 0) {\n+\t\tfprintf(stderr, \"filter-rand: %d...\\n\",\n+\t\t\t(ctx->blobCount + ctx->treeCount + 1));\n+\t}\n+\n+\tswitch (filter_situation) {\n+\tdefault:\n+\t\tdie(\"filter-rand: unknown filter_situation: %d\", filter_situation);\n+\n+\tcase LOFS_BEGIN_TREE:\n+\t\tctx->treeCount++;\n+\t\t/* always include all tree objects */\n+\t\treturn LOFR_MARK_SEEN | LOFR_DO_SHOW;\n+\n+\tcase LOFS_END_TREE:\n+\t\treturn LOFR_ZERO;\n+\n+\tcase LOFS_BLOB:\n+\t\tctx->blobCount++;\n+\n+\t\tif ((rand() % 100) < ctx->percentageMatch) {\n+\t\t\tctx->matchCount++;\n+\t\t\ttrace_printf_key(&trace_filter,\n+\t\t\t\t\"match: %s %s\\n\",\n+\t\t\t\toid_to_hex(&obj->oid),\n+\t\t\t\tpathname\n+\t\t\t);\n+\t\t\treturn LOFR_MARK_SEEN | LOFR_DO_SHOW;\n+\t\t} else {\n+\t\t\t*omit = LOFO_OMIT;\n+\t\t\treturn LOFR_MARK_SEEN; /* hard omit */\n+\t\t}\n+\t}\n+}\n+\n+static void rand_free(const struct repository *r, void *context)\n+{\n+\tstruct rand_context *ctx = (struct rand_context*)(context);\n+\tdouble elapsed = (getnanotime() - ctx->started_at)/1E9;\n+\tint count = ctx->blobCount + ctx->treeCount;\n+\n+\tfprintf(stderr, \"filter-rand: done: count=%d (blob=%d tree=%d) \"\n+\t\t\"matched=%d elapsed=%fs rate=%0.1f/s average=%0.1fus\\n\",\n+\t\tcount, ctx->blobCount, ctx->treeCount, ctx->matchCount,\n+\t\telapsed, count/elapsed, elapsed/count*1E6);\n+\n+\tfree(ctx);\n+}\n+\n+const struct filter_extension filter_extension_rand = {\n+\t\"rand\",\n+\t&rand_init,\n+\t&rand_filter_object,\n+\t&rand_free,\n+};\ndiff --git a/contrib/filter-extensions/rand_cpp/.gitignore b/contrib/filter-extensions/rand_cpp/.gitignore\nnew file mode 100644\nindex 00000000000..9eca6c88cf2\n--- /dev/null\n+++ b/contrib/filter-extensions/rand_cpp/.gitignore\n@@ -0,0 +1,2 @@\n+*.a\n+*.o\ndiff --git a/contrib/filter-extensions/rand_cpp/Makefile b/contrib/filter-extensions/rand_cpp/Makefile\nnew file mode 100644\nindex 00000000000..278121e3d5a\n--- /dev/null\n+++ b/contrib/filter-extensions/rand_cpp/Makefile\n@@ -0,0 +1,34 @@\n+# Run this via `FILTER_EXTENSIONS=contrib/filter-extensions/rand_cpp/rand_cpp.a make`\n+# from the main git directory. That way we inherit useful variables.\n+\n+ifneq ($(findstring s,$(MAKEFLAGS)),s)\n+ifndef V\n+\tQUIET_CC       = @echo '   ' CC $@;\n+\tQUIET_CXX       = @echo '   ' CXX $@;\n+\tQUIET_AR       = @echo '   ' AR $@;\n+endif\n+endif\n+\n+FILTER_STATIC_LIB = rand_cpp.a\n+\n+ALL_CXXFLAGS += -std=c++11\n+\n+all: $(FILTER_STATIC_LIB)\n+ifeq ($(MAKELEVEL),0)\n+\t$(error \"Run via parent git make\")\n+endif\n+\t@:\n+\n+$(FILTER_STATIC_LIB): rand.o adapter_functions.o\n+\t$(QUIET_AR)$(AR) $(ARFLAGS) $@ $^\n+\n+rand.o: rand.cpp\n+\t$(QUIET_CXX)$(CXX) -c $(ALL_CFLAGS) $(ALL_CXXFLAGS) $<\n+\n+adapter_functions.o: adapter_functions.c\n+\t$(QUIET_CC)$(CC) -c $(ALL_CFLAGS) $<\n+\n+clean:\n+\t$(RM) $(FILTER_STATIC_LIB) rand.o\n+\n+.PHONY: all clean\ndiff --git a/contrib/filter-extensions/rand_cpp/adapter_functions.c b/contrib/filter-extensions/rand_cpp/adapter_functions.c\nnew file mode 100644\nindex 00000000000..0d9d2a2aa96\n--- /dev/null\n+++ b/contrib/filter-extensions/rand_cpp/adapter_functions.c\n@@ -0,0 +1,6 @@\n+#include \"../../../git-compat-util.h\"\n+#include \"../../../object.h\"\n+\n+char *obj_to_hex_oid(struct object *obj) {\n+    return oid_to_hex(&obj->oid);\n+}\ndiff --git a/contrib/filter-extensions/rand_cpp/adapter_functions.h b/contrib/filter-extensions/rand_cpp/adapter_functions.h\nnew file mode 100644\nindex 00000000000..1150c21a258\n--- /dev/null\n+++ b/contrib/filter-extensions/rand_cpp/adapter_functions.h\n@@ -0,0 +1,10 @@\n+#ifndef RAND_CPP_ADAPTER_FUNCTIONS_H\n+#define RAND_CPP_ADAPTER_FUNCTIONS_H\n+\n+struct object;\n+\n+uint64_t getnanotime(void);\n+\n+char *obj_to_hex_oid(struct object *obj);\n+\n+#endif /* RAND_CPP_ADAPTER_FUNCTIONS_H */\ndiff --git a/contrib/filter-extensions/rand_cpp/rand.cpp b/contrib/filter-extensions/rand_cpp/rand.cpp\nnew file mode 100644\nindex 00000000000..cb608d14ed9\n--- /dev/null\n+++ b/contrib/filter-extensions/rand_cpp/rand.cpp\n@@ -0,0 +1,103 @@\n+#include <iomanip>\n+#include <iostream>\n+#include <sstream>\n+\n+#include <time.h>\n+\n+extern \"C\" {\n+\t#include \"../../../list-objects-filter-extensions.h\"\n+\t#include \"adapter_functions.h\"\n+}\n+\n+namespace {\n+\n+struct rand_context {\n+\tint percentageMatch = 0;\n+\tint matchCount = 0;\n+\tint blobCount = 0;\n+\tint treeCount = 0;\n+\tuint64_t started_at = 0;\n+};\n+\n+static int rand_init(\n+\tconst struct repository *r,\n+\tconst char *filter_arg,\n+\tvoid **context)\n+{\n+\tstruct rand_context *ctx = new rand_context();\n+\n+\tctx->percentageMatch = atoi(filter_arg);\n+\tif (ctx->percentageMatch > 100 || ctx->percentageMatch < 0) {\n+\t\tstd::cerr << \"filter-rand-cpp: warning: invalid match %: \" << filter_arg << \"\\n\";\n+\t\tctx->percentageMatch = 1;  // default 1%\n+\t}\n+\tstd::cerr << \"filter-rand-cpp: matching \" << ctx->percentageMatch << \"%\\n\";\n+\tctx->started_at = getnanotime();\n+\n+\treturn 0;\n+}\n+\n+enum list_objects_filter_result rand_filter_object(\n+\tconst struct repository *r,\n+\tconst enum list_objects_filter_situation filter_situation,\n+\tstruct object *obj,\n+\tconst char *pathname,\n+\tconst char *filename,\n+\tenum list_objects_filter_omit *omit,\n+\tvoid *context)\n+{\n+\tstruct rand_context *ctx = static_cast<struct rand_context*>(context);\n+\n+\tif ((ctx->blobCount + ctx->treeCount + 1) % 100000 == 0) {\n+\t\tstd::cerr << \"filter-rand-cpp: \" << (ctx->blobCount + ctx->treeCount + 1) << \"...\\n\";\n+\t}\n+\tswitch (filter_situation) {\n+\tdefault:\n+\t\tstd::cerr << \"filter-rand-cpp: unknown filter_situation: \" << filter_situation << \"\\n\";\n+\t\tabort();\n+\n+\tcase LOFS_BEGIN_TREE:\n+\t\tctx->treeCount++;\n+\t\t/* always include all tree objects */\n+\t\treturn static_cast<list_objects_filter_result>(LOFR_MARK_SEEN | LOFR_DO_SHOW);\n+\n+\tcase LOFS_END_TREE:\n+\t\treturn LOFR_ZERO;\n+\n+\tcase LOFS_BLOB:\n+\t\tctx->blobCount++;\n+\n+\t\tif ((rand() % 100) < ctx->percentageMatch) {\n+\t\t\tctx->matchCount++;\n+\t\t\tstd::cout << \"match: \" << obj_to_hex_oid(obj) << pathname << \"\\n\";\n+\t\t\treturn static_cast<list_objects_filter_result>(LOFR_MARK_SEEN | LOFR_DO_SHOW);\n+\t\t} else {\n+\t\t\t*omit = LOFO_OMIT;\n+\t\t\treturn LOFR_MARK_SEEN; /* but not LOFR_DO_SHOW (hard omit) */\n+\t\t}\n+\t}\n+}\n+\n+void rand_free(const struct repository *r, void *context) {\n+\tstruct rand_context *ctx = static_cast<struct rand_context*>(context);\n+\tdouble elapsed = (getnanotime() - ctx->started_at)/1E9;\n+\tint count = ctx->blobCount + ctx->treeCount;\n+\n+\tstd::cerr << \"filter-rand-cpp: done: count=\" << count\n+\t\t<< \" (blob=\" << ctx->blobCount << \" tree=\" << ctx->treeCount << \")\"\n+\t\t<< \" matched=\" << ctx->matchCount\n+\t\t<< \" elapsed=\" << elapsed << \"s\"\n+\t\t<< \" rate=\" << count/elapsed << \"/s\"\n+\t\t<< \" average=\" << elapsed/count*1E6 << \"us\\n\";\n+\n+\tdelete ctx;\n+}\n+\n+} // namespace\n+\n+extern const struct filter_extension filter_extension_rand_cpp = {\n+\t\"rand_cpp\",\n+\t&rand_init,\n+\t&rand_filter_object,\n+\t&rand_free,\n+};\n-- \ngitgitgadget\n\n"},{"id":"434744","messageId":"43415de761ac029b67645d9131b38e761ef6ae21.1630885899.git.gitgitgadget@gmail.com","threadId":"56447","inReplyTo":"pull.1031.git.1630885899.gitgitgadget@gmail.com","subject":"[PATCH 2/4] Makefile for list-object-filter extensions","fromName":"Andrew Olsen via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2021-09-05T23:51:37Z","receivedAt":"2021-09-05T23:52:36Z","isPatch":true,"sender":{"key":"andrew.olsen@koordinates.com","avatar":"https://gravatar.com/avatar/f316140b761c57e8a68f9f93ea80a2aceac0c1d37df1b4aec24997025d441a34?d=mp&s=160"},"body":"From: Andrew Olsen <andrew.olsen@koordinates.com>\n\nCustom list-object-filter extensions can be compiled into Git using the\nFILTER_EXTENSIONS Makefile argument.\n\nSigned-off-by: Andrew Olsen <andrew.olsen@koordinates.com>\n---\n Makefile                            | 35 +++++++++++++++++++++++++++--\n compat/vcbuild/README               |  5 +++--\n config.mak.uname                    |  6 ++---\n contrib/buildsystems/CMakeLists.txt |  7 ++++++\n 4 files changed, 46 insertions(+), 7 deletions(-)\n\ndiff --git a/Makefile b/Makefile\nindex 429c276058d..0b1d0be81a5 100644\n--- a/Makefile\n+++ b/Makefile\n@@ -471,6 +471,11 @@ all::\n # directory, and the JSON compilation database 'compile_commands.json' will be\n # created at the root of the repository.\n #\n+# Define FILTER_EXTENSIONS to a space-separated list of static library plugins\n+# that implement the filter-object-list extension API. Each of this filter\n+# extensions will then be available in addition to the builtin ones such as\n+# \"blob:limit\" and \"object:type\". See contrib/filter-extensions/README.txt\n+#\n # Define DEVELOPER to enable more compiler warnings. Compiler version\n # and family are auto detected, but could be overridden by defining\n # COMPILER_FEATURES (see config.mak.dev). You can still set\n@@ -824,6 +829,7 @@ XDIFF_LIB = xdiff/lib.a\n \n GENERATED_H += command-list.h\n GENERATED_H += config-list.h\n+GENERATED_C += list-objects-filter-extensions.c\n \n LIB_H := $(sort $(patsubst ./%,%,$(shell git ls-files '*.h' ':!t/' ':!Documentation/' 2>/dev/null || \\\n \t$(FIND) . \\\n@@ -916,6 +922,7 @@ LIB_OBJS += levenshtein.o\n LIB_OBJS += line-log.o\n LIB_OBJS += line-range.o\n LIB_OBJS += linear-assignment.o\n+LIB_OBJS += list-objects-filter-extensions.o\n LIB_OBJS += list-objects-filter-options.o\n LIB_OBJS += list-objects-filter.o\n LIB_OBJS += list-objects.o\n@@ -2116,6 +2123,19 @@ ifdef DEFAULT_HELP_FORMAT\n BASIC_CFLAGS += -DDEFAULT_HELP_FORMAT='\"$(DEFAULT_HELP_FORMAT)\"'\n endif\n \n+ifneq ($(FILTER_EXTENSIONS),)\n+FILTER_EXT_PATHS = $(dir $(FILTER_EXTENSIONS))\n+\n+$(FILTER_EXTENSIONS): $(FILTER_EXT_PATHS)\n+\t$(QUIET_SUBDIR0)$(@D) $(QUIET_SUBDIR1) \\\n+\t\tALL_CFLAGS='$(subst ','\\'',$(ALL_CFLAGS))' \\\n+\t\tALL_LDFLAGS='$(subst ','\\'',$(ALL_LDFLAGS))' \\\n+\t\tPROFILE_DIR='$(subst ','\\'',$(PROFILE_DIR))' \\\n+\t\t$(@F)\n+\n+GITLIBS += $(FILTER_EXTENSIONS)\n+endif\n+\n PAGER_ENV_SQ = $(subst ','\\'',$(PAGER_ENV))\n PAGER_ENV_CQ = \"$(subst \",\\\",$(subst \\,\\\\,$(PAGER_ENV)))\"\n PAGER_ENV_CQ_SQ = $(subst ','\\'',$(PAGER_ENV_CQ))\n@@ -2222,7 +2242,7 @@ git.sp git.s git.o: EXTRA_CPPFLAGS = \\\n \t'-DGIT_MAN_PATH=\"$(mandir_relative_SQ)\"' \\\n \t'-DGIT_INFO_PATH=\"$(infodir_relative_SQ)\"'\n \n-git$X: git.o GIT-LDFLAGS $(BUILTIN_OBJS) $(GITLIBS)\n+git$X: git.o GIT-LDFLAGS $(BUILTIN_OBJS) $(GITLIBS) $(EXTENSION_LIBS)\n \t$(QUIET_LINK)$(CC) $(ALL_CFLAGS) -o $@ $(ALL_LDFLAGS) \\\n \t\t$(filter %.o,$^) $(LIBS)\n \n@@ -2261,6 +2281,10 @@ command-list.h: $(wildcard Documentation/git*.txt)\n \t\t$(patsubst %,--exclude-program %,$(EXCLUDED_PROGRAMS)) \\\n \t\tcommand-list.txt >$@+ && mv $@+ $@\n \n+list-objects-filter-extensions.c: generate-list-objects-filter-extensions.sh GIT-BUILD-OPTIONS\n+\t$(QUIET_GEN)$(SHELL_PATH) ./generate-list-objects-filter-extensions.sh \\\n+\t\t$(FILTER_EXTENSIONS) > $@+ && mv $@+ $@\n+\n SCRIPT_DEFINES = $(SHELL_PATH_SQ):$(DIFF_SQ):$(GIT_VERSION):\\\n \t$(localedir_SQ):$(NO_CURL):$(USE_GETTEXT_SCHEME):$(SANE_TOOL_PATH_SQ):\\\n \t$(gitwebdir_SQ):$(PERL_PATH_SQ):$(SANE_TEXT_GREP):$(PAGER_ENV):\\\n@@ -2612,6 +2636,7 @@ $(LIB_FILE): $(LIB_OBJS)\n $(XDIFF_LIB): $(XDIFF_OBJS)\n \t$(QUIET_AR)$(RM) $@ && $(AR) $(ARFLAGS) $@ $^\n \n+\n export DEFAULT_EDITOR DEFAULT_PAGER\n \n Documentation/GIT-EXCLUDED-PROGRAMS: FORCE\n@@ -2857,6 +2882,9 @@ ifdef RUNTIME_PREFIX\n \t@echo RUNTIME_PREFIX=\\'true\\' >>$@+\n else\n \t@echo RUNTIME_PREFIX=\\'false\\' >>$@+\n+endif\n+ifdef FILTER_EXTENSIONS\n+\t@echo FILTER_EXTENSIONS=\\''$(subst ','\\'',$(subst ','\\'',$(FILTER_EXTENSIONS)))'\\' >>$@+\n endif\n \t@if cmp $@+ $@ >/dev/null 2>&1; then $(RM) $@+; else mv $@+ $@; fi\n \n@@ -3241,7 +3269,7 @@ clean: profile-clean coverage-clean cocciclean\n \t$(RM) $(HCC)\n \t$(RM) -r bin-wrappers $(dep_dirs) $(compdb_dir) compile_commands.json\n \t$(RM) -r po/build/\n-\t$(RM) *.pyc *.pyo */*.pyc */*.pyo $(GENERATED_H) $(ETAGS_TARGET) tags cscope*\n+\t$(RM) *.pyc *.pyo */*.pyc */*.pyo $(GENERATED_H) $(GENERATED_C) $(ETAGS_TARGET) tags cscope*\n \t$(RM) -r .dist-tmp-dir .doc-tmp-dir\n \t$(RM) $(GIT_TARNAME).tar.gz\n \t$(RM) $(htmldocs).tar.gz $(manpages).tar.gz\n@@ -3256,6 +3284,9 @@ endif\n ifndef NO_TCLTK\n \t$(MAKE) -C gitk-git clean\n \t$(MAKE) -C git-gui clean\n+endif\n+ifneq ($(FILTER_EXTENSIONS),)\n+\t$(foreach FP,$(FILTER_EXTENSIONS),$(MAKE) -C $(dir $(FP)) clean && ) true\n endif\n \t$(RM) GIT-VERSION-FILE GIT-CFLAGS GIT-LDFLAGS GIT-BUILD-OPTIONS\n \t$(RM) GIT-USER-AGENT GIT-PREFIX\ndiff --git a/compat/vcbuild/README b/compat/vcbuild/README\nindex 51fb083dbbe..5e39022eade 100644\n--- a/compat/vcbuild/README\n+++ b/compat/vcbuild/README\n@@ -92,8 +92,9 @@ The Steps of Build Git with VS2008\n    the git operations.\n \n 3. Inside Git's directory run the command:\n-       make command-list.h config-list.h\n-   to generate the header file needed to compile git.\n+       make command-list.h config-list.h list-objects-filter-extensions.c\n+   to generate those source files that are not included in the repo, but\n+   instead are automatically generated from other files.\n \n 4. Then either build Git with the GNU Make Makefile in the Git projects\n    root\ndiff --git a/config.mak.uname b/config.mak.uname\nindex 76516aaa9a5..405e7d91e7a 100644\n--- a/config.mak.uname\n+++ b/config.mak.uname\n@@ -735,9 +735,9 @@ vcxproj:\n \t echo '</Project>') >git-remote-http/LinkOrCopyRemoteHttp.targets\n \tgit add -f git/LinkOrCopyBuiltins.targets git-remote-http/LinkOrCopyRemoteHttp.targets\n \n-\t# Add command-list.h and config-list.h\n-\t$(MAKE) MSVC=1 SKIP_VCPKG=1 prefix=/mingw64 config-list.h command-list.h\n-\tgit add -f config-list.h command-list.h\n+\t# Add command-list.h, config-list.h list-objects-filter-extensions.c\n+\t$(MAKE) MSVC=1 SKIP_VCPKG=1 prefix=/mingw64 config-list.h command-list.h list-objects-filter-extensions.c\n+\tgit add -f config-list.h command-list.h list-objects-filter-extensions.c\n \n \t# Add scripts\n \trm -f perl/perl.mak\ndiff --git a/contrib/buildsystems/CMakeLists.txt b/contrib/buildsystems/CMakeLists.txt\nindex 171b4124afe..60627a2892f 100644\n--- a/contrib/buildsystems/CMakeLists.txt\n+++ b/contrib/buildsystems/CMakeLists.txt\n@@ -624,6 +624,13 @@ if(NOT EXISTS ${CMAKE_BINARY_DIR}/config-list.h)\n \t\t\tOUTPUT_FILE ${CMAKE_BINARY_DIR}/config-list.h)\n endif()\n \n+if(NOT EXISTS ${CMAKE_BINARY_DIR}/list-objects-filter-extensions.c)\n+\tmessage(\"Generating list-objects-filter-extensions.c\")\n+\texecute_process(COMMAND ${SH_EXE} ${CMAKE_SOURCE_DIR}/generate-list-objects-filter-extensions.sh\n+\t\t\tWORKING_DIRECTORY ${CMAKE_SOURCE_DIR}\n+\t\t\tOUTPUT_FILE ${CMAKE_BINARY_DIR}/list-objects-filter-extensions.c)\n+endif()\n+\n include_directories(${CMAKE_BINARY_DIR})\n \n #build\n-- \ngitgitgadget\n\n"},{"id":"434748","messageId":"87fsuiplv2.fsf@evledraar.gmail.com","threadId":"56447","inReplyTo":"pull.1031.git.1630885899.gitgitgadget@gmail.com","subject":"Re: [PATCH 0/4] Compile-time extensions for list-object-filter","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2021-09-06T00:49:38Z","receivedAt":"2021-09-06T00:51:36Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"\nOn Sun, Sep 05 2021, Andrew Olsen via GitGitGadget wrote:\n\n> Adds an extension: option to list-object-filters, these are implemented by\n> static libraries that must be compiled into Git. The Makefile argument\n> FILTER_EXTENSIONS makes it easier to compile these extensions into a custom\n> build of Git. When no custom filter-extensions are supplied, Git works as\n> normal.\n\nHaving skimmed this and the added documentation I think what's really\nmissing is a \"why\"? What concrete use-case is this going to serve?\n\nI.e. what is an extension you have in mind that's useful, but not so\nuseful as to even suggest it for inclusion in git.git before coming up\nwith this plug-in API mechanism?\n\nAlso, for such plug-ins the license is going to be GPL-v2 too I assume?\nBut that aspect isn't covered at all.\n"},{"id":"434762","messageId":"20713339-ba53-38e2-b20c-befe68dc147e@gmail.com","threadId":"56447","inReplyTo":"43415de761ac029b67645d9131b38e761ef6ae21.1630885899.git.gitgitgadget@gmail.com","subject":"Re: [PATCH 2/4] Makefile for list-object-filter extensions","fromName":"Bagas Sanjaya","fromEmail":"bagasdotme@gmail.com","sentAt":"2021-09-06T06:15:29Z","receivedAt":"2021-09-06T06:15:39Z","isPatch":true,"sender":{"key":"bagasdotme@gmail.com","avatar":"https://avatars.githubusercontent.com/u/40219486?v=4"},"body":"On 06/09/21 06.51, Andrew Olsen via GitGitGadget wrote:\n> From: Andrew Olsen <andrew.olsen@koordinates.com>\n> \n> Custom list-object-filter extensions can be compiled into Git using the\n> FILTER_EXTENSIONS Makefile argument.\n> \n\nThis can be squashed to previous patch.\n\n-- \nAn old man doll... just what I always wanted! - Clara\n"},{"id":"434763","messageId":"066e186a-ffa5-3548-1b7d-a769bbd296f2@gmail.com","threadId":"56447","inReplyTo":"pull.1031.git.1630885899.gitgitgadget@gmail.com","subject":"Re: [PATCH 0/4] Compile-time extensions for list-object-filter","fromName":"Bagas Sanjaya","fromEmail":"bagasdotme@gmail.com","sentAt":"2021-09-06T06:18:59Z","receivedAt":"2021-09-06T06:19:08Z","isPatch":true,"sender":{"key":"bagasdotme@gmail.com","avatar":"https://avatars.githubusercontent.com/u/40219486?v=4"},"body":"On 06/09/21 06.51, Andrew Olsen via GitGitGadget wrote:\n> Adds an extension: option to list-object-filters, these are implemented by\n> static libraries that must be compiled into Git. The Makefile argument\n> FILTER_EXTENSIONS makes it easier to compile these extensions into a custom\n> build of Git. When no custom filter-extensions are supplied, Git works as\n> normal.\n\nI don't see why this series is useful (use cases?).\n\n-- \nAn old man doll... just what I always wanted! - Clara\n"},{"id":"434812","messageId":"CAPJmHpV2c_EubO4-GpV5Yi3RLpgvcaake=fz0rfgD_SZZcawOg@mail.gmail.com","threadId":"56447","inReplyTo":"066e186a-ffa5-3548-1b7d-a769bbd296f2@gmail.com","subject":"Re: [PATCH 0/4] Compile-time extensions for list-object-filter","fromName":"Andrew Olsen","fromEmail":"andrew.olsen@koordinates.com","sentAt":"2021-09-07T00:37:50Z","receivedAt":"2021-09-07T00:38:03Z","isPatch":true,"sender":{"key":"andrew.olsen@koordinates.com","avatar":"https://gravatar.com/avatar/f316140b761c57e8a68f9f93ea80a2aceac0c1d37df1b4aec24997025d441a34?d=mp&s=160"},"body":"Good point - sorry I sent this out without accompanying explanation. I'm still\nlearning about contributing to Git.\n\nThe filter extension that I want to implement is a spatial filter - it will\nreturn blobs that store geometries that intersect with a given geometry, eg,\n\"only return blobs in North America\". This is useful to us at kartproject.org,\n\"distributed version control for geospatial data\", which is built on Git. But\nsafe to say that this functionality is not generally useful to Git users.\n\nHowever, the idea we have is that there will be others who want to implement\ncustom filters also - perhaps like the spatial filter, these could be\ndomain-specific filters that are not useful to most Git users, but allow for\na custom Git to be more powerful when storing data from a particular domain.\nWe could just fork git and do what we want with the fork, but defining a plugin\ninterface makes it possible for us to keep using Git at master, instead of\nmaintaining a fork indefinitely.\n\nMy colleague Robert Coup coded this up once already as a plugin library\ninterface that could be loaded at runtime, and I've been tasked with rewriting\nit as a compile-time interface, which he thought was \"more likely\" (but of\ncourse not guaranteed) to be accepted as a worthwhile change to Git. He's\nunfortunately on the other side of the world to me and not working today, but\nI hope when he reappears he'll be able to say something more in defence of this\nidea, and perhaps give a history of the reasoning for this particular solution.\n\nRegarding licenses: the sample extensions I'm contributing will be covered by\nGit's GPL-v2 (I assume), if they make it into the Git repository. Any other\nextensions that may be written by third party authors and are maintained\nelsewhere could be licensed as those authors see fit, as long as they take care\nnot to violate the terms of Git's GPL-v2 when they distribute the extension or\nGit and the extension together. I could add a link to the GPL-v2 in the README\nwarning developers to check it before distributing any kind of extension to Git.\nI'm not a lawyer and wouldn't want to give more specific advice than that.\n\nOn Tue, Sep 7, 2021 at 11:24 AM Bagas Sanjaya <bagasdotme@gmail.com> wrote:\n>\n> On 06/09/21 06.51, Andrew Olsen via GitGitGadget wrote:\n> > Adds an extension: option to list-object-filters, these are implemented by\n> > static libraries that must be compiled into Git. The Makefile argument\n> > FILTER_EXTENSIONS makes it easier to compile these extensions into a custom\n> > build of Git. When no custom filter-extensions are supplied, Git works as\n> > normal.\n>\n> I don't see why this series is useful (use cases?).\n>\n> --\n> An old man doll... just what I always wanted! - Clara\n>\n"},{"id":"434820","messageId":"87r1e0n3yi.fsf@evledraar.gmail.com","threadId":"56447","inReplyTo":"CAPJmHpV2c_EubO4-GpV5Yi3RLpgvcaake=fz0rfgD_SZZcawOg@mail.gmail.com","subject":"Re: [PATCH 0/4] Compile-time extensions for list-object-filter","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2021-09-07T08:59:15Z","receivedAt":"2021-09-07T09:13:30Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"\nOn Tue, Sep 07 2021, Andrew Olsen wrote:\n\n> Good point - sorry I sent this out without accompanying explanation. I'm still\n> learning about contributing to Git.\n>\n> The filter extension that I want to implement is a spatial filter - it will\n> return blobs that store geometries that intersect with a given geometry, eg,\n> \"only return blobs in North America\". This is useful to us at kartproject.org,\n> \"distributed version control for geospatial data\", which is built on Git. But\n> safe to say that this functionality is not generally useful to Git users.\n\nThat's interesting. I think you're probably right that returning blobs\nby a GEO filter is not going to be generally useful to git (I assume\nit's aware of some specially-encoded blobs?), but a mechanism that\nenables that might be...\n\n> However, the idea we have is that there will be others who want to implement\n> custom filters also - perhaps like the spatial filter, these could be\n> domain-specific filters that are not useful to most Git users, but allow for\n> a custom Git to be more powerful when storing data from a particular domain.\n> We could just fork git and do what we want with the fork, but defining a plugin\n> interface makes it possible for us to keep using Git at master, instead of\n> maintaining a fork indefinitely.\n>\n> [...]\n>\n> My colleague Robert Coup coded this up once already as a plugin library\n> interface that could be loaded at runtime, and I've been tasked with rewriting\n> it as a compile-time interface, which he thought was \"more likely\" (but of\n> course not guaranteed) to be accepted as a worthwhile change to Git. He's\n> unfortunately on the other side of the world to me and not working today, but\n> I hope when he reappears he'll be able to say something more in defence of this\n> idea, and perhaps give a history of the reasoning for this particular solution.\n\nWhile it would be easier for you it would leave this project stuck\nmaintaining a C API interface, and indeed your documentation suggests\nthat not only should users use the narrow C API provided here, but any\narbitrary internal structs in git.git.\n\nPersonally I'm not per-se opposed to such a thing, but I think that we\nshould really be considering and trying something like the clean/smudge\nhook interface first rather than a full C API.\n\nThis seems like a perfect fit for such an IPC interface, i.e. we'd have\na hook to register custom filters, and when it came to filtering objects\ngit would communicate with that hook, which in turn would query data\nwith something like the \"git cat-file --batch\" interface.\n\n> Regarding licenses: the sample extensions I'm contributing will be covered by\n> Git's GPL-v2 (I assume), if they make it into the Git repository. Any other\n> extensions that may be written by third party authors and are maintained\n> elsewhere could be licensed as those authors see fit, as long as they take care\n> not to violate the terms of Git's GPL-v2 when they distribute the extension or\n> Git and the extension together. I could add a link to the GPL-v2 in the README\n> warning developers to check it before distributing any kind of extension to Git.\n> I'm not a lawyer and wouldn't want to give more specific advice than that.\n\nI believe you've misunderstood how the GPL works, those third party\nauthors would not be free to license their plugins as they see fit. The\nreason the LGPL license exists is to allow what you're describing, but\ngit uses the full GPL v2.\n\nSee https://en.wikipedia.org/wiki/GPL_linking_exception and\nhttps://www.gnu.org/licenses/gpl-faq.html#LinkingWithGPL\n\nThe project you've linked to even has a GPL linking exception of its own\n(but git.git does not):\nhttps://github.com/koordinates/kart/blob/2934f2b951d61233cbaab9ff627aa3c8cbfb82bc/COPYING#L7-L16\n\n> On Tue, Sep 7, 2021 at 11:24 AM Bagas Sanjaya <bagasdotme@gmail.com> wrote:\n>>\n>> On 06/09/21 06.51, Andrew Olsen via GitGitGadget wrote:\n>> > Adds an extension: option to list-object-filters, these are implemented by\n>> > static libraries that must be compiled into Git. The Makefile argument\n>> > FILTER_EXTENSIONS makes it easier to compile these extensions into a custom\n>> > build of Git. When no custom filter-extensions are supplied, Git works as\n>> > normal.\n>>\n>> I don't see why this series is useful (use cases?).\n>>\n>> --\n>> An old man doll... just what I always wanted! - Clara\n>>\n\n"},{"id":"435094","messageId":"CAFLLRpJGORS4S-FkqY7+c9rkV2RL=wSrqmmKv4Ef-Y=JjOa9Kw@mail.gmail.com","threadId":"56447","inReplyTo":"CAPJmHpV2c_EubO4-GpV5Yi3RLpgvcaake=fz0rfgD_SZZcawOg@mail.gmail.com","subject":"Re: [PATCH 0/4] Compile-time extensions for list-object-filter","fromName":"Robert Coup","fromEmail":"robert.coup@koordinates.com","sentAt":"2021-09-08T14:23:08Z","receivedAt":"2021-09-08T14:23:28Z","isPatch":true,"sender":{"key":"robert.coup@koordinates.com","avatar":"https://gravatar.com/avatar/d1a87d63ffb562b791992d8a119ebbdd742e703109d23333ca3fca51306ee95c?d=mp&s=160"},"body":"Hi all,\n\nSorry, life got in the way at an unfortunate moment. And it should\nvery much be tagged \"RFC\" — thanks Ævar and Bagas for reading. Here's\nthe additional background you could have used earlier on — I've\nbundled it together, but I'll happily follow up specific questions\nindividually. I've CCed in a couple of other people who might find it\ninteresting too.\n\nSo Andrew & my motivation here is to provide some specialised\nfiltering at clone/fetch time. In Kart[1] datasets are organised\n(simplistically) by primary key, but for spatial data we want to\nprovide an orthogonal spatial extent filter which isn't part of the\ntree path, so we can't reuse the work done in the sparse filters. For\na fetch obviously the server-side will require support for any\nindexing and ultimately deciding whether a particular blob should be\npart of the tree or not.\n\nIn the original filter implementation [2], various \"profiles\" were\nalluded to as a case where the server operator might know a lot more\nabout how the developer would want to use the repository than the\nclient does, and a named profile for the server to interpret would be\na reasonably clean approach. Referred again in [3]. Sparse filters,\nsubject to the performance issues hopefully being improved by the\ncone-mode changes, cater to a lot of them. The existing built-in\nfilters are fairly simple and there's a relatively simple interface\nfor them to implement, extending them seems like a reasonable approach\nto me — potentially allowing people doing interesting things with\npartial clones to take it and run in a general way without too much\neffort.\n\nSo the key element to clarify/understand for this proposal is that the\nmain change to Git is the ability to use\n`--filter=extension:<name>[=<param>]` which passes through to\ngit-upload-pack on the server side, to rev-list, which looks up /\nvalidates the filter name/parameter and applies it. So if you want to\noffer a custom filter, you build & set it up on the server and any Git\nclient (if this is merged) can make use of it without any additional\ncode.\n\nWrt IPC, my very first proof of concept used an external process that\nrev-list launched, passed a series of oids/types via stdin, receiving\nyes/no responses via stdout. Even after quite a lot of OS-specific\nefforts to optimise the data flow across the pipes it was slow for\nnon-trivial sized repositories (where it matters) — essentially\nboiling down to too much context switching between processes.\nReorganising the existing filtering approach to do batching with\ndeferred responses and parallelising the filtering into threads seemed\nlike an awful lot of effort for potentially little gain, in a niche\nuse case.\n\nMoving it in-process made it perform well: CPU use moves into the\n\"deciding whether this object is in or out\" phase rather than burning\nit in IPC & context-switching. I did build up a basic runtime-loadable\nplugin approach, but there was a reasonable amount of the internal git\nAPI that the filters need/touched (even things like hash sizes add a\npile of complexity to it) unless it was reduced back to passing\noids+types. My approach for plugins was basically \"could I potentially\nimplement the existing filters?\" Without more of the git API I don't\nthink this would be feasible. Plus Git would have to agree on and\nsupport a public ABI going forward, which for a potentially niche use\ncase didn't seem reasonable to propose.\n\nHence compile time: simpler; no ABI issues; the internal API doesn't\nchange that much wrt things that filters are likely to do — if someone\ncreates a plugin then it's on them to keep it building across git\nupgrades on their server; platform support is simpler; and if others\nfind exciting uses for it then a runtime-loadable plugin API is always\npossible in future. And only the server ever needs any custom\nbinaries.\n\nLicensing — yes, any filters would need to be GPL-licensed since\nthey're compiled with Git. Only the server operator needs to concern\nthemselves with complying with this (& associated licensing for any\nexternal libraries/etc a plugin might need) since that's where the\nplugin code is linked & runs. With the usual issue around internal use\nwithin an organisation not qualifying as \"distribution\" under the GPL.\nFWIW, for Kart we'll be GPL-licensing the server-side spatial filter\nplugin code for anyone who's interested.\n\nHope this clarifies a bit.\n\nRob :)\n\n[1] https://kartproject.org — building on Git to version geospatial\ndatasets. Not sure if the videos ever got released (thanks Covid), but\nI did a talk at Git Merge 2020 on it when we released the first alpha.\n[2] https://public-inbox.org/git/1488999039-37631-1-git-send-email-git@jeffhostetler.com/\n[3] https://public-inbox.org/git/79b06312-75ca-5a50-c337-dc6715305edb@jeffhostetler.com/\n"}]}