{"thread":{"id":"59921","subject":"[RFC PATCH 0/8] Introduce Git Standard Library","startedAt":"2023-06-27T19:53:10Z","lastAt":"2024-03-07T21:13:12Z","messageCount":111,"participants":["Calvin Wan","Junio C Hamano","Glen Choo","Victoria Dye","Phillip Wood","Linus Arver","Jeff Hostetler","Jonathan Tan","phillip.wood123@gmail.com","Kyle Lippincott","Jeff King"],"isPatch":true,"patchVersion":1,"patchTotal":8},"messages":[{"id":"478919","messageId":"20230627195251.1973421-1-calvinwan@google.com","threadId":"59921","inReplyTo":null,"subject":"[RFC PATCH 0/8] Introduce Git Standard Library","fromName":"Calvin Wan","fromEmail":"calvinwan@google.com","sentAt":"2023-06-27T19:52:43Z","receivedAt":"2023-06-27T19:53:10Z","isPatch":true,"sender":{"key":"calvinwan@google.com","avatar":"https://avatars.githubusercontent.com/u/92547554?v=4"},"body":"Introduction / Pre-reading\n================\n\nThe Git Standard Library intends to serve as the foundational library\nand root dependency that other libraries in Git will be built off of.\nThat is to say, suppose we have libraries X and Y; a user that wants to\nuse X and Y would need to include X, Y, and this Git Standard Library.\nThis cover letter will explain the rationale behind having a root\ndependency that encompasses many files in the form of a standard library\nrather than many root dependencies/libraries of those files. This does\nnot mean that the Git Standard Library will be the only possible root\ndependency in the future, but rather the most significant and widely\nused one. I will also explain why each file was chosen to be a part of\nGit Standard Library v1. I will not explain entirely why we would like\nto libify parts of Git -- see here[1] for that context.\n\nBefore looking at this series, it probably makes sense to look at the\nother series that this is built on top of since that is the state I will\nbe referring to in this cover letter:\n\n - Elijah's final cache.h cleanup series[2]\n - my strbuf cleanup series[3]\n - my git-compat-util cleanup series[4]\n\nMost importantly, in the git-compat-util series, the declarations for\nfunctions implemented in wrapper.c and usage.c have been moved to their\nrespective header files, wrapper.h and usage.h, from git-compat-util.h.\nAlso config.[ch] had its general parsing code moved to parse.[ch].\n\nDependency graph in libified Git\n================\n\nIf you look in the Git Makefile, all of the objects defined in the Git\nlibrary are compiled and archived into a singular file, libgit.a, which\nis linked against by common-main.o with other external dependencies and\nturned into the Git executable. In other words, the Git executable has\ndependencies on libgit.a and a couple of external libraries. While our\nefforts to libify Git will not affect this current build flow, it will\nprovide an alternate method for building Git.\n\nWith our current method of building Git, we can imagine the dependency\ngraph as such:\n\n Git\n /\\\n / \\\n / \\\n libgit.a ext deps\n\nIn libifying parts of Git, we want to shrink the dependency graph to\nonly the minimal set of dependencies, so libraries should not use\nlibgit.a. Instead, it would look like:\n\n Git\n /\\\n / \\\n / \\\n libgit.a ext deps\n /\\\n / \\\n / \\\nobject-store.a (other lib)\n | /\n | /\n | /\n config.a / \n | /\n | /\n | /\ngit-std-lib.a\n\nInstead of containing all of the objects in Git, libgit.a would contain\nobjects that are not built by libraries it links against. Consequently,\nif someone wanted their own custom build of Git with their own custom\nimplementation of the object store, they would only have to swap out\nobject-store.a rather than do a hard fork of Git.\n\nRationale behind Git Standard Library\n================\n\nThe rationale behind Git Standard Library essentially is the result of\ntwo observations within the Git codebase: every file includes\ngit-compat-util.h which defines functions in a couple of different\nfiles, and wrapper.c + usage.c have difficult-to-separate circular\ndependencies with each other and other files.\n\nUbiquity of git-compat-util.h and circular dependencies\n========\n\nEvery file in the Git codebase includes git-compat-util.h. It serves as\n\"a compatibility aid that isolates the knowledge of platform specific\ninclusion order and what feature macros to define before including which\nsystem header\" (Junio[5]). Since every file includes git-compat-util.h, and\ngit-compat-util.h includes wrapper.h and usage.h, it would make sense\nfor wrapper.c and usage.c to be a part of the root library. They have\ndifficult to separate circular dependencies with each other so they\ncan't be independent libraries. Wrapper.c has dependencies on parse.c,\nabspath.c, strbuf.c, which in turn also have dependencies on usage.c and\nwrapper.c -- more circular dependencies. \n\nTradeoff between swappability and refactoring\n========\n\nFrom the above dependency graph, we can see that git-std-lib.a could be\nmany smaller libraries rather than a singular library. So why choose a\nsingular library when multiple libraries can be individually easier to\nswap and are more modular? A singular library requires less work to\nseparate out circular dependencies within itself so it becomes a\ntradeoff question between work and reward. While there may be a point in\nthe future where a file like usage.c would want its own library so that\nsomeone can have custom die() or error(), the work required to refactor\nout the circular dependencies in some files would be enormous due to\ntheir ubiquity so therefore I believe it is not worth the tradeoff\ncurrently. Additionally, we can in the future choose to do this refactor\nand change the API for the library if there becomes enough of a reason\nto do so (remember we are avoiding promising stability of the interfaces\nof those libraries).\n\nReuse of compatibility functions in git-compat-util.h\n========\n\nMost functions defined in git-compat-util.h are implemented in compat/\nand have dependencies limited to strbuf.h and wrapper.h so they can be\neasily included in git-std-lib.a, which as a root dependency means that\nhigher level libraries do not have to worry about compatibility files in\ncompat/. The rest of the functions defined in git-compat-util.h are\nimplemented in top level files and, in this patch set, are hidden behind\nan #ifdef if their implementation is not in git-std-lib.a.\n\nRationale summary\n========\n\nThe Git Standard Library allows us to get the libification ball rolling\nwith other libraries in Git (such as Glen's removal of global state from\nconfig iteration[6] prepares a config library). By not spending many\nmore months attempting to refactor difficult circular dependencies and\ninstead spending that time getting to a state where we can test out\nswapping a library out such as config or object store, we can prove the\nviability of Git libification on a much faster time scale. Additionally\nthe code cleanups that have happened so far have been minor and\nbeneficial for the codebase. It is probable that making large movements\nwould negatively affect code clarity.\n\nGit Standard Library boundary\n================\n\nWhile I have described above some useful heuristics for identifying\npotential candidates for git-std-lib.a, a standard library should not\nhave a shaky definition for what belongs in it.\n\n - Low-level files (aka operates only on other primitive types) that are\n used everywhere within the codebase (wrapper.c, usage.c, strbuf.c)\n - Dependencies that are low-level and widely used\n (abspath.c, date.c, hex-ll.c, parse.c, utf8.c)\n - low-level git/* files with functions defined in git-compat-util.h\n (ctype.c)\n - compat/*\n\nThere are other files that might fit this definition, but that does not\nmean it should belong in git-std-lib.a. Those files should start as\ntheir own separate library since any file added to git-std-lib.a loses\nits flexibility of being easily swappable.\n\nFiles inside of Git Standard Library\n================\n\nThe initial set of files in git-std-lib.a are:\nabspath.c\nctype.c\ndate.c\nhex-ll.c\nparse.c\nstrbuf.c\nusage.c\nutf8.c\nwrapper.c\nrelevant compat/ files\n\nPitfalls\n================\n\nIn patch 7, I use #ifdef GIT_STD_LIB to both stub out code and hide\ncertain function headers. As other parts of Git are libified, if we\nhave to use more ifdefs for each different library, then the codebase\nwill become uglier and harder to understand. \n\nThere are a small amount of files under compat/* that have dependencies\nnot inside of git-std-lib.a. While those functions are not called on\nLinux, other OSes might call those problematic functions. I don't see\nthis as a major problem, just moreso an observation that libification in\ngeneral may also require some minor compatibility work in the future.\n\nTesting\n================\n\nPatch 8 introduces a temporary test file which will be replaced with\nunit tests once a unit testing framework is decided upon[7]. It simply\nproves that all of the functions in git-std-lib.a do not have any\nmissing dependencies and can stand up by itself.\n\nI have not yet tested building Git with git-std-lib.a yet (basically\nremoving the objects in git-std-lib.a from LIB_OBJS and linking against\ngit-std-lib.a instead), but I intend on testing this in a future version\nof this patch. As an RFC, I want to showcase git-std-lib.a as an\nexperimental dependency that other executables can include in order to\nuse Git binaries. Internally we have tested building and calling\nfunctions in git-std-lib.a from other programs.\n\nUnit tests should catch any breakages caused by changes to files in\ngit-std-lib.a (i.e. introduction of a out of scope dependency) and new\nfunctions introduced to git-std-lib.a will require unit tests written\nfor them.\n\nSeries structure\n================\n\nWhile my strbuf and git-compat-util series can stand alone, they also\nfunction as preparatory patches for this series. There are more cleanup\npatches in this series, but since most of them have marginal benefits\nprobably not worth the churn on its own, I decided not to split them\ninto a separate series like with strbuf and git-compat-util. As an RFC,\nI am looking for comments on whether the rationale behind git-std-lib\nmakes sense as well as whether there are better ways to build and enable\ngit-std-lib in patch 7, specifically regarding Makefile rules and the\nusage of ifdef's to stub out certain functions and headers. \n\nThe patch series is structured as follows:\n\nPatches 1-6 are cleanup patches to remove the last few extraneous\ndependencies from git-std-lib.a. Here's a short summary of the\ndependencies that are specifically removed from git-std-lib.a since some\nof the commit messages and diffs showcase dependency cleanups for other\nfiles not directly related to git-std-lib.a:\n - Patch 1 removes trace2.h and repository.h dependencies from wrapper.c\n - Patch 2 removes the repository.h dependency from strbuf.c inherited from\n hex.c by separating it into hex-ll.c and hex.c\n - Patch 3 removes the object.h dependency from wrapper.c\n - Patch 4 is a bug fix that sets up the next patch. This importantly\n removes the git_config_bool() call from git_env_bool() so that env\n parsing can go in a separate file\n - Patch 5 removes the config.h dependency from wrapper.c and swaps it\n with a dependency to parse.h, which doesn't have extraneous\n dependencies to files outside of git-std-lib.a\n - Patch 6 removes the pager.h dependency from date.c\n\nPatch 7 introduces Git standard library.\n\nPatch 8 introduces a temporary test file for Git standard library. The\ntest file directly or indirectly calls all functions in git-std-lib.a to\nshowcase that the functions don't reference missing objects and that\ngit-std-lib.a can stand on its own.\n\n[1] https://lore.kernel.org/git/CAJoAoZ=Cig_kLocxKGax31sU7Xe4==BGzC__Bg2_pr7krNq6MA@mail.gmail.com/\n[2] https://lore.kernel.org/git/pull.1525.v3.git.1684218848.gitgitgadget@gmail.com/\n[3] https://lore.kernel.org/git/20230606194720.2053551-1-calvinwan@google.com/\n[4] https://lore.kernel.org/git/20230606170711.912972-1-calvinwan@google.com/\n[5] https://lore.kernel.org/git/xmqqwn17sydw.fsf@gitster.g/\n[6] https://lore.kernel.org/git/pull.1497.v3.git.git.1687290231.gitgitgadget@gmail.com/\n[7] https://lore.kernel.org/git/8afdb215d7e10ca16a2ce8226b4127b3d8a2d971.1686352386.git.steadmon@google.com/\n\nCalvin Wan (8):\n trace2: log fsync stats in trace2 rather than wrapper\n hex-ll: split out functionality from hex\n object: move function to object.c\n config: correct bad boolean env value error message\n parse: create new library for parsing strings and env values\n pager: remove pager_in_use()\n git-std-lib: introduce git standard library\n git-std-lib: add test file to call git-std-lib.a functions\n\n Documentation/technical/git-std-lib.txt | 182 ++++++++++++++++++\n Makefile | 30 ++-\n attr.c | 2 +-\n builtin/log.c | 2 +-\n color.c | 4 +-\n column.c | 2 +-\n config.c | 173 +----------------\n config.h | 14 +-\n date.c | 4 +-\n git-compat-util.h | 7 +-\n git.c | 2 +-\n hex-ll.c | 49 +++++\n hex-ll.h | 27 +++\n hex.c | 47 -----\n hex.h | 24 +--\n mailinfo.c | 2 +-\n object.c | 5 +\n object.h | 6 +\n pack-objects.c | 2 +-\n pack-revindex.c | 2 +-\n pager.c | 5 -\n pager.h | 1 -\n parse-options.c | 3 +-\n parse.c | 182 ++++++++++++++++++\n parse.h | 20 ++\n pathspec.c | 2 +-\n preload-index.c | 2 +-\n progress.c | 2 +-\n prompt.c | 2 +-\n rebase.c | 2 +-\n strbuf.c | 2 +-\n symlinks.c | 2 +\n t/Makefile | 4 +\n t/helper/test-env-helper.c | 2 +-\n t/stdlib-test.c | 239 ++++++++++++++++++++++++\n trace2.c | 13 ++\n trace2.h | 5 +\n unpack-trees.c | 2 +-\n url.c | 2 +-\n urlmatch.c | 2 +-\n usage.c | 8 +\n wrapper.c | 25 +--\n wrapper.h | 9 +-\n write-or-die.c | 2 +-\n 44 files changed, 813 insertions(+), 311 deletions(-)\n create mode 100644 Documentation/technical/git-std-lib.txt\n create mode 100644 hex-ll.c\n create mode 100644 hex-ll.h\n create mode 100644 parse.c\n create mode 100644 parse.h\n create mode 100644 t/stdlib-test.c\n\n-- \n2.41.0.162.gfafddb0af9-goog\n\n"},{"id":"478920","messageId":"20230627195251.1973421-2-calvinwan@google.com","threadId":"59921","inReplyTo":"20230627195251.1973421-1-calvinwan@google.com","subject":"[RFC PATCH 1/8] trace2: log fsync stats in trace2 rather than wrapper","fromName":"Calvin Wan","fromEmail":"calvinwan@google.com","sentAt":"2023-06-27T19:52:44Z","receivedAt":"2023-06-27T19:53:13Z","isPatch":true,"sender":{"key":"calvinwan@google.com","avatar":"https://avatars.githubusercontent.com/u/92547554?v=4"},"body":"As a library boundary, wrapper.c should not directly log trace2\nstatistics, but instead provide those statistics upon\nrequest. Therefore, move the trace2 logging code to trace2.[ch.]. This\nalso allows wrapper.c to not be dependent on trace2.h and repository.h.\n\nSigned-off-by: Calvin Wan \n---\n trace2.c | 13 +++++++++++++\n trace2.h | 5 +++++\n wrapper.c | 17 ++++++-----------\n wrapper.h | 4 ++--\n 4 files changed, 26 insertions(+), 13 deletions(-)\n\ndiff --git a/trace2.c b/trace2.c\nindex 0efc4e7b95..f367a1ce31 100644\n--- a/trace2.c\n+++ b/trace2.c\n@@ -915,3 +915,16 @@ const char *trace2_session_id(void)\n {\n \treturn tr2_sid_get();\n }\n+\n+static void log_trace_fsync_if(const char *key)\n+{\n+\tintmax_t value = get_trace_git_fsync_stats(key);\n+\tif (value)\n+\t\ttrace2_data_intmax(\"fsync\", the_repository, key, value);\n+}\n+\n+void trace_git_fsync_stats(void)\n+{\n+\tlog_trace_fsync_if(\"fsync/writeout-only\");\n+\tlog_trace_fsync_if(\"fsync/hardware-flush\");\n+}\ndiff --git a/trace2.h b/trace2.h\nindex 4ced30c0db..689e9a4027 100644\n--- a/trace2.h\n+++ b/trace2.h\n@@ -581,4 +581,9 @@ void trace2_collect_process_info(enum trace2_process_info_reason reason);\n \n const char *trace2_session_id(void);\n \n+/*\n+ * Writes out trace statistics for fsync\n+ */\n+void trace_git_fsync_stats(void);\n+\n #endif /* TRACE2_H */\ndiff --git a/wrapper.c b/wrapper.c\nindex 22be9812a7..bd7f0a9752 100644\n--- a/wrapper.c\n+++ b/wrapper.c\n@@ -6,9 +6,7 @@\n #include \"config.h\"\n #include \"gettext.h\"\n #include \"object.h\"\n-#include \"repository.h\"\n #include \"strbuf.h\"\n-#include \"trace2.h\"\n \n static intmax_t count_fsync_writeout_only;\n static intmax_t count_fsync_hardware_flush;\n@@ -600,16 +598,13 @@ int git_fsync(int fd, enum fsync_action action)\n \t}\n }\n \n-static void log_trace_fsync_if(const char *key, intmax_t value)\n+intmax_t get_trace_git_fsync_stats(const char *key)\n {\n-\tif (value)\n-\t\ttrace2_data_intmax(\"fsync\", the_repository, key, value);\n-}\n-\n-void trace_git_fsync_stats(void)\n-{\n-\tlog_trace_fsync_if(\"fsync/writeout-only\", count_fsync_writeout_only);\n-\tlog_trace_fsync_if(\"fsync/hardware-flush\", count_fsync_hardware_flush);\n+\tif (!strcmp(key, \"fsync/writeout-only\"))\n+\t\treturn count_fsync_writeout_only;\n+\tif (!strcmp(key, \"fsync/hardware-flush\"))\n+\t\treturn count_fsync_hardware_flush;\n+\treturn 0;\n }\n \n static int warn_if_unremovable(const char *op, const char *file, int rc)\ndiff --git a/wrapper.h b/wrapper.h\nindex c85b1328d1..db1bc109ed 100644\n--- a/wrapper.h\n+++ b/wrapper.h\n@@ -88,9 +88,9 @@ enum fsync_action {\n int git_fsync(int fd, enum fsync_action action);\n \n /*\n- * Writes out trace statistics for fsync using the trace2 API.\n+ * Returns trace statistics for fsync using the trace2 API.\n */\n-void trace_git_fsync_stats(void);\n+intmax_t get_trace_git_fsync_stats(const char *key);\n \n /*\n * Preserves errno, prints a message, but gives no warning for ENOENT.\n-- \n2.41.0.162.gfafddb0af9-goog\n\n"},{"id":"478921","messageId":"20230627195251.1973421-3-calvinwan@google.com","threadId":"59921","inReplyTo":"20230627195251.1973421-1-calvinwan@google.com","subject":"[RFC PATCH 2/8] hex-ll: split out functionality from hex","fromName":"Calvin Wan","fromEmail":"calvinwan@google.com","sentAt":"2023-06-27T19:52:45Z","receivedAt":"2023-06-27T19:53:16Z","isPatch":true,"sender":{"key":"calvinwan@google.com","avatar":"https://avatars.githubusercontent.com/u/92547554?v=4"},"body":"Separate out hex functionality that doesn't require a hash algo into\nhex-ll.[ch]. Since the hash algo is currently a global that sits in\nrepository, this separation removes that dependency for files that only\nneed basic hex manipulation functions.\n\nSigned-off-by: Calvin Wan \n---\n Makefile | 1 +\n color.c | 2 +-\n hex-ll.c | 49 +++++++++++++++++++++++++++++++++++++++++++++++++\n hex-ll.h | 27 +++++++++++++++++++++++++++\n hex.c | 47 -----------------------------------------------\n hex.h | 24 +-----------------------\n mailinfo.c | 2 +-\n strbuf.c | 2 +-\n url.c | 2 +-\n urlmatch.c | 2 +-\n 10 files changed, 83 insertions(+), 75 deletions(-)\n create mode 100644 hex-ll.c\n create mode 100644 hex-ll.h\n\ndiff --git a/Makefile b/Makefile\nindex 045e2187c4..83b385b0be 100644\n--- a/Makefile\n+++ b/Makefile\n@@ -1040,6 +1040,7 @@ LIB_OBJS += hash-lookup.o\n LIB_OBJS += hashmap.o\n LIB_OBJS += help.o\n LIB_OBJS += hex.o\n+LIB_OBJS += hex-ll.o\n LIB_OBJS += hook.o\n LIB_OBJS += ident.o\n LIB_OBJS += json-writer.o\ndiff --git a/color.c b/color.c\nindex 83abb11eda..f3c0a4659b 100644\n--- a/color.c\n+++ b/color.c\n@@ -3,7 +3,7 @@\n #include \"color.h\"\n #include \"editor.h\"\n #include \"gettext.h\"\n-#include \"hex.h\"\n+#include \"hex-ll.h\"\n #include \"pager.h\"\n #include \"strbuf.h\"\n \ndiff --git a/hex-ll.c b/hex-ll.c\nnew file mode 100644\nindex 0000000000..4d7ece1de5\n--- /dev/null\n+++ b/hex-ll.c\n@@ -0,0 +1,49 @@\n+#include \"git-compat-util.h\"\n+#include \"hex-ll.h\"\n+\n+const signed char hexval_table[256] = {\n+\t -1, -1, -1, -1, -1, -1, -1, -1,\t\t/* 00-07 */\n+\t -1, -1, -1, -1, -1, -1, -1, -1,\t\t/* 08-0f */\n+\t -1, -1, -1, -1, -1, -1, -1, -1,\t\t/* 10-17 */\n+\t -1, -1, -1, -1, -1, -1, -1, -1,\t\t/* 18-1f */\n+\t -1, -1, -1, -1, -1, -1, -1, -1,\t\t/* 20-27 */\n+\t -1, -1, -1, -1, -1, -1, -1, -1,\t\t/* 28-2f */\n+\t 0, 1, 2, 3, 4, 5, 6, 7,\t\t/* 30-37 */\n+\t 8, 9, -1, -1, -1, -1, -1, -1,\t\t/* 38-3f */\n+\t -1, 10, 11, 12, 13, 14, 15, -1,\t\t/* 40-47 */\n+\t -1, -1, -1, -1, -1, -1, -1, -1,\t\t/* 48-4f */\n+\t -1, -1, -1, -1, -1, -1, -1, -1,\t\t/* 50-57 */\n+\t -1, -1, -1, -1, -1, -1, -1, -1,\t\t/* 58-5f */\n+\t -1, 10, 11, 12, 13, 14, 15, -1,\t\t/* 60-67 */\n+\t -1, -1, -1, -1, -1, -1, -1, -1,\t\t/* 68-67 */\n+\t -1, -1, -1, -1, -1, -1, -1, -1,\t\t/* 70-77 */\n+\t -1, -1, -1, -1, -1, -1, -1, -1,\t\t/* 78-7f */\n+\t -1, -1, -1, -1, -1, -1, -1, -1,\t\t/* 80-87 */\n+\t -1, -1, -1, -1, -1, -1, -1, -1,\t\t/* 88-8f */\n+\t -1, -1, -1, -1, -1, -1, -1, -1,\t\t/* 90-97 */\n+\t -1, -1, -1, -1, -1, -1, -1, -1,\t\t/* 98-9f */\n+\t -1, -1, -1, -1, -1, -1, -1, -1,\t\t/* a0-a7 */\n+\t -1, -1, -1, -1, -1, -1, -1, -1,\t\t/* a8-af */\n+\t -1, -1, -1, -1, -1, -1, -1, -1,\t\t/* b0-b7 */\n+\t -1, -1, -1, -1, -1, -1, -1, -1,\t\t/* b8-bf */\n+\t -1, -1, -1, -1, -1, -1, -1, -1,\t\t/* c0-c7 */\n+\t -1, -1, -1, -1, -1, -1, -1, -1,\t\t/* c8-cf */\n+\t -1, -1, -1, -1, -1, -1, -1, -1,\t\t/* d0-d7 */\n+\t -1, -1, -1, -1, -1, -1, -1, -1,\t\t/* d8-df */\n+\t -1, -1, -1, -1, -1, -1, -1, -1,\t\t/* e0-e7 */\n+\t -1, -1, -1, -1, -1, -1, -1, -1,\t\t/* e8-ef */\n+\t -1, -1, -1, -1, -1, -1, -1, -1,\t\t/* f0-f7 */\n+\t -1, -1, -1, -1, -1, -1, -1, -1,\t\t/* f8-ff */\n+};\n+\n+int hex_to_bytes(unsigned char *binary, const char *hex, size_t len)\n+{\n+\tfor (; len; len--, hex += 2) {\n+\t\tunsigned int val = (hexval(hex[0]) << 4) | hexval(hex[1]);\n+\n+\t\tif (val & ~0xff)\n+\t\t\treturn -1;\n+\t\t*binary++ = val;\n+\t}\n+\treturn 0;\n+}\ndiff --git a/hex-ll.h b/hex-ll.h\nnew file mode 100644\nindex 0000000000..a381fa8556\n--- /dev/null\n+++ b/hex-ll.h\n@@ -0,0 +1,27 @@\n+#ifndef HEX_LL_H\n+#define HEX_LL_H\n+\n+extern const signed char hexval_table[256];\n+static inline unsigned int hexval(unsigned char c)\n+{\n+\treturn hexval_table[c];\n+}\n+\n+/*\n+ * Convert two consecutive hexadecimal digits into a char. Return a\n+ * negative value on error. Don't run over the end of short strings.\n+ */\n+static inline int hex2chr(const char *s)\n+{\n+\tunsigned int val = hexval(s[0]);\n+\treturn (val & ~0xf) ? val : (val << 4) | hexval(s[1]);\n+}\n+\n+/*\n+ * Read `len` pairs of hexadecimal digits from `hex` and write the\n+ * values to `binary` as `len` bytes. Return 0 on success, or -1 if\n+ * the input does not consist of hex digits).\n+ */\n+int hex_to_bytes(unsigned char *binary, const char *hex, size_t len);\n+\n+#endif\ndiff --git a/hex.c b/hex.c\nindex 7bb440e794..03e55841ed 100644\n--- a/hex.c\n+++ b/hex.c\n@@ -2,53 +2,6 @@\n #include \"hash.h\"\n #include \"hex.h\"\n \n-const signed char hexval_table[256] = {\n-\t -1, -1, -1, -1, -1, -1, -1, -1,\t\t/* 00-07 */\n-\t -1, -1, -1, -1, -1, -1, -1, -1,\t\t/* 08-0f */\n-\t -1, -1, -1, -1, -1, -1, -1, -1,\t\t/* 10-17 */\n-\t -1, -1, -1, -1, -1, -1, -1, -1,\t\t/* 18-1f */\n-\t -1, -1, -1, -1, -1, -1, -1, -1,\t\t/* 20-27 */\n-\t -1, -1, -1, -1, -1, -1, -1, -1,\t\t/* 28-2f */\n-\t 0, 1, 2, 3, 4, 5, 6, 7,\t\t/* 30-37 */\n-\t 8, 9, -1, -1, -1, -1, -1, -1,\t\t/* 38-3f */\n-\t -1, 10, 11, 12, 13, 14, 15, -1,\t\t/* 40-47 */\n-\t -1, -1, -1, -1, -1, -1, -1, -1,\t\t/* 48-4f */\n-\t -1, -1, -1, -1, -1, -1, -1, -1,\t\t/* 50-57 */\n-\t -1, -1, -1, -1, -1, -1, -1, -1,\t\t/* 58-5f */\n-\t -1, 10, 11, 12, 13, 14, 15, -1,\t\t/* 60-67 */\n-\t -1, -1, -1, -1, -1, -1, -1, -1,\t\t/* 68-67 */\n-\t -1, -1, -1, -1, -1, -1, -1, -1,\t\t/* 70-77 */\n-\t -1, -1, -1, -1, -1, -1, -1, -1,\t\t/* 78-7f */\n-\t -1, -1, -1, -1, -1, -1, -1, -1,\t\t/* 80-87 */\n-\t -1, -1, -1, -1, -1, -1, -1, -1,\t\t/* 88-8f */\n-\t -1, -1, -1, -1, -1, -1, -1, -1,\t\t/* 90-97 */\n-\t -1, -1, -1, -1, -1, -1, -1, -1,\t\t/* 98-9f */\n-\t -1, -1, -1, -1, -1, -1, -1, -1,\t\t/* a0-a7 */\n-\t -1, -1, -1, -1, -1, -1, -1, -1,\t\t/* a8-af */\n-\t -1, -1, -1, -1, -1, -1, -1, -1,\t\t/* b0-b7 */\n-\t -1, -1, -1, -1, -1, -1, -1, -1,\t\t/* b8-bf */\n-\t -1, -1, -1, -1, -1, -1, -1, -1,\t\t/* c0-c7 */\n-\t -1, -1, -1, -1, -1, -1, -1, -1,\t\t/* c8-cf */\n-\t -1, -1, -1, -1, -1, -1, -1, -1,\t\t/* d0-d7 */\n-\t -1, -1, -1, -1, -1, -1, -1, -1,\t\t/* d8-df */\n-\t -1, -1, -1, -1, -1, -1, -1, -1,\t\t/* e0-e7 */\n-\t -1, -1, -1, -1, -1, -1, -1, -1,\t\t/* e8-ef */\n-\t -1, -1, -1, -1, -1, -1, -1, -1,\t\t/* f0-f7 */\n-\t -1, -1, -1, -1, -1, -1, -1, -1,\t\t/* f8-ff */\n-};\n-\n-int hex_to_bytes(unsigned char *binary, const char *hex, size_t len)\n-{\n-\tfor (; len; len--, hex += 2) {\n-\t\tunsigned int val = (hexval(hex[0]) << 4) | hexval(hex[1]);\n-\n-\t\tif (val & ~0xff)\n-\t\t\treturn -1;\n-\t\t*binary++ = val;\n-\t}\n-\treturn 0;\n-}\n-\n static int get_hash_hex_algop(const char *hex, unsigned char *hash,\n \t\t\t const struct git_hash_algo *algop)\n {\ndiff --git a/hex.h b/hex.h\nindex 7df4b3c460..c07c8b34c2 100644\n--- a/hex.h\n+++ b/hex.h\n@@ -2,22 +2,7 @@\n #define HEX_H\n \n #include \"hash-ll.h\"\n-\n-extern const signed char hexval_table[256];\n-static inline unsigned int hexval(unsigned char c)\n-{\n-\treturn hexval_table[c];\n-}\n-\n-/*\n- * Convert two consecutive hexadecimal digits into a char. Return a\n- * negative value on error. Don't run over the end of short strings.\n- */\n-static inline int hex2chr(const char *s)\n-{\n-\tunsigned int val = hexval(s[0]);\n-\treturn (val & ~0xf) ? val : (val << 4) | hexval(s[1]);\n-}\n+#include \"hex-ll.h\"\n \n /*\n * Try to read a SHA1 in hexadecimal format from the 40 characters\n@@ -32,13 +17,6 @@ int get_oid_hex(const char *hex, struct object_id *sha1);\n /* Like get_oid_hex, but for an arbitrary hash algorithm. */\n int get_oid_hex_algop(const char *hex, struct object_id *oid, const struct git_hash_algo *algop);\n \n-/*\n- * Read `len` pairs of hexadecimal digits from `hex` and write the\n- * values to `binary` as `len` bytes. Return 0 on success, or -1 if\n- * the input does not consist of hex digits).\n- */\n-int hex_to_bytes(unsigned char *binary, const char *hex, size_t len);\n-\n /*\n * Convert a binary hash in \"unsigned char []\" or an object name in\n * \"struct object_id *\" to its hex equivalent. The `_r` variant is reentrant,\ndiff --git a/mailinfo.c b/mailinfo.c\nindex 2aeb20e5e6..eb34c30be7 100644\n--- a/mailinfo.c\n+++ b/mailinfo.c\n@@ -1,7 +1,7 @@\n #include \"git-compat-util.h\"\n #include \"config.h\"\n #include \"gettext.h\"\n-#include \"hex.h\"\n+#include \"hex-ll.h\"\n #include \"utf8.h\"\n #include \"strbuf.h\"\n #include \"mailinfo.h\"\ndiff --git a/strbuf.c b/strbuf.c\nindex 8dac52b919..a2a05fe168 100644\n--- a/strbuf.c\n+++ b/strbuf.c\n@@ -1,6 +1,6 @@\n #include \"git-compat-util.h\"\n #include \"gettext.h\"\n-#include \"hex.h\"\n+#include \"hex-ll.h\"\n #include \"strbuf.h\"\n #include \"string-list.h\"\n #include \"utf8.h\"\ndiff --git a/url.c b/url.c\nindex 2e1a9f6fee..282b12495a 100644\n--- a/url.c\n+++ b/url.c\n@@ -1,5 +1,5 @@\n #include \"git-compat-util.h\"\n-#include \"hex.h\"\n+#include \"hex-ll.h\"\n #include \"strbuf.h\"\n #include \"url.h\"\n \ndiff --git a/urlmatch.c b/urlmatch.c\nindex eba0bdd77f..f1aa87d1dd 100644\n--- a/urlmatch.c\n+++ b/urlmatch.c\n@@ -1,6 +1,6 @@\n #include \"git-compat-util.h\"\n #include \"gettext.h\"\n-#include \"hex.h\"\n+#include \"hex-ll.h\"\n #include \"strbuf.h\"\n #include \"urlmatch.h\"\n \n-- \n2.41.0.162.gfafddb0af9-goog\n\n"},{"id":"478922","messageId":"20230627195251.1973421-4-calvinwan@google.com","threadId":"59921","inReplyTo":"20230627195251.1973421-1-calvinwan@google.com","subject":"[RFC PATCH 3/8] object: move function to object.c","fromName":"Calvin Wan","fromEmail":"calvinwan@google.com","sentAt":"2023-06-27T19:52:46Z","receivedAt":"2023-06-27T19:53:22Z","isPatch":true,"sender":{"key":"calvinwan@google.com","avatar":"https://avatars.githubusercontent.com/u/92547554?v=4"},"body":"While remove_or_warn() is a simple ternary operator to call two other\nwrapper functions, it creates an unnecessary dependency to object.h in\nwrapper.c. Therefore move the function to object.[ch] where the concept\nof GITLINKs is first defined.\n\nSigned-off-by: Calvin Wan \n---\n object.c | 5 +++++\n object.h | 6 ++++++\n wrapper.c | 6 ------\n wrapper.h | 5 -----\n 4 files changed, 11 insertions(+), 11 deletions(-)\n\ndiff --git a/object.c b/object.c\nindex 60f954194f..cb29fcc304 100644\n--- a/object.c\n+++ b/object.c\n@@ -617,3 +617,8 @@ void parsed_object_pool_clear(struct parsed_object_pool *o)\n \tFREE_AND_NULL(o->object_state);\n \tFREE_AND_NULL(o->shallow_stat);\n }\n+\n+int remove_or_warn(unsigned int mode, const char *file)\n+{\n+\treturn S_ISGITLINK(mode) ? rmdir_or_warn(file) : unlink_or_warn(file);\n+}\ndiff --git a/object.h b/object.h\nindex 5871615fee..e908ef6515 100644\n--- a/object.h\n+++ b/object.h\n@@ -284,4 +284,10 @@ void clear_object_flags(unsigned flags);\n */\n void repo_clear_commit_marks(struct repository *r, unsigned int flags);\n \n+/*\n+ * Calls the correct function out of {unlink,rmdir}_or_warn based on\n+ * the supplied file mode.\n+ */\n+int remove_or_warn(unsigned int mode, const char *path);\n+\n #endif /* OBJECT_H */\ndiff --git a/wrapper.c b/wrapper.c\nindex bd7f0a9752..62c04aeb17 100644\n--- a/wrapper.c\n+++ b/wrapper.c\n@@ -5,7 +5,6 @@\n #include \"abspath.h\"\n #include \"config.h\"\n #include \"gettext.h\"\n-#include \"object.h\"\n #include \"strbuf.h\"\n \n static intmax_t count_fsync_writeout_only;\n@@ -642,11 +641,6 @@ int rmdir_or_warn(const char *file)\n \treturn warn_if_unremovable(\"rmdir\", file, rmdir(file));\n }\n \n-int remove_or_warn(unsigned int mode, const char *file)\n-{\n-\treturn S_ISGITLINK(mode) ? rmdir_or_warn(file) : unlink_or_warn(file);\n-}\n-\n static int access_error_is_ok(int err, unsigned flag)\n {\n \treturn (is_missing_file_error(err) ||\ndiff --git a/wrapper.h b/wrapper.h\nindex db1bc109ed..166740ae60 100644\n--- a/wrapper.h\n+++ b/wrapper.h\n@@ -111,11 +111,6 @@ int unlink_or_msg(const char *file, struct strbuf *err);\n * not exist.\n */\n int rmdir_or_warn(const char *path);\n-/*\n- * Calls the correct function out of {unlink,rmdir}_or_warn based on\n- * the supplied file mode.\n- */\n-int remove_or_warn(unsigned int mode, const char *path);\n \n /*\n * Call access(2), but warn for any error except \"missing file\"\n-- \n2.41.0.162.gfafddb0af9-goog\n\n"},{"id":"478923","messageId":"20230627195251.1973421-5-calvinwan@google.com","threadId":"59921","inReplyTo":"20230627195251.1973421-1-calvinwan@google.com","subject":"[RFC PATCH 4/8] config: correct bad boolean env value error message","fromName":"Calvin Wan","fromEmail":"calvinwan@google.com","sentAt":"2023-06-27T19:52:47Z","receivedAt":"2023-06-27T19:53:24Z","isPatch":true,"sender":{"key":"calvinwan@google.com","avatar":"https://avatars.githubusercontent.com/u/92547554?v=4"},"body":"An incorrectly defined boolean environment value would result in the\nfollowing error message:\n\nbad boolean config value '%s' for '%s'\n\nThis is a misnomer since environment value != config value. Instead of\ncalling git_config_bool() to parse the environment value, mimic the\nfunctionality inside of git_config_bool() but with the correct error\nmessage.\n\nSigned-off-by: Calvin Wan \n---\n config.c | 9 ++++++++-\n 1 file changed, 8 insertions(+), 1 deletion(-)\n\ndiff --git a/config.c b/config.c\nindex 09851a6909..5b71ef1624 100644\n--- a/config.c\n+++ b/config.c\n@@ -2172,7 +2172,14 @@ void git_global_config(char **user_out, char **xdg_out)\n int git_env_bool(const char *k, int def)\n {\n \tconst char *v = getenv(k);\n-\treturn v ? git_config_bool(k, v) : def;\n+\tint val;\n+\tif (!v)\n+\t\treturn def;\n+\tval = git_parse_maybe_bool(v);\n+\tif (val < 0)\n+\t\tdie(_(\"bad boolean environment value '%s' for '%s'\"),\n+\t\t v, k);\n+\treturn val;\n }\n \n /*\n-- \n2.41.0.162.gfafddb0af9-goog\n\n"},{"id":"478924","messageId":"20230627195251.1973421-6-calvinwan@google.com","threadId":"59921","inReplyTo":"20230627195251.1973421-1-calvinwan@google.com","subject":"[RFC PATCH 5/8] parse: create new library for parsing strings and env values","fromName":"Calvin Wan","fromEmail":"calvinwan@google.com","sentAt":"2023-06-27T19:52:48Z","receivedAt":"2023-06-27T19:53:26Z","isPatch":true,"sender":{"key":"calvinwan@google.com","avatar":"https://avatars.githubusercontent.com/u/92547554?v=4"},"body":"While string and environment value parsing is mainly consumed by\nconfig.c, there are other files that only need parsing functionality and\nnot config functionality. By separating out string and environment value\nparsing from config, those files can instead be dependent on parse,\nwhich has a much smaller dependency chain than config.\n\nMove general string and env parsing functions from config.[ch] to\nparse.[ch].\n\nSigned-off-by: Calvin Wan \n---\n Makefile | 1 +\n attr.c | 2 +-\n config.c | 180 +-----------------------------------\n config.h | 14 +--\n pack-objects.c | 2 +-\n pack-revindex.c | 2 +-\n parse-options.c | 3 +-\n parse.c | 182 +++++++++++++++++++++++++++++++++++++\n parse.h | 20 ++++\n pathspec.c | 2 +-\n preload-index.c | 2 +-\n progress.c | 2 +-\n prompt.c | 2 +-\n rebase.c | 2 +-\n t/helper/test-env-helper.c | 2 +-\n unpack-trees.c | 2 +-\n wrapper.c | 2 +-\n write-or-die.c | 2 +-\n 18 files changed, 219 insertions(+), 205 deletions(-)\n create mode 100644 parse.c\n create mode 100644 parse.h\n\ndiff --git a/Makefile b/Makefile\nindex 83b385b0be..e9ad9f9ef1 100644\n--- a/Makefile\n+++ b/Makefile\n@@ -1091,6 +1091,7 @@ LIB_OBJS += pack-write.o\n LIB_OBJS += packfile.o\n LIB_OBJS += pager.o\n LIB_OBJS += parallel-checkout.o\n+LIB_OBJS += parse.o\n LIB_OBJS += parse-options-cb.o\n LIB_OBJS += parse-options.o\n LIB_OBJS += patch-delta.o\ndiff --git a/attr.c b/attr.c\nindex e9c81b6e07..cb047b4618 100644\n--- a/attr.c\n+++ b/attr.c\n@@ -7,7 +7,7 @@\n */\n \n #include \"git-compat-util.h\"\n-#include \"config.h\"\n+#include \"parse.h\"\n #include \"environment.h\"\n #include \"exec-cmd.h\"\n #include \"attr.h\"\ndiff --git a/config.c b/config.c\nindex 5b71ef1624..cdd70999aa 100644\n--- a/config.c\n+++ b/config.c\n@@ -11,6 +11,7 @@\n #include \"date.h\"\n #include \"branch.h\"\n #include \"config.h\"\n+#include \"parse.h\"\n #include \"convert.h\"\n #include \"environment.h\"\n #include \"gettext.h\"\n@@ -1204,129 +1205,6 @@ static int git_parse_source(struct config_source *cs, config_fn_t fn,\n \treturn error_return;\n }\n \n-static uintmax_t get_unit_factor(const char *end)\n-{\n-\tif (!*end)\n-\t\treturn 1;\n-\telse if (!strcasecmp(end, \"k\"))\n-\t\treturn 1024;\n-\telse if (!strcasecmp(end, \"m\"))\n-\t\treturn 1024 * 1024;\n-\telse if (!strcasecmp(end, \"g\"))\n-\t\treturn 1024 * 1024 * 1024;\n-\treturn 0;\n-}\n-\n-static int git_parse_signed(const char *value, intmax_t *ret, intmax_t max)\n-{\n-\tif (value && *value) {\n-\t\tchar *end;\n-\t\tintmax_t val;\n-\t\tintmax_t factor;\n-\n-\t\tif (max < 0)\n-\t\t\tBUG(\"max must be a positive integer\");\n-\n-\t\terrno = 0;\n-\t\tval = strtoimax(value, &end, 0);\n-\t\tif (errno == ERANGE)\n-\t\t\treturn 0;\n-\t\tif (end == value) {\n-\t\t\terrno = EINVAL;\n-\t\t\treturn 0;\n-\t\t}\n-\t\tfactor = get_unit_factor(end);\n-\t\tif (!factor) {\n-\t\t\terrno = EINVAL;\n-\t\t\treturn 0;\n-\t\t}\n-\t\tif ((val < 0 && -max / factor > val) ||\n-\t\t (val > 0 && max / factor < val)) {\n-\t\t\terrno = ERANGE;\n-\t\t\treturn 0;\n-\t\t}\n-\t\tval *= factor;\n-\t\t*ret = val;\n-\t\treturn 1;\n-\t}\n-\terrno = EINVAL;\n-\treturn 0;\n-}\n-\n-static int git_parse_unsigned(const char *value, uintmax_t *ret, uintmax_t max)\n-{\n-\tif (value && *value) {\n-\t\tchar *end;\n-\t\tuintmax_t val;\n-\t\tuintmax_t factor;\n-\n-\t\t/* negative values would be accepted by strtoumax */\n-\t\tif (strchr(value, '-')) {\n-\t\t\terrno = EINVAL;\n-\t\t\treturn 0;\n-\t\t}\n-\t\terrno = 0;\n-\t\tval = strtoumax(value, &end, 0);\n-\t\tif (errno == ERANGE)\n-\t\t\treturn 0;\n-\t\tif (end == value) {\n-\t\t\terrno = EINVAL;\n-\t\t\treturn 0;\n-\t\t}\n-\t\tfactor = get_unit_factor(end);\n-\t\tif (!factor) {\n-\t\t\terrno = EINVAL;\n-\t\t\treturn 0;\n-\t\t}\n-\t\tif (unsigned_mult_overflows(factor, val) ||\n-\t\t factor * val > max) {\n-\t\t\terrno = ERANGE;\n-\t\t\treturn 0;\n-\t\t}\n-\t\tval *= factor;\n-\t\t*ret = val;\n-\t\treturn 1;\n-\t}\n-\terrno = EINVAL;\n-\treturn 0;\n-}\n-\n-int git_parse_int(const char *value, int *ret)\n-{\n-\tintmax_t tmp;\n-\tif (!git_parse_signed(value, &tmp, maximum_signed_value_of_type(int)))\n-\t\treturn 0;\n-\t*ret = tmp;\n-\treturn 1;\n-}\n-\n-static int git_parse_int64(const char *value, int64_t *ret)\n-{\n-\tintmax_t tmp;\n-\tif (!git_parse_signed(value, &tmp, maximum_signed_value_of_type(int64_t)))\n-\t\treturn 0;\n-\t*ret = tmp;\n-\treturn 1;\n-}\n-\n-int git_parse_ulong(const char *value, unsigned long *ret)\n-{\n-\tuintmax_t tmp;\n-\tif (!git_parse_unsigned(value, &tmp, maximum_unsigned_value_of_type(long)))\n-\t\treturn 0;\n-\t*ret = tmp;\n-\treturn 1;\n-}\n-\n-int git_parse_ssize_t(const char *value, ssize_t *ret)\n-{\n-\tintmax_t tmp;\n-\tif (!git_parse_signed(value, &tmp, maximum_signed_value_of_type(ssize_t)))\n-\t\treturn 0;\n-\t*ret = tmp;\n-\treturn 1;\n-}\n-\n static int reader_config_name(struct config_reader *reader, const char **out);\n static int reader_origin_type(struct config_reader *reader,\n \t\t\t enum config_origin_type *type);\n@@ -1404,23 +1282,6 @@ ssize_t git_config_ssize_t(const char *name, const char *value)\n \treturn ret;\n }\n \n-static int git_parse_maybe_bool_text(const char *value)\n-{\n-\tif (!value)\n-\t\treturn 1;\n-\tif (!*value)\n-\t\treturn 0;\n-\tif (!strcasecmp(value, \"true\")\n-\t || !strcasecmp(value, \"yes\")\n-\t || !strcasecmp(value, \"on\"))\n-\t\treturn 1;\n-\tif (!strcasecmp(value, \"false\")\n-\t || !strcasecmp(value, \"no\")\n-\t || !strcasecmp(value, \"off\"))\n-\t\treturn 0;\n-\treturn -1;\n-}\n-\n static const struct fsync_component_name {\n \tconst char *name;\n \tenum fsync_component component_bits;\n@@ -1495,16 +1356,6 @@ static enum fsync_component parse_fsync_components(const char *var, const char *\n \treturn (current & ~negative) | positive;\n }\n \n-int git_parse_maybe_bool(const char *value)\n-{\n-\tint v = git_parse_maybe_bool_text(value);\n-\tif (0 <= v)\n-\t\treturn v;\n-\tif (git_parse_int(value, &v))\n-\t\treturn !!v;\n-\treturn -1;\n-}\n-\n int git_config_bool_or_int(const char *name, const char *value, int *is_bool)\n {\n \tint v = git_parse_maybe_bool_text(value);\n@@ -2165,35 +2016,6 @@ void git_global_config(char **user_out, char **xdg_out)\n \t*xdg_out = xdg_config;\n }\n \n-/*\n- * Parse environment variable 'k' as a boolean (in various\n- * possible spellings); if missing, use the default value 'def'.\n- */\n-int git_env_bool(const char *k, int def)\n-{\n-\tconst char *v = getenv(k);\n-\tint val;\n-\tif (!v)\n-\t\treturn def;\n-\tval = git_parse_maybe_bool(v);\n-\tif (val < 0)\n-\t\tdie(_(\"bad boolean environment value '%s' for '%s'\"),\n-\t\t v, k);\n-\treturn val;\n-}\n-\n-/*\n- * Parse environment variable 'k' as ulong with possibly a unit\n- * suffix; if missing, use the default value 'val'.\n- */\n-unsigned long git_env_ulong(const char *k, unsigned long val)\n-{\n-\tconst char *v = getenv(k);\n-\tif (v && !git_parse_ulong(v, &val))\n-\t\tdie(_(\"failed to parse %s\"), k);\n-\treturn val;\n-}\n-\n int git_config_system(void)\n {\n \treturn !git_env_bool(\"GIT_CONFIG_NOSYSTEM\", 0);\ndiff --git a/config.h b/config.h\nindex 247b572b37..7a7f53e503 100644\n--- a/config.h\n+++ b/config.h\n@@ -3,7 +3,7 @@\n \n #include \"hashmap.h\"\n #include \"string-list.h\"\n-\n+#include \"parse.h\"\n \n /**\n * The config API gives callers a way to access Git configuration files\n@@ -205,16 +205,6 @@ int config_with_options(config_fn_t fn, void *,\n * The following helper functions aid in parsing string values\n */\n \n-int git_parse_ssize_t(const char *, ssize_t *);\n-int git_parse_ulong(const char *, unsigned long *);\n-int git_parse_int(const char *value, int *ret);\n-\n-/**\n- * Same as `git_config_bool`, except that it returns -1 on error rather\n- * than dying.\n- */\n-int git_parse_maybe_bool(const char *);\n-\n /**\n * Parse the string to an integer, including unit factors. Dies on error;\n * otherwise, returns the parsed result.\n@@ -343,8 +333,6 @@ int git_config_rename_section(const char *, const char *);\n int git_config_rename_section_in_file(const char *, const char *, const char *);\n int git_config_copy_section(const char *, const char *);\n int git_config_copy_section_in_file(const char *, const char *, const char *);\n-int git_env_bool(const char *, int);\n-unsigned long git_env_ulong(const char *, unsigned long);\n int git_config_system(void);\n int config_error_nonbool(const char *);\n #if defined(__GNUC__)\ndiff --git a/pack-objects.c b/pack-objects.c\nindex 1b8052bece..f403ca6986 100644\n--- a/pack-objects.c\n+++ b/pack-objects.c\n@@ -3,7 +3,7 @@\n #include \"pack.h\"\n #include \"pack-objects.h\"\n #include \"packfile.h\"\n-#include \"config.h\"\n+#include \"parse.h\"\n \n static uint32_t locate_object_entry_hash(struct packing_data *pdata,\n \t\t\t\t\t const struct object_id *oid,\ndiff --git a/pack-revindex.c b/pack-revindex.c\nindex 7fffcad912..a01a2a4640 100644\n--- a/pack-revindex.c\n+++ b/pack-revindex.c\n@@ -6,7 +6,7 @@\n #include \"packfile.h\"\n #include \"strbuf.h\"\n #include \"trace2.h\"\n-#include \"config.h\"\n+#include \"parse.h\"\n #include \"midx.h\"\n #include \"csum-file.h\"\n \ndiff --git a/parse-options.c b/parse-options.c\nindex f8a155ee13..9f542950a7 100644\n--- a/parse-options.c\n+++ b/parse-options.c\n@@ -1,11 +1,12 @@\n #include \"git-compat-util.h\"\n #include \"parse-options.h\"\n #include \"abspath.h\"\n-#include \"config.h\"\n+#include \"parse.h\"\n #include \"commit.h\"\n #include \"color.h\"\n #include \"gettext.h\"\n #include \"strbuf.h\"\n+#include \"string-list.h\"\n #include \"utf8.h\"\n \n static int disallow_abbreviated_options;\ndiff --git a/parse.c b/parse.c\nnew file mode 100644\nindex 0000000000..42d691a0fb\n--- /dev/null\n+++ b/parse.c\n@@ -0,0 +1,182 @@\n+#include \"git-compat-util.h\"\n+#include \"gettext.h\"\n+#include \"parse.h\"\n+\n+static uintmax_t get_unit_factor(const char *end)\n+{\n+\tif (!*end)\n+\t\treturn 1;\n+\telse if (!strcasecmp(end, \"k\"))\n+\t\treturn 1024;\n+\telse if (!strcasecmp(end, \"m\"))\n+\t\treturn 1024 * 1024;\n+\telse if (!strcasecmp(end, \"g\"))\n+\t\treturn 1024 * 1024 * 1024;\n+\treturn 0;\n+}\n+\n+int git_parse_signed(const char *value, intmax_t *ret, intmax_t max)\n+{\n+\tif (value && *value) {\n+\t\tchar *end;\n+\t\tintmax_t val;\n+\t\tintmax_t factor;\n+\n+\t\tif (max < 0)\n+\t\t\tBUG(\"max must be a positive integer\");\n+\n+\t\terrno = 0;\n+\t\tval = strtoimax(value, &end, 0);\n+\t\tif (errno == ERANGE)\n+\t\t\treturn 0;\n+\t\tif (end == value) {\n+\t\t\terrno = EINVAL;\n+\t\t\treturn 0;\n+\t\t}\n+\t\tfactor = get_unit_factor(end);\n+\t\tif (!factor) {\n+\t\t\terrno = EINVAL;\n+\t\t\treturn 0;\n+\t\t}\n+\t\tif ((val < 0 && -max / factor > val) ||\n+\t\t (val > 0 && max / factor < val)) {\n+\t\t\terrno = ERANGE;\n+\t\t\treturn 0;\n+\t\t}\n+\t\tval *= factor;\n+\t\t*ret = val;\n+\t\treturn 1;\n+\t}\n+\terrno = EINVAL;\n+\treturn 0;\n+}\n+\n+static int git_parse_unsigned(const char *value, uintmax_t *ret, uintmax_t max)\n+{\n+\tif (value && *value) {\n+\t\tchar *end;\n+\t\tuintmax_t val;\n+\t\tuintmax_t factor;\n+\n+\t\t/* negative values would be accepted by strtoumax */\n+\t\tif (strchr(value, '-')) {\n+\t\t\terrno = EINVAL;\n+\t\t\treturn 0;\n+\t\t}\n+\t\terrno = 0;\n+\t\tval = strtoumax(value, &end, 0);\n+\t\tif (errno == ERANGE)\n+\t\t\treturn 0;\n+\t\tif (end == value) {\n+\t\t\terrno = EINVAL;\n+\t\t\treturn 0;\n+\t\t}\n+\t\tfactor = get_unit_factor(end);\n+\t\tif (!factor) {\n+\t\t\terrno = EINVAL;\n+\t\t\treturn 0;\n+\t\t}\n+\t\tif (unsigned_mult_overflows(factor, val) ||\n+\t\t factor * val > max) {\n+\t\t\terrno = ERANGE;\n+\t\t\treturn 0;\n+\t\t}\n+\t\tval *= factor;\n+\t\t*ret = val;\n+\t\treturn 1;\n+\t}\n+\terrno = EINVAL;\n+\treturn 0;\n+}\n+\n+int git_parse_int(const char *value, int *ret)\n+{\n+\tintmax_t tmp;\n+\tif (!git_parse_signed(value, &tmp, maximum_signed_value_of_type(int)))\n+\t\treturn 0;\n+\t*ret = tmp;\n+\treturn 1;\n+}\n+\n+int git_parse_int64(const char *value, int64_t *ret)\n+{\n+\tintmax_t tmp;\n+\tif (!git_parse_signed(value, &tmp, maximum_signed_value_of_type(int64_t)))\n+\t\treturn 0;\n+\t*ret = tmp;\n+\treturn 1;\n+}\n+\n+int git_parse_ulong(const char *value, unsigned long *ret)\n+{\n+\tuintmax_t tmp;\n+\tif (!git_parse_unsigned(value, &tmp, maximum_unsigned_value_of_type(long)))\n+\t\treturn 0;\n+\t*ret = tmp;\n+\treturn 1;\n+}\n+\n+int git_parse_ssize_t(const char *value, ssize_t *ret)\n+{\n+\tintmax_t tmp;\n+\tif (!git_parse_signed(value, &tmp, maximum_signed_value_of_type(ssize_t)))\n+\t\treturn 0;\n+\t*ret = tmp;\n+\treturn 1;\n+}\n+\n+int git_parse_maybe_bool_text(const char *value)\n+{\n+\tif (!value)\n+\t\treturn 1;\n+\tif (!*value)\n+\t\treturn 0;\n+\tif (!strcasecmp(value, \"true\")\n+\t || !strcasecmp(value, \"yes\")\n+\t || !strcasecmp(value, \"on\"))\n+\t\treturn 1;\n+\tif (!strcasecmp(value, \"false\")\n+\t || !strcasecmp(value, \"no\")\n+\t || !strcasecmp(value, \"off\"))\n+\t\treturn 0;\n+\treturn -1;\n+}\n+\n+int git_parse_maybe_bool(const char *value)\n+{\n+\tint v = git_parse_maybe_bool_text(value);\n+\tif (0 <= v)\n+\t\treturn v;\n+\tif (git_parse_int(value, &v))\n+\t\treturn !!v;\n+\treturn -1;\n+}\n+\n+/*\n+ * Parse environment variable 'k' as a boolean (in various\n+ * possible spellings); if missing, use the default value 'def'.\n+ */\n+int git_env_bool(const char *k, int def)\n+{\n+\tconst char *v = getenv(k);\n+\tint val;\n+\tif (!v)\n+\t\treturn def;\n+\tval = git_parse_maybe_bool(v);\n+\tif (val < 0)\n+\t\tdie(_(\"bad boolean environment value '%s' for '%s'\"),\n+\t\t v, k);\n+\treturn val;\n+}\n+\n+/*\n+ * Parse environment variable 'k' as ulong with possibly a unit\n+ * suffix; if missing, use the default value 'val'.\n+ */\n+unsigned long git_env_ulong(const char *k, unsigned long val)\n+{\n+\tconst char *v = getenv(k);\n+\tif (v && !git_parse_ulong(v, &val))\n+\t\tdie(_(\"failed to parse %s\"), k);\n+\treturn val;\n+}\ndiff --git a/parse.h b/parse.h\nnew file mode 100644\nindex 0000000000..07d2193d69\n--- /dev/null\n+++ b/parse.h\n@@ -0,0 +1,20 @@\n+#ifndef PARSE_H\n+#define PARSE_H\n+\n+int git_parse_signed(const char *value, intmax_t *ret, intmax_t max);\n+int git_parse_ssize_t(const char *, ssize_t *);\n+int git_parse_ulong(const char *, unsigned long *);\n+int git_parse_int(const char *value, int *ret);\n+int git_parse_int64(const char *value, int64_t *ret);\n+\n+/**\n+ * Same as `git_config_bool`, except that it returns -1 on error rather\n+ * than dying.\n+ */\n+int git_parse_maybe_bool(const char *);\n+int git_parse_maybe_bool_text(const char *value);\n+\n+int git_env_bool(const char *, int);\n+unsigned long git_env_ulong(const char *, unsigned long);\n+\n+#endif /* PARSE_H */\ndiff --git a/pathspec.c b/pathspec.c\nindex 4991455281..39337999d4 100644\n--- a/pathspec.c\n+++ b/pathspec.c\n@@ -1,6 +1,6 @@\n #include \"git-compat-util.h\"\n #include \"abspath.h\"\n-#include \"config.h\"\n+#include \"parse.h\"\n #include \"dir.h\"\n #include \"environment.h\"\n #include \"gettext.h\"\ndiff --git a/preload-index.c b/preload-index.c\nindex e44530c80c..63fd35d64b 100644\n--- a/preload-index.c\n+++ b/preload-index.c\n@@ -7,7 +7,7 @@\n #include \"environment.h\"\n #include \"fsmonitor.h\"\n #include \"gettext.h\"\n-#include \"config.h\"\n+#include \"parse.h\"\n #include \"preload-index.h\"\n #include \"progress.h\"\n #include \"read-cache.h\"\ndiff --git a/progress.c b/progress.c\nindex f695798aca..c83cb60bf1 100644\n--- a/progress.c\n+++ b/progress.c\n@@ -17,7 +17,7 @@\n #include \"trace.h\"\n #include \"trace2.h\"\n #include \"utf8.h\"\n-#include \"config.h\"\n+#include \"parse.h\"\n \n #define TP_IDX_MAX 8\n \ndiff --git a/prompt.c b/prompt.c\nindex 3baa33f63d..8935fe4dfb 100644\n--- a/prompt.c\n+++ b/prompt.c\n@@ -1,5 +1,5 @@\n #include \"git-compat-util.h\"\n-#include \"config.h\"\n+#include \"parse.h\"\n #include \"environment.h\"\n #include \"run-command.h\"\n #include \"strbuf.h\"\ndiff --git a/rebase.c b/rebase.c\nindex 17a570f1ff..69a1822da3 100644\n--- a/rebase.c\n+++ b/rebase.c\n@@ -1,6 +1,6 @@\n #include \"git-compat-util.h\"\n #include \"rebase.h\"\n-#include \"config.h\"\n+#include \"parse.h\"\n #include \"gettext.h\"\n \n /*\ndiff --git a/t/helper/test-env-helper.c b/t/helper/test-env-helper.c\nindex 66c88b8ff3..1c486888a4 100644\n--- a/t/helper/test-env-helper.c\n+++ b/t/helper/test-env-helper.c\n@@ -1,5 +1,5 @@\n #include \"test-tool.h\"\n-#include \"config.h\"\n+#include \"parse.h\"\n #include \"parse-options.h\"\n \n static char const * const env__helper_usage[] = {\ndiff --git a/unpack-trees.c b/unpack-trees.c\nindex 87517364dc..761562a96e 100644\n--- a/unpack-trees.c\n+++ b/unpack-trees.c\n@@ -2,7 +2,7 @@\n #include \"advice.h\"\n #include \"strvec.h\"\n #include \"repository.h\"\n-#include \"config.h\"\n+#include \"parse.h\"\n #include \"dir.h\"\n #include \"environment.h\"\n #include \"gettext.h\"\ndiff --git a/wrapper.c b/wrapper.c\nindex 62c04aeb17..3e554f50c6 100644\n--- a/wrapper.c\n+++ b/wrapper.c\n@@ -3,7 +3,7 @@\n */\n #include \"git-compat-util.h\"\n #include \"abspath.h\"\n-#include \"config.h\"\n+#include \"parse.h\"\n #include \"gettext.h\"\n #include \"strbuf.h\"\n \ndiff --git a/write-or-die.c b/write-or-die.c\nindex d8355c0c3e..42a2dc73cd 100644\n--- a/write-or-die.c\n+++ b/write-or-die.c\n@@ -1,5 +1,5 @@\n #include \"git-compat-util.h\"\n-#include \"config.h\"\n+#include \"parse.h\"\n #include \"run-command.h\"\n #include \"write-or-die.h\"\n \n-- \n2.41.0.162.gfafddb0af9-goog\n\n"},{"id":"478925","messageId":"20230627195251.1973421-7-calvinwan@google.com","threadId":"59921","inReplyTo":"20230627195251.1973421-1-calvinwan@google.com","subject":"[RFC PATCH 6/8] pager: remove pager_in_use()","fromName":"Calvin Wan","fromEmail":"calvinwan@google.com","sentAt":"2023-06-27T19:52:49Z","receivedAt":"2023-06-27T19:53:28Z","isPatch":true,"sender":{"key":"calvinwan@google.com","avatar":"https://avatars.githubusercontent.com/u/92547554?v=4"},"body":"pager_in_use() is simply a wrapper around\ngit_env_bool(\"GIT_PAGER_IN_USE\", 0). Other places that call\ngit_env_bool() in this fashion also do not have a wrapper function\naround it. By removing pager_in_use(), we can also get rid of the\npager.h dependency from a few files.\n\nSigned-off-by: Calvin Wan \n---\n builtin/log.c | 2 +-\n color.c | 2 +-\n column.c | 2 +-\n date.c | 4 ++--\n git.c | 2 +-\n pager.c | 5 -----\n pager.h | 1 -\n 7 files changed, 6 insertions(+), 12 deletions(-)\n\ndiff --git a/builtin/log.c b/builtin/log.c\nindex 03954fb749..d5e979932f 100644\n--- a/builtin/log.c\n+++ b/builtin/log.c\n@@ -82,7 +82,7 @@ struct line_opt_callback_data {\n \n static int session_is_interactive(void)\n {\n-\treturn isatty(1) || pager_in_use();\n+\treturn isatty(1) || git_env_bool(\"GIT_PAGER_IN_USE\", 0);\n }\n \n static int auto_decoration_style(void)\ndiff --git a/color.c b/color.c\nindex f3c0a4659b..dd6f26b8db 100644\n--- a/color.c\n+++ b/color.c\n@@ -388,7 +388,7 @@ static int check_auto_color(int fd)\n \tint *is_tty_p = fd == 1 ? &color_stdout_is_tty : &color_stderr_is_tty;\n \tif (*is_tty_p < 0)\n \t\t*is_tty_p = isatty(fd);\n-\tif (*is_tty_p || (fd == 1 && pager_in_use() && pager_use_color)) {\n+\tif (*is_tty_p || (fd == 1 && git_env_bool(\"GIT_PAGER_IN_USE\", 0) && pager_use_color)) {\n \t\tif (!is_terminal_dumb())\n \t\t\treturn 1;\n \t}\ndiff --git a/column.c b/column.c\nindex ff2f0abf39..e15ca70f36 100644\n--- a/column.c\n+++ b/column.c\n@@ -214,7 +214,7 @@ int finalize_colopts(unsigned int *colopts, int stdout_is_tty)\n \t\tif (stdout_is_tty < 0)\n \t\t\tstdout_is_tty = isatty(1);\n \t\t*colopts &= ~COL_ENABLE_MASK;\n-\t\tif (stdout_is_tty || pager_in_use())\n+\t\tif (stdout_is_tty || git_env_bool(\"GIT_PAGER_IN_USE\", 0))\n \t\t\t*colopts |= COL_ENABLED;\n \t}\n \treturn 0;\ndiff --git a/date.c b/date.c\nindex 619ada5b20..95c0f568ba 100644\n--- a/date.c\n+++ b/date.c\n@@ -7,7 +7,7 @@\n #include \"git-compat-util.h\"\n #include \"date.h\"\n #include \"gettext.h\"\n-#include \"pager.h\"\n+#include \"parse.h\"\n #include \"strbuf.h\"\n \n /*\n@@ -1009,7 +1009,7 @@ void parse_date_format(const char *format, struct date_mode *mode)\n \n \t/* \"auto:foo\" is \"if tty/pager, then foo, otherwise normal\" */\n \tif (skip_prefix(format, \"auto:\", &p)) {\n-\t\tif (isatty(1) || pager_in_use())\n+\t\tif (isatty(1) || git_env_bool(\"GIT_PAGER_IN_USE\", 0))\n \t\t\tformat = p;\n \t\telse\n \t\t\tformat = \"default\";\ndiff --git a/git.c b/git.c\nindex eb69f4f997..3bfb673a4c 100644\n--- a/git.c\n+++ b/git.c\n@@ -131,7 +131,7 @@ static void commit_pager_choice(void)\n \n void setup_auto_pager(const char *cmd, int def)\n {\n-\tif (use_pager != -1 || pager_in_use())\n+\tif (use_pager != -1 || git_env_bool(\"GIT_PAGER_IN_USE\", 0))\n \t\treturn;\n \tuse_pager = check_pager_config(cmd);\n \tif (use_pager == -1)\ndiff --git a/pager.c b/pager.c\nindex 63055d0873..9b392622d2 100644\n--- a/pager.c\n+++ b/pager.c\n@@ -149,11 +149,6 @@ void setup_pager(void)\n \tatexit(wait_for_pager_atexit);\n }\n \n-int pager_in_use(void)\n-{\n-\treturn git_env_bool(\"GIT_PAGER_IN_USE\", 0);\n-}\n-\n /*\n * Return cached value (if set) or $COLUMNS environment variable (if\n * set and positive) or ioctl(1, TIOCGWINSZ).ws_col (if positive),\ndiff --git a/pager.h b/pager.h\nindex b77433026d..6832c6168d 100644\n--- a/pager.h\n+++ b/pager.h\n@@ -5,7 +5,6 @@ struct child_process;\n \n const char *git_pager(int stdout_is_tty);\n void setup_pager(void);\n-int pager_in_use(void);\n int term_columns(void);\n void term_clear_line(void);\n int decimal_width(uintmax_t);\n-- \n2.41.0.162.gfafddb0af9-goog\n\n"},{"id":"478926","messageId":"20230627195251.1973421-9-calvinwan@google.com","threadId":"59921","inReplyTo":"20230627195251.1973421-1-calvinwan@google.com","subject":"[RFC PATCH 8/8] git-std-lib: add test file to call git-std-lib.a functions","fromName":"Calvin Wan","fromEmail":"calvinwan@google.com","sentAt":"2023-06-27T19:52:51Z","receivedAt":"2023-06-27T19:53:32Z","isPatch":true,"sender":{"key":"calvinwan@google.com","avatar":"https://avatars.githubusercontent.com/u/92547554?v=4"},"body":"Add test file that directly or indirectly calls all functions defined in\ngit-std-lib.a object files to showcase that they do not reference\nmissing objects and that git-std-lib.a can stand on its own.\n\nCertain functions that cause the program to exit or are already called\nby other functions are commented out.\n\nTODO: replace with unit tests\nSigned-off-by: Calvin Wan \n---\n t/Makefile | 4 +\n t/stdlib-test.c | 239 ++++++++++++++++++++++++++++++++++++++++++++++++\n 2 files changed, 243 insertions(+)\n create mode 100644 t/stdlib-test.c\n\ndiff --git a/t/Makefile b/t/Makefile\nindex 3e00cdd801..b6d0bc9daa 100644\n--- a/t/Makefile\n+++ b/t/Makefile\n@@ -150,3 +150,7 @@ perf:\n \n .PHONY: pre-clean $(T) aggregate-results clean valgrind perf \\\n \tcheck-chainlint clean-chainlint test-chainlint\n+\n+test-git-std-lib:\n+\tcc -It -o stdlib-test stdlib-test.c -L. -l:../git-std-lib.a\n+\t./stdlib-test\ndiff --git a/t/stdlib-test.c b/t/stdlib-test.c\nnew file mode 100644\nindex 0000000000..0e4f6d5807\n--- /dev/null\n+++ b/t/stdlib-test.c\n@@ -0,0 +1,239 @@\n+#include \"../git-compat-util.h\"\n+#include \"../abspath.h\"\n+#include \"../hex-ll.h\"\n+#include \"../parse.h\"\n+#include \"../strbuf.h\"\n+#include \"../string-list.h\"\n+\n+/*\n+ * Calls all functions from git-std-lib\n+ * Some inline/trivial functions are skipped\n+ */\n+\n+void abspath_funcs(void) {\n+\tstruct strbuf sb = STRBUF_INIT;\n+\n+\tfprintf(stderr, \"calling abspath functions\\n\");\n+\tis_directory(\"foo\");\n+\tstrbuf_realpath(&sb, \"foo\", 0);\n+\tstrbuf_realpath_forgiving(&sb, \"foo\", 0);\n+\treal_pathdup(\"foo\", 0);\n+\tabsolute_path(\"foo\");\n+\tabsolute_pathdup(\"foo\");\n+\tprefix_filename(\"foo/\", \"bar\");\n+\tprefix_filename_except_for_dash(\"foo/\", \"bar\");\n+\tis_absolute_path(\"foo\");\n+\tstrbuf_add_absolute_path(&sb, \"foo\");\n+\tstrbuf_add_real_path(&sb, \"foo\");\n+}\n+\n+void hex_ll_funcs(void) {\n+\tunsigned char c;\n+\n+\tfprintf(stderr, \"calling hex-ll functions\\n\");\n+\n+\thexval('c');\n+\thex2chr(\"A1\");\n+\thex_to_bytes(&c, \"A1\", 2);\n+}\n+\n+void parse_funcs(void) {\n+\tintmax_t foo;\n+\tssize_t foo1 = -1;\n+\tunsigned long foo2;\n+\tint foo3;\n+\tint64_t foo4;\n+\n+\tfprintf(stderr, \"calling parse functions\\n\");\n+\n+\tgit_parse_signed(\"42\", &foo, maximum_signed_value_of_type(int));\n+\tgit_parse_ssize_t(\"42\", &foo1);\n+\tgit_parse_ulong(\"42\", &foo2);\n+\tgit_parse_int(\"42\", &foo3);\n+\tgit_parse_int64(\"42\", &foo4);\n+\tgit_parse_maybe_bool(\"foo\");\n+\tgit_parse_maybe_bool_text(\"foo\");\n+\tgit_env_bool(\"foo\", 1);\n+\tgit_env_ulong(\"foo\", 1);\n+}\n+\n+static int allow_unencoded_fn(char ch) {\n+\treturn 0;\n+}\n+\n+void strbuf_funcs(void) {\n+\tstruct strbuf *sb = xmalloc(sizeof(void*));\n+\tstruct strbuf *sb2 = xmalloc(sizeof(void*));\n+\tstruct strbuf sb3 = STRBUF_INIT;\n+\tstruct string_list list = STRING_LIST_INIT_NODUP;\n+\tchar *buf = \"foo\";\n+\tstruct strbuf_expand_dict_entry dict[] = {\n+\t\t{ \"foo\", NULL, },\n+\t\t{ \"bar\", NULL, },\n+\t};\n+\tint fd = open(\"/dev/null\", O_RDONLY);\n+\n+\tfprintf(stderr, \"calling strbuf functions\\n\");\n+\n+\tstarts_with(\"foo\", \"bar\");\n+\tistarts_with(\"foo\", \"bar\");\n+\t// skip_to_optional_arg_default(const char *str, const char *prefix,\n+\t// \t\t\t const char **arg, const char *def)\n+\tstrbuf_init(sb, 0);\n+\tstrbuf_init(sb2, 0);\n+\tstrbuf_release(sb);\n+\tstrbuf_attach(sb, strbuf_detach(sb, NULL), 0, 0); // calls strbuf_grow\n+\tstrbuf_swap(sb, sb2);\n+\tstrbuf_setlen(sb, 0);\n+\tstrbuf_trim(sb); // calls strbuf_rtrim, strbuf_ltrim\n+\t// strbuf_rtrim() called by strbuf_trim()\n+\t// strbuf_ltrim() called by strbuf_trim()\n+\tstrbuf_trim_trailing_dir_sep(sb);\n+\tstrbuf_trim_trailing_newline(sb);\n+\tstrbuf_reencode(sb, \"foo\", \"bar\");\n+\tstrbuf_tolower(sb);\n+\tstrbuf_add_separated_string_list(sb, \" \", &list);\n+\tstrbuf_list_free(strbuf_split_buf(\"foo bar\", 8, ' ', -1));\n+\tstrbuf_cmp(sb, sb2);\n+\tstrbuf_addch(sb, 1);\n+\tstrbuf_splice(sb, 0, 1, \"foo\", 3);\n+\tstrbuf_insert(sb, 0, \"foo\", 3);\n+\t// strbuf_vinsertf() called by strbuf_insertf\n+\tstrbuf_insertf(sb, 0, \"%s\", \"foo\"); \n+\tstrbuf_remove(sb, 0, 1);\n+\tstrbuf_add(sb, \"foo\", 3);\n+\tstrbuf_addbuf(sb, sb2);\n+\tstrbuf_join_argv(sb, 0, NULL, ' ');\n+\tstrbuf_addchars(sb, 1, 1);\n+\tstrbuf_addf(sb, \"%s\", \"foo\");\n+\tstrbuf_add_commented_lines(sb, \"foo\", 3, '#');\n+\tstrbuf_commented_addf(sb, '#', \"%s\", \"foo\");\n+\t// strbuf_vaddf() called by strbuf_addf()\n+\tstrbuf_expand(sb, \"%s\", strbuf_expand_literal_cb, NULL);\n+\tstrbuf_expand(sb, \"%s\", strbuf_expand_dict_cb, &dict);\n+\t// strbuf_expand_literal_cb() called by strbuf_expand()\n+\t// strbuf_expand_dict_cb() called by strbuf_expand()\n+\tstrbuf_addbuf_percentquote(sb, &sb3);\n+\tstrbuf_add_percentencode(sb, \"foo\", STRBUF_ENCODE_SLASH);\n+\tstrbuf_fread(sb, 0, stdin);\n+\tstrbuf_read(sb, fd, 0);\n+\tstrbuf_read_once(sb, fd, 0);\n+\tstrbuf_write(sb, stderr);\n+\tstrbuf_readlink(sb, \"/dev/null\", 0);\n+\tstrbuf_getcwd(sb);\n+\tstrbuf_getwholeline(sb, stderr, '\\n');\n+\tstrbuf_appendwholeline(sb, stderr, '\\n');\n+\tstrbuf_getline(sb, stderr);\n+\tstrbuf_getline_lf(sb, stderr);\n+\tstrbuf_getline_nul(sb, stderr);\n+\tstrbuf_getwholeline_fd(sb, fd, '\\n');\n+\tstrbuf_read_file(sb, \"/dev/null\", 0);\n+\tstrbuf_add_lines(sb, \"foo\", \"bar\", 0);\n+\tstrbuf_addstr_xml_quoted(sb, \"foo\");\n+\tstrbuf_addstr_urlencode(sb, \"foo\", allow_unencoded_fn);\n+\tstrbuf_humanise_bytes(sb, 42);\n+\tstrbuf_humanise_rate(sb, 42);\n+\tprintf_ln(\"%s\", sb);\n+\tfprintf_ln(stderr, \"%s\", sb);\n+\txstrdup_tolower(\"foo\");\n+\txstrdup_toupper(\"foo\");\n+\t// xstrvfmt() called by xstrfmt()\n+\txstrfmt(\"%s\", \"foo\");\n+\t// strbuf_addftime(struct strbuf *sb, const char *fmt, const struct tm *tm,\n+\t// \t int tz_offset, int suppress_tz_name)\n+\t// strbuf_stripspace(struct strbuf *sb, char comment_line_char)\n+\t// strbuf_strip_suffix(struct strbuf *sb, const char *suffix)\n+\t// strbuf_strip_file_from_path(struct strbuf *sb)\n+}\n+\n+static void error_builtin(const char *err, va_list params) {}\n+static void warn_builtin(const char *err, va_list params) {}\n+\n+static report_fn error_routine = error_builtin;\n+static report_fn warn_routine = warn_builtin;\n+\n+void usage_funcs(void) {\n+\tfprintf(stderr, \"calling usage functions\\n\");\n+\t// Functions that call exit() are commented out\n+\n+\t// usage()\n+\t// usagef()\n+\t// die()\n+\t// die_errno();\n+\terror(\"foo\");\n+\terror_errno(\"foo\");\n+\tdie_message(\"foo\");\n+\tdie_message_errno(\"foo\");\n+\twarning(\"foo\");\n+\twarning_errno(\"foo\");\n+\n+\t// set_die_routine();\n+\tget_die_message_routine();\n+\tset_error_routine(error_builtin);\n+\tget_error_routine();\n+\tset_warn_routine(warn_builtin);\n+\tget_warn_routine();\n+\t// set_die_is_recursing_routine();\n+}\n+\n+void wrapper_funcs(void) {\n+\tvoid *ptr = xmalloc(1);\n+\tint fd = open(\"/dev/null\", O_RDONLY);\n+\tstruct strbuf sb = STRBUF_INIT;\n+\tint mode = 0444;\n+\tchar host[PATH_MAX], path[PATH_MAX], path1[PATH_MAX];\n+\txsnprintf(path, sizeof(path), \"out-XXXXXX\");\n+\txsnprintf(path1, sizeof(path1), \"out-XXXXXX\");\n+\tint tmp;\n+\n+\tfprintf(stderr, \"calling wrapper functions\\n\");\n+\n+\txstrdup(\"foo\");\n+\txmalloc(1);\n+\txmallocz(1);\n+\txmallocz_gently(1);\n+\txmemdupz(\"foo\", 3);\n+\txstrndup(\"foo\", 3);\n+\txrealloc(ptr, 2);\n+\txcalloc(1, 1);\n+\txsetenv(\"foo\", \"bar\", 0);\n+\txopen(\"/dev/null\", O_RDONLY);\n+\txread(fd, &sb, 1);\n+\txwrite(fd, &sb, 1);\n+\txpread(fd, &sb, 1, 0);\n+\txdup(fd);\n+\txfopen(\"/dev/null\", \"r\");\n+\txfdopen(fd, \"r\");\n+\ttmp = xmkstemp(path);\n+\tclose(tmp);\n+\tunlink(path);\n+\ttmp = xmkstemp_mode(path1, mode);\n+\tclose(tmp);\n+\tunlink(path1);\n+\txgetcwd();\n+\tfopen_for_writing(path);\n+\tfopen_or_warn(path, \"r\");\n+\txstrncmpz(\"foo\", \"bar\", 3);\n+\t// xsnprintf() called above\n+\txgethostname(host, 3);\n+\ttmp = git_mkstemps_mode(path, 1, mode);\n+\tclose(tmp);\n+\tunlink(path);\n+\ttmp = git_mkstemp_mode(path, mode);\n+\tclose(tmp);\n+\tunlink(path);\n+\tread_in_full(fd, &sb, 1);\n+\twrite_in_full(fd, &sb, 1);\n+\tpread_in_full(fd, &sb, 1, 0);\t\n+}\n+\n+int main() {\n+\tabspath_funcs();\n+\thex_ll_funcs();\n+\tparse_funcs();\n+\tstrbuf_funcs();\n+\tusage_funcs();\n+\twrapper_funcs();\n+\tfprintf(stderr, \"all git-std-lib functions finished calling\\n\");\n+\treturn 0;\n+}\n\\ No newline at end of file\n-- \n2.41.0.162.gfafddb0af9-goog\n\n"},{"id":"478927","messageId":"20230627195251.1973421-8-calvinwan@google.com","threadId":"59921","inReplyTo":"20230627195251.1973421-1-calvinwan@google.com","subject":"[RFC PATCH 7/8] git-std-lib: introduce git standard library","fromName":"Calvin Wan","fromEmail":"calvinwan@google.com","sentAt":"2023-06-27T19:52:50Z","receivedAt":"2023-06-27T19:53:35Z","isPatch":true,"sender":{"key":"calvinwan@google.com","avatar":"https://avatars.githubusercontent.com/u/92547554?v=4"},"body":"The Git Standard Library intends to serve as the foundational library\nand root dependency that other libraries in Git will be built off of.\nThat is to say, suppose we have libraries X and Y; a user that wants to\nuse X and Y would need to include X, Y, and this Git Standard Library.\n\nAdd Documentation/technical/git-std-lib.txt to further explain the\ndesign and rationale.\n\nSigned-off-by: Calvin Wan \n---\n Documentation/technical/git-std-lib.txt | 182 ++++++++++++++++++++++++\n Makefile | 28 +++-\n git-compat-util.h | 7 +-\n symlinks.c | 2 +\n usage.c | 8 ++\n 5 files changed, 225 insertions(+), 2 deletions(-)\n create mode 100644 Documentation/technical/git-std-lib.txt\n\ndiff --git a/Documentation/technical/git-std-lib.txt b/Documentation/technical/git-std-lib.txt\nnew file mode 100644\nindex 0000000000..3dce36c9f9\n--- /dev/null\n+++ b/Documentation/technical/git-std-lib.txt\n@@ -0,0 +1,182 @@\n+Git Standard Library\n+================\n+\n+The Git Standard Library intends to serve as the foundational library\n+and root dependency that other libraries in Git will be built off of.\n+That is to say, suppose we have libraries X and Y; a user that wants to\n+use X and Y would need to include X, Y, and this Git Standard Library.\n+This does not mean that the Git Standard Library will be the only\n+possible root dependency in the future, but rather the most significant\n+and widely used one.\n+\n+Dependency graph in libified Git\n+================\n+\n+If you look in the Git Makefile, all of the objects defined in the Git\n+library are compiled and archived into a singular file, libgit.a, which\n+is linked against by common-main.o with other external dependencies and\n+turned into the Git executable. In other words, the Git executable has\n+dependencies on libgit.a and a couple of external libraries. The\n+libfication of Git will not affect this current build flow, but instead\n+will provide an alternate method for building Git.\n+\n+With our current method of building Git, we can imagine the dependency\n+graph as such:\n+\n+ Git\n+ /\\\n+ / \\\n+ / \\\n+ libgit.a ext deps\n+\n+In libifying parts of Git, we want to shrink the dependency graph to\n+only the minimal set of dependencies, so libraries should not use\n+libgit.a. Instead, it would look like:\n+\n+ Git\n+ /\\\n+ / \\\n+ / \\\n+ libgit.a ext deps\n+ /\\\n+ / \\\n+ / \\\n+object-store.a (other lib)\n+ | /\n+ | /\n+ | /\n+ config.a / \n+ | /\n+ | /\n+ | /\n+git-std-lib.a\n+\n+Instead of containing all of the objects in Git, libgit.a would contain\n+objects that are not built by libraries it links against. Consequently,\n+if someone wanted their own custom build of Git with their own custom\n+implementation of the object store, they would only have to swap out\n+object-store.a rather than do a hard fork of Git.\n+\n+Rationale behind Git Standard Library\n+================\n+\n+The rationale behind Git Standard Library essentially is the result of\n+two observations within the Git codebase: every file includes\n+git-compat-util.h which defines functions in a couple of different\n+files, and wrapper.c + usage.c have difficult-to-separate circular\n+dependencies with each other and other files.\n+\n+Ubiquity of git-compat-util.h and circular dependencies\n+========\n+\n+Every file in the Git codebase includes git-compat-util.h. It serves as\n+\"a compatibility aid that isolates the knowledge of platform specific\n+inclusion order and what feature macros to define before including which\n+system header\" (Junio[1]). Since every file includes git-compat-util.h, and\n+git-compat-util.h includes wrapper.h and usage.h, it would make sense\n+for wrapper.c and usage.c to be a part of the root library. They have\n+difficult to separate circular dependencies with each other so they\n+can't be independent libraries. Wrapper.c has dependencies on parse.c,\n+abspath.c, strbuf.c, which in turn also have dependencies on usage.c and\n+wrapper.c -- more circular dependencies. \n+\n+Tradeoff between swappability and refactoring\n+========\n+\n+From the above dependency graph, we can see that git-std-lib.a could be\n+many smaller libraries rather than a singular library. So why choose a\n+singular library when multiple libraries can be individually easier to\n+swap and are more modular? A singular library requires less work to\n+separate out circular dependencies within itself so it becomes a\n+tradeoff question between work and reward. While there may be a point in\n+the future where a file like usage.c would want its own library so that\n+someone can have custom die() or error(), the work required to refactor\n+out the circular dependencies in some files would be enormous due to\n+their ubiquity so therefore I believe it is not worth the tradeoff\n+currently. Additionally, we can in the future choose to do this refactor\n+and change the API for the library if there becomes enough of a reason\n+to do so (remember we are avoiding promising stability of the interfaces\n+of those libraries).\n+\n+Reuse of compatibility functions in git-compat-util.h\n+========\n+\n+Most functions defined in git-compat-util.h are implemented in compat/\n+and have dependencies limited to strbuf.h and wrapper.h so they can be\n+easily included in git-std-lib.a, which as a root dependency means that\n+higher level libraries do not have to worry about compatibility files in\n+compat/. The rest of the functions defined in git-compat-util.h are\n+implemented in top level files and, in this patch set, are hidden behind\n+an #ifdef if their implementation is not in git-std-lib.a.\n+\n+Rationale summary\n+========\n+\n+The Git Standard Library allows us to get the libification ball rolling\n+with other libraries in Git. By not spending many\n+more months attempting to refactor difficult circular dependencies and\n+instead spending that time getting to a state where we can test out\n+swapping a library out such as config or object store, we can prove the\n+viability of Git libification on a much faster time scale. Additionally\n+the code cleanups that have happened so far have been minor and\n+beneficial for the codebase. It is probable that making large movements\n+would negatively affect code clarity.\n+\n+Git Standard Library boundary\n+================\n+\n+While I have described above some useful heuristics for identifying\n+potential candidates for git-std-lib.a, a standard library should not\n+have a shaky definition for what belongs in it.\n+\n+ - Low-level files (aka operates only on other primitive types) that are\n+ used everywhere within the codebase (wrapper.c, usage.c, strbuf.c)\n+ - Dependencies that are low-level and widely used\n+ (abspath.c, date.c, hex-ll.c, parse.c, utf8.c)\n+ - low-level git/* files with functions defined in git-compat-util.h\n+ (ctype.c)\n+ - compat/*\n+\n+There are other files that might fit this definition, but that does not\n+mean it should belong in git-std-lib.a. Those files should start as\n+their own separate library since any file added to git-std-lib.a loses\n+its flexibility of being easily swappable.\n+\n+Files inside of Git Standard Library\n+================\n+\n+The initial set of files in git-std-lib.a are:\n+abspath.c\n+ctype.c\n+date.c\n+hex-ll.c\n+parse.c\n+strbuf.c\n+usage.c\n+utf8.c\n+wrapper.c\n+relevant compat/ files\n+\n+Pitfalls\n+================\n+\n+In patch 7, I use #ifdef GIT_STD_LIB to both stub out code and hide\n+certain function headers. As other parts of Git are libified, if we\n+have to use more ifdefs for each different library, then the codebase\n+will become uglier and harder to understand. \n+\n+There are a small amount of files under compat/* that have dependencies\n+not inside of git-std-lib.a. While those functions are not called on\n+Linux, other OSes might call those problematic functions. I don't see\n+this as a major problem, just moreso an observation that libification in\n+general may also require some minor compatibility work in the future.\n+\n+Testing\n+================\n+\n+Unit tests should catch any breakages caused by changes to files in\n+git-std-lib.a (i.e. introduction of a out of scope dependency) and new\n+functions introduced to git-std-lib.a will require unit tests written\n+for them.\n+\n+[1] https://lore.kernel.org/git/xmqqwn17sydw.fsf@gitster.g/\n\\ No newline at end of file\ndiff --git a/Makefile b/Makefile\nindex e9ad9f9ef1..255bd10b82 100644\n--- a/Makefile\n+++ b/Makefile\n@@ -2162,6 +2162,11 @@ ifdef FSMONITOR_OS_SETTINGS\n \tCOMPAT_OBJS += compat/fsmonitor/fsm-path-utils-$(FSMONITOR_OS_SETTINGS).o\n endif\n \n+ifdef GIT_STD_LIB\n+\tBASIC_CFLAGS += -DGIT_STD_LIB\n+\tBASIC_CFLAGS += -DNO_GETTEXT\n+endif\n+\n ifeq ($(TCLTK_PATH),)\n NO_TCLTK = NoThanks\n endif\n@@ -3654,7 +3659,7 @@ clean: profile-clean coverage-clean cocciclean\n \t$(RM) po/git.pot po/git-core.pot\n \t$(RM) git.res\n \t$(RM) $(OBJECTS)\n-\t$(RM) $(LIB_FILE) $(XDIFF_LIB) $(REFTABLE_LIB) $(REFTABLE_TEST_LIB)\n+\t$(RM) $(LIB_FILE) $(XDIFF_LIB) $(REFTABLE_LIB) $(REFTABLE_TEST_LIB) $(STD_LIB_FILE)\n \t$(RM) $(ALL_PROGRAMS) $(SCRIPT_LIB) $(BUILT_INS) $(OTHER_PROGRAMS)\n \t$(RM) $(TEST_PROGRAMS)\n \t$(RM) $(FUZZ_PROGRAMS)\n@@ -3834,3 +3839,24 @@ $(FUZZ_PROGRAMS): all\n \t\t$(XDIFF_OBJS) $(EXTLIBS) git.o $@.o $(LIB_FUZZING_ENGINE) -o $@\n \n fuzz-all: $(FUZZ_PROGRAMS)\n+\n+### Libified Git rules\n+\n+# git-std-lib\n+# `make git-std-lib GIT_STD_LIB=YesPlease`\n+STD_LIB = git-std-lib.a\n+\n+GIT_STD_LIB_OBJS += abspath.o\n+GIT_STD_LIB_OBJS += ctype.o\n+GIT_STD_LIB_OBJS += date.o\n+GIT_STD_LIB_OBJS += hex-ll.o\n+GIT_STD_LIB_OBJS += parse.o\n+GIT_STD_LIB_OBJS += strbuf.o\n+GIT_STD_LIB_OBJS += usage.o\n+GIT_STD_LIB_OBJS += utf8.o\n+GIT_STD_LIB_OBJS += wrapper.o\n+\n+$(STD_LIB): $(GIT_STD_LIB_OBJS) $(COMPAT_OBJS)\n+\t$(QUIET_AR)$(RM) $@ && $(AR) $(ARFLAGS) $@ $^\n+\n+git-std-lib: $(STD_LIB)\ndiff --git a/git-compat-util.h b/git-compat-util.h\nindex 481dac22b0..75aa9b263e 100644\n--- a/git-compat-util.h\n+++ b/git-compat-util.h\n@@ -396,8 +396,8 @@ static inline int noop_core_config(const char *var UNUSED,\n #define platform_core_config noop_core_config\n #endif\n \n+#if !defined(__MINGW32__) && !defined(_MSC_VER) && !defined(GIT_STD_LIB)\n int lstat_cache_aware_rmdir(const char *path);\n-#if !defined(__MINGW32__) && !defined(_MSC_VER)\n #define rmdir lstat_cache_aware_rmdir\n #endif\n \n@@ -787,9 +787,11 @@ const char *inet_ntop(int af, const void *src, char *dst, size_t size);\n #endif\n \n #ifdef NO_PTHREADS\n+#ifdef GIT_STD_LIB\n #define atexit git_atexit\n int git_atexit(void (*handler)(void));\n #endif\n+#endif\n \n /*\n * Limit size of IO chunks, because huge chunks only cause pain. OS X\n@@ -951,14 +953,17 @@ int git_access(const char *path, int mode);\n # endif\n #endif\n \n+#ifndef GIT_STD_LIB\n int cmd_main(int, const char **);\n \n /*\n * Intercept all calls to exit() and route them to trace2 to\n * optionally emit a message before calling the real exit().\n */\n+\n int common_exit(const char *file, int line, int code);\n #define exit(code) exit(common_exit(__FILE__, __LINE__, (code)))\n+#endif\n \n /*\n * You can mark a stack variable with UNLEAK(var) to avoid it being\ndiff --git a/symlinks.c b/symlinks.c\nindex b29e340c2d..bced721a0c 100644\n--- a/symlinks.c\n+++ b/symlinks.c\n@@ -337,6 +337,7 @@ void invalidate_lstat_cache(void)\n \treset_lstat_cache(&default_cache);\n }\n \n+#if !defined(__MINGW32__) && !defined(_MSC_VER) && !defined(GIT_STD_LIB)\n #undef rmdir\n int lstat_cache_aware_rmdir(const char *path)\n {\n@@ -348,3 +349,4 @@ int lstat_cache_aware_rmdir(const char *path)\n \n \treturn ret;\n }\n+#endif\ndiff --git a/usage.c b/usage.c\nindex 09f0ed509b..58994e0d5c 100644\n--- a/usage.c\n+++ b/usage.c\n@@ -5,7 +5,15 @@\n */\n #include \"git-compat-util.h\"\n #include \"gettext.h\"\n+\n+#ifdef GIT_STD_LIB\n+#undef trace2_cmd_name\n+#undef trace2_cmd_error_va\n+#define trace2_cmd_name(x) \n+#define trace2_cmd_error_va(x, y)\n+#else\n #include \"trace2.h\"\n+#endif\n \n static void vreportf(const char *prefix, const char *err, va_list params)\n {\n-- \n2.41.0.162.gfafddb0af9-goog\n\n"},{"id":"478938","messageId":"xmqq5y78fr7c.fsf@gitster.g","threadId":"59921","inReplyTo":"20230627195251.1973421-6-calvinwan@google.com","subject":"Re: [RFC PATCH 5/8] parse: create new library for parsing strings and env values","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2023-06-27T22:58:15Z","receivedAt":"2023-06-27T22:58:22Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Calvin Wan writes:\n\n> While string and environment value parsing is mainly consumed by\n> config.c, there are other files that only need parsing functionality and\n> not config functionality. By separating out string and environment value\n> parsing from config, those files can instead be dependent on parse,\n> which has a much smaller dependency chain than config.\n>\n> Move general string and env parsing functions from config.[ch] to\n> parse.[ch].\n\nQuite sensible and ...\n\n>\n> Signed-off-by: Calvin Wan \n> ---\n> Makefile | 1 +\n> attr.c | 2 +-\n> config.c | 180 +-----------------------------------\n\n... long overdue to have this.\n\n> config.h | 14 +--\n> pack-objects.c | 2 +-\n> pack-revindex.c | 2 +-\n> parse-options.c | 3 +-\n> parse.c | 182 +++++++++++++++++++++++++++++++++++++\n> parse.h | 20 ++++\n> pathspec.c | 2 +-\n> preload-index.c | 2 +-\n> progress.c | 2 +-\n> prompt.c | 2 +-\n> rebase.c | 2 +-\n> t/helper/test-env-helper.c | 2 +-\n> unpack-trees.c | 2 +-\n> wrapper.c | 2 +-\n> write-or-die.c | 2 +-\n> 18 files changed, 219 insertions(+), 205 deletions(-)\n> create mode 100644 parse.c\n> create mode 100644 parse.h\n\nIt is somewhat surprising and very pleasing to see so many *.c files\nhad and now can lose dependency on . Very nice.\n\n"},{"id":"478939","messageId":"xmqq1qhwfr46.fsf@gitster.g","threadId":"59921","inReplyTo":"20230627195251.1973421-7-calvinwan@google.com","subject":"Re: [RFC PATCH 6/8] pager: remove pager_in_use()","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2023-06-27T23:00:09Z","receivedAt":"2023-06-27T23:00:19Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Calvin Wan writes:\n\n> pager_in_use() is simply a wrapper around\n> git_env_bool(\"GIT_PAGER_IN_USE\", 0). Other places that call\n> git_env_bool() in this fashion also do not have a wrapper function\n> around it. By removing pager_in_use(), we can also get rid of the\n> pager.h dependency from a few files.\n>\n> Signed-off-by: Calvin Wan \n> ---\n> builtin/log.c | 2 +-\n> color.c | 2 +-\n> column.c | 2 +-\n> date.c | 4 ++--\n> git.c | 2 +-\n> pager.c | 5 -----\n> pager.h | 1 -\n> 7 files changed, 6 insertions(+), 12 deletions(-)\n\nWith so many (read: more than 3) callsites, I am not sure if this is\nan improvement. pager_in_use() cannot be misspelt without getting\nnoticed by compilers, but git_env_bool(\"GIT_PAGOR_IN_USE\", 0) will\ngo silently unnoticed. Is there no other way to lose the dependency\nyou do not like?\n\n"},{"id":"478940","messageId":"CAFySSZCpEy1ZF5b9kjbxfh7ANiYbVO7W1t=oazAJzGtOZofyew@mail.gmail.com","threadId":"59921","inReplyTo":"xmqq1qhwfr46.fsf@gitster.g","subject":"Re: [RFC PATCH 6/8] pager: remove pager_in_use()","fromName":"Calvin Wan","fromEmail":"calvinwan@google.com","sentAt":"2023-06-27T23:18:54Z","receivedAt":"2023-06-27T23:19:09Z","isPatch":true,"sender":{"key":"calvinwan@google.com","avatar":"https://avatars.githubusercontent.com/u/92547554?v=4"},"body":"> With so many (read: more than 3) callsites, I am not sure if this is\n> an improvement. pager_in_use() cannot be misspelt without getting\n> noticed by compilers, but git_env_bool(\"GIT_PAGOR_IN_USE\", 0) will\n> go silently unnoticed. Is there no other way to lose the dependency\n> you do not like?\n\nI thought about only changing this call site, but that creates an\ninconsistency that shouldn't exist. The other way is to move this\nfunction into a different file, but it is also unclear to me which\nfile that would be. It would be awkward in parse.c and if it was in\nenvironment.c then we would have many more inherited dependencies from\nthat. I agree that the value of this patch is dubious in and of\nitself, which is why it's coupled together with this series rather\nthan in a separate standalone cleanup series.\n"},{"id":"478942","messageId":"kl6lzg4kqw7v.fsf@chooglen-macbookpro.roam.corp.google.com","threadId":"59921","inReplyTo":"20230627195251.1973421-1-calvinwan@google.com","subject":"Re: [RFC PATCH 0/8] Introduce Git Standard Library","fromName":"Glen Choo","fromEmail":"chooglen@google.com","sentAt":"2023-06-28T00:14:28Z","receivedAt":"2023-06-28T00:14:34Z","isPatch":true,"sender":{"key":"glencbz@gmail.com","avatar":"https://avatars.githubusercontent.com/u/58092771?v=4"},"body":"I see that this doesn't apply cleanly to 'master'. Do you have a base\ncommit that reviewers can easily apply this to?\n\nCalvin Wan writes:\n\n> Before looking at this series, it probably makes sense to look at the\n> other series that this is built on top of since that is the state I will\n> be referring to in this cover letter:\n>\n> - Elijah's final cache.h cleanup series[2]\n> - my strbuf cleanup series[3]\n> - my git-compat-util cleanup series[4]\n\nUnfortunately, not all of these series apply cleanly to 'master' either,\nso I went digging for the topic branches, which I think are:\n\n- en/header-split-cache-h-part-3\n- cw/header-compat-util-shuffle\n- cw/strbuf-cleanup\n\nAnd then I tried merging them, but it looks like they don't merge\ncleanly either :/\n\n(Btw Junio, I think cw/header-compat-util-shuffle didn't get called out\nin What's Cooking.)\n\n> [2] https://lore.kernel.org/git/pull.1525.v3.git.1684218848.gitgitgadget@gmail.com/\n> [3] https://lore.kernel.org/git/20230606194720.2053551-1-calvinwan@google.com/\n> [4] https://lore.kernel.org/git/20230606170711.912972-1-calvinwan@google.com/\n"},{"id":"478943","messageId":"kl6lv8f8qvhd.fsf@chooglen-macbookpro.roam.corp.google.com","threadId":"59921","inReplyTo":"xmqq1qhwfr46.fsf@gitster.g","subject":"Re: [RFC PATCH 6/8] pager: remove pager_in_use()","fromName":"Glen Choo","fromEmail":"chooglen@google.com","sentAt":"2023-06-28T00:30:22Z","receivedAt":"2023-06-28T00:30:28Z","isPatch":true,"sender":{"key":"glencbz@gmail.com","avatar":"https://avatars.githubusercontent.com/u/58092771?v=4"},"body":"Junio C Hamano writes:\n\n>> pager_in_use() is simply a wrapper around\n>> git_env_bool(\"GIT_PAGER_IN_USE\", 0). Other places that call\n>> git_env_bool() in this fashion also do not have a wrapper function\n>> around it. By removing pager_in_use(), we can also get rid of the\n>> pager.h dependency from a few files.\n>\n> With so many (read: more than 3) callsites, I am not sure if this is\n> an improvement. pager_in_use() cannot be misspelt without getting\n> noticed by compilers, but git_env_bool(\"GIT_PAGOR_IN_USE\", 0) will\n> go silently unnoticed. Is there no other way to lose the dependency\n> you do not like?\n\nHaving the function isn't just nice for typo prevention - it's also a\nreasonable boundary around the pager subsystem. We could imagine a\nworld where we wanted to track the pager status using a static\nvar instead of an env var (not that we'd even want that :P), and this\ninlining makes that harder.\n\nFrom the cover letter, it seems like we only need this to remove\n\"#include pager.h\" from date.c, and that's only used in\nparse_date_format(). Could we add a is_pager/pager_in_use to that\nfunction and push the pager.h dependency upwards?\n"},{"id":"478946","messageId":"05df2971-0f26-6da3-093b-7de5d7a5dbca@github.com","threadId":"59921","inReplyTo":"20230627195251.1973421-2-calvinwan@google.com","subject":"Re: [RFC PATCH 1/8] trace2: log fsync stats in trace2 rather than wrapper","fromName":"Victoria Dye","fromEmail":"vdye@github.com","sentAt":"2023-06-28T02:05:47Z","receivedAt":"2023-06-28T02:05:56Z","isPatch":true,"sender":{"key":"vdye@github.com","avatar":"https://avatars.githubusercontent.com/u/3619353?v=4"},"body":"Calvin Wan wrote:\n> As a library boundary, wrapper.c should not directly log trace2\n> statistics, but instead provide those statistics upon\n> request. Therefore, move the trace2 logging code to trace2.[ch.]. This\n> also allows wrapper.c to not be dependent on trace2.h and repository.h.\n> \n\n...\n\n> diff --git a/trace2.h b/trace2.h\n> index 4ced30c0db..689e9a4027 100644\n> --- a/trace2.h\n> +++ b/trace2.h\n> @@ -581,4 +581,9 @@ void trace2_collect_process_info(enum trace2_process_info_reason reason);\n> \n> const char *trace2_session_id(void);\n> \n> +/*\n> + * Writes out trace statistics for fsync\n> + */\n> +void trace_git_fsync_stats(void);\n> +\n\nThis function does not belong in 'trace2.h', IMO. The purpose of that file\nis to contain the generic API for Trace2 (e.g., 'trace2_printf()',\n'trace2_region_(enter|exit)'), whereas this function is effectively a\nwrapper around a specific invocation of that API. \n\nYou note in the commit message that \"wrapper.c should not directly log\ntrace2 statistics\" with the reasoning of \"[it's] a library boundary,\" but I\nsuspect the unstated underlying reason is \"because it tracks 'count_fsync_*'\nin static variables.\" This case would be better handled, then, by replacing\nthe usage in 'wrapper.c' with a new Trace2 counter (API introduced in [1]).\nThat keeps this usage consistent with the API already established for\nTrace2, rather than starting an unsustainable trend of creating ad-hoc,\nper-metric wrappers in 'trace2.[c|h]'.\n\nAn added note re: the commit message - it's extremely important that\nfunctions _anywhere in Git_ are able to use the Trace2 API directly. A\ndeveloper could reasonably want to measure performance, keep track of an\ninteresting metric, log when a region is entered in the larger trace,\ncapture error information, etc. for any function, regardless of where in\nfalls in the internal library organization. To that end, I think either the\ncommit message should be rephrased to remove that statement (if the issue is\nreally \"we're using a static variable and we want to avoid that\"), or the\nlibification effort should be updated to accommodate use of Trace2 anywhere\nin Git. \n\n[1] https://lore.kernel.org/git/pull.1373.v4.git.1666618868.gitgitgadget@gmail.com/\n\n"},{"id":"478958","messageId":"e08d9319-9a4f-bf16-242f-d976aeae32bc@gmail.com","threadId":"59921","inReplyTo":"20230627195251.1973421-3-calvinwan@google.com","subject":"Re: [RFC PATCH 2/8] hex-ll: split out functionality from hex","fromName":"Phillip Wood","fromEmail":"phillip.wood123@gmail.com","sentAt":"2023-06-28T13:15:08Z","receivedAt":"2023-06-28T13:15:18Z","isPatch":true,"sender":{"key":"phillip.wood@dunelm.org.uk","avatar":null},"body":"Hi Calvin\n\nOn 27/06/2023 20:52, Calvin Wan wrote:\n> Separate out hex functionality that doesn't require a hash algo into\n> hex-ll.[ch]. Since the hash algo is currently a global that sits in\n> repository, this separation removes that dependency for files that only\n> need basic hex manipulation functions.\n>\n> diff --git a/hex.h b/hex.h\n> index 7df4b3c460..c07c8b34c2 100644\n> --- a/hex.h\n> +++ b/hex.h\n> @@ -2,22 +2,7 @@\n> #define HEX_H\n> \n> #include \"hash-ll.h\"\n> -\n> -extern const signed char hexval_table[256];\n> -static inline unsigned int hexval(unsigned char c)\n> -{\n> -\treturn hexval_table[c];\n> -}\n> -\n> -/*\n> - * Convert two consecutive hexadecimal digits into a char. Return a\n> - * negative value on error. Don't run over the end of short strings.\n> - */\n> -static inline int hex2chr(const char *s)\n> -{\n> -\tunsigned int val = hexval(s[0]);\n> -\treturn (val & ~0xf) ? val : (val << 4) | hexval(s[1]);\n> -}\n> +#include \"hex-ll.h\"\n\nI don't think any of the remaining declarations in hex.h depend on the \nones that are moved to \"hex-ll.h\" so this include should probably be in \n\"hex.c\" rather than \"hex.h\"\n\nBest Wishes\n\nPhillip\n\n"},{"id":"478959","messageId":"d77f99df-f50a-39fa-a7c5-1bb958f196fb@gmail.com","threadId":"59921","inReplyTo":"20230627195251.1973421-8-calvinwan@google.com","subject":"Re: [RFC PATCH 7/8] git-std-lib: introduce git standard library","fromName":"Phillip Wood","fromEmail":"phillip.wood123@gmail.com","sentAt":"2023-06-28T13:27:04Z","receivedAt":"2023-06-28T13:28:46Z","isPatch":true,"sender":{"key":"phillip.wood@dunelm.org.uk","avatar":null},"body":"Hi Calvin\n\nOn 27/06/2023 20:52, Calvin Wan wrote:\n> The Git Standard Library intends to serve as the foundational library\n> and root dependency that other libraries in Git will be built off of.\n> That is to say, suppose we have libraries X and Y; a user that wants to\n> use X and Y would need to include X, Y, and this Git Standard Library.\n\nI think having a library of commonly used functions and structures is a \ngood idea. While I appreciate that we don't want to include everything \nI'm surprised to see it does not include things like \"hashmap.c\" and \n\"string-list.c\" that will be required by the config library as well as \nother code in \"libgit.a\". I don't think we want \"libgitconfig.a\" and \n\"libgit.a\" to both contain a copy of \"hashmap.o\" and \"string-list.o\"\n\n> diff --git a/Makefile b/Makefile\n> index e9ad9f9ef1..255bd10b82 100644\n> --- a/Makefile\n> +++ b/Makefile\n> @@ -2162,6 +2162,11 @@ ifdef FSMONITOR_OS_SETTINGS\n> \tCOMPAT_OBJS += compat/fsmonitor/fsm-path-utils-$(FSMONITOR_OS_SETTINGS).o\n> endif\n> \n> +ifdef GIT_STD_LIB\n> +\tBASIC_CFLAGS += -DGIT_STD_LIB\n> +\tBASIC_CFLAGS += -DNO_GETTEXT\n\nI can see other projects may want to build git-std-lib without gettext \nsupport but if we're going to use git-std-lib within git it needs to be \nable to be built with that support. The same goes for the trace \nfunctions that you are redefining in usage.h\n\n> diff --git a/git-compat-util.h b/git-compat-util.h\n> index 481dac22b0..75aa9b263e 100644\n> --- a/git-compat-util.h\n> +++ b/git-compat-util.h\n> @@ -396,8 +396,8 @@ static inline int noop_core_config(const char *var UNUSED,\n> #define platform_core_config noop_core_config\n> #endif\n> \n> +#if !defined(__MINGW32__) && !defined(_MSC_VER) && !defined(GIT_STD_LIB)\n> int lstat_cache_aware_rmdir(const char *path);\n> -#if !defined(__MINGW32__) && !defined(_MSC_VER)\n> #define rmdir lstat_cache_aware_rmdir\n> #endif\n\nI'm not sure why the existing condition is being moved here\n\nThanks for posting this RFC. I've only really given it a quick glance \nbut on the whole it seems to make sense.\n\nBest Wishes\n\nPhillip\n\n"},{"id":"478963","messageId":"CAFySSZDkc5xN5cNz0UyKyKpS8Eq2y_517hrLe8HixzAwQ+E4Dg@mail.gmail.com","threadId":"59921","inReplyTo":"kl6lzg4kqw7v.fsf@chooglen-macbookpro.roam.corp.google.com","subject":"Re: [RFC PATCH 0/8] Introduce Git Standard Library","fromName":"Calvin Wan","fromEmail":"calvinwan@google.com","sentAt":"2023-06-28T16:30:05Z","receivedAt":"2023-06-28T16:31:36Z","isPatch":true,"sender":{"key":"calvinwan@google.com","avatar":"https://avatars.githubusercontent.com/u/92547554?v=4"},"body":"Ah I failed to mention that this is built on top of 2.41. You can also\nget this series with the correctly applied patches from:\nhttps://github.com/calvin-wan-google/git/tree/git-std-lib-rfc\n"},{"id":"478964","messageId":"kl6lpm5fr19f.fsf@chooglen-macbookpro.roam.corp.google.com","threadId":"59921","inReplyTo":"kl6lv8f8qvhd.fsf@chooglen-macbookpro.roam.corp.google.com","subject":"Re: [RFC PATCH 6/8] pager: remove pager_in_use()","fromName":"Glen Choo","fromEmail":"chooglen@google.com","sentAt":"2023-06-28T16:37:48Z","receivedAt":"2023-06-28T16:37:55Z","isPatch":true,"sender":{"key":"glencbz@gmail.com","avatar":"https://avatars.githubusercontent.com/u/58092771?v=4"},"body":"Glen Choo writes:\n\n> Could we add a is_pager/pager_in_use to that\n> function and push the pager.h dependency upwards?\n\nBleh, I meant \"Could we add a new is_pager/pager_in_use parameter to\nthat function?\"\n"},{"id":"478966","messageId":"CAFySSZBBAatvBmfwn9vc=v2hdWX_0Q4g4txqLOwtqiFZFW7uiA@mail.gmail.com","threadId":"59921","inReplyTo":"kl6lpm5fr19f.fsf@chooglen-macbookpro.roam.corp.google.com","subject":"Re: [RFC PATCH 6/8] pager: remove pager_in_use()","fromName":"Calvin Wan","fromEmail":"calvinwan@google.com","sentAt":"2023-06-28T16:44:59Z","receivedAt":"2023-06-28T16:45:17Z","isPatch":true,"sender":{"key":"calvinwan@google.com","avatar":"https://avatars.githubusercontent.com/u/92547554?v=4"},"body":"> Glen Choo writes:\n>\n> > Could we add a is_pager/pager_in_use to that\n> > function and push the pager.h dependency upwards?\n>\n> Bleh, I meant \"Could we add a new is_pager/pager_in_use parameter to\n> that function?\"\n\nRefactoring the function signature to:\n\nparse_date_format(const char *format, struct date_mode *mode, int pager_in_use)\n\nas you suggested is a much better solution, thanks! I'll make that\nchange in the next reroll.\n"},{"id":"478967","messageId":"CAFySSZA8kC5mN5ZgAnz+RiBXKXQq=z4DZ01QPnmkD50EbQjeeQ@mail.gmail.com","threadId":"59921","inReplyTo":"e08d9319-9a4f-bf16-242f-d976aeae32bc@gmail.com","subject":"Re: [RFC PATCH 2/8] hex-ll: split out functionality from hex","fromName":"Calvin Wan","fromEmail":"calvinwan@google.com","sentAt":"2023-06-28T16:55:44Z","receivedAt":"2023-06-28T16:56:00Z","isPatch":true,"sender":{"key":"calvinwan@google.com","avatar":"https://avatars.githubusercontent.com/u/92547554?v=4"},"body":"> I don't think any of the remaining declarations in hex.h depend on the\n> ones that are moved to \"hex-ll.h\" so this include should probably be in\n> \"hex.c\" rather than \"hex.h\"\n\nThe reason why hex-ll.h is included in hex.h isn't because there might\nbe other declarations in hex.h that depend on it. It is for files that\ninclude hex.h to also inherit the inclusion of hex-ll.h. If we moved\nthe inclusion of hex-ll.h to hex.c rather than hex.h, then those files\nwould have to include both hex.h and hex-ll.h. It clarifies whether a\nfile needs all of hex or just the low level functionality of hex.\n"},{"id":"478970","messageId":"xmqqr0pvcx58.fsf@gitster.g","threadId":"59921","inReplyTo":"CAFySSZBBAatvBmfwn9vc=v2hdWX_0Q4g4txqLOwtqiFZFW7uiA@mail.gmail.com","subject":"Re: [RFC PATCH 6/8] pager: remove pager_in_use()","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2023-06-28T17:30:27Z","receivedAt":"2023-06-28T17:30:41Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Calvin Wan writes:\n\n>> Glen Choo writes:\n>>\n>> > Could we add a is_pager/pager_in_use to that\n>> > function and push the pager.h dependency upwards?\n>>\n>> Bleh, I meant \"Could we add a new is_pager/pager_in_use parameter to\n>> that function?\"\n>\n> Refactoring the function signature to:\n>\n> parse_date_format(const char *format, struct date_mode *mode, int pager_in_use)\n>\n> as you suggested is a much better solution, thanks! I'll make that\n> change in the next reroll.\n\nYeah, the date format \"auto:\" that changes behaviour between the\noutput medium feels a serious layering violation, but given the\nconstraints, it looks like the best thing to do.\n\nThanks.\n"},{"id":"478990","messageId":"xmqqfs6bcnic.fsf@gitster.g","threadId":"59921","inReplyTo":"kl6lv8f8qvhd.fsf@chooglen-macbookpro.roam.corp.google.com","subject":"Re: [RFC PATCH 6/8] pager: remove pager_in_use()","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2023-06-28T20:58:35Z","receivedAt":"2023-06-28T20:58:42Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Glen Choo writes:\n\n> Having the function isn't just nice for typo prevention - it's also a\n> reasonable boundary around the pager subsystem. We could imagine a\n> world where we wanted to track the pager status using a static\n> var instead of an env var (not that we'd even want that :P), and this\n> inlining makes that harder.\n>\n> From the cover letter, it seems like we only need this to remove\n> \"#include pager.h\" from date.c, and that's only used in\n> parse_date_format(). Could we add a is_pager/pager_in_use to that\n> function and push the pager.h dependency upwards?\n\nThanks---I think that may show a good direction. parse_date_format()\nreacts to \"auto:foo\" and as long as that feature needs to be there,\npager_in_use() must be available to the function.\n"},{"id":"478992","messageId":"CAFySSZBMng9nEdCkuT5+fc6rfFgaFfU2E0NP3=jUQC1yRcUE6Q@mail.gmail.com","threadId":"59921","inReplyTo":"d77f99df-f50a-39fa-a7c5-1bb958f196fb@gmail.com","subject":"Re: [RFC PATCH 7/8] git-std-lib: introduce git standard library","fromName":"Calvin Wan","fromEmail":"calvinwan@google.com","sentAt":"2023-06-28T21:15:38Z","receivedAt":"2023-06-28T21:21:59Z","isPatch":true,"sender":{"key":"calvinwan@google.com","avatar":"https://avatars.githubusercontent.com/u/92547554?v=4"},"body":"> On 27/06/2023 20:52, Calvin Wan wrote:\n> > The Git Standard Library intends to serve as the foundational library\n> > and root dependency that other libraries in Git will be built off of.\n> > That is to say, suppose we have libraries X and Y; a user that wants to\n> > use X and Y would need to include X, Y, and this Git Standard Library.\n>\n> I think having a library of commonly used functions and structures is a\n> good idea. While I appreciate that we don't want to include everything\n> I'm surprised to see it does not include things like \"hashmap.c\" and\n> \"string-list.c\" that will be required by the config library as well as\n> other code in \"libgit.a\". I don't think we want \"libgitconfig.a\" and\n> \"libgit.a\" to both contain a copy of \"hashmap.o\" and \"string-list.o\"\n\nI chose not to include hashmap and string-list in git-std-lib.a in the\nfirst pass since they can exist as libraries built on top of\ngit-std-lib.a. There is no harm starting off with more libraries than\nfewer besides having something like the config library be dependent on\nlib-hashmap.a, lib-string-list.a, and git-std-lib.a rather than only\ngit-std-lib.a. They can always be added into git-std-lib.a in the\nfuture. That being said, I do find it extremely unlikely that someone\nwould want to swap out the implementation for hashmap or string-list\nso it is also very reasonable to include them into git-std-lib.a\n\n>\n> > diff --git a/Makefile b/Makefile\n> > index e9ad9f9ef1..255bd10b82 100644\n> > --- a/Makefile\n> > +++ b/Makefile\n> > @@ -2162,6 +2162,11 @@ ifdef FSMONITOR_OS_SETTINGS\n> > COMPAT_OBJS += compat/fsmonitor/fsm-path-utils-$(FSMONITOR_OS_SETTINGS).o\n> > endif\n> >\n> > +ifdef GIT_STD_LIB\n> > + BASIC_CFLAGS += -DGIT_STD_LIB\n> > + BASIC_CFLAGS += -DNO_GETTEXT\n>\n> I can see other projects may want to build git-std-lib without gettext\n> support but if we're going to use git-std-lib within git it needs to be\n> able to be built with that support. The same goes for the trace\n> functions that you are redefining in usage.h\n\nTaking a closer look at gettext.[ch], I believe I can also include it\ninto git-std-lib.a with a couple of minor changes. I'm currently\nthinking about how the trace functions should interact with\ngit-std-lib.a since Victoria had similar comments on patch 1. I'll\nreply to that thread when I come up with an answer.\n\n>\n> > diff --git a/git-compat-util.h b/git-compat-util.h\n> > index 481dac22b0..75aa9b263e 100644\n> > --- a/git-compat-util.h\n> > +++ b/git-compat-util.h\n> > @@ -396,8 +396,8 @@ static inline int noop_core_config(const char *var UNUSED,\n> > #define platform_core_config noop_core_config\n> > #endif\n> >\n> > +#if !defined(__MINGW32__) && !defined(_MSC_VER) && !defined(GIT_STD_LIB)\n> > int lstat_cache_aware_rmdir(const char *path);\n> > -#if !defined(__MINGW32__) && !defined(_MSC_VER)\n> > #define rmdir lstat_cache_aware_rmdir\n> > #endif\n>\n> I'm not sure why the existing condition is being moved here\n\nAh I see that this changes behavior for callers of\nlstat_cache_aware_rmdir if those conditions aren't satisfied. I\nshould've added an extra #if for GIT_STD_LIB instead of adding it to\nthe end of the current check and moving it. Thanks for spotting this.\n"},{"id":"479044","messageId":"owly4jmp8mdf.fsf@fine.c.googlers.com","threadId":"59921","inReplyTo":"20230627195251.1973421-1-calvinwan@google.com","subject":"Re: [RFC PATCH 0/8] Introduce Git Standard Library","fromName":"Linus Arver","fromEmail":"linusa@google.com","sentAt":"2023-06-30T07:01:16Z","receivedAt":"2023-06-30T07:01:23Z","isPatch":true,"sender":{"key":"linus@ucla.edu","avatar":null},"body":"Hello Calvin,\n\nCalvin Wan writes:\n> With our current method of building Git, we can imagine the dependency\n> graph as such:\n>\n> Git\n> /\\\n> / \\\n> / \\\n> libgit.a ext deps\n>\n> In libifying parts of Git, we want to shrink the dependency graph to\n> only the minimal set of dependencies, so libraries should not use\n> libgit.a. Instead, it would look like:\n>\n> Git\n> /\\\n> / \\\n> / \\\n> libgit.a ext deps\n> /\\\n> / \\\n> / \\\n> object-store.a (other lib)\n> | /\n> | /\n> | /\n> config.a / \n> | /\n> | /\n> | /\n> git-std-lib.a\n>\n> Instead of containing all of the objects in Git, libgit.a would contain\n> objects that are not built by libraries it links against. Consequently,\n> if someone wanted their own custom build of Git with their own custom\n> implementation of the object store, they would only have to swap out\n> object-store.a rather than do a hard fork of Git.\n\nWhat about the case where someone wants to build program Foo which just\npulls in some bits of Git? For example, I am thinking of trailer.[ch]\nwhich could be refactored to expose a public API. Then the Foo program\ncould pull this public trailer manipulation API in as a library\ndependency (so that Foo can parse trailers in commit messages without\nre-implementing that logic in Foo's own codebase). With the proposed Git\nStandard Library (GSL) model above, would my Foo program also have to\npull in GSL? If so, isn't this onerous because of the additional bloat?\nThe Foo developers just want the banana, not the gorilla holding the\nbanana in the jungle, so to speak.\n\n> Rationale behind Git Standard Library\n> ================\n>\n> The rationale behind Git Standard Library essentially is the result of\n> two observations within the Git codebase: every file includes\n> git-compat-util.h which defines functions in a couple of different\n> files, and wrapper.c + usage.c have difficult-to-separate circular\n> dependencies with each other and other files.\n>\n> Ubiquity of git-compat-util.h and circular dependencies\n> ========\n>\n> Every file in the Git codebase includes git-compat-util.h. It serves as\n> \"a compatibility aid that isolates the knowledge of platform specific\n> inclusion order and what feature macros to define before including which\n> system header\" (Junio[5]). Since every file includes git-compat-util.h, and\n> git-compat-util.h includes wrapper.h and usage.h, it would make sense\n> for wrapper.c and usage.c to be a part of the root library. They have\n> difficult to separate circular dependencies with each other so they\n\ns/difficult to separate/difficult-to-separate\n\n> can't be independent libraries. Wrapper.c has dependencies on parse.c,\n> abspath.c, strbuf.c, which in turn also have dependencies on usage.c and\n> wrapper.c -- more circular dependencies. \n>\n> Tradeoff between swappability and refactoring\n> ========\n>\n> From the above dependency graph, we can see that git-std-lib.a could be\n> many smaller libraries rather than a singular library. So why choose a\n> singular library when multiple libraries can be individually easier to\n> swap and are more modular? A singular library requires less work to\n> separate out circular dependencies within itself so it becomes a\n> tradeoff question between work and reward. While there may be a point in\n> the future where a file like usage.c would want its own library so that\n> someone can have custom die() or error(), the work required to refactor\n> out the circular dependencies in some files would be enormous due to\n> their ubiquity so therefore I believe it is not worth the tradeoff\n> currently. Additionally, we can in the future choose to do this refactor\n> and change the API for the library if there becomes enough of a reason\n> to do so (remember we are avoiding promising stability of the interfaces\n> of those libraries).\n\nWould getting us down the currently proposed path make it even more\ndifficult to do this refactor? If so, I think it's worth mentioning.\n\n> Reuse of compatibility functions in git-compat-util.h\n> ========\n>\n> Most functions defined in git-compat-util.h are implemented in compat/\n> and have dependencies limited to strbuf.h and wrapper.h so they can be\n> easily included in git-std-lib.a, which as a root dependency means that\n> higher level libraries do not have to worry about compatibility files in\n> compat/. The rest of the functions defined in git-compat-util.h are\n> implemented in top level files and, in this patch set, are hidden behind\n> an #ifdef if their implementation is not in git-std-lib.a.\n>\n> Rationale summary\n> ========\n>\n> The Git Standard Library allows us to get the libification ball rolling\n> with other libraries in Git (such as Glen's removal of global state from\n> config iteration[6] prepares a config library). By not spending many\n> more months attempting to refactor difficult circular dependencies and\n> instead spending that time getting to a state where we can test out\n> swapping a library out such as config or object store, we can prove the\n> viability of Git libification on a much faster time scale. Additionally\n> the code cleanups that have happened so far have been minor and\n> beneficial for the codebase. It is probable that making large movements\n> would negatively affect code clarity.\n\nIt sounds like the circular dependencies are so difficult to untangle that they\nare the primary motivation behind grouping these tightly-coupled libraries\ntogether into the Git Standard Library (GSL) banner. Still, I think it would\nhelp reviewers if you explain what tradeoffs we are making by accepting the\ncircular dependencies as they are instead of untangling them. Conversely, if we\nassume that there are no circular dependencies, what kind of benefits do we get\nwhen designing the GSL from this (improved) position? Would there be little to\nno additional benefits? If so, then I think it would be easier to support the\ncurrent approach (as removing the circularities would not give us significant\nadvantages for libification).\n\n> Git Standard Library boundary\n> ================\n>\n> While I have described above some useful heuristics for identifying\n> potential candidates for git-std-lib.a, a standard library should not\n> have a shaky definition for what belongs in it.\n>\n> - Low-level files (aka operates only on other primitive types) that are\n> used everywhere within the codebase (wrapper.c, usage.c, strbuf.c)\n> - Dependencies that are low-level and widely used\n> (abspath.c, date.c, hex-ll.c, parse.c, utf8.c)\n> - low-level git/* files with functions defined in git-compat-util.h\n> (ctype.c)\n> - compat/*\n\nI'm confused. Is the list above an example of a shaky definition, or the\nopposite? IOW, do you mean that the list above should be the initial set\nof content to include in the GSL? Or _not_ to include?\n\n> Series structure\n> ================\n>\n> While my strbuf and git-compat-util series can stand alone, they also\n> function as preparatory patches for this series. There are more cleanup\n> patches in this series, but since most of them have marginal benefits\n> probably not worth the churn on its own, I decided not to split them\n> into a separate series like with strbuf and git-compat-util. As an RFC,\n> I am looking for comments on whether the rationale behind git-std-lib\n> makes sense as well as whether there are better ways to build and enable\n> git-std-lib in patch 7, specifically regarding Makefile rules and the\n> usage of ifdef's to stub out certain functions and headers. \n\nIf the cleanups are independent I think it would be simpler to put them\nin a separate series.\n\nIn general, I think the doc would make a stronger case if it expanded\nthe discussions around alternative approaches to the one proposed, with\nthe reasons why they were rejected.\n\nMinor nits:\n- Documentation/technical/git-std-lib.txt: (style) prefer \"we\" over \"I\" (\"we\n believe\" instead of \"I believe\").\n- There are some \"\\ No newline at end of file\" warnings in this series.\n\nThanks,\nLinus\n"},{"id":"479047","messageId":"3029e4c8-5b42-5193-f9ce-7f4bb22fccab@gmail.com","threadId":"59921","inReplyTo":"CAFySSZBMng9nEdCkuT5+fc6rfFgaFfU2E0NP3=jUQC1yRcUE6Q@mail.gmail.com","subject":"Re: [RFC PATCH 7/8] git-std-lib: introduce git standard library","fromName":"Phillip Wood","fromEmail":"phillip.wood123@gmail.com","sentAt":"2023-06-30T10:00:52Z","receivedAt":"2023-06-30T10:01:20Z","isPatch":true,"sender":{"key":"phillip.wood@dunelm.org.uk","avatar":null},"body":"Hi Calvin\n\nOn 28/06/2023 22:15, Calvin Wan wrote:\n>> On 27/06/2023 20:52, Calvin Wan wrote:\n>>> The Git Standard Library intends to serve as the foundational library\n>>> and root dependency that other libraries in Git will be built off of.\n>>> That is to say, suppose we have libraries X and Y; a user that wants to\n>>> use X and Y would need to include X, Y, and this Git Standard Library.\n>>\n>> I think having a library of commonly used functions and structures is a\n>> good idea. While I appreciate that we don't want to include everything\n>> I'm surprised to see it does not include things like \"hashmap.c\" and\n>> \"string-list.c\" that will be required by the config library as well as\n>> other code in \"libgit.a\". I don't think we want \"libgitconfig.a\" and\n>> \"libgit.a\" to both contain a copy of \"hashmap.o\" and \"string-list.o\"\n> \n> I chose not to include hashmap and string-list in git-std-lib.a in the\n> first pass since they can exist as libraries built on top of\n> git-std-lib.a. There is no harm starting off with more libraries than\n> fewer besides having something like the config library be dependent on\n> lib-hashmap.a, lib-string-list.a, and git-std-lib.a rather than only\n> git-std-lib.a. They can always be added into git-std-lib.a in the\n> future. That being said, I do find it extremely unlikely that someone\n> would want to swap out the implementation for hashmap or string-list\n> so it is also very reasonable to include them into git-std-lib.a\n\nFinding the right boundary for git-std-lib is a bit of a judgement call. \nWe certainly could have separate libraries for things like hashmap, \nstring-list, strvec, strmap and wildmatch but there is some overhead \nadding each one to the Makefile. I think their use is common enough that \nit would be continent to have them in git-std-lib but we can always add \nthem later.\n\n>>> diff --git a/Makefile b/Makefile\n>>> index e9ad9f9ef1..255bd10b82 100644\n>>> --- a/Makefile\n>>> +++ b/Makefile\n>>> @@ -2162,6 +2162,11 @@ ifdef FSMONITOR_OS_SETTINGS\n>>> COMPAT_OBJS += compat/fsmonitor/fsm-path-utils-$(FSMONITOR_OS_SETTINGS).o\n>>> endif\n>>>\n>>> +ifdef GIT_STD_LIB\n>>> + BASIC_CFLAGS += -DGIT_STD_LIB\n>>> + BASIC_CFLAGS += -DNO_GETTEXT\n>>\n>> I can see other projects may want to build git-std-lib without gettext\n>> support but if we're going to use git-std-lib within git it needs to be\n>> able to be built with that support. The same goes for the trace\n>> functions that you are redefining in usage.h\n> \n> Taking a closer look at gettext.[ch], I believe I can also include it\n> into git-std-lib.a with a couple of minor changes.\n\nThat's great\n\n> I'm currently\n> thinking about how the trace functions should interact with\n> git-std-lib.a since Victoria had similar comments on patch 1. I'll\n> reply to that thread when I come up with an answer.\n\nOne thought I had was to have a compile time flag so someone building \ngit-std-lib for an external project could build it with\n\n\tmake git-std-lib NO_TRACE2=YesPlease\n\nand then we'd either compile against a stub version of trace2 that does \nnothing or use some #define magic to get rid of the calls if that is not \ntoo invasive.\n\nBest Wishes\n\nPhillip\n\n"},{"id":"479216","messageId":"CAFySSZCYpxNcK6Tfagk1ZBs6cOT9XvWm2b3DJhN0fSPsQCWR1g@mail.gmail.com","threadId":"59921","inReplyTo":"05df2971-0f26-6da3-093b-7de5d7a5dbca@github.com","subject":"Re: [RFC PATCH 1/8] trace2: log fsync stats in trace2 rather than wrapper","fromName":"Calvin Wan","fromEmail":"calvinwan@google.com","sentAt":"2023-07-05T17:57:08Z","receivedAt":"2023-07-05T17:57:24Z","isPatch":true,"sender":{"key":"calvinwan@google.com","avatar":"https://avatars.githubusercontent.com/u/92547554?v=4"},"body":"> This function does not belong in 'trace2.h', IMO. The purpose of that file\n> is to contain the generic API for Trace2 (e.g., 'trace2_printf()',\n> 'trace2_region_(enter|exit)'), whereas this function is effectively a\n> wrapper around a specific invocation of that API.\n>\n> You note in the commit message that \"wrapper.c should not directly log\n> trace2 statistics\" with the reasoning of \"[it's] a library boundary,\" but I\n> suspect the unstated underlying reason is \"because it tracks 'count_fsync_*'\n> in static variables.\" This case would be better handled, then, by replacing\n> the usage in 'wrapper.c' with a new Trace2 counter (API introduced in [1]).\n> That keeps this usage consistent with the API already established for\n> Trace2, rather than starting an unsustainable trend of creating ad-hoc,\n> per-metric wrappers in 'trace2.[c|h]'.\n\nThe underlying reason is for removing the trace2 dependency from\nwrapper.c so that when git-std-lib is compiled, there isn't a missing\nobject for trace_git_fsync_stats(), resulting in a compilation error.\nHowever I do agree that the method I chose to do so by creating an\nad-hoc wrapper is unsustainable and I will come up with a better\nmethod for doing so.\n\n>\n> An added note re: the commit message - it's extremely important that\n> functions _anywhere in Git_ are able to use the Trace2 API directly. A\n> developer could reasonably want to measure performance, keep track of an\n> interesting metric, log when a region is entered in the larger trace,\n> capture error information, etc. for any function, regardless of where in\n> falls in the internal library organization.\n\nI don't quite agree that functions _anywhere in Git_ are able to use\nthe Trace2 API directly for the same reason that we don't have the\nability to log functions in external libraries -- logging common,\nlow-level functionality creates an unnecessary amount of log churn and\nthose logs generally contain practically useless information. However,\nthat does not mean that all of the functions in git-std-lib fall into\nthat category (usage has certain functions definitely worth logging).\nThis means that files like usage.c could instead be separated into its\nown library and git-std-lib would only contain files that we deem\n\"should never be logged\".\n\n> To that end, I think either the\n> commit message should be rephrased to remove that statement (if the issue is\n> really \"we're using a static variable and we want to avoid that\"), or the\n> libification effort should be updated to accommodate use of Trace2 anywhere\n> in Git.\n\nBesides potentially redrawing the boundaries of git-std-lib to\naccommodate Trace2, we're also looking into the possibility of\nstubbing out tracing in git-std-lib so that it and other libraries can\nbe built and tested, and then when Trace2 is turned into a library,\nit's full functionality can be linked to.\n"},{"id":"479222","messageId":"151e7426-e5b2-b078-7b51-63a4bd99dfed@github.com","threadId":"59921","inReplyTo":"CAFySSZCYpxNcK6Tfagk1ZBs6cOT9XvWm2b3DJhN0fSPsQCWR1g@mail.gmail.com","subject":"Re: [RFC PATCH 1/8] trace2: log fsync stats in trace2 rather than wrapper","fromName":"Victoria Dye","fromEmail":"vdye@github.com","sentAt":"2023-07-05T18:22:42Z","receivedAt":"2023-07-05T18:23:35Z","isPatch":true,"sender":{"key":"vdye@github.com","avatar":"https://avatars.githubusercontent.com/u/3619353?v=4"},"body":"Calvin Wan wrote:\n>> An added note re: the commit message - it's extremely important that\n>> functions _anywhere in Git_ are able to use the Trace2 API directly. A\n>> developer could reasonably want to measure performance, keep track of an\n>> interesting metric, log when a region is entered in the larger trace,\n>> capture error information, etc. for any function, regardless of where in\n>> falls in the internal library organization.\n> \n> I don't quite agree that functions _anywhere in Git_ are able to use\n> the Trace2 API directly for the same reason that we don't have the\n> ability to log functions in external libraries -- logging common,\n> low-level functionality creates an unnecessary amount of log churn and\n> those logs generally contain practically useless information. \n\nThat may be true in your use cases, but it isn't in mine and may not be for\nothers'. In fact, I was just using these exact fsync metrics a couple weeks\nago to do some performance analysis; I could easily imagine doing something\nsimilar for another \"low level\" function. It's unreasonable - and unfair to\nfuture development - to make an absolute declaration about \"what's useful\nvs. useless\" and use that decision to justify severely limiting our future\nflexibility on the matter.\n\n> However,\n> that does not mean that all of the functions in git-std-lib fall into\n> that category (usage has certain functions definitely worth logging).\n> This means that files like usage.c could instead be separated into its\n> own library and git-std-lib would only contain files that we deem\n> \"should never be logged\".\n\nHow do you make that determination? What about if/when someone realizes,\nsomewhere down the line, that one of those \"should never be logged\" files\nwould actually benefit from some aggregate metric, e.g. a Trace2 timer? This\nisn't a case of extracting an extraneous dependency (where a function really\ndoesn't _need_ something it has access to); tracing & logging is a core\nfunctionality in Git, and should not be artificially constrained in the name\nof organization. \n\n>> To that end, I think either the\n>> commit message should be rephrased to remove that statement (if the issue is\n>> really \"we're using a static variable and we want to avoid that\"), or the\n>> libification effort should be updated to accommodate use of Trace2 anywhere\n>> in Git.\n> \n> Besides potentially redrawing the boundaries of git-std-lib to\n> accommodate Trace2, we're also looking into the possibility of\n> stubbing out tracing in git-std-lib so that it and other libraries can\n> be built and tested, and then when Trace2 is turned into a library,\n> it's full functionality can be linked to.\n\nIf that allows you to meet your libification goals without limiting Trace2's\naccessibility throughout the codebase, that works for me.\n\n"},{"id":"479394","messageId":"7c8d6012-5bb5-9ee5-eda1-80e46c4b6981@jeffhostetler.com","threadId":"59921","inReplyTo":"20230627195251.1973421-2-calvinwan@google.com","subject":"Re: [RFC PATCH 1/8] trace2: log fsync stats in trace2 rather than wrapper","fromName":"Jeff Hostetler","fromEmail":"git@jeffhostetler.com","sentAt":"2023-07-11T20:07:15Z","receivedAt":"2023-07-11T20:15:30Z","isPatch":true,"sender":{"key":"git@jeffhostetler.com","avatar":null},"body":"\n\nOn 6/27/23 3:52 PM, Calvin Wan wrote:\n> As a library boundary, wrapper.c should not directly log trace2\n> statistics, but instead provide those statistics upon\n> request. Therefore, move the trace2 logging code to trace2.[ch.]. This\n> also allows wrapper.c to not be dependent on trace2.h and repository.h.\n> \n> Signed-off-by: Calvin Wan \n> ---\n> trace2.c | 13 +++++++++++++\n> trace2.h | 5 +++++\n> wrapper.c | 17 ++++++-----------\n> wrapper.h | 4 ++--\n> 4 files changed, 26 insertions(+), 13 deletions(-)\n> \n> diff --git a/trace2.c b/trace2.c\n> index 0efc4e7b95..f367a1ce31 100644\n> --- a/trace2.c\n> +++ b/trace2.c\n> @@ -915,3 +915,16 @@ const char *trace2_session_id(void)\n> {\n> \treturn tr2_sid_get();\n> }\n> +\n> +static void log_trace_fsync_if(const char *key)\n> +{\n> +\tintmax_t value = get_trace_git_fsync_stats(key);\n> +\tif (value)\n> +\t\ttrace2_data_intmax(\"fsync\", the_repository, key, value);\n> +}\n> +\n> +void trace_git_fsync_stats(void)\n> +{\n> +\tlog_trace_fsync_if(\"fsync/writeout-only\");\n> +\tlog_trace_fsync_if(\"fsync/hardware-flush\");\n> +}\n> diff --git a/trace2.h b/trace2.h\n> index 4ced30c0db..689e9a4027 100644\n> --- a/trace2.h\n> +++ b/trace2.h\n> @@ -581,4 +581,9 @@ void trace2_collect_process_info(enum trace2_process_info_reason reason);\n> \n> const char *trace2_session_id(void);\n> \n> +/*\n> + * Writes out trace statistics for fsync\n> + */\n> +void trace_git_fsync_stats(void);\n> +\n> #endif /* TRACE2_H */\n\nSorry to be late to this party, but none of this belongs\nin trace2.[ch].\n\nAs Victoria stated, you can/should use the new \"timers and counters\"\nfeature in Trace2 to collect and log these stats.\n\nAnd then you don't need specific log_trace_* functions or wrappers\n-- just use the trace2_timer_start()/stop() or trace2_counter_add()\nfunctions as necessary around the various fsync operations.\n\n\nI haven't really followed the lib-ification effort, so I'm just going\nto GUESS that all of the Trace2_ and tr2_ prefixed functions and data\nstructures will need to be in the lowest-level .a so that it can be\ncalled from the main .exe and any other .a's between them.\n\nJeff\n\n"},{"id":"480446","messageId":"20230810163346.274132-1-calvinwan@google.com","threadId":"59921","inReplyTo":"20230627195251.1973421-1-calvinwan@google.com","subject":"[RFC PATCH v2 0/7] Introduce Git Standard Library","fromName":"Calvin Wan","fromEmail":"calvinwan@google.com","sentAt":"2023-08-10T16:33:45Z","receivedAt":"2023-08-10T16:34:30Z","isPatch":true,"sender":{"key":"calvinwan@google.com","avatar":"https://avatars.githubusercontent.com/u/92547554?v=4"},"body":"Original cover letter:\nhttps://lore.kernel.org/git/20230627195251.1973421-1-calvinwan@google.com/\n\nIn the initial RFC, I had a patch that removed the trace2 dependency\nfrom usage.c so that git-std-lib.a would not have dependencies outside\nof git-std-lib.a files. Consequently this meant that tracing would not\nbe possible in git-std-lib.a files for other developers of Git, and it\nis not a good idea for the libification effort to close the door on\ntracing in certain files for future development (thanks Victoria for\npointing this out). That patch has been removed and instead I introduce\nstubbed out versions of repository.[ch] and trace2.[ch] that are swapped\nin during compilation time (I'm no Makefile expert so any advice on how\non I could do this better would be much appreciated). These stubbed out\nfiles contain no implementations and therefore do not have any\nadditional dependencies, allowing git-std-lib.a to compile with only the\nstubs as additional dependencies. This also has the added benefit of\nremoving `#ifdef GIT_STD_LIB` macros in C files for specific library\ncompilation rules. Libification shouldn't pollute C files with these\nmacros. The boundaries for git-std-lib.a have also been updated to\ncontain these stubbed out files.\n\nI have also made some additional changes to the Makefile to piggy back\noff of our existing build rules for .c/.o targets and their\ndependencies. As I learn more about Makefiles, I am continuing to look\nfor ways to improve these rules. Eventually I would like to be able to\nhave a set of rules that future libraries can emulate and is scalable\nin the sense of not creating additional toil for developers that are not\ninterested in libification.\n\nCalvin Wan (7):\n hex-ll: split out functionality from hex\n object: move function to object.c\n config: correct bad boolean env value error message\n parse: create new library for parsing strings and env values\n date: push pager.h dependency up\n git-std-lib: introduce git standard library\n git-std-lib: add test file to call git-std-lib.a functions\n\n Documentation/technical/git-std-lib.txt | 186 ++++++++++++++++++\n Makefile | 64 ++++++-\n attr.c | 2 +-\n builtin/blame.c | 2 +-\n builtin/log.c | 2 +-\n color.c | 2 +-\n config.c | 173 +----------------\n config.h | 14 +-\n date.c | 5 +-\n date.h | 2 +-\n git-compat-util.h | 7 +-\n hex-ll.c | 49 +++++\n hex-ll.h | 27 +++\n hex.c | 47 -----\n hex.h | 24 +--\n mailinfo.c | 2 +-\n object.c | 5 +\n object.h | 6 +\n pack-objects.c | 2 +-\n pack-revindex.c | 2 +-\n parse-options.c | 3 +-\n parse.c | 182 ++++++++++++++++++\n parse.h | 20 ++\n pathspec.c | 2 +-\n preload-index.c | 2 +-\n progress.c | 2 +-\n prompt.c | 2 +-\n rebase.c | 2 +-\n ref-filter.c | 3 +-\n revision.c | 3 +-\n strbuf.c | 2 +-\n stubs/repository.c | 4 +\n stubs/repository.h | 8 +\n stubs/trace2.c | 22 +++\n stubs/trace2.h | 69 +++++++\n symlinks.c | 2 +\n t/Makefile | 4 +\n t/helper/test-date.c | 3 +-\n t/helper/test-env-helper.c | 2 +-\n t/stdlib-test.c | 239 ++++++++++++++++++++++++\n unpack-trees.c | 2 +-\n url.c | 2 +-\n urlmatch.c | 2 +-\n wrapper.c | 8 +-\n wrapper.h | 5 -\n write-or-die.c | 2 +-\n 46 files changed, 925 insertions(+), 295 deletions(-)\n create mode 100644 Documentation/technical/git-std-lib.txt\n create mode 100644 hex-ll.c\n create mode 100644 hex-ll.h\n create mode 100644 parse.c\n create mode 100644 parse.h\n create mode 100644 stubs/repository.c\n create mode 100644 stubs/repository.h\n create mode 100644 stubs/trace2.c\n create mode 100644 stubs/trace2.h\n create mode 100644 t/stdlib-test.c\n\nRange-diff against v1:\n1: f7abe7a239 < -: ---------- trace2: log fsync stats in trace2 rather than wrapper\n2: c302ae0052 = 1: 78634bc406 hex-ll: split out functionality from hex\n3: 74e8e35ae2 ! 2: 21ec1d276e object: move function to object.c\n @@ wrapper.c\n #include \"config.h\"\n #include \"gettext.h\"\n -#include \"object.h\"\n + #include \"repository.h\"\n #include \"strbuf.h\"\n - \n - static intmax_t count_fsync_writeout_only;\n + #include \"trace2.h\"\n @@ wrapper.c: int rmdir_or_warn(const char *file)\n \treturn warn_if_unremovable(\"rmdir\", file, rmdir(file));\n }\n4: 419c702633 = 3: 41dcf8107c config: correct bad boolean env value error message\n5: a325002438 ! 4: 3e800a41c4 parse: create new library for parsing strings and env values\n @@ wrapper.c\n -#include \"config.h\"\n +#include \"parse.h\"\n #include \"gettext.h\"\n + #include \"repository.h\"\n #include \"strbuf.h\"\n - \n \n ## write-or-die.c ##\n @@\n6: 475190310a < -: ---------- pager: remove pager_in_use()\n-: ---------- > 5: 7a4a088bc3 date: push pager.h dependency up\n7: d7f4d4a137 ! 6: c9002734d0 git-std-lib: introduce git standard library\n @@ Documentation/technical/git-std-lib.txt (new)\n +easily included in git-std-lib.a, which as a root dependency means that\n +higher level libraries do not have to worry about compatibility files in\n +compat/. The rest of the functions defined in git-compat-util.h are\n -+implemented in top level files and, in this patch set, are hidden behind\n ++implemented in top level files and are hidden behind\n +an #ifdef if their implementation is not in git-std-lib.a.\n +\n +Rationale summary\n @@ Documentation/technical/git-std-lib.txt (new)\n + - low-level git/* files with functions defined in git-compat-util.h\n + (ctype.c)\n + - compat/*\n ++ - stubbed out dependencies in stubs/ (stubs/repository.c, stubs/trace2.c)\n +\n +There are other files that might fit this definition, but that does not\n +mean it should belong in git-std-lib.a. Those files should start as\n +their own separate library since any file added to git-std-lib.a loses\n +its flexibility of being easily swappable.\n +\n ++Wrapper.c and usage.c have dependencies on repository and trace2 that are\n ++possible to remove at the cost of sacrificing the ability for standard Git\n ++to be able to trace functions in those files and other files in git-std-lib.a.\n ++In order for git-std-lib.a to compile with those dependencies, stubbed out\n ++versions of those files are implemented and swapped in during compilation time.\n ++\n +Files inside of Git Standard Library\n +================\n +\n @@ Documentation/technical/git-std-lib.txt (new)\n +usage.c\n +utf8.c\n +wrapper.c\n ++stubs/repository.c\n ++stubs/trace2.c\n +relevant compat/ files\n +\n +Pitfalls\n +================\n +\n -+In patch 7, I use #ifdef GIT_STD_LIB to both stub out code and hide\n -+certain function headers. As other parts of Git are libified, if we\n -+have to use more ifdefs for each different library, then the codebase\n -+will become uglier and harder to understand. \n -+\n +There are a small amount of files under compat/* that have dependencies\n +not inside of git-std-lib.a. While those functions are not called on\n +Linux, other OSes might call those problematic functions. I don't see\n @@ Documentation/technical/git-std-lib.txt (new)\n \\ No newline at end of file\n \n ## Makefile ##\n +@@ Makefile: FUZZ_PROGRAMS =\n + GIT_OBJS =\n + LIB_OBJS =\n + SCALAR_OBJS =\n ++STUB_OBJS =\n + OBJECTS =\n + OTHER_PROGRAMS =\n + PROGRAM_OBJS =\n +@@ Makefile: COCCI_SOURCES = $(filter-out $(THIRD_PARTY_SOURCES),$(FOUND_C_SOURCES))\n + \n + LIB_H = $(FOUND_H_SOURCES)\n + \n ++ifndef GIT_STD_LIB\n + LIB_OBJS += abspath.o\n + LIB_OBJS += add-interactive.o\n + LIB_OBJS += add-patch.o\n +@@ Makefile: LIB_OBJS += write-or-die.o\n + LIB_OBJS += ws.o\n + LIB_OBJS += wt-status.o\n + LIB_OBJS += xdiff-interface.o\n ++else ifdef GIT_STD_LIB\n ++LIB_OBJS += abspath.o\n ++LIB_OBJS += ctype.o\n ++LIB_OBJS += date.o\n ++LIB_OBJS += hex-ll.o\n ++LIB_OBJS += parse.o\n ++LIB_OBJS += strbuf.o\n ++LIB_OBJS += usage.o\n ++LIB_OBJS += utf8.o\n ++LIB_OBJS += wrapper.o\n ++\n ++ifdef STUB_REPOSITORY\n ++STUB_OBJS += stubs/repository.o\n ++endif\n ++\n ++ifdef STUB_TRACE2\n ++STUB_OBJS += stubs/trace2.o\n ++endif\n ++\n ++LIB_OBJS += $(STUB_OBJS)\n ++endif\n + \n + BUILTIN_OBJS += builtin/add.o\n + BUILTIN_OBJS += builtin/am.o\n @@ Makefile: ifdef FSMONITOR_OS_SETTINGS\n \tCOMPAT_OBJS += compat/fsmonitor/fsm-path-utils-$(FSMONITOR_OS_SETTINGS).o\n endif\n @@ Makefile: $(FUZZ_PROGRAMS): all\n +### Libified Git rules\n +\n +# git-std-lib\n -+# `make git-std-lib GIT_STD_LIB=YesPlease`\n ++# `make git-std-lib GIT_STD_LIB=YesPlease STUB_REPOSITORY=YesPlease STUB_TRACE2=YesPlease`\n +STD_LIB = git-std-lib.a\n +\n -+GIT_STD_LIB_OBJS += abspath.o\n -+GIT_STD_LIB_OBJS += ctype.o\n -+GIT_STD_LIB_OBJS += date.o\n -+GIT_STD_LIB_OBJS += hex-ll.o\n -+GIT_STD_LIB_OBJS += parse.o\n -+GIT_STD_LIB_OBJS += strbuf.o\n -+GIT_STD_LIB_OBJS += usage.o\n -+GIT_STD_LIB_OBJS += utf8.o\n -+GIT_STD_LIB_OBJS += wrapper.o\n -+\n -+$(STD_LIB): $(GIT_STD_LIB_OBJS) $(COMPAT_OBJS)\n ++$(STD_LIB): $(LIB_OBJS) $(COMPAT_OBJS) $(STUB_OBJS)\n +\t$(QUIET_AR)$(RM) $@ && $(AR) $(ARFLAGS) $@ $^\n +\n -+git-std-lib: $(STD_LIB)\n ++TEMP_HEADERS = temp_headers/\n ++\n ++git-std-lib:\n ++# Move headers to temporary folder and replace them with stubbed headers.\n ++# After building, move headers and stubbed headers back.\n ++ifneq ($(STUB_OBJS),)\n ++\tmkdir -p $(TEMP_HEADERS); \\\n ++\tfor d in $(STUB_OBJS); do \\\n ++\t\tBASE=$${d%.*}; \\\n ++\t\tmv $${BASE##*/}.h $(TEMP_HEADERS)$${BASE##*/}.h; \\\n ++\t\tmv $${BASE}.h $${BASE##*/}.h; \\\n ++\tdone; \\\n ++\t$(MAKE) $(STD_LIB); \\\n ++\tfor d in $(STUB_OBJS); do \\\n ++\t\tBASE=$${d%.*}; \\\n ++\t\tmv $${BASE##*/}.h $${BASE}.h; \\\n ++\t\tmv $(TEMP_HEADERS)$${BASE##*/}.h $${BASE##*/}.h; \\\n ++\tdone; \\\n ++\trm -rf temp_headers\n ++else\n ++\t$(MAKE) $(STD_LIB)\n ++endif\n \n ## git-compat-util.h ##\n @@ git-compat-util.h: static inline int noop_core_config(const char *var UNUSED,\n @@ git-compat-util.h: int git_access(const char *path, int mode);\n /*\n * You can mark a stack variable with UNLEAK(var) to avoid it being\n \n + ## stubs/repository.c (new) ##\n +@@\n ++#include \"git-compat-util.h\"\n ++#include \"repository.h\"\n ++\n ++struct repository *the_repository;\n +\n + ## stubs/repository.h (new) ##\n +@@\n ++#ifndef REPOSITORY_H\n ++#define REPOSITORY_H\n ++\n ++struct repository { int stub; };\n ++\n ++extern struct repository *the_repository;\n ++\n ++#endif /* REPOSITORY_H */\n +\n + ## stubs/trace2.c (new) ##\n +@@\n ++#include \"git-compat-util.h\"\n ++#include \"trace2.h\"\n ++\n ++void trace2_region_enter_fl(const char *file, int line, const char *category,\n ++\t\t\t const char *label, const struct repository *repo, ...) { }\n ++void trace2_region_leave_fl(const char *file, int line, const char *category,\n ++\t\t\t const char *label, const struct repository *repo, ...) { }\n ++void trace2_data_string_fl(const char *file, int line, const char *category,\n ++\t\t\t const struct repository *repo, const char *key,\n ++\t\t\t const char *value) { }\n ++void trace2_cmd_ancestry_fl(const char *file, int line, const char **parent_names) { }\n ++void trace2_cmd_error_va_fl(const char *file, int line, const char *fmt,\n ++\t\t\t va_list ap) { }\n ++void trace2_cmd_name_fl(const char *file, int line, const char *name) { }\n ++void trace2_thread_start_fl(const char *file, int line,\n ++\t\t\t const char *thread_base_name) { }\n ++void trace2_thread_exit_fl(const char *file, int line) { }\n ++void trace2_data_intmax_fl(const char *file, int line, const char *category,\n ++\t\t\t const struct repository *repo, const char *key,\n ++\t\t\t intmax_t value) { }\n ++int trace2_is_enabled(void) { return 0; }\n ++void trace2_collect_process_info(enum trace2_process_info_reason reason) { }\n +\n + ## stubs/trace2.h (new) ##\n +@@\n ++#ifndef TRACE2_H\n ++#define TRACE2_H\n ++\n ++struct child_process { int stub; };\n ++struct repository;\n ++struct json_writer { int stub; };\n ++\n ++void trace2_region_enter_fl(const char *file, int line, const char *category,\n ++\t\t\t const char *label, const struct repository *repo, ...);\n ++\n ++#define trace2_region_enter(category, label, repo) \\\n ++\ttrace2_region_enter_fl(__FILE__, __LINE__, (category), (label), (repo))\n ++\n ++void trace2_region_leave_fl(const char *file, int line, const char *category,\n ++\t\t\t const char *label, const struct repository *repo, ...);\n ++\n ++#define trace2_region_leave(category, label, repo) \\\n ++\ttrace2_region_leave_fl(__FILE__, __LINE__, (category), (label), (repo))\n ++\n ++void trace2_data_string_fl(const char *file, int line, const char *category,\n ++\t\t\t const struct repository *repo, const char *key,\n ++\t\t\t const char *value);\n ++\n ++#define trace2_data_string(category, repo, key, value) \\\n ++\ttrace2_data_string_fl(__FILE__, __LINE__, (category), (repo), (key), \\\n ++\t\t\t (value))\n ++\n ++void trace2_cmd_ancestry_fl(const char *file, int line, const char **parent_names);\n ++\n ++#define trace2_cmd_ancestry(v) trace2_cmd_ancestry_fl(__FILE__, __LINE__, (v))\n ++\n ++void trace2_cmd_error_va_fl(const char *file, int line, const char *fmt,\n ++\t\t\t va_list ap);\n ++\n ++#define trace2_cmd_error_va(fmt, ap) \\\n ++\ttrace2_cmd_error_va_fl(__FILE__, __LINE__, (fmt), (ap))\n ++\n ++\n ++void trace2_cmd_name_fl(const char *file, int line, const char *name);\n ++\n ++#define trace2_cmd_name(v) trace2_cmd_name_fl(__FILE__, __LINE__, (v))\n ++\n ++void trace2_thread_start_fl(const char *file, int line,\n ++\t\t\t const char *thread_base_name);\n ++\n ++#define trace2_thread_start(thread_base_name) \\\n ++\ttrace2_thread_start_fl(__FILE__, __LINE__, (thread_base_name))\n ++\n ++void trace2_thread_exit_fl(const char *file, int line);\n ++\n ++#define trace2_thread_exit() trace2_thread_exit_fl(__FILE__, __LINE__)\n ++\n ++void trace2_data_intmax_fl(const char *file, int line, const char *category,\n ++\t\t\t const struct repository *repo, const char *key,\n ++\t\t\t intmax_t value);\n ++\n ++#define trace2_data_intmax(category, repo, key, value) \\\n ++\ttrace2_data_intmax_fl(__FILE__, __LINE__, (category), (repo), (key), \\\n ++\t\t\t (value))\n ++\n ++enum trace2_process_info_reason {\n ++\tTRACE2_PROCESS_INFO_STARTUP,\n ++\tTRACE2_PROCESS_INFO_EXIT,\n ++};\n ++int trace2_is_enabled(void);\n ++void trace2_collect_process_info(enum trace2_process_info_reason reason);\n ++\n ++#endif /* TRACE2_H */\n ++\n +\n ## symlinks.c ##\n @@ symlinks.c: void invalidate_lstat_cache(void)\n \treset_lstat_cache(&default_cache);\n @@ symlinks.c: int lstat_cache_aware_rmdir(const char *path)\n \treturn ret;\n }\n +#endif\n -\n - ## usage.c ##\n -@@\n - */\n - #include \"git-compat-util.h\"\n - #include \"gettext.h\"\n -+\n -+#ifdef GIT_STD_LIB\n -+#undef trace2_cmd_name\n -+#undef trace2_cmd_error_va\n -+#define trace2_cmd_name(x) \n -+#define trace2_cmd_error_va(x, y)\n -+#else\n - #include \"trace2.h\"\n -+#endif\n - \n - static void vreportf(const char *prefix, const char *err, va_list params)\n - {\n8: cb96e67774 ! 7: 0bead8f980 git-std-lib: add test file to call git-std-lib.a functions\n @@ t/stdlib-test.c (new)\n +\tstrbuf_splice(sb, 0, 1, \"foo\", 3);\n +\tstrbuf_insert(sb, 0, \"foo\", 3);\n +\t// strbuf_vinsertf() called by strbuf_insertf\n -+\tstrbuf_insertf(sb, 0, \"%s\", \"foo\"); \n ++\tstrbuf_insertf(sb, 0, \"%s\", \"foo\");\n +\tstrbuf_remove(sb, 0, 1);\n +\tstrbuf_add(sb, \"foo\", 3);\n +\tstrbuf_addbuf(sb, sb2);\n @@ t/stdlib-test.c (new)\n +\tunlink(path);\n +\tread_in_full(fd, &sb, 1);\n +\twrite_in_full(fd, &sb, 1);\n -+\tpread_in_full(fd, &sb, 1, 0);\t\n ++\tpread_in_full(fd, &sb, 1, 0);\n +}\n +\n +int main() {\n @@ t/stdlib-test.c (new)\n +\tfprintf(stderr, \"all git-std-lib functions finished calling\\n\");\n +\treturn 0;\n +}\n - \\ No newline at end of file\n-- \n2.41.0.640.ga95def55d0-goog\n\n"},{"id":"480450","messageId":"20230810163654.275023-2-calvinwan@google.com","threadId":"59921","inReplyTo":"20230810163346.274132-1-calvinwan@google.com","subject":"[RFC PATCH v2 2/7] object: move function to object.c","fromName":"Calvin Wan","fromEmail":"calvinwan@google.com","sentAt":"2023-08-10T16:36:49Z","receivedAt":"2023-08-10T16:37:43Z","isPatch":true,"sender":{"key":"calvinwan@google.com","avatar":"https://avatars.githubusercontent.com/u/92547554?v=4"},"body":"While remove_or_warn() is a simple ternary operator to call two other\nwrapper functions, it creates an unnecessary dependency to object.h in\nwrapper.c. Therefore move the function to object.[ch] where the concept\nof GITLINKs is first defined.\n\nSigned-off-by: Calvin Wan \n---\n object.c | 5 +++++\n object.h | 6 ++++++\n wrapper.c | 6 ------\n wrapper.h | 5 -----\n 4 files changed, 11 insertions(+), 11 deletions(-)\n\ndiff --git a/object.c b/object.c\nindex 60f954194f..cb29fcc304 100644\n--- a/object.c\n+++ b/object.c\n@@ -617,3 +617,8 @@ void parsed_object_pool_clear(struct parsed_object_pool *o)\n \tFREE_AND_NULL(o->object_state);\n \tFREE_AND_NULL(o->shallow_stat);\n }\n+\n+int remove_or_warn(unsigned int mode, const char *file)\n+{\n+\treturn S_ISGITLINK(mode) ? rmdir_or_warn(file) : unlink_or_warn(file);\n+}\ndiff --git a/object.h b/object.h\nindex 5871615fee..e908ef6515 100644\n--- a/object.h\n+++ b/object.h\n@@ -284,4 +284,10 @@ void clear_object_flags(unsigned flags);\n */\n void repo_clear_commit_marks(struct repository *r, unsigned int flags);\n \n+/*\n+ * Calls the correct function out of {unlink,rmdir}_or_warn based on\n+ * the supplied file mode.\n+ */\n+int remove_or_warn(unsigned int mode, const char *path);\n+\n #endif /* OBJECT_H */\ndiff --git a/wrapper.c b/wrapper.c\nindex 22be9812a7..118d3033de 100644\n--- a/wrapper.c\n+++ b/wrapper.c\n@@ -5,7 +5,6 @@\n #include \"abspath.h\"\n #include \"config.h\"\n #include \"gettext.h\"\n-#include \"object.h\"\n #include \"repository.h\"\n #include \"strbuf.h\"\n #include \"trace2.h\"\n@@ -647,11 +646,6 @@ int rmdir_or_warn(const char *file)\n \treturn warn_if_unremovable(\"rmdir\", file, rmdir(file));\n }\n \n-int remove_or_warn(unsigned int mode, const char *file)\n-{\n-\treturn S_ISGITLINK(mode) ? rmdir_or_warn(file) : unlink_or_warn(file);\n-}\n-\n static int access_error_is_ok(int err, unsigned flag)\n {\n \treturn (is_missing_file_error(err) ||\ndiff --git a/wrapper.h b/wrapper.h\nindex c85b1328d1..272795f863 100644\n--- a/wrapper.h\n+++ b/wrapper.h\n@@ -111,11 +111,6 @@ int unlink_or_msg(const char *file, struct strbuf *err);\n * not exist.\n */\n int rmdir_or_warn(const char *path);\n-/*\n- * Calls the correct function out of {unlink,rmdir}_or_warn based on\n- * the supplied file mode.\n- */\n-int remove_or_warn(unsigned int mode, const char *path);\n \n /*\n * Call access(2), but warn for any error except \"missing file\"\n-- \n2.41.0.640.ga95def55d0-goog\n\n"},{"id":"480451","messageId":"20230810163654.275023-1-calvinwan@google.com","threadId":"59921","inReplyTo":"20230810163346.274132-1-calvinwan@google.com","subject":"[RFC PATCH v2 1/7] hex-ll: split out functionality from hex","fromName":"Calvin Wan","fromEmail":"calvinwan@google.com","sentAt":"2023-08-10T16:36:48Z","receivedAt":"2023-08-10T16:37:46Z","isPatch":true,"sender":{"key":"calvinwan@google.com","avatar":"https://avatars.githubusercontent.com/u/92547554?v=4"},"body":"Separate out hex functionality that doesn't require a hash algo into\nhex-ll.[ch]. Since the hash algo is currently a global that sits in\nrepository, this separation removes that dependency for files that only\nneed basic hex manipulation functions.\n\nSigned-off-by: Calvin Wan \n---\n Makefile | 1 +\n color.c | 2 +-\n hex-ll.c | 49 +++++++++++++++++++++++++++++++++++++++++++++++++\n hex-ll.h | 27 +++++++++++++++++++++++++++\n hex.c | 47 -----------------------------------------------\n hex.h | 24 +-----------------------\n mailinfo.c | 2 +-\n strbuf.c | 2 +-\n url.c | 2 +-\n urlmatch.c | 2 +-\n 10 files changed, 83 insertions(+), 75 deletions(-)\n create mode 100644 hex-ll.c\n create mode 100644 hex-ll.h\n\ndiff --git a/Makefile b/Makefile\nindex 045e2187c4..83b385b0be 100644\n--- a/Makefile\n+++ b/Makefile\n@@ -1040,6 +1040,7 @@ LIB_OBJS += hash-lookup.o\n LIB_OBJS += hashmap.o\n LIB_OBJS += help.o\n LIB_OBJS += hex.o\n+LIB_OBJS += hex-ll.o\n LIB_OBJS += hook.o\n LIB_OBJS += ident.o\n LIB_OBJS += json-writer.o\ndiff --git a/color.c b/color.c\nindex 83abb11eda..f3c0a4659b 100644\n--- a/color.c\n+++ b/color.c\n@@ -3,7 +3,7 @@\n #include \"color.h\"\n #include \"editor.h\"\n #include \"gettext.h\"\n-#include \"hex.h\"\n+#include \"hex-ll.h\"\n #include \"pager.h\"\n #include \"strbuf.h\"\n \ndiff --git a/hex-ll.c b/hex-ll.c\nnew file mode 100644\nindex 0000000000..4d7ece1de5\n--- /dev/null\n+++ b/hex-ll.c\n@@ -0,0 +1,49 @@\n+#include \"git-compat-util.h\"\n+#include \"hex-ll.h\"\n+\n+const signed char hexval_table[256] = {\n+\t -1, -1, -1, -1, -1, -1, -1, -1,\t\t/* 00-07 */\n+\t -1, -1, -1, -1, -1, -1, -1, -1,\t\t/* 08-0f */\n+\t -1, -1, -1, -1, -1, -1, -1, -1,\t\t/* 10-17 */\n+\t -1, -1, -1, -1, -1, -1, -1, -1,\t\t/* 18-1f */\n+\t -1, -1, -1, -1, -1, -1, -1, -1,\t\t/* 20-27 */\n+\t -1, -1, -1, -1, -1, -1, -1, -1,\t\t/* 28-2f */\n+\t 0, 1, 2, 3, 4, 5, 6, 7,\t\t/* 30-37 */\n+\t 8, 9, -1, -1, -1, -1, -1, -1,\t\t/* 38-3f */\n+\t -1, 10, 11, 12, 13, 14, 15, -1,\t\t/* 40-47 */\n+\t -1, -1, -1, -1, -1, -1, -1, -1,\t\t/* 48-4f */\n+\t -1, -1, -1, -1, -1, -1, -1, -1,\t\t/* 50-57 */\n+\t -1, -1, -1, -1, -1, -1, -1, -1,\t\t/* 58-5f */\n+\t -1, 10, 11, 12, 13, 14, 15, -1,\t\t/* 60-67 */\n+\t -1, -1, -1, -1, -1, -1, -1, -1,\t\t/* 68-67 */\n+\t -1, -1, -1, -1, -1, -1, -1, -1,\t\t/* 70-77 */\n+\t -1, -1, -1, -1, -1, -1, -1, -1,\t\t/* 78-7f */\n+\t -1, -1, -1, -1, -1, -1, -1, -1,\t\t/* 80-87 */\n+\t -1, -1, -1, -1, -1, -1, -1, -1,\t\t/* 88-8f */\n+\t -1, -1, -1, -1, -1, -1, -1, -1,\t\t/* 90-97 */\n+\t -1, -1, -1, -1, -1, -1, -1, -1,\t\t/* 98-9f */\n+\t -1, -1, -1, -1, -1, -1, -1, -1,\t\t/* a0-a7 */\n+\t -1, -1, -1, -1, -1, -1, -1, -1,\t\t/* a8-af */\n+\t -1, -1, -1, -1, -1, -1, -1, -1,\t\t/* b0-b7 */\n+\t -1, -1, -1, -1, -1, -1, -1, -1,\t\t/* b8-bf */\n+\t -1, -1, -1, -1, -1, -1, -1, -1,\t\t/* c0-c7 */\n+\t -1, -1, -1, -1, -1, -1, -1, -1,\t\t/* c8-cf */\n+\t -1, -1, -1, -1, -1, -1, -1, -1,\t\t/* d0-d7 */\n+\t -1, -1, -1, -1, -1, -1, -1, -1,\t\t/* d8-df */\n+\t -1, -1, -1, -1, -1, -1, -1, -1,\t\t/* e0-e7 */\n+\t -1, -1, -1, -1, -1, -1, -1, -1,\t\t/* e8-ef */\n+\t -1, -1, -1, -1, -1, -1, -1, -1,\t\t/* f0-f7 */\n+\t -1, -1, -1, -1, -1, -1, -1, -1,\t\t/* f8-ff */\n+};\n+\n+int hex_to_bytes(unsigned char *binary, const char *hex, size_t len)\n+{\n+\tfor (; len; len--, hex += 2) {\n+\t\tunsigned int val = (hexval(hex[0]) << 4) | hexval(hex[1]);\n+\n+\t\tif (val & ~0xff)\n+\t\t\treturn -1;\n+\t\t*binary++ = val;\n+\t}\n+\treturn 0;\n+}\ndiff --git a/hex-ll.h b/hex-ll.h\nnew file mode 100644\nindex 0000000000..a381fa8556\n--- /dev/null\n+++ b/hex-ll.h\n@@ -0,0 +1,27 @@\n+#ifndef HEX_LL_H\n+#define HEX_LL_H\n+\n+extern const signed char hexval_table[256];\n+static inline unsigned int hexval(unsigned char c)\n+{\n+\treturn hexval_table[c];\n+}\n+\n+/*\n+ * Convert two consecutive hexadecimal digits into a char. Return a\n+ * negative value on error. Don't run over the end of short strings.\n+ */\n+static inline int hex2chr(const char *s)\n+{\n+\tunsigned int val = hexval(s[0]);\n+\treturn (val & ~0xf) ? val : (val << 4) | hexval(s[1]);\n+}\n+\n+/*\n+ * Read `len` pairs of hexadecimal digits from `hex` and write the\n+ * values to `binary` as `len` bytes. Return 0 on success, or -1 if\n+ * the input does not consist of hex digits).\n+ */\n+int hex_to_bytes(unsigned char *binary, const char *hex, size_t len);\n+\n+#endif\ndiff --git a/hex.c b/hex.c\nindex 7bb440e794..03e55841ed 100644\n--- a/hex.c\n+++ b/hex.c\n@@ -2,53 +2,6 @@\n #include \"hash.h\"\n #include \"hex.h\"\n \n-const signed char hexval_table[256] = {\n-\t -1, -1, -1, -1, -1, -1, -1, -1,\t\t/* 00-07 */\n-\t -1, -1, -1, -1, -1, -1, -1, -1,\t\t/* 08-0f */\n-\t -1, -1, -1, -1, -1, -1, -1, -1,\t\t/* 10-17 */\n-\t -1, -1, -1, -1, -1, -1, -1, -1,\t\t/* 18-1f */\n-\t -1, -1, -1, -1, -1, -1, -1, -1,\t\t/* 20-27 */\n-\t -1, -1, -1, -1, -1, -1, -1, -1,\t\t/* 28-2f */\n-\t 0, 1, 2, 3, 4, 5, 6, 7,\t\t/* 30-37 */\n-\t 8, 9, -1, -1, -1, -1, -1, -1,\t\t/* 38-3f */\n-\t -1, 10, 11, 12, 13, 14, 15, -1,\t\t/* 40-47 */\n-\t -1, -1, -1, -1, -1, -1, -1, -1,\t\t/* 48-4f */\n-\t -1, -1, -1, -1, -1, -1, -1, -1,\t\t/* 50-57 */\n-\t -1, -1, -1, -1, -1, -1, -1, -1,\t\t/* 58-5f */\n-\t -1, 10, 11, 12, 13, 14, 15, -1,\t\t/* 60-67 */\n-\t -1, -1, -1, -1, -1, -1, -1, -1,\t\t/* 68-67 */\n-\t -1, -1, -1, -1, -1, -1, -1, -1,\t\t/* 70-77 */\n-\t -1, -1, -1, -1, -1, -1, -1, -1,\t\t/* 78-7f */\n-\t -1, -1, -1, -1, -1, -1, -1, -1,\t\t/* 80-87 */\n-\t -1, -1, -1, -1, -1, -1, -1, -1,\t\t/* 88-8f */\n-\t -1, -1, -1, -1, -1, -1, -1, -1,\t\t/* 90-97 */\n-\t -1, -1, -1, -1, -1, -1, -1, -1,\t\t/* 98-9f */\n-\t -1, -1, -1, -1, -1, -1, -1, -1,\t\t/* a0-a7 */\n-\t -1, -1, -1, -1, -1, -1, -1, -1,\t\t/* a8-af */\n-\t -1, -1, -1, -1, -1, -1, -1, -1,\t\t/* b0-b7 */\n-\t -1, -1, -1, -1, -1, -1, -1, -1,\t\t/* b8-bf */\n-\t -1, -1, -1, -1, -1, -1, -1, -1,\t\t/* c0-c7 */\n-\t -1, -1, -1, -1, -1, -1, -1, -1,\t\t/* c8-cf */\n-\t -1, -1, -1, -1, -1, -1, -1, -1,\t\t/* d0-d7 */\n-\t -1, -1, -1, -1, -1, -1, -1, -1,\t\t/* d8-df */\n-\t -1, -1, -1, -1, -1, -1, -1, -1,\t\t/* e0-e7 */\n-\t -1, -1, -1, -1, -1, -1, -1, -1,\t\t/* e8-ef */\n-\t -1, -1, -1, -1, -1, -1, -1, -1,\t\t/* f0-f7 */\n-\t -1, -1, -1, -1, -1, -1, -1, -1,\t\t/* f8-ff */\n-};\n-\n-int hex_to_bytes(unsigned char *binary, const char *hex, size_t len)\n-{\n-\tfor (; len; len--, hex += 2) {\n-\t\tunsigned int val = (hexval(hex[0]) << 4) | hexval(hex[1]);\n-\n-\t\tif (val & ~0xff)\n-\t\t\treturn -1;\n-\t\t*binary++ = val;\n-\t}\n-\treturn 0;\n-}\n-\n static int get_hash_hex_algop(const char *hex, unsigned char *hash,\n \t\t\t const struct git_hash_algo *algop)\n {\ndiff --git a/hex.h b/hex.h\nindex 7df4b3c460..c07c8b34c2 100644\n--- a/hex.h\n+++ b/hex.h\n@@ -2,22 +2,7 @@\n #define HEX_H\n \n #include \"hash-ll.h\"\n-\n-extern const signed char hexval_table[256];\n-static inline unsigned int hexval(unsigned char c)\n-{\n-\treturn hexval_table[c];\n-}\n-\n-/*\n- * Convert two consecutive hexadecimal digits into a char. Return a\n- * negative value on error. Don't run over the end of short strings.\n- */\n-static inline int hex2chr(const char *s)\n-{\n-\tunsigned int val = hexval(s[0]);\n-\treturn (val & ~0xf) ? val : (val << 4) | hexval(s[1]);\n-}\n+#include \"hex-ll.h\"\n \n /*\n * Try to read a SHA1 in hexadecimal format from the 40 characters\n@@ -32,13 +17,6 @@ int get_oid_hex(const char *hex, struct object_id *sha1);\n /* Like get_oid_hex, but for an arbitrary hash algorithm. */\n int get_oid_hex_algop(const char *hex, struct object_id *oid, const struct git_hash_algo *algop);\n \n-/*\n- * Read `len` pairs of hexadecimal digits from `hex` and write the\n- * values to `binary` as `len` bytes. Return 0 on success, or -1 if\n- * the input does not consist of hex digits).\n- */\n-int hex_to_bytes(unsigned char *binary, const char *hex, size_t len);\n-\n /*\n * Convert a binary hash in \"unsigned char []\" or an object name in\n * \"struct object_id *\" to its hex equivalent. The `_r` variant is reentrant,\ndiff --git a/mailinfo.c b/mailinfo.c\nindex 2aeb20e5e6..eb34c30be7 100644\n--- a/mailinfo.c\n+++ b/mailinfo.c\n@@ -1,7 +1,7 @@\n #include \"git-compat-util.h\"\n #include \"config.h\"\n #include \"gettext.h\"\n-#include \"hex.h\"\n+#include \"hex-ll.h\"\n #include \"utf8.h\"\n #include \"strbuf.h\"\n #include \"mailinfo.h\"\ndiff --git a/strbuf.c b/strbuf.c\nindex 8dac52b919..a2a05fe168 100644\n--- a/strbuf.c\n+++ b/strbuf.c\n@@ -1,6 +1,6 @@\n #include \"git-compat-util.h\"\n #include \"gettext.h\"\n-#include \"hex.h\"\n+#include \"hex-ll.h\"\n #include \"strbuf.h\"\n #include \"string-list.h\"\n #include \"utf8.h\"\ndiff --git a/url.c b/url.c\nindex 2e1a9f6fee..282b12495a 100644\n--- a/url.c\n+++ b/url.c\n@@ -1,5 +1,5 @@\n #include \"git-compat-util.h\"\n-#include \"hex.h\"\n+#include \"hex-ll.h\"\n #include \"strbuf.h\"\n #include \"url.h\"\n \ndiff --git a/urlmatch.c b/urlmatch.c\nindex eba0bdd77f..f1aa87d1dd 100644\n--- a/urlmatch.c\n+++ b/urlmatch.c\n@@ -1,6 +1,6 @@\n #include \"git-compat-util.h\"\n #include \"gettext.h\"\n-#include \"hex.h\"\n+#include \"hex-ll.h\"\n #include \"strbuf.h\"\n #include \"urlmatch.h\"\n \n-- \n2.41.0.640.ga95def55d0-goog\n\n"},{"id":"480452","messageId":"20230810163654.275023-3-calvinwan@google.com","threadId":"59921","inReplyTo":"20230810163346.274132-1-calvinwan@google.com","subject":"[RFC PATCH v2 3/7] config: correct bad boolean env value error message","fromName":"Calvin Wan","fromEmail":"calvinwan@google.com","sentAt":"2023-08-10T16:36:50Z","receivedAt":"2023-08-10T16:38:14Z","isPatch":true,"sender":{"key":"calvinwan@google.com","avatar":"https://avatars.githubusercontent.com/u/92547554?v=4"},"body":"An incorrectly defined boolean environment value would result in the\nfollowing error message:\n\nbad boolean config value '%s' for '%s'\n\nThis is a misnomer since environment value != config value. Instead of\ncalling git_config_bool() to parse the environment value, mimic the\nfunctionality inside of git_config_bool() but with the correct error\nmessage.\n\nSigned-off-by: Calvin Wan \n---\n config.c | 9 ++++++++-\n 1 file changed, 8 insertions(+), 1 deletion(-)\n\ndiff --git a/config.c b/config.c\nindex 09851a6909..5b71ef1624 100644\n--- a/config.c\n+++ b/config.c\n@@ -2172,7 +2172,14 @@ void git_global_config(char **user_out, char **xdg_out)\n int git_env_bool(const char *k, int def)\n {\n \tconst char *v = getenv(k);\n-\treturn v ? git_config_bool(k, v) : def;\n+\tint val;\n+\tif (!v)\n+\t\treturn def;\n+\tval = git_parse_maybe_bool(v);\n+\tif (val < 0)\n+\t\tdie(_(\"bad boolean environment value '%s' for '%s'\"),\n+\t\t v, k);\n+\treturn val;\n }\n \n /*\n-- \n2.41.0.640.ga95def55d0-goog\n\n"},{"id":"480453","messageId":"20230810163654.275023-5-calvinwan@google.com","threadId":"59921","inReplyTo":"20230810163346.274132-1-calvinwan@google.com","subject":"[RFC PATCH v2 5/7] date: push pager.h dependency up","fromName":"Calvin Wan","fromEmail":"calvinwan@google.com","sentAt":"2023-08-10T16:36:52Z","receivedAt":"2023-08-10T16:38:15Z","isPatch":true,"sender":{"key":"calvinwan@google.com","avatar":"https://avatars.githubusercontent.com/u/92547554?v=4"},"body":"In order for date.c to be included in git-std-lib, the dependency to\npager.h must be removed since it has dependencies on many other files\nnot in git-std-lib. We achieve this by passing a boolean for\n\"pager_in_use\", rather than checking for it in parse_date_format() so\ncallers of the function will have that dependency.\n\nSigned-off-by: Calvin Wan \n---\n builtin/blame.c | 2 +-\n builtin/log.c | 2 +-\n date.c | 5 ++---\n date.h | 2 +-\n ref-filter.c | 3 ++-\n revision.c | 3 ++-\n t/helper/test-date.c | 3 ++-\n 7 files changed, 11 insertions(+), 9 deletions(-)\n\ndiff --git a/builtin/blame.c b/builtin/blame.c\nindex 9a3f9facea..665511570d 100644\n--- a/builtin/blame.c\n+++ b/builtin/blame.c\n@@ -714,7 +714,7 @@ static int git_blame_config(const char *var, const char *value, void *cb)\n \tif (!strcmp(var, \"blame.date\")) {\n \t\tif (!value)\n \t\t\treturn config_error_nonbool(var);\n-\t\tparse_date_format(value, &blame_date_mode);\n+\t\tparse_date_format(value, &blame_date_mode, pager_in_use());\n \t\treturn 0;\n \t}\n \tif (!strcmp(var, \"blame.ignorerevsfile\")) {\ndiff --git a/builtin/log.c b/builtin/log.c\nindex 03954fb749..a72ce30c2e 100644\n--- a/builtin/log.c\n+++ b/builtin/log.c\n@@ -185,7 +185,7 @@ static void cmd_log_init_defaults(struct rev_info *rev)\n \trev->diffopt.flags.allow_textconv = 1;\n \n \tif (default_date_mode)\n-\t\tparse_date_format(default_date_mode, &rev->date_mode);\n+\t\tparse_date_format(default_date_mode, &rev->date_mode, pager_in_use());\n }\n \n static void set_default_decoration_filter(struct decoration_filter *decoration_filter)\ndiff --git a/date.c b/date.c\nindex 619ada5b20..55f73ce2e0 100644\n--- a/date.c\n+++ b/date.c\n@@ -7,7 +7,6 @@\n #include \"git-compat-util.h\"\n #include \"date.h\"\n #include \"gettext.h\"\n-#include \"pager.h\"\n #include \"strbuf.h\"\n \n /*\n@@ -1003,13 +1002,13 @@ static enum date_mode_type parse_date_type(const char *format, const char **end)\n \tdie(\"unknown date format %s\", format);\n }\n \n-void parse_date_format(const char *format, struct date_mode *mode)\n+void parse_date_format(const char *format, struct date_mode *mode, int pager_in_use)\n {\n \tconst char *p;\n \n \t/* \"auto:foo\" is \"if tty/pager, then foo, otherwise normal\" */\n \tif (skip_prefix(format, \"auto:\", &p)) {\n-\t\tif (isatty(1) || pager_in_use())\n+\t\tif (isatty(1) || pager_in_use)\n \t\t\tformat = p;\n \t\telse\n \t\t\tformat = \"default\";\ndiff --git a/date.h b/date.h\nindex 6136212a19..d9bd6dc09f 100644\n--- a/date.h\n+++ b/date.h\n@@ -53,7 +53,7 @@ const char *show_date(timestamp_t time, int timezone, const struct date_mode *mo\n * be used with strbuf_addftime(), in which case you'll need to call\n * date_mode_release() later.\n */\n-void parse_date_format(const char *format, struct date_mode *mode);\n+void parse_date_format(const char *format, struct date_mode *mode, int pager_in_use);\n \n /**\n * Release a \"struct date_mode\", currently only required if\ndiff --git a/ref-filter.c b/ref-filter.c\nindex 2ed0ecf260..1b96bb7822 100644\n--- a/ref-filter.c\n+++ b/ref-filter.c\n@@ -28,6 +28,7 @@\n #include \"worktree.h\"\n #include \"hashmap.h\"\n #include \"strvec.h\"\n+#include \"pager.h\"\n \n static struct ref_msg {\n \tconst char *gone;\n@@ -1323,7 +1324,7 @@ static void grab_date(const char *buf, struct atom_value *v, const char *atomnam\n \tformatp = strchr(atomname, ':');\n \tif (formatp) {\n \t\tformatp++;\n-\t\tparse_date_format(formatp, &date_mode);\n+\t\tparse_date_format(formatp, &date_mode, pager_in_use());\n \t}\n \n \tif (!eoemail)\ndiff --git a/revision.c b/revision.c\nindex 985b8b2f51..c7efd11914 100644\n--- a/revision.c\n+++ b/revision.c\n@@ -46,6 +46,7 @@\n #include \"resolve-undo.h\"\n #include \"parse-options.h\"\n #include \"wildmatch.h\"\n+#include \"pager.h\"\n \n volatile show_early_output_fn_t show_early_output;\n \n@@ -2577,7 +2578,7 @@ static int handle_revision_opt(struct rev_info *revs, int argc, const char **arg\n \t\trevs->date_mode.type = DATE_RELATIVE;\n \t\trevs->date_mode_explicit = 1;\n \t} else if ((argcount = parse_long_opt(\"date\", argv, &optarg))) {\n-\t\tparse_date_format(optarg, &revs->date_mode);\n+\t\tparse_date_format(optarg, &revs->date_mode, pager_in_use());\n \t\trevs->date_mode_explicit = 1;\n \t\treturn argcount;\n \t} else if (!strcmp(arg, \"--log-size\")) {\ndiff --git a/t/helper/test-date.c b/t/helper/test-date.c\nindex 0683d46574..b3927a95b3 100644\n--- a/t/helper/test-date.c\n+++ b/t/helper/test-date.c\n@@ -1,5 +1,6 @@\n #include \"test-tool.h\"\n #include \"date.h\"\n+#include \"pager.h\"\n #include \"trace.h\"\n \n static const char *usage_msg = \"\\n\"\n@@ -37,7 +38,7 @@ static void show_dates(const char **argv, const char *format)\n {\n \tstruct date_mode mode = DATE_MODE_INIT;\n \n-\tparse_date_format(format, &mode);\n+\tparse_date_format(format, &mode, pager_in_use());\n \tfor (; *argv; argv++) {\n \t\tchar *arg;\n \t\ttimestamp_t t;\n-- \n2.41.0.640.ga95def55d0-goog\n\n"},{"id":"480454","messageId":"20230810163654.275023-4-calvinwan@google.com","threadId":"59921","inReplyTo":"20230810163346.274132-1-calvinwan@google.com","subject":"[RFC PATCH v2 4/7] parse: create new library for parsing strings and env values","fromName":"Calvin Wan","fromEmail":"calvinwan@google.com","sentAt":"2023-08-10T16:36:51Z","receivedAt":"2023-08-10T16:38:46Z","isPatch":true,"sender":{"key":"calvinwan@google.com","avatar":"https://avatars.githubusercontent.com/u/92547554?v=4"},"body":"While string and environment value parsing is mainly consumed by\nconfig.c, there are other files that only need parsing functionality and\nnot config functionality. By separating out string and environment value\nparsing from config, those files can instead be dependent on parse,\nwhich has a much smaller dependency chain than config.\n\nMove general string and env parsing functions from config.[ch] to\nparse.[ch].\n\nSigned-off-by: Calvin Wan \n---\n Makefile | 1 +\n attr.c | 2 +-\n config.c | 180 +-----------------------------------\n config.h | 14 +--\n pack-objects.c | 2 +-\n pack-revindex.c | 2 +-\n parse-options.c | 3 +-\n parse.c | 182 +++++++++++++++++++++++++++++++++++++\n parse.h | 20 ++++\n pathspec.c | 2 +-\n preload-index.c | 2 +-\n progress.c | 2 +-\n prompt.c | 2 +-\n rebase.c | 2 +-\n t/helper/test-env-helper.c | 2 +-\n unpack-trees.c | 2 +-\n wrapper.c | 2 +-\n write-or-die.c | 2 +-\n 18 files changed, 219 insertions(+), 205 deletions(-)\n create mode 100644 parse.c\n create mode 100644 parse.h\n\ndiff --git a/Makefile b/Makefile\nindex 83b385b0be..e9ad9f9ef1 100644\n--- a/Makefile\n+++ b/Makefile\n@@ -1091,6 +1091,7 @@ LIB_OBJS += pack-write.o\n LIB_OBJS += packfile.o\n LIB_OBJS += pager.o\n LIB_OBJS += parallel-checkout.o\n+LIB_OBJS += parse.o\n LIB_OBJS += parse-options-cb.o\n LIB_OBJS += parse-options.o\n LIB_OBJS += patch-delta.o\ndiff --git a/attr.c b/attr.c\nindex e9c81b6e07..cb047b4618 100644\n--- a/attr.c\n+++ b/attr.c\n@@ -7,7 +7,7 @@\n */\n \n #include \"git-compat-util.h\"\n-#include \"config.h\"\n+#include \"parse.h\"\n #include \"environment.h\"\n #include \"exec-cmd.h\"\n #include \"attr.h\"\ndiff --git a/config.c b/config.c\nindex 5b71ef1624..cdd70999aa 100644\n--- a/config.c\n+++ b/config.c\n@@ -11,6 +11,7 @@\n #include \"date.h\"\n #include \"branch.h\"\n #include \"config.h\"\n+#include \"parse.h\"\n #include \"convert.h\"\n #include \"environment.h\"\n #include \"gettext.h\"\n@@ -1204,129 +1205,6 @@ static int git_parse_source(struct config_source *cs, config_fn_t fn,\n \treturn error_return;\n }\n \n-static uintmax_t get_unit_factor(const char *end)\n-{\n-\tif (!*end)\n-\t\treturn 1;\n-\telse if (!strcasecmp(end, \"k\"))\n-\t\treturn 1024;\n-\telse if (!strcasecmp(end, \"m\"))\n-\t\treturn 1024 * 1024;\n-\telse if (!strcasecmp(end, \"g\"))\n-\t\treturn 1024 * 1024 * 1024;\n-\treturn 0;\n-}\n-\n-static int git_parse_signed(const char *value, intmax_t *ret, intmax_t max)\n-{\n-\tif (value && *value) {\n-\t\tchar *end;\n-\t\tintmax_t val;\n-\t\tintmax_t factor;\n-\n-\t\tif (max < 0)\n-\t\t\tBUG(\"max must be a positive integer\");\n-\n-\t\terrno = 0;\n-\t\tval = strtoimax(value, &end, 0);\n-\t\tif (errno == ERANGE)\n-\t\t\treturn 0;\n-\t\tif (end == value) {\n-\t\t\terrno = EINVAL;\n-\t\t\treturn 0;\n-\t\t}\n-\t\tfactor = get_unit_factor(end);\n-\t\tif (!factor) {\n-\t\t\terrno = EINVAL;\n-\t\t\treturn 0;\n-\t\t}\n-\t\tif ((val < 0 && -max / factor > val) ||\n-\t\t (val > 0 && max / factor < val)) {\n-\t\t\terrno = ERANGE;\n-\t\t\treturn 0;\n-\t\t}\n-\t\tval *= factor;\n-\t\t*ret = val;\n-\t\treturn 1;\n-\t}\n-\terrno = EINVAL;\n-\treturn 0;\n-}\n-\n-static int git_parse_unsigned(const char *value, uintmax_t *ret, uintmax_t max)\n-{\n-\tif (value && *value) {\n-\t\tchar *end;\n-\t\tuintmax_t val;\n-\t\tuintmax_t factor;\n-\n-\t\t/* negative values would be accepted by strtoumax */\n-\t\tif (strchr(value, '-')) {\n-\t\t\terrno = EINVAL;\n-\t\t\treturn 0;\n-\t\t}\n-\t\terrno = 0;\n-\t\tval = strtoumax(value, &end, 0);\n-\t\tif (errno == ERANGE)\n-\t\t\treturn 0;\n-\t\tif (end == value) {\n-\t\t\terrno = EINVAL;\n-\t\t\treturn 0;\n-\t\t}\n-\t\tfactor = get_unit_factor(end);\n-\t\tif (!factor) {\n-\t\t\terrno = EINVAL;\n-\t\t\treturn 0;\n-\t\t}\n-\t\tif (unsigned_mult_overflows(factor, val) ||\n-\t\t factor * val > max) {\n-\t\t\terrno = ERANGE;\n-\t\t\treturn 0;\n-\t\t}\n-\t\tval *= factor;\n-\t\t*ret = val;\n-\t\treturn 1;\n-\t}\n-\terrno = EINVAL;\n-\treturn 0;\n-}\n-\n-int git_parse_int(const char *value, int *ret)\n-{\n-\tintmax_t tmp;\n-\tif (!git_parse_signed(value, &tmp, maximum_signed_value_of_type(int)))\n-\t\treturn 0;\n-\t*ret = tmp;\n-\treturn 1;\n-}\n-\n-static int git_parse_int64(const char *value, int64_t *ret)\n-{\n-\tintmax_t tmp;\n-\tif (!git_parse_signed(value, &tmp, maximum_signed_value_of_type(int64_t)))\n-\t\treturn 0;\n-\t*ret = tmp;\n-\treturn 1;\n-}\n-\n-int git_parse_ulong(const char *value, unsigned long *ret)\n-{\n-\tuintmax_t tmp;\n-\tif (!git_parse_unsigned(value, &tmp, maximum_unsigned_value_of_type(long)))\n-\t\treturn 0;\n-\t*ret = tmp;\n-\treturn 1;\n-}\n-\n-int git_parse_ssize_t(const char *value, ssize_t *ret)\n-{\n-\tintmax_t tmp;\n-\tif (!git_parse_signed(value, &tmp, maximum_signed_value_of_type(ssize_t)))\n-\t\treturn 0;\n-\t*ret = tmp;\n-\treturn 1;\n-}\n-\n static int reader_config_name(struct config_reader *reader, const char **out);\n static int reader_origin_type(struct config_reader *reader,\n \t\t\t enum config_origin_type *type);\n@@ -1404,23 +1282,6 @@ ssize_t git_config_ssize_t(const char *name, const char *value)\n \treturn ret;\n }\n \n-static int git_parse_maybe_bool_text(const char *value)\n-{\n-\tif (!value)\n-\t\treturn 1;\n-\tif (!*value)\n-\t\treturn 0;\n-\tif (!strcasecmp(value, \"true\")\n-\t || !strcasecmp(value, \"yes\")\n-\t || !strcasecmp(value, \"on\"))\n-\t\treturn 1;\n-\tif (!strcasecmp(value, \"false\")\n-\t || !strcasecmp(value, \"no\")\n-\t || !strcasecmp(value, \"off\"))\n-\t\treturn 0;\n-\treturn -1;\n-}\n-\n static const struct fsync_component_name {\n \tconst char *name;\n \tenum fsync_component component_bits;\n@@ -1495,16 +1356,6 @@ static enum fsync_component parse_fsync_components(const char *var, const char *\n \treturn (current & ~negative) | positive;\n }\n \n-int git_parse_maybe_bool(const char *value)\n-{\n-\tint v = git_parse_maybe_bool_text(value);\n-\tif (0 <= v)\n-\t\treturn v;\n-\tif (git_parse_int(value, &v))\n-\t\treturn !!v;\n-\treturn -1;\n-}\n-\n int git_config_bool_or_int(const char *name, const char *value, int *is_bool)\n {\n \tint v = git_parse_maybe_bool_text(value);\n@@ -2165,35 +2016,6 @@ void git_global_config(char **user_out, char **xdg_out)\n \t*xdg_out = xdg_config;\n }\n \n-/*\n- * Parse environment variable 'k' as a boolean (in various\n- * possible spellings); if missing, use the default value 'def'.\n- */\n-int git_env_bool(const char *k, int def)\n-{\n-\tconst char *v = getenv(k);\n-\tint val;\n-\tif (!v)\n-\t\treturn def;\n-\tval = git_parse_maybe_bool(v);\n-\tif (val < 0)\n-\t\tdie(_(\"bad boolean environment value '%s' for '%s'\"),\n-\t\t v, k);\n-\treturn val;\n-}\n-\n-/*\n- * Parse environment variable 'k' as ulong with possibly a unit\n- * suffix; if missing, use the default value 'val'.\n- */\n-unsigned long git_env_ulong(const char *k, unsigned long val)\n-{\n-\tconst char *v = getenv(k);\n-\tif (v && !git_parse_ulong(v, &val))\n-\t\tdie(_(\"failed to parse %s\"), k);\n-\treturn val;\n-}\n-\n int git_config_system(void)\n {\n \treturn !git_env_bool(\"GIT_CONFIG_NOSYSTEM\", 0);\ndiff --git a/config.h b/config.h\nindex 247b572b37..7a7f53e503 100644\n--- a/config.h\n+++ b/config.h\n@@ -3,7 +3,7 @@\n \n #include \"hashmap.h\"\n #include \"string-list.h\"\n-\n+#include \"parse.h\"\n \n /**\n * The config API gives callers a way to access Git configuration files\n@@ -205,16 +205,6 @@ int config_with_options(config_fn_t fn, void *,\n * The following helper functions aid in parsing string values\n */\n \n-int git_parse_ssize_t(const char *, ssize_t *);\n-int git_parse_ulong(const char *, unsigned long *);\n-int git_parse_int(const char *value, int *ret);\n-\n-/**\n- * Same as `git_config_bool`, except that it returns -1 on error rather\n- * than dying.\n- */\n-int git_parse_maybe_bool(const char *);\n-\n /**\n * Parse the string to an integer, including unit factors. Dies on error;\n * otherwise, returns the parsed result.\n@@ -343,8 +333,6 @@ int git_config_rename_section(const char *, const char *);\n int git_config_rename_section_in_file(const char *, const char *, const char *);\n int git_config_copy_section(const char *, const char *);\n int git_config_copy_section_in_file(const char *, const char *, const char *);\n-int git_env_bool(const char *, int);\n-unsigned long git_env_ulong(const char *, unsigned long);\n int git_config_system(void);\n int config_error_nonbool(const char *);\n #if defined(__GNUC__)\ndiff --git a/pack-objects.c b/pack-objects.c\nindex 1b8052bece..f403ca6986 100644\n--- a/pack-objects.c\n+++ b/pack-objects.c\n@@ -3,7 +3,7 @@\n #include \"pack.h\"\n #include \"pack-objects.h\"\n #include \"packfile.h\"\n-#include \"config.h\"\n+#include \"parse.h\"\n \n static uint32_t locate_object_entry_hash(struct packing_data *pdata,\n \t\t\t\t\t const struct object_id *oid,\ndiff --git a/pack-revindex.c b/pack-revindex.c\nindex 7fffcad912..a01a2a4640 100644\n--- a/pack-revindex.c\n+++ b/pack-revindex.c\n@@ -6,7 +6,7 @@\n #include \"packfile.h\"\n #include \"strbuf.h\"\n #include \"trace2.h\"\n-#include \"config.h\"\n+#include \"parse.h\"\n #include \"midx.h\"\n #include \"csum-file.h\"\n \ndiff --git a/parse-options.c b/parse-options.c\nindex f8a155ee13..9f542950a7 100644\n--- a/parse-options.c\n+++ b/parse-options.c\n@@ -1,11 +1,12 @@\n #include \"git-compat-util.h\"\n #include \"parse-options.h\"\n #include \"abspath.h\"\n-#include \"config.h\"\n+#include \"parse.h\"\n #include \"commit.h\"\n #include \"color.h\"\n #include \"gettext.h\"\n #include \"strbuf.h\"\n+#include \"string-list.h\"\n #include \"utf8.h\"\n \n static int disallow_abbreviated_options;\ndiff --git a/parse.c b/parse.c\nnew file mode 100644\nindex 0000000000..42d691a0fb\n--- /dev/null\n+++ b/parse.c\n@@ -0,0 +1,182 @@\n+#include \"git-compat-util.h\"\n+#include \"gettext.h\"\n+#include \"parse.h\"\n+\n+static uintmax_t get_unit_factor(const char *end)\n+{\n+\tif (!*end)\n+\t\treturn 1;\n+\telse if (!strcasecmp(end, \"k\"))\n+\t\treturn 1024;\n+\telse if (!strcasecmp(end, \"m\"))\n+\t\treturn 1024 * 1024;\n+\telse if (!strcasecmp(end, \"g\"))\n+\t\treturn 1024 * 1024 * 1024;\n+\treturn 0;\n+}\n+\n+int git_parse_signed(const char *value, intmax_t *ret, intmax_t max)\n+{\n+\tif (value && *value) {\n+\t\tchar *end;\n+\t\tintmax_t val;\n+\t\tintmax_t factor;\n+\n+\t\tif (max < 0)\n+\t\t\tBUG(\"max must be a positive integer\");\n+\n+\t\terrno = 0;\n+\t\tval = strtoimax(value, &end, 0);\n+\t\tif (errno == ERANGE)\n+\t\t\treturn 0;\n+\t\tif (end == value) {\n+\t\t\terrno = EINVAL;\n+\t\t\treturn 0;\n+\t\t}\n+\t\tfactor = get_unit_factor(end);\n+\t\tif (!factor) {\n+\t\t\terrno = EINVAL;\n+\t\t\treturn 0;\n+\t\t}\n+\t\tif ((val < 0 && -max / factor > val) ||\n+\t\t (val > 0 && max / factor < val)) {\n+\t\t\terrno = ERANGE;\n+\t\t\treturn 0;\n+\t\t}\n+\t\tval *= factor;\n+\t\t*ret = val;\n+\t\treturn 1;\n+\t}\n+\terrno = EINVAL;\n+\treturn 0;\n+}\n+\n+static int git_parse_unsigned(const char *value, uintmax_t *ret, uintmax_t max)\n+{\n+\tif (value && *value) {\n+\t\tchar *end;\n+\t\tuintmax_t val;\n+\t\tuintmax_t factor;\n+\n+\t\t/* negative values would be accepted by strtoumax */\n+\t\tif (strchr(value, '-')) {\n+\t\t\terrno = EINVAL;\n+\t\t\treturn 0;\n+\t\t}\n+\t\terrno = 0;\n+\t\tval = strtoumax(value, &end, 0);\n+\t\tif (errno == ERANGE)\n+\t\t\treturn 0;\n+\t\tif (end == value) {\n+\t\t\terrno = EINVAL;\n+\t\t\treturn 0;\n+\t\t}\n+\t\tfactor = get_unit_factor(end);\n+\t\tif (!factor) {\n+\t\t\terrno = EINVAL;\n+\t\t\treturn 0;\n+\t\t}\n+\t\tif (unsigned_mult_overflows(factor, val) ||\n+\t\t factor * val > max) {\n+\t\t\terrno = ERANGE;\n+\t\t\treturn 0;\n+\t\t}\n+\t\tval *= factor;\n+\t\t*ret = val;\n+\t\treturn 1;\n+\t}\n+\terrno = EINVAL;\n+\treturn 0;\n+}\n+\n+int git_parse_int(const char *value, int *ret)\n+{\n+\tintmax_t tmp;\n+\tif (!git_parse_signed(value, &tmp, maximum_signed_value_of_type(int)))\n+\t\treturn 0;\n+\t*ret = tmp;\n+\treturn 1;\n+}\n+\n+int git_parse_int64(const char *value, int64_t *ret)\n+{\n+\tintmax_t tmp;\n+\tif (!git_parse_signed(value, &tmp, maximum_signed_value_of_type(int64_t)))\n+\t\treturn 0;\n+\t*ret = tmp;\n+\treturn 1;\n+}\n+\n+int git_parse_ulong(const char *value, unsigned long *ret)\n+{\n+\tuintmax_t tmp;\n+\tif (!git_parse_unsigned(value, &tmp, maximum_unsigned_value_of_type(long)))\n+\t\treturn 0;\n+\t*ret = tmp;\n+\treturn 1;\n+}\n+\n+int git_parse_ssize_t(const char *value, ssize_t *ret)\n+{\n+\tintmax_t tmp;\n+\tif (!git_parse_signed(value, &tmp, maximum_signed_value_of_type(ssize_t)))\n+\t\treturn 0;\n+\t*ret = tmp;\n+\treturn 1;\n+}\n+\n+int git_parse_maybe_bool_text(const char *value)\n+{\n+\tif (!value)\n+\t\treturn 1;\n+\tif (!*value)\n+\t\treturn 0;\n+\tif (!strcasecmp(value, \"true\")\n+\t || !strcasecmp(value, \"yes\")\n+\t || !strcasecmp(value, \"on\"))\n+\t\treturn 1;\n+\tif (!strcasecmp(value, \"false\")\n+\t || !strcasecmp(value, \"no\")\n+\t || !strcasecmp(value, \"off\"))\n+\t\treturn 0;\n+\treturn -1;\n+}\n+\n+int git_parse_maybe_bool(const char *value)\n+{\n+\tint v = git_parse_maybe_bool_text(value);\n+\tif (0 <= v)\n+\t\treturn v;\n+\tif (git_parse_int(value, &v))\n+\t\treturn !!v;\n+\treturn -1;\n+}\n+\n+/*\n+ * Parse environment variable 'k' as a boolean (in various\n+ * possible spellings); if missing, use the default value 'def'.\n+ */\n+int git_env_bool(const char *k, int def)\n+{\n+\tconst char *v = getenv(k);\n+\tint val;\n+\tif (!v)\n+\t\treturn def;\n+\tval = git_parse_maybe_bool(v);\n+\tif (val < 0)\n+\t\tdie(_(\"bad boolean environment value '%s' for '%s'\"),\n+\t\t v, k);\n+\treturn val;\n+}\n+\n+/*\n+ * Parse environment variable 'k' as ulong with possibly a unit\n+ * suffix; if missing, use the default value 'val'.\n+ */\n+unsigned long git_env_ulong(const char *k, unsigned long val)\n+{\n+\tconst char *v = getenv(k);\n+\tif (v && !git_parse_ulong(v, &val))\n+\t\tdie(_(\"failed to parse %s\"), k);\n+\treturn val;\n+}\ndiff --git a/parse.h b/parse.h\nnew file mode 100644\nindex 0000000000..07d2193d69\n--- /dev/null\n+++ b/parse.h\n@@ -0,0 +1,20 @@\n+#ifndef PARSE_H\n+#define PARSE_H\n+\n+int git_parse_signed(const char *value, intmax_t *ret, intmax_t max);\n+int git_parse_ssize_t(const char *, ssize_t *);\n+int git_parse_ulong(const char *, unsigned long *);\n+int git_parse_int(const char *value, int *ret);\n+int git_parse_int64(const char *value, int64_t *ret);\n+\n+/**\n+ * Same as `git_config_bool`, except that it returns -1 on error rather\n+ * than dying.\n+ */\n+int git_parse_maybe_bool(const char *);\n+int git_parse_maybe_bool_text(const char *value);\n+\n+int git_env_bool(const char *, int);\n+unsigned long git_env_ulong(const char *, unsigned long);\n+\n+#endif /* PARSE_H */\ndiff --git a/pathspec.c b/pathspec.c\nindex 4991455281..39337999d4 100644\n--- a/pathspec.c\n+++ b/pathspec.c\n@@ -1,6 +1,6 @@\n #include \"git-compat-util.h\"\n #include \"abspath.h\"\n-#include \"config.h\"\n+#include \"parse.h\"\n #include \"dir.h\"\n #include \"environment.h\"\n #include \"gettext.h\"\ndiff --git a/preload-index.c b/preload-index.c\nindex e44530c80c..63fd35d64b 100644\n--- a/preload-index.c\n+++ b/preload-index.c\n@@ -7,7 +7,7 @@\n #include \"environment.h\"\n #include \"fsmonitor.h\"\n #include \"gettext.h\"\n-#include \"config.h\"\n+#include \"parse.h\"\n #include \"preload-index.h\"\n #include \"progress.h\"\n #include \"read-cache.h\"\ndiff --git a/progress.c b/progress.c\nindex f695798aca..c83cb60bf1 100644\n--- a/progress.c\n+++ b/progress.c\n@@ -17,7 +17,7 @@\n #include \"trace.h\"\n #include \"trace2.h\"\n #include \"utf8.h\"\n-#include \"config.h\"\n+#include \"parse.h\"\n \n #define TP_IDX_MAX 8\n \ndiff --git a/prompt.c b/prompt.c\nindex 3baa33f63d..8935fe4dfb 100644\n--- a/prompt.c\n+++ b/prompt.c\n@@ -1,5 +1,5 @@\n #include \"git-compat-util.h\"\n-#include \"config.h\"\n+#include \"parse.h\"\n #include \"environment.h\"\n #include \"run-command.h\"\n #include \"strbuf.h\"\ndiff --git a/rebase.c b/rebase.c\nindex 17a570f1ff..69a1822da3 100644\n--- a/rebase.c\n+++ b/rebase.c\n@@ -1,6 +1,6 @@\n #include \"git-compat-util.h\"\n #include \"rebase.h\"\n-#include \"config.h\"\n+#include \"parse.h\"\n #include \"gettext.h\"\n \n /*\ndiff --git a/t/helper/test-env-helper.c b/t/helper/test-env-helper.c\nindex 66c88b8ff3..1c486888a4 100644\n--- a/t/helper/test-env-helper.c\n+++ b/t/helper/test-env-helper.c\n@@ -1,5 +1,5 @@\n #include \"test-tool.h\"\n-#include \"config.h\"\n+#include \"parse.h\"\n #include \"parse-options.h\"\n \n static char const * const env__helper_usage[] = {\ndiff --git a/unpack-trees.c b/unpack-trees.c\nindex 87517364dc..761562a96e 100644\n--- a/unpack-trees.c\n+++ b/unpack-trees.c\n@@ -2,7 +2,7 @@\n #include \"advice.h\"\n #include \"strvec.h\"\n #include \"repository.h\"\n-#include \"config.h\"\n+#include \"parse.h\"\n #include \"dir.h\"\n #include \"environment.h\"\n #include \"gettext.h\"\ndiff --git a/wrapper.c b/wrapper.c\nindex 118d3033de..a6249cc30e 100644\n--- a/wrapper.c\n+++ b/wrapper.c\n@@ -3,7 +3,7 @@\n */\n #include \"git-compat-util.h\"\n #include \"abspath.h\"\n-#include \"config.h\"\n+#include \"parse.h\"\n #include \"gettext.h\"\n #include \"repository.h\"\n #include \"strbuf.h\"\ndiff --git a/write-or-die.c b/write-or-die.c\nindex d8355c0c3e..42a2dc73cd 100644\n--- a/write-or-die.c\n+++ b/write-or-die.c\n@@ -1,5 +1,5 @@\n #include \"git-compat-util.h\"\n-#include \"config.h\"\n+#include \"parse.h\"\n #include \"run-command.h\"\n #include \"write-or-die.h\"\n \n-- \n2.41.0.640.ga95def55d0-goog\n\n"},{"id":"480455","messageId":"20230810163654.275023-7-calvinwan@google.com","threadId":"59921","inReplyTo":"20230810163346.274132-1-calvinwan@google.com","subject":"[RFC PATCH v2 7/7] git-std-lib: add test file to call git-std-lib.a functions","fromName":"Calvin Wan","fromEmail":"calvinwan@google.com","sentAt":"2023-08-10T16:36:54Z","receivedAt":"2023-08-10T16:38:46Z","isPatch":true,"sender":{"key":"calvinwan@google.com","avatar":"https://avatars.githubusercontent.com/u/92547554?v=4"},"body":"Add test file that directly or indirectly calls all functions defined in\ngit-std-lib.a object files to showcase that they do not reference\nmissing objects and that git-std-lib.a can stand on its own.\n\nCertain functions that cause the program to exit or are already called\nby other functions are commented out.\n\nTODO: replace with unit tests\nSigned-off-by: Calvin Wan \n---\n t/Makefile | 4 +\n t/stdlib-test.c | 239 ++++++++++++++++++++++++++++++++++++++++++++++++\n 2 files changed, 243 insertions(+)\n create mode 100644 t/stdlib-test.c\n\ndiff --git a/t/Makefile b/t/Makefile\nindex 3e00cdd801..b6d0bc9daa 100644\n--- a/t/Makefile\n+++ b/t/Makefile\n@@ -150,3 +150,7 @@ perf:\n \n .PHONY: pre-clean $(T) aggregate-results clean valgrind perf \\\n \tcheck-chainlint clean-chainlint test-chainlint\n+\n+test-git-std-lib:\n+\tcc -It -o stdlib-test stdlib-test.c -L. -l:../git-std-lib.a\n+\t./stdlib-test\ndiff --git a/t/stdlib-test.c b/t/stdlib-test.c\nnew file mode 100644\nindex 0000000000..a5d7374e2f\n--- /dev/null\n+++ b/t/stdlib-test.c\n@@ -0,0 +1,239 @@\n+#include \"../git-compat-util.h\"\n+#include \"../abspath.h\"\n+#include \"../hex-ll.h\"\n+#include \"../parse.h\"\n+#include \"../strbuf.h\"\n+#include \"../string-list.h\"\n+\n+/*\n+ * Calls all functions from git-std-lib\n+ * Some inline/trivial functions are skipped\n+ */\n+\n+void abspath_funcs(void) {\n+\tstruct strbuf sb = STRBUF_INIT;\n+\n+\tfprintf(stderr, \"calling abspath functions\\n\");\n+\tis_directory(\"foo\");\n+\tstrbuf_realpath(&sb, \"foo\", 0);\n+\tstrbuf_realpath_forgiving(&sb, \"foo\", 0);\n+\treal_pathdup(\"foo\", 0);\n+\tabsolute_path(\"foo\");\n+\tabsolute_pathdup(\"foo\");\n+\tprefix_filename(\"foo/\", \"bar\");\n+\tprefix_filename_except_for_dash(\"foo/\", \"bar\");\n+\tis_absolute_path(\"foo\");\n+\tstrbuf_add_absolute_path(&sb, \"foo\");\n+\tstrbuf_add_real_path(&sb, \"foo\");\n+}\n+\n+void hex_ll_funcs(void) {\n+\tunsigned char c;\n+\n+\tfprintf(stderr, \"calling hex-ll functions\\n\");\n+\n+\thexval('c');\n+\thex2chr(\"A1\");\n+\thex_to_bytes(&c, \"A1\", 2);\n+}\n+\n+void parse_funcs(void) {\n+\tintmax_t foo;\n+\tssize_t foo1 = -1;\n+\tunsigned long foo2;\n+\tint foo3;\n+\tint64_t foo4;\n+\n+\tfprintf(stderr, \"calling parse functions\\n\");\n+\n+\tgit_parse_signed(\"42\", &foo, maximum_signed_value_of_type(int));\n+\tgit_parse_ssize_t(\"42\", &foo1);\n+\tgit_parse_ulong(\"42\", &foo2);\n+\tgit_parse_int(\"42\", &foo3);\n+\tgit_parse_int64(\"42\", &foo4);\n+\tgit_parse_maybe_bool(\"foo\");\n+\tgit_parse_maybe_bool_text(\"foo\");\n+\tgit_env_bool(\"foo\", 1);\n+\tgit_env_ulong(\"foo\", 1);\n+}\n+\n+static int allow_unencoded_fn(char ch) {\n+\treturn 0;\n+}\n+\n+void strbuf_funcs(void) {\n+\tstruct strbuf *sb = xmalloc(sizeof(void*));\n+\tstruct strbuf *sb2 = xmalloc(sizeof(void*));\n+\tstruct strbuf sb3 = STRBUF_INIT;\n+\tstruct string_list list = STRING_LIST_INIT_NODUP;\n+\tchar *buf = \"foo\";\n+\tstruct strbuf_expand_dict_entry dict[] = {\n+\t\t{ \"foo\", NULL, },\n+\t\t{ \"bar\", NULL, },\n+\t};\n+\tint fd = open(\"/dev/null\", O_RDONLY);\n+\n+\tfprintf(stderr, \"calling strbuf functions\\n\");\n+\n+\tstarts_with(\"foo\", \"bar\");\n+\tistarts_with(\"foo\", \"bar\");\n+\t// skip_to_optional_arg_default(const char *str, const char *prefix,\n+\t// \t\t\t const char **arg, const char *def)\n+\tstrbuf_init(sb, 0);\n+\tstrbuf_init(sb2, 0);\n+\tstrbuf_release(sb);\n+\tstrbuf_attach(sb, strbuf_detach(sb, NULL), 0, 0); // calls strbuf_grow\n+\tstrbuf_swap(sb, sb2);\n+\tstrbuf_setlen(sb, 0);\n+\tstrbuf_trim(sb); // calls strbuf_rtrim, strbuf_ltrim\n+\t// strbuf_rtrim() called by strbuf_trim()\n+\t// strbuf_ltrim() called by strbuf_trim()\n+\tstrbuf_trim_trailing_dir_sep(sb);\n+\tstrbuf_trim_trailing_newline(sb);\n+\tstrbuf_reencode(sb, \"foo\", \"bar\");\n+\tstrbuf_tolower(sb);\n+\tstrbuf_add_separated_string_list(sb, \" \", &list);\n+\tstrbuf_list_free(strbuf_split_buf(\"foo bar\", 8, ' ', -1));\n+\tstrbuf_cmp(sb, sb2);\n+\tstrbuf_addch(sb, 1);\n+\tstrbuf_splice(sb, 0, 1, \"foo\", 3);\n+\tstrbuf_insert(sb, 0, \"foo\", 3);\n+\t// strbuf_vinsertf() called by strbuf_insertf\n+\tstrbuf_insertf(sb, 0, \"%s\", \"foo\");\n+\tstrbuf_remove(sb, 0, 1);\n+\tstrbuf_add(sb, \"foo\", 3);\n+\tstrbuf_addbuf(sb, sb2);\n+\tstrbuf_join_argv(sb, 0, NULL, ' ');\n+\tstrbuf_addchars(sb, 1, 1);\n+\tstrbuf_addf(sb, \"%s\", \"foo\");\n+\tstrbuf_add_commented_lines(sb, \"foo\", 3, '#');\n+\tstrbuf_commented_addf(sb, '#', \"%s\", \"foo\");\n+\t// strbuf_vaddf() called by strbuf_addf()\n+\tstrbuf_expand(sb, \"%s\", strbuf_expand_literal_cb, NULL);\n+\tstrbuf_expand(sb, \"%s\", strbuf_expand_dict_cb, &dict);\n+\t// strbuf_expand_literal_cb() called by strbuf_expand()\n+\t// strbuf_expand_dict_cb() called by strbuf_expand()\n+\tstrbuf_addbuf_percentquote(sb, &sb3);\n+\tstrbuf_add_percentencode(sb, \"foo\", STRBUF_ENCODE_SLASH);\n+\tstrbuf_fread(sb, 0, stdin);\n+\tstrbuf_read(sb, fd, 0);\n+\tstrbuf_read_once(sb, fd, 0);\n+\tstrbuf_write(sb, stderr);\n+\tstrbuf_readlink(sb, \"/dev/null\", 0);\n+\tstrbuf_getcwd(sb);\n+\tstrbuf_getwholeline(sb, stderr, '\\n');\n+\tstrbuf_appendwholeline(sb, stderr, '\\n');\n+\tstrbuf_getline(sb, stderr);\n+\tstrbuf_getline_lf(sb, stderr);\n+\tstrbuf_getline_nul(sb, stderr);\n+\tstrbuf_getwholeline_fd(sb, fd, '\\n');\n+\tstrbuf_read_file(sb, \"/dev/null\", 0);\n+\tstrbuf_add_lines(sb, \"foo\", \"bar\", 0);\n+\tstrbuf_addstr_xml_quoted(sb, \"foo\");\n+\tstrbuf_addstr_urlencode(sb, \"foo\", allow_unencoded_fn);\n+\tstrbuf_humanise_bytes(sb, 42);\n+\tstrbuf_humanise_rate(sb, 42);\n+\tprintf_ln(\"%s\", sb);\n+\tfprintf_ln(stderr, \"%s\", sb);\n+\txstrdup_tolower(\"foo\");\n+\txstrdup_toupper(\"foo\");\n+\t// xstrvfmt() called by xstrfmt()\n+\txstrfmt(\"%s\", \"foo\");\n+\t// strbuf_addftime(struct strbuf *sb, const char *fmt, const struct tm *tm,\n+\t// \t int tz_offset, int suppress_tz_name)\n+\t// strbuf_stripspace(struct strbuf *sb, char comment_line_char)\n+\t// strbuf_strip_suffix(struct strbuf *sb, const char *suffix)\n+\t// strbuf_strip_file_from_path(struct strbuf *sb)\n+}\n+\n+static void error_builtin(const char *err, va_list params) {}\n+static void warn_builtin(const char *err, va_list params) {}\n+\n+static report_fn error_routine = error_builtin;\n+static report_fn warn_routine = warn_builtin;\n+\n+void usage_funcs(void) {\n+\tfprintf(stderr, \"calling usage functions\\n\");\n+\t// Functions that call exit() are commented out\n+\n+\t// usage()\n+\t// usagef()\n+\t// die()\n+\t// die_errno();\n+\terror(\"foo\");\n+\terror_errno(\"foo\");\n+\tdie_message(\"foo\");\n+\tdie_message_errno(\"foo\");\n+\twarning(\"foo\");\n+\twarning_errno(\"foo\");\n+\n+\t// set_die_routine();\n+\tget_die_message_routine();\n+\tset_error_routine(error_builtin);\n+\tget_error_routine();\n+\tset_warn_routine(warn_builtin);\n+\tget_warn_routine();\n+\t// set_die_is_recursing_routine();\n+}\n+\n+void wrapper_funcs(void) {\n+\tvoid *ptr = xmalloc(1);\n+\tint fd = open(\"/dev/null\", O_RDONLY);\n+\tstruct strbuf sb = STRBUF_INIT;\n+\tint mode = 0444;\n+\tchar host[PATH_MAX], path[PATH_MAX], path1[PATH_MAX];\n+\txsnprintf(path, sizeof(path), \"out-XXXXXX\");\n+\txsnprintf(path1, sizeof(path1), \"out-XXXXXX\");\n+\tint tmp;\n+\n+\tfprintf(stderr, \"calling wrapper functions\\n\");\n+\n+\txstrdup(\"foo\");\n+\txmalloc(1);\n+\txmallocz(1);\n+\txmallocz_gently(1);\n+\txmemdupz(\"foo\", 3);\n+\txstrndup(\"foo\", 3);\n+\txrealloc(ptr, 2);\n+\txcalloc(1, 1);\n+\txsetenv(\"foo\", \"bar\", 0);\n+\txopen(\"/dev/null\", O_RDONLY);\n+\txread(fd, &sb, 1);\n+\txwrite(fd, &sb, 1);\n+\txpread(fd, &sb, 1, 0);\n+\txdup(fd);\n+\txfopen(\"/dev/null\", \"r\");\n+\txfdopen(fd, \"r\");\n+\ttmp = xmkstemp(path);\n+\tclose(tmp);\n+\tunlink(path);\n+\ttmp = xmkstemp_mode(path1, mode);\n+\tclose(tmp);\n+\tunlink(path1);\n+\txgetcwd();\n+\tfopen_for_writing(path);\n+\tfopen_or_warn(path, \"r\");\n+\txstrncmpz(\"foo\", \"bar\", 3);\n+\t// xsnprintf() called above\n+\txgethostname(host, 3);\n+\ttmp = git_mkstemps_mode(path, 1, mode);\n+\tclose(tmp);\n+\tunlink(path);\n+\ttmp = git_mkstemp_mode(path, mode);\n+\tclose(tmp);\n+\tunlink(path);\n+\tread_in_full(fd, &sb, 1);\n+\twrite_in_full(fd, &sb, 1);\n+\tpread_in_full(fd, &sb, 1, 0);\n+}\n+\n+int main() {\n+\tabspath_funcs();\n+\thex_ll_funcs();\n+\tparse_funcs();\n+\tstrbuf_funcs();\n+\tusage_funcs();\n+\twrapper_funcs();\n+\tfprintf(stderr, \"all git-std-lib functions finished calling\\n\");\n+\treturn 0;\n+}\n-- \n2.41.0.640.ga95def55d0-goog\n\n"},{"id":"480456","messageId":"20230810163654.275023-6-calvinwan@google.com","threadId":"59921","inReplyTo":"20230810163346.274132-1-calvinwan@google.com","subject":"[RFC PATCH v2 6/7] git-std-lib: introduce git standard library","fromName":"Calvin Wan","fromEmail":"calvinwan@google.com","sentAt":"2023-08-10T16:36:53Z","receivedAt":"2023-08-10T16:38:46Z","isPatch":true,"sender":{"key":"calvinwan@google.com","avatar":"https://avatars.githubusercontent.com/u/92547554?v=4"},"body":"The Git Standard Library intends to serve as the foundational library\nand root dependency that other libraries in Git will be built off of.\nThat is to say, suppose we have libraries X and Y; a user that wants to\nuse X and Y would need to include X, Y, and this Git Standard Library.\n\nAdd Documentation/technical/git-std-lib.txt to further explain the\ndesign and rationale.\n\nSigned-off-by: Calvin Wan \n---\n Documentation/technical/git-std-lib.txt | 186 ++++++++++++++++++++++++\n Makefile | 62 +++++++-\n git-compat-util.h | 7 +-\n stubs/repository.c | 4 +\n stubs/repository.h | 8 +\n stubs/trace2.c | 22 +++\n stubs/trace2.h | 69 +++++++++\n symlinks.c | 2 +\n 8 files changed, 358 insertions(+), 2 deletions(-)\n create mode 100644 Documentation/technical/git-std-lib.txt\n create mode 100644 stubs/repository.c\n create mode 100644 stubs/repository.h\n create mode 100644 stubs/trace2.c\n create mode 100644 stubs/trace2.h\n\ndiff --git a/Documentation/technical/git-std-lib.txt b/Documentation/technical/git-std-lib.txt\nnew file mode 100644\nindex 0000000000..3d901a89b0\n--- /dev/null\n+++ b/Documentation/technical/git-std-lib.txt\n@@ -0,0 +1,186 @@\n+Git Standard Library\n+================\n+\n+The Git Standard Library intends to serve as the foundational library\n+and root dependency that other libraries in Git will be built off of.\n+That is to say, suppose we have libraries X and Y; a user that wants to\n+use X and Y would need to include X, Y, and this Git Standard Library.\n+This does not mean that the Git Standard Library will be the only\n+possible root dependency in the future, but rather the most significant\n+and widely used one.\n+\n+Dependency graph in libified Git\n+================\n+\n+If you look in the Git Makefile, all of the objects defined in the Git\n+library are compiled and archived into a singular file, libgit.a, which\n+is linked against by common-main.o with other external dependencies and\n+turned into the Git executable. In other words, the Git executable has\n+dependencies on libgit.a and a couple of external libraries. The\n+libfication of Git will not affect this current build flow, but instead\n+will provide an alternate method for building Git.\n+\n+With our current method of building Git, we can imagine the dependency\n+graph as such:\n+\n+ Git\n+ /\\\n+ / \\\n+ / \\\n+ libgit.a ext deps\n+\n+In libifying parts of Git, we want to shrink the dependency graph to\n+only the minimal set of dependencies, so libraries should not use\n+libgit.a. Instead, it would look like:\n+\n+ Git\n+ /\\\n+ / \\\n+ / \\\n+ libgit.a ext deps\n+ /\\\n+ / \\\n+ / \\\n+object-store.a (other lib)\n+ | /\n+ | /\n+ | /\n+ config.a / \n+ | /\n+ | /\n+ | /\n+git-std-lib.a\n+\n+Instead of containing all of the objects in Git, libgit.a would contain\n+objects that are not built by libraries it links against. Consequently,\n+if someone wanted their own custom build of Git with their own custom\n+implementation of the object store, they would only have to swap out\n+object-store.a rather than do a hard fork of Git.\n+\n+Rationale behind Git Standard Library\n+================\n+\n+The rationale behind Git Standard Library essentially is the result of\n+two observations within the Git codebase: every file includes\n+git-compat-util.h which defines functions in a couple of different\n+files, and wrapper.c + usage.c have difficult-to-separate circular\n+dependencies with each other and other files.\n+\n+Ubiquity of git-compat-util.h and circular dependencies\n+========\n+\n+Every file in the Git codebase includes git-compat-util.h. It serves as\n+\"a compatibility aid that isolates the knowledge of platform specific\n+inclusion order and what feature macros to define before including which\n+system header\" (Junio[1]). Since every file includes git-compat-util.h, and\n+git-compat-util.h includes wrapper.h and usage.h, it would make sense\n+for wrapper.c and usage.c to be a part of the root library. They have\n+difficult to separate circular dependencies with each other so they\n+can't be independent libraries. Wrapper.c has dependencies on parse.c,\n+abspath.c, strbuf.c, which in turn also have dependencies on usage.c and\n+wrapper.c -- more circular dependencies. \n+\n+Tradeoff between swappability and refactoring\n+========\n+\n+From the above dependency graph, we can see that git-std-lib.a could be\n+many smaller libraries rather than a singular library. So why choose a\n+singular library when multiple libraries can be individually easier to\n+swap and are more modular? A singular library requires less work to\n+separate out circular dependencies within itself so it becomes a\n+tradeoff question between work and reward. While there may be a point in\n+the future where a file like usage.c would want its own library so that\n+someone can have custom die() or error(), the work required to refactor\n+out the circular dependencies in some files would be enormous due to\n+their ubiquity so therefore I believe it is not worth the tradeoff\n+currently. Additionally, we can in the future choose to do this refactor\n+and change the API for the library if there becomes enough of a reason\n+to do so (remember we are avoiding promising stability of the interfaces\n+of those libraries).\n+\n+Reuse of compatibility functions in git-compat-util.h\n+========\n+\n+Most functions defined in git-compat-util.h are implemented in compat/\n+and have dependencies limited to strbuf.h and wrapper.h so they can be\n+easily included in git-std-lib.a, which as a root dependency means that\n+higher level libraries do not have to worry about compatibility files in\n+compat/. The rest of the functions defined in git-compat-util.h are\n+implemented in top level files and are hidden behind\n+an #ifdef if their implementation is not in git-std-lib.a.\n+\n+Rationale summary\n+========\n+\n+The Git Standard Library allows us to get the libification ball rolling\n+with other libraries in Git. By not spending many\n+more months attempting to refactor difficult circular dependencies and\n+instead spending that time getting to a state where we can test out\n+swapping a library out such as config or object store, we can prove the\n+viability of Git libification on a much faster time scale. Additionally\n+the code cleanups that have happened so far have been minor and\n+beneficial for the codebase. It is probable that making large movements\n+would negatively affect code clarity.\n+\n+Git Standard Library boundary\n+================\n+\n+While I have described above some useful heuristics for identifying\n+potential candidates for git-std-lib.a, a standard library should not\n+have a shaky definition for what belongs in it.\n+\n+ - Low-level files (aka operates only on other primitive types) that are\n+ used everywhere within the codebase (wrapper.c, usage.c, strbuf.c)\n+ - Dependencies that are low-level and widely used\n+ (abspath.c, date.c, hex-ll.c, parse.c, utf8.c)\n+ - low-level git/* files with functions defined in git-compat-util.h\n+ (ctype.c)\n+ - compat/*\n+ - stubbed out dependencies in stubs/ (stubs/repository.c, stubs/trace2.c)\n+\n+There are other files that might fit this definition, but that does not\n+mean it should belong in git-std-lib.a. Those files should start as\n+their own separate library since any file added to git-std-lib.a loses\n+its flexibility of being easily swappable.\n+\n+Wrapper.c and usage.c have dependencies on repository and trace2 that are\n+possible to remove at the cost of sacrificing the ability for standard Git\n+to be able to trace functions in those files and other files in git-std-lib.a.\n+In order for git-std-lib.a to compile with those dependencies, stubbed out\n+versions of those files are implemented and swapped in during compilation time.\n+\n+Files inside of Git Standard Library\n+================\n+\n+The initial set of files in git-std-lib.a are:\n+abspath.c\n+ctype.c\n+date.c\n+hex-ll.c\n+parse.c\n+strbuf.c\n+usage.c\n+utf8.c\n+wrapper.c\n+stubs/repository.c\n+stubs/trace2.c\n+relevant compat/ files\n+\n+Pitfalls\n+================\n+\n+There are a small amount of files under compat/* that have dependencies\n+not inside of git-std-lib.a. While those functions are not called on\n+Linux, other OSes might call those problematic functions. I don't see\n+this as a major problem, just moreso an observation that libification in\n+general may also require some minor compatibility work in the future.\n+\n+Testing\n+================\n+\n+Unit tests should catch any breakages caused by changes to files in\n+git-std-lib.a (i.e. introduction of a out of scope dependency) and new\n+functions introduced to git-std-lib.a will require unit tests written\n+for them.\n+\n+[1] https://lore.kernel.org/git/xmqqwn17sydw.fsf@gitster.g/\n\\ No newline at end of file\ndiff --git a/Makefile b/Makefile\nindex e9ad9f9ef1..82510cf50e 100644\n--- a/Makefile\n+++ b/Makefile\n@@ -669,6 +669,7 @@ FUZZ_PROGRAMS =\n GIT_OBJS =\n LIB_OBJS =\n SCALAR_OBJS =\n+STUB_OBJS =\n OBJECTS =\n OTHER_PROGRAMS =\n PROGRAM_OBJS =\n@@ -956,6 +957,7 @@ COCCI_SOURCES = $(filter-out $(THIRD_PARTY_SOURCES),$(FOUND_C_SOURCES))\n \n LIB_H = $(FOUND_H_SOURCES)\n \n+ifndef GIT_STD_LIB\n LIB_OBJS += abspath.o\n LIB_OBJS += add-interactive.o\n LIB_OBJS += add-patch.o\n@@ -1196,6 +1198,27 @@ LIB_OBJS += write-or-die.o\n LIB_OBJS += ws.o\n LIB_OBJS += wt-status.o\n LIB_OBJS += xdiff-interface.o\n+else ifdef GIT_STD_LIB\n+LIB_OBJS += abspath.o\n+LIB_OBJS += ctype.o\n+LIB_OBJS += date.o\n+LIB_OBJS += hex-ll.o\n+LIB_OBJS += parse.o\n+LIB_OBJS += strbuf.o\n+LIB_OBJS += usage.o\n+LIB_OBJS += utf8.o\n+LIB_OBJS += wrapper.o\n+\n+ifdef STUB_REPOSITORY\n+STUB_OBJS += stubs/repository.o\n+endif\n+\n+ifdef STUB_TRACE2\n+STUB_OBJS += stubs/trace2.o\n+endif\n+\n+LIB_OBJS += $(STUB_OBJS)\n+endif\n \n BUILTIN_OBJS += builtin/add.o\n BUILTIN_OBJS += builtin/am.o\n@@ -2162,6 +2185,11 @@ ifdef FSMONITOR_OS_SETTINGS\n \tCOMPAT_OBJS += compat/fsmonitor/fsm-path-utils-$(FSMONITOR_OS_SETTINGS).o\n endif\n \n+ifdef GIT_STD_LIB\n+\tBASIC_CFLAGS += -DGIT_STD_LIB\n+\tBASIC_CFLAGS += -DNO_GETTEXT\n+endif\n+\n ifeq ($(TCLTK_PATH),)\n NO_TCLTK = NoThanks\n endif\n@@ -3654,7 +3682,7 @@ clean: profile-clean coverage-clean cocciclean\n \t$(RM) po/git.pot po/git-core.pot\n \t$(RM) git.res\n \t$(RM) $(OBJECTS)\n-\t$(RM) $(LIB_FILE) $(XDIFF_LIB) $(REFTABLE_LIB) $(REFTABLE_TEST_LIB)\n+\t$(RM) $(LIB_FILE) $(XDIFF_LIB) $(REFTABLE_LIB) $(REFTABLE_TEST_LIB) $(STD_LIB_FILE)\n \t$(RM) $(ALL_PROGRAMS) $(SCRIPT_LIB) $(BUILT_INS) $(OTHER_PROGRAMS)\n \t$(RM) $(TEST_PROGRAMS)\n \t$(RM) $(FUZZ_PROGRAMS)\n@@ -3834,3 +3862,35 @@ $(FUZZ_PROGRAMS): all\n \t\t$(XDIFF_OBJS) $(EXTLIBS) git.o $@.o $(LIB_FUZZING_ENGINE) -o $@\n \n fuzz-all: $(FUZZ_PROGRAMS)\n+\n+### Libified Git rules\n+\n+# git-std-lib\n+# `make git-std-lib GIT_STD_LIB=YesPlease STUB_REPOSITORY=YesPlease STUB_TRACE2=YesPlease`\n+STD_LIB = git-std-lib.a\n+\n+$(STD_LIB): $(LIB_OBJS) $(COMPAT_OBJS) $(STUB_OBJS)\n+\t$(QUIET_AR)$(RM) $@ && $(AR) $(ARFLAGS) $@ $^\n+\n+TEMP_HEADERS = temp_headers/\n+\n+git-std-lib:\n+# Move headers to temporary folder and replace them with stubbed headers.\n+# After building, move headers and stubbed headers back.\n+ifneq ($(STUB_OBJS),)\n+\tmkdir -p $(TEMP_HEADERS); \\\n+\tfor d in $(STUB_OBJS); do \\\n+\t\tBASE=$${d%.*}; \\\n+\t\tmv $${BASE##*/}.h $(TEMP_HEADERS)$${BASE##*/}.h; \\\n+\t\tmv $${BASE}.h $${BASE##*/}.h; \\\n+\tdone; \\\n+\t$(MAKE) $(STD_LIB); \\\n+\tfor d in $(STUB_OBJS); do \\\n+\t\tBASE=$${d%.*}; \\\n+\t\tmv $${BASE##*/}.h $${BASE}.h; \\\n+\t\tmv $(TEMP_HEADERS)$${BASE##*/}.h $${BASE##*/}.h; \\\n+\tdone; \\\n+\trm -rf temp_headers\n+else\n+\t$(MAKE) $(STD_LIB)\n+endif\ndiff --git a/git-compat-util.h b/git-compat-util.h\nindex 481dac22b0..75aa9b263e 100644\n--- a/git-compat-util.h\n+++ b/git-compat-util.h\n@@ -396,8 +396,8 @@ static inline int noop_core_config(const char *var UNUSED,\n #define platform_core_config noop_core_config\n #endif\n \n+#if !defined(__MINGW32__) && !defined(_MSC_VER) && !defined(GIT_STD_LIB)\n int lstat_cache_aware_rmdir(const char *path);\n-#if !defined(__MINGW32__) && !defined(_MSC_VER)\n #define rmdir lstat_cache_aware_rmdir\n #endif\n \n@@ -787,9 +787,11 @@ const char *inet_ntop(int af, const void *src, char *dst, size_t size);\n #endif\n \n #ifdef NO_PTHREADS\n+#ifdef GIT_STD_LIB\n #define atexit git_atexit\n int git_atexit(void (*handler)(void));\n #endif\n+#endif\n \n /*\n * Limit size of IO chunks, because huge chunks only cause pain. OS X\n@@ -951,14 +953,17 @@ int git_access(const char *path, int mode);\n # endif\n #endif\n \n+#ifndef GIT_STD_LIB\n int cmd_main(int, const char **);\n \n /*\n * Intercept all calls to exit() and route them to trace2 to\n * optionally emit a message before calling the real exit().\n */\n+\n int common_exit(const char *file, int line, int code);\n #define exit(code) exit(common_exit(__FILE__, __LINE__, (code)))\n+#endif\n \n /*\n * You can mark a stack variable with UNLEAK(var) to avoid it being\ndiff --git a/stubs/repository.c b/stubs/repository.c\nnew file mode 100644\nindex 0000000000..f81520d083\n--- /dev/null\n+++ b/stubs/repository.c\n@@ -0,0 +1,4 @@\n+#include \"git-compat-util.h\"\n+#include \"repository.h\"\n+\n+struct repository *the_repository;\ndiff --git a/stubs/repository.h b/stubs/repository.h\nnew file mode 100644\nindex 0000000000..18262d748e\n--- /dev/null\n+++ b/stubs/repository.h\n@@ -0,0 +1,8 @@\n+#ifndef REPOSITORY_H\n+#define REPOSITORY_H\n+\n+struct repository { int stub; };\n+\n+extern struct repository *the_repository;\n+\n+#endif /* REPOSITORY_H */\ndiff --git a/stubs/trace2.c b/stubs/trace2.c\nnew file mode 100644\nindex 0000000000..efc3f9c1f3\n--- /dev/null\n+++ b/stubs/trace2.c\n@@ -0,0 +1,22 @@\n+#include \"git-compat-util.h\"\n+#include \"trace2.h\"\n+\n+void trace2_region_enter_fl(const char *file, int line, const char *category,\n+\t\t\t const char *label, const struct repository *repo, ...) { }\n+void trace2_region_leave_fl(const char *file, int line, const char *category,\n+\t\t\t const char *label, const struct repository *repo, ...) { }\n+void trace2_data_string_fl(const char *file, int line, const char *category,\n+\t\t\t const struct repository *repo, const char *key,\n+\t\t\t const char *value) { }\n+void trace2_cmd_ancestry_fl(const char *file, int line, const char **parent_names) { }\n+void trace2_cmd_error_va_fl(const char *file, int line, const char *fmt,\n+\t\t\t va_list ap) { }\n+void trace2_cmd_name_fl(const char *file, int line, const char *name) { }\n+void trace2_thread_start_fl(const char *file, int line,\n+\t\t\t const char *thread_base_name) { }\n+void trace2_thread_exit_fl(const char *file, int line) { }\n+void trace2_data_intmax_fl(const char *file, int line, const char *category,\n+\t\t\t const struct repository *repo, const char *key,\n+\t\t\t intmax_t value) { }\n+int trace2_is_enabled(void) { return 0; }\n+void trace2_collect_process_info(enum trace2_process_info_reason reason) { }\ndiff --git a/stubs/trace2.h b/stubs/trace2.h\nnew file mode 100644\nindex 0000000000..88ad7387ff\n--- /dev/null\n+++ b/stubs/trace2.h\n@@ -0,0 +1,69 @@\n+#ifndef TRACE2_H\n+#define TRACE2_H\n+\n+struct child_process { int stub; };\n+struct repository;\n+struct json_writer { int stub; };\n+\n+void trace2_region_enter_fl(const char *file, int line, const char *category,\n+\t\t\t const char *label, const struct repository *repo, ...);\n+\n+#define trace2_region_enter(category, label, repo) \\\n+\ttrace2_region_enter_fl(__FILE__, __LINE__, (category), (label), (repo))\n+\n+void trace2_region_leave_fl(const char *file, int line, const char *category,\n+\t\t\t const char *label, const struct repository *repo, ...);\n+\n+#define trace2_region_leave(category, label, repo) \\\n+\ttrace2_region_leave_fl(__FILE__, __LINE__, (category), (label), (repo))\n+\n+void trace2_data_string_fl(const char *file, int line, const char *category,\n+\t\t\t const struct repository *repo, const char *key,\n+\t\t\t const char *value);\n+\n+#define trace2_data_string(category, repo, key, value) \\\n+\ttrace2_data_string_fl(__FILE__, __LINE__, (category), (repo), (key), \\\n+\t\t\t (value))\n+\n+void trace2_cmd_ancestry_fl(const char *file, int line, const char **parent_names);\n+\n+#define trace2_cmd_ancestry(v) trace2_cmd_ancestry_fl(__FILE__, __LINE__, (v))\n+\n+void trace2_cmd_error_va_fl(const char *file, int line, const char *fmt,\n+\t\t\t va_list ap);\n+\n+#define trace2_cmd_error_va(fmt, ap) \\\n+\ttrace2_cmd_error_va_fl(__FILE__, __LINE__, (fmt), (ap))\n+\n+\n+void trace2_cmd_name_fl(const char *file, int line, const char *name);\n+\n+#define trace2_cmd_name(v) trace2_cmd_name_fl(__FILE__, __LINE__, (v))\n+\n+void trace2_thread_start_fl(const char *file, int line,\n+\t\t\t const char *thread_base_name);\n+\n+#define trace2_thread_start(thread_base_name) \\\n+\ttrace2_thread_start_fl(__FILE__, __LINE__, (thread_base_name))\n+\n+void trace2_thread_exit_fl(const char *file, int line);\n+\n+#define trace2_thread_exit() trace2_thread_exit_fl(__FILE__, __LINE__)\n+\n+void trace2_data_intmax_fl(const char *file, int line, const char *category,\n+\t\t\t const struct repository *repo, const char *key,\n+\t\t\t intmax_t value);\n+\n+#define trace2_data_intmax(category, repo, key, value) \\\n+\ttrace2_data_intmax_fl(__FILE__, __LINE__, (category), (repo), (key), \\\n+\t\t\t (value))\n+\n+enum trace2_process_info_reason {\n+\tTRACE2_PROCESS_INFO_STARTUP,\n+\tTRACE2_PROCESS_INFO_EXIT,\n+};\n+int trace2_is_enabled(void);\n+void trace2_collect_process_info(enum trace2_process_info_reason reason);\n+\n+#endif /* TRACE2_H */\n+\ndiff --git a/symlinks.c b/symlinks.c\nindex b29e340c2d..bced721a0c 100644\n--- a/symlinks.c\n+++ b/symlinks.c\n@@ -337,6 +337,7 @@ void invalidate_lstat_cache(void)\n \treset_lstat_cache(&default_cache);\n }\n \n+#if !defined(__MINGW32__) && !defined(_MSC_VER) && !defined(GIT_STD_LIB)\n #undef rmdir\n int lstat_cache_aware_rmdir(const char *path)\n {\n@@ -348,3 +349,4 @@ int lstat_cache_aware_rmdir(const char *path)\n \n \treturn ret;\n }\n+#endif\n-- \n2.41.0.640.ga95def55d0-goog\n\n"},{"id":"480466","messageId":"xmqqy1iir645.fsf@gitster.g","threadId":"59921","inReplyTo":"20230810163654.275023-2-calvinwan@google.com","subject":"Re: [RFC PATCH v2 2/7] object: move function to object.c","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2023-08-10T20:32:10Z","receivedAt":"2023-08-10T20:32:25Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Calvin Wan writes:\n\n> While remove_or_warn() is a simple ternary operator to call two other\n> wrapper functions, it creates an unnecessary dependency to object.h in\n> wrapper.c. Therefore move the function to object.[ch] where the concept\n> of GITLINKs is first defined.\n\nAn untold assumption here is that we would want to make wrapper.[ch]\nindependent of Git's internals?\n\nIf so, where the thing is moved to (i.e. object.c) is much less\ninteresting than the fact that the goal of this function is to make\nwrapper.[ch] less dependent on Git, so the title should reflect\nthat, no?\n\n> +/*\n> + * Calls the correct function out of {unlink,rmdir}_or_warn based on\n> + * the supplied file mode.\n> + */\n> +int remove_or_warn(unsigned int mode, const char *path);\n\nOK. That \"file mode\" thing is not a regular \"struct stat .st_mode\",\nbut knows Git's internals, hence it makes sense to have it on our\nside, not on the wrapper.[ch] side. That makes sense.\n\n> #endif /* OBJECT_H */\n> diff --git a/wrapper.c b/wrapper.c\n> index 22be9812a7..118d3033de 100644\n> --- a/wrapper.c\n> +++ b/wrapper.c\n> @@ -5,7 +5,6 @@\n> #include \"abspath.h\"\n> #include \"config.h\"\n> #include \"gettext.h\"\n> -#include \"object.h\"\n> #include \"repository.h\"\n> #include \"strbuf.h\"\n> #include \"trace2.h\"\n> @@ -647,11 +646,6 @@ int rmdir_or_warn(const char *file)\n> \treturn warn_if_unremovable(\"rmdir\", file, rmdir(file));\n> }\n> \n> -int remove_or_warn(unsigned int mode, const char *file)\n> -{\n> -\treturn S_ISGITLINK(mode) ? rmdir_or_warn(file) : unlink_or_warn(file);\n> -}\n> -\n> static int access_error_is_ok(int err, unsigned flag)\n> {\n> \treturn (is_missing_file_error(err) ||\n> diff --git a/wrapper.h b/wrapper.h\n> index c85b1328d1..272795f863 100644\n> --- a/wrapper.h\n> +++ b/wrapper.h\n> @@ -111,11 +111,6 @@ int unlink_or_msg(const char *file, struct strbuf *err);\n> * not exist.\n> */\n> int rmdir_or_warn(const char *path);\n> -/*\n> - * Calls the correct function out of {unlink,rmdir}_or_warn based on\n> - * the supplied file mode.\n> - */\n> -int remove_or_warn(unsigned int mode, const char *path);\n> \n> /*\n> * Call access(2), but warn for any error except \"missing file\"\n"},{"id":"480467","messageId":"xmqqttt6r5xc.fsf@gitster.g","threadId":"59921","inReplyTo":"20230810163654.275023-3-calvinwan@google.com","subject":"Re: [RFC PATCH v2 3/7] config: correct bad boolean env value error message","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2023-08-10T20:36:15Z","receivedAt":"2023-08-10T20:36:22Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Calvin Wan writes:\n\n> An incorrectly defined boolean environment value would result in the\n> following error message:\n>\n> bad boolean config value '%s' for '%s'\n>\n> This is a misnomer since environment value != config value. Instead of\n> calling git_config_bool() to parse the environment value, mimic the\n> functionality inside of git_config_bool() but with the correct error\n> message.\n\nMakes sense.\n\n>\n> Signed-off-by: Calvin Wan \n> ---\n> config.c | 9 ++++++++-\n> 1 file changed, 8 insertions(+), 1 deletion(-)\n>\n> diff --git a/config.c b/config.c\n> index 09851a6909..5b71ef1624 100644\n> --- a/config.c\n> +++ b/config.c\n> @@ -2172,7 +2172,14 @@ void git_global_config(char **user_out, char **xdg_out)\n> int git_env_bool(const char *k, int def)\n> {\n> \tconst char *v = getenv(k);\n> -\treturn v ? git_config_bool(k, v) : def;\n> +\tint val;\n> +\tif (!v)\n> +\t\treturn def;\n> +\tval = git_parse_maybe_bool(v);\n> +\tif (val < 0)\n> +\t\tdie(_(\"bad boolean environment value '%s' for '%s'\"),\n> +\t\t v, k);\n> +\treturn val;\n> }\n> \n> /*\n"},{"id":"480508","messageId":"kl6lmsyy8sfj.fsf@chooglen-macbookpro.roam.corp.google.com","threadId":"59921","inReplyTo":"20230810163346.274132-1-calvinwan@google.com","subject":"Re: [RFC PATCH v2 0/7] Introduce Git Standard Library","fromName":"Glen Choo","fromEmail":"chooglen@google.com","sentAt":"2023-08-10T22:05:04Z","receivedAt":"2023-08-10T22:05:15Z","isPatch":true,"sender":{"key":"glencbz@gmail.com","avatar":"https://avatars.githubusercontent.com/u/58092771?v=4"},"body":"Calvin Wan writes:\n\n> Calvin Wan (7):\n> hex-ll: split out functionality from hex\n> object: move function to object.c\n> config: correct bad boolean env value error message\n> parse: create new library for parsing strings and env values\n> date: push pager.h dependency up\n> git-std-lib: introduce git standard library\n> git-std-lib: add test file to call git-std-lib.a functions\n\nThis doesn't seem to apply to 'master'. Do you have a base commit that\nreviewers could apply the patches to?\n"},{"id":"480509","messageId":"kl6ljzu28qzt.fsf@chooglen-macbookpro.roam.corp.google.com","threadId":"59921","inReplyTo":"20230810163654.275023-2-calvinwan@google.com","subject":"Re: [RFC PATCH v2 2/7] object: move function to object.c","fromName":"Glen Choo","fromEmail":"chooglen@google.com","sentAt":"2023-08-10T22:36:06Z","receivedAt":"2023-08-10T22:36:11Z","isPatch":true,"sender":{"key":"glencbz@gmail.com","avatar":"https://avatars.githubusercontent.com/u/58092771?v=4"},"body":"Calvin Wan writes:\n\n> While remove_or_warn() is a simple ternary operator to call two other\n> wrapper functions, it creates an unnecessary dependency to object.h in\n> wrapper.c. Therefore move the function to object.[ch] where the concept\n> of GITLINKs is first defined.\n\nAs Junio mentioned elsewhere, I think we need to establish that\nwrapper.c should be free of Git-specific internals.\n\n> diff --git a/object.c b/object.c\n> index 60f954194f..cb29fcc304 100644\n> --- a/object.c\n> +++ b/object.c\n> @@ -617,3 +617,8 @@ void parsed_object_pool_clear(struct parsed_object_pool *o)\n> \tFREE_AND_NULL(o->object_state);\n> \tFREE_AND_NULL(o->shallow_stat);\n> }\n> +\n> +int remove_or_warn(unsigned int mode, const char *file)\n> +{\n> +\treturn S_ISGITLINK(mode) ? rmdir_or_warn(file) : unlink_or_warn(file);\n> +}\n\nSince this function really needs S_ISGITLINK (I tried to see if we could\njust replace it with S_ISDIR and get the same behavior, but we can't),\nthis really is a Git-specific thing, so yes, this should be moved out of\nwrapper.c.\n\nMinor point: I think a better home might be entry.[ch], because those\nfiles care about performing changes on the worktree based on the\nGit-specific file modes in the index, whereas object.[ch] seems more\nconcerned about the format of objects.\n"},{"id":"480510","messageId":"xmqq5y5mr023.fsf@gitster.g","threadId":"59921","inReplyTo":"kl6ljzu28qzt.fsf@chooglen-macbookpro.roam.corp.google.com","subject":"Re: [RFC PATCH v2 2/7] object: move function to object.c","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2023-08-10T22:43:00Z","receivedAt":"2023-08-10T22:43:08Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Glen Choo writes:\n\n> Minor point: I think a better home might be entry.[ch], because those\n> files care about performing changes on the worktree based on the\n> Git-specific file modes in the index, whereas object.[ch] seems more\n> concerned about the format of objects.\n\nYeah, I wasn't paying much attention on that point while reading the\npatch, and I do agree with you that entry.[ch] may be a better fit.\n\nThanks.\n\n"},{"id":"480513","messageId":"kl6ledka8owj.fsf@chooglen-macbookpro.roam.corp.google.com","threadId":"59921","inReplyTo":"20230810163654.275023-4-calvinwan@google.com","subject":"Re: [RFC PATCH v2 4/7] parse: create new library for parsing strings and env values","fromName":"Glen Choo","fromEmail":"chooglen@google.com","sentAt":"2023-08-10T23:21:16Z","receivedAt":"2023-08-10T23:21:20Z","isPatch":true,"sender":{"key":"glencbz@gmail.com","avatar":"https://avatars.githubusercontent.com/u/58092771?v=4"},"body":"Calvin Wan writes:\n\n> While string and environment value parsing is mainly consumed by\n> config.c, there are other files that only need parsing functionality and\n> not config functionality. By separating out string and environment value\n> parsing from config, those files can instead be dependent on parse,\n> which has a much smaller dependency chain than config.\n>\n> Move general string and env parsing functions from config.[ch] to\n> parse.[ch].\n\nAn unstated purpose of this patch is that parse.[ch] becomes part of\ngit-std-lib, but not config.[ch], right?\n\nI think it's reasonable to have the string value parsing logic in\ngit-std-lib, e.g. this parsing snippet from diff.c seems like a good\nthing to put into a library that wants to accept user input:\n\n static int parse_color_moved(const char *arg)\n {\n switch (git_parse_maybe_bool(arg)) {\n case 0:\n return COLOR_MOVED_NO;\n case 1:\n return COLOR_MOVED_DEFAULT;\n default:\n break;\n }\n\n if (!strcmp(arg, \"no\"))\n return COLOR_MOVED_NO;\n else if (!strcmp(arg, \"plain\"))\n return COLOR_MOVED_PLAIN;\n else if (!strcmp(arg, \"blocks\"))\n return COLOR_MOVED_BLOCKS;\n /* ... */\n }\n\nBut, I don't see a why a non-Git caller would want environment value\nparsing in git-std-lib. I wouldn't think that libraries should be\nreading Git-formatted environment variables. If I had to guess, you\narranged it this way because you want to keep xmalloc in git-std-lib,\nwhich has a dependency on env var parsing here:\n\n static int memory_limit_check(size_t size, int gentle)\n {\n static size_t limit = 0;\n if (!limit) {\n limit = git_env_ulong(\"GIT_ALLOC_LIMIT\", 0);\n if (!limit)\n limit = SIZE_MAX;\n }\n if (size > limit) {\n if (gentle) {\n error(\"attempting to allocate %\"PRIuMAX\" over limit %\"PRIuMAX,\n (uintmax_t)size, (uintmax_t)limit);\n return -1;\n } else\n die(\"attempting to allocate %\"PRIuMAX\" over limit %\"PRIuMAX,\n (uintmax_t)size, (uintmax_t)limit);\n }\n return 0;\n }\n\nIf we libified this as-is, wouldn't our caller start paying attention to\nthe GIT_ALLOC_LIMIT environment variable? That seems like an undesirable\nside effect.\n\nI see later in the series that you have \"stubs\", which are presumably\nentrypoints for the caller to specify their own implementations of\nGit-specific things. If so, then an alternative would be to provide a\n\"stub\" to get the memory limit, something like:\n\n /* wrapper.h aka the things to stub */\n size_t git_get_memory_limit(void);\n\n /* stub-wrapper-or-something.c aka Git's implementation of the stub */\n\n #include \"wrapper.h\"\n size_t git_get_memory_limit(void)\n {\n return git_env_ulong(\"GIT_ALLOC_LIMIT\", 0);\n }\n\n /* wrapper.c aka the thing in git-stb-lib */\n static int memory_limit_check(size_t size, int gentle)\n {\n static size_t limit = 0;\n if (!limit) {\n limit = git_get_memory_limit();\n if (!limit)\n limit = SIZE_MAX;\n }\n if (size > limit) {\n if (gentle) {\n error(\"attempting to allocate %\"PRIuMAX\" over limit %\"PRIuMAX,\n (uintmax_t)size, (uintmax_t)limit);\n return -1;\n } else\n die(\"attempting to allocate %\"PRIuMAX\" over limit %\"PRIuMAX,\n (uintmax_t)size, (uintmax_t)limit);\n }\n return 0;\n }\n"},{"id":"480514","messageId":"kl6lbkfe8nyb.fsf@chooglen-macbookpro.roam.corp.google.com","threadId":"59921","inReplyTo":"20230810163654.275023-5-calvinwan@google.com","subject":"Re: [RFC PATCH v2 5/7] date: push pager.h dependency up","fromName":"Glen Choo","fromEmail":"chooglen@google.com","sentAt":"2023-08-10T23:41:48Z","receivedAt":"2023-08-10T23:41:53Z","isPatch":true,"sender":{"key":"glencbz@gmail.com","avatar":"https://avatars.githubusercontent.com/u/58092771?v=4"},"body":"Calvin Wan writes:\n\n> In order for date.c to be included in git-std-lib, the dependency to\n> pager.h must be removed since it has dependencies on many other files\n> not in git-std-lib.\n\nDependencies aside, I doubt callers of Git libraries want Git's\npager-handling logic bundled in git-std-lib ;)\n\n> @@ -1003,13 +1002,13 @@ static enum date_mode_type parse_date_type(const char *format, const char **end)\n> \tdie(\"unknown date format %s\", format);\n> }\n> \n> -void parse_date_format(const char *format, struct date_mode *mode)\n> +void parse_date_format(const char *format, struct date_mode *mode, int pager_in_use)\n> {\n> \tconst char *p;\n> \n> \t/* \"auto:foo\" is \"if tty/pager, then foo, otherwise normal\" */\n> \tif (skip_prefix(format, \"auto:\", &p)) {\n> -\t\tif (isatty(1) || pager_in_use())\n> +\t\tif (isatty(1) || pager_in_use)\n> \t\t\tformat = p;\n> \t\telse\n> \t\t\tformat = \"default\";\n\nHm, it feels odd to ship a parsing option that changes based on whether\nthe caller isatty or not. Ideally we would stub this \"switch the value\nof auto\" logic too.\n\nWithout reading ahead, I'm not sure if there are other sorts of \"library\ninfluencing process-wide\" oddities like the one here and in the previous\npatch. I think it would be okay for us to merge this series with these,\nas long as we advertise to callers that the library boundary isn't very\nclean yet, and we eventually clean it up.\n"},{"id":"480515","messageId":"xmqqy1iipip4.fsf@gitster.g","threadId":"59921","inReplyTo":"kl6ledka8owj.fsf@chooglen-macbookpro.roam.corp.google.com","subject":"Re: [RFC PATCH v2 4/7] parse: create new library for parsing strings and env values","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2023-08-10T23:43:19Z","receivedAt":"2023-08-10T23:43:26Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Glen Choo writes:\n\n> I think it's reasonable to have the string value parsing logic in\n> git-std-lib, e.g. this parsing snippet from diff.c seems like a good\n> thing to put into a library that wants to accept user input:\n>\n> static int parse_color_moved(const char *arg)\n> {\n> switch (git_parse_maybe_bool(arg)) {\n> case 0:\n> return COLOR_MOVED_NO;\n> case 1:\n> return COLOR_MOVED_DEFAULT;\n> default:\n> break;\n> }\n>\n> if (!strcmp(arg, \"no\"))\n> return COLOR_MOVED_NO;\n> else if (!strcmp(arg, \"plain\"))\n> return COLOR_MOVED_PLAIN;\n> else if (!strcmp(arg, \"blocks\"))\n> return COLOR_MOVED_BLOCKS;\n> /* ... */\n> }\n>\n> But, I don't see a why a non-Git caller would want environment value\n> parsing in git-std-lib.\n\nIt also is debatable why a non-Git caller wants to parse the value\nto the \"--color-moved\" option (or a configuration variable) to begin\nwith. Its vocabulary is closely tied to what the diff machinery in\nGit can do, isn't it?\n"},{"id":"480660","messageId":"20230814220910.60930-1-jonathantanmy@google.com","threadId":"59921","inReplyTo":"20230810163654.275023-4-calvinwan@google.com","subject":"Re: [RFC PATCH v2 4/7] parse: create new library for parsing strings and env values","fromName":"Jonathan Tan","fromEmail":"jonathantanmy@google.com","sentAt":"2023-08-14T22:09:10Z","receivedAt":"2023-08-14T22:09:59Z","isPatch":true,"sender":{"key":"jonathantanmy@fastmail.com","avatar":null},"body":"Calvin Wan writes:\n> While string and environment value parsing is mainly consumed by\n> config.c, there are other files that only need parsing functionality and\n> not config functionality. By separating out string and environment value\n> parsing from config, those files can instead be dependent on parse,\n> which has a much smaller dependency chain than config.\n> \n> Move general string and env parsing functions from config.[ch] to\n> parse.[ch].\n> \n> Signed-off-by: Calvin Wan \n\nThanks - I think that patches 1 through 4 are worth merging even now.\nOne thing we hoped to accomplish through the libification effort is to\nmake changes that are beneficial even outside the libification context,\nand it seems that this is one of them. Previously, code needed to\ninclude config.h even when it didn't use the main functionality that\nconfig.h provides (config), but now it no longer needs to do so. (And\nsame argument for hex, although on a smaller scale.)\n"},{"id":"480661","messageId":"20230814221514.62111-1-jonathantanmy@google.com","threadId":"59921","inReplyTo":"kl6ledka8owj.fsf@chooglen-macbookpro.roam.corp.google.com","subject":"Re: [RFC PATCH v2 4/7] parse: create new library for parsing strings and env values","fromName":"Jonathan Tan","fromEmail":"jonathantanmy@google.com","sentAt":"2023-08-14T22:15:14Z","receivedAt":"2023-08-14T22:15:55Z","isPatch":true,"sender":{"key":"jonathantanmy@fastmail.com","avatar":null},"body":"Glen Choo writes:\n> But, I don't see a why a non-Git caller would want environment value\n> parsing in git-std-lib. I wouldn't think that libraries should be\n> reading Git-formatted environment variables.\n\nI think environment parsing in git-std-lib is fine, at least for the\nshort term. First, currently we expect a lot from a user of our library\n(including tolerating breaking changes in API), so I think it is\nreasonable for such a user to be aware that some functionality can be\nchanged by an environment variable. Second, the purpose of the library\nis to provide functionality that currently is only accessible through\nCLI in library form, and if the CLI deems that some functionality\nshould be accessible through an environment variable instead of a config\nvariable or CLI parameter for whatever reason, we should reflect that in\nthe library as well.\n \n"},{"id":"480662","messageId":"20230814221709.62539-1-jonathantanmy@google.com","threadId":"59921","inReplyTo":"20230810163654.275023-5-calvinwan@google.com","subject":"Re: [RFC PATCH v2 5/7] date: push pager.h dependency up","fromName":"Jonathan Tan","fromEmail":"jonathantanmy@google.com","sentAt":"2023-08-14T22:17:09Z","receivedAt":"2023-08-14T22:18:06Z","isPatch":true,"sender":{"key":"jonathantanmy@fastmail.com","avatar":null},"body":"Calvin Wan writes:\n> In order for date.c to be included in git-std-lib, the dependency to\n> pager.h must be removed since it has dependencies on many other files\n> not in git-std-lib. We achieve this by passing a boolean for\n> \"pager_in_use\", rather than checking for it in parse_date_format() so\n> callers of the function will have that dependency.\n\nInstead of doing it as you describe here, could this be another stub\ninstead? That way, we don't need to change the code here.\n\nI don't feel strongly about this, though, so if other reviewers think\nthat the approach in this patch makes the code better, I'm OK with that.\n \n"},{"id":"480663","messageId":"xmqqzg2tjmgu.fsf@gitster.g","threadId":"59921","inReplyTo":"20230814220910.60930-1-jonathantanmy@google.com","subject":"Re: [RFC PATCH v2 4/7] parse: create new library for parsing strings and env values","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2023-08-14T22:19:45Z","receivedAt":"2023-08-14T22:20:47Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Jonathan Tan writes:\n\n> Thanks - I think that patches 1 through 4 are worth merging even now.\n> One thing we hoped to accomplish through the libification effort is to\n> make changes that are beneficial even outside the libification context,\n> and it seems that this is one of them. Previously, code needed to\n> include config.h even when it didn't use the main functionality that\n> config.h provides (config), but now it no longer needs to do so. (And\n> same argument for hex, although on a smaller scale.)\n\nThanks for writing this down. The parser is shared across handling\ndata that come from config, environ, and options, and separating it\nas a component different from the config does make sense.\n\n"},{"id":"480664","messageId":"20230814222641.64695-1-jonathantanmy@google.com","threadId":"59921","inReplyTo":"20230810163654.275023-6-calvinwan@google.com","subject":"Re: [RFC PATCH v2 6/7] git-std-lib: introduce git standard library","fromName":"Jonathan Tan","fromEmail":"jonathantanmy@google.com","sentAt":"2023-08-14T22:26:41Z","receivedAt":"2023-08-14T22:27:19Z","isPatch":true,"sender":{"key":"jonathantanmy@fastmail.com","avatar":null},"body":"Calvin Wan writes:\n> +Rationale behind Git Standard Library\n> +================\n\nWould it be clearer to write \"Rationale behind what's in and what's not\nin the Git Standard Library\"? Or maybe that is too much of a mouthful.\n\n> +Files inside of Git Standard Library\n> +================\n> +\n> +The initial set of files in git-std-lib.a are:\n> +abspath.c\n> +ctype.c\n> +date.c\n> +hex-ll.c\n> +parse.c\n> +strbuf.c\n> +usage.c\n> +utf8.c\n> +wrapper.c\n> +stubs/repository.c\n> +stubs/trace2.c\n> +relevant compat/ files\n\nI noticed that an earlier version did not have the \"stubs\" lines and\nthis version does, but could not find a comment about why these were\nadded. For me, what would make sense is to remove the \"stubs\" lines,\nand then say \"When these files are compiled together with the following\nfiles (or user-provided files that provide the same functions), they\nform a complete library\", and then list the stubs after.\n\n> diff --git a/git-compat-util.h b/git-compat-util.h\n> index 481dac22b0..75aa9b263e 100644\n> --- a/git-compat-util.h\n> +++ b/git-compat-util.h\n> @@ -396,8 +396,8 @@ static inline int noop_core_config(const char *var UNUSED,\n> #define platform_core_config noop_core_config\n> #endif\n> \n> +#if !defined(__MINGW32__) && !defined(_MSC_VER) && !defined(GIT_STD_LIB)\n> int lstat_cache_aware_rmdir(const char *path);\n> -#if !defined(__MINGW32__) && !defined(_MSC_VER)\n> #define rmdir lstat_cache_aware_rmdir\n> #endif\n\n(and other changes that use defined(GIT_STD_LIB))\n\nOne alternative is to add stubs for lstat_cache_aware_rmdir that call\nthe \"real\" rmdir, but I guess that would be unnecessarily confusing.\nAlso, it would be strange if a user included a header file that\nredefined a standard library function, so I guess we do need such a\n\"defined()\" guard.\n"},{"id":"480665","messageId":"20230814222823.65107-1-jonathantanmy@google.com","threadId":"59921","inReplyTo":"20230810163654.275023-7-calvinwan@google.com","subject":"Re: [RFC PATCH v2 7/7] git-std-lib: add test file to call git-std-lib.a functions","fromName":"Jonathan Tan","fromEmail":"jonathantanmy@google.com","sentAt":"2023-08-14T22:28:23Z","receivedAt":"2023-08-14T22:29:29Z","isPatch":true,"sender":{"key":"jonathantanmy@fastmail.com","avatar":null},"body":"Calvin Wan writes:\n> Add test file that directly or indirectly calls all functions defined in\n> git-std-lib.a object files to showcase that they do not reference\n> missing objects and that git-std-lib.a can stand on its own.\n> \n> Certain functions that cause the program to exit or are already called\n> by other functions are commented out.\n> \n> TODO: replace with unit tests\n> Signed-off-by: Calvin Wan \n\nThanks for this patch - it's useful for reviewers to see what this\npatch set accomplishes (a way to compile a subset of files in Git that\ncan provide library functionality). I don't think we should merge it\nas-is but should wait until we have a unit test that also exercises\nfunctions, and then merge that instead (I think your TODO expresses the\nsame sentiment).\n \n"},{"id":"480672","messageId":"56d0838d-12a5-8a8a-3c3e-7d473f2977e8@gmail.com","threadId":"59921","inReplyTo":"kl6lmsyy8sfj.fsf@chooglen-macbookpro.roam.corp.google.com","subject":"Re: [RFC PATCH v2 0/7] Introduce Git Standard Library","fromName":"Phillip Wood","fromEmail":"phillip.wood123@gmail.com","sentAt":"2023-08-15T09:20:43Z","receivedAt":"2023-08-15T09:23:41Z","isPatch":true,"sender":{"key":"phillip.wood@dunelm.org.uk","avatar":null},"body":"On 10/08/2023 23:05, Glen Choo wrote:\n> Calvin Wan writes:\n> \n>> Calvin Wan (7):\n>> hex-ll: split out functionality from hex\n>> object: move function to object.c\n>> config: correct bad boolean env value error message\n>> parse: create new library for parsing strings and env values\n>> date: push pager.h dependency up\n>> git-std-lib: introduce git standard library\n>> git-std-lib: add test file to call git-std-lib.a functions\n> \n> This doesn't seem to apply to 'master'. Do you have a base commit that\n> reviewers could apply the patches to?\n\nI don't know what they are based on, but I did manage to apply them to \nmaster by using \"am -3\" and resolving the conflicts. The result is at \nhttps://github.com/phillipwood/git/tree/cw/git-std-lib/rfc-v2 if anyone \nis interested.\n\nBest Wishes\n\nPhillip\n\n"},{"id":"480673","messageId":"a0f04bd7-3a1e-b303-fd52-eee2af4d38b3@gmail.com","threadId":"59921","inReplyTo":"20230810163346.274132-1-calvinwan@google.com","subject":"Re: [RFC PATCH v2 0/7] Introduce Git Standard Library","fromName":"Phillip Wood","fromEmail":"phillip.wood123@gmail.com","sentAt":"2023-08-15T09:41:38Z","receivedAt":"2023-08-15T09:42:38Z","isPatch":true,"sender":{"key":"phillip.wood@dunelm.org.uk","avatar":null},"body":"Hi Calvin\n\nOn 10/08/2023 17:33, Calvin Wan wrote:\n> Original cover letter:\n> https://lore.kernel.org/git/20230627195251.1973421-1-calvinwan@google.com/\n> \n> In the initial RFC, I had a patch that removed the trace2 dependency\n> from usage.c so that git-std-lib.a would not have dependencies outside\n> of git-std-lib.a files. Consequently this meant that tracing would not\n> be possible in git-std-lib.a files for other developers of Git, and it\n> is not a good idea for the libification effort to close the door on\n> tracing in certain files for future development (thanks Victoria for\n> pointing this out). That patch has been removed and instead I introduce\n> stubbed out versions of repository.[ch] and trace2.[ch] that are swapped\n> in during compilation time (I'm no Makefile expert so any advice on how\n> on I could do this better would be much appreciated). These stubbed out\n> files contain no implementations and therefore do not have any\n> additional dependencies, allowing git-std-lib.a to compile with only the\n> stubs as additional dependencies.\n\nI think stubbing out trace2 is a sensible approach. I don't think we\nneed separate headers when using the stub though, or a stub for\nrepository.c as we don't call any of the functions declared in that\nheader. I've appended a patch that shows a simplified stub. It also\nremoves the recursive make call as it no-longer needs to juggle the\nheader files.\n\n> This also has the added benefit of\n> removing `#ifdef GIT_STD_LIB` macros in C files for specific library\n> compilation rules. Libification shouldn't pollute C files with these\n> macros. The boundaries for git-std-lib.a have also been updated to\n> contain these stubbed out files.\n\nDo you have any plans to support building with gettext support so we\ncan use git-std-lib.a as a dependency of libgit.a?\n \n> I have also made some additional changes to the Makefile to piggy back\n> off of our existing build rules for .c/.o targets and their\n> dependencies. As I learn more about Makefiles, I am continuing to look\n> for ways to improve these rules. Eventually I would like to be able to\n> have a set of rules that future libraries can emulate and is scalable\n> in the sense of not creating additional toil for developers that are not\n> interested in libification.\n\nI'm not sure reusing LIB_OBJS for different targets is a good idea.\nOnce libgit.a starts to depend on git-std-lib.a we'll want to build them\nboth with a single make invocation without resorting to recursive make\ncalls. I think we could perhaps make a template function to create the\ncompilation rules for each library - see the end of\nhttps://wingolog.org/archives/2023/08/08/a-negative-result\n\nBest Wishes\n\nPhillip\n\n---- >8 -----\n From 194403e42f116cc3c6ed8eb8b03d6933b24067e4 Mon Sep 17 00:00:00 2001\nFrom: Phillip Wood \nDate: Sat, 12 Aug 2023 17:27:23 +0100\nSubject: [PATCH] git-std-lib: simplify sub implementation\n\nThe code in std-lib does not depend directly on the functions declared\nin repository.h and so it does not need to provide stub\nimplementations of the functions declared in repository.h. There is a\ntransitive dependency on `struct repository` from the functions\ndeclared in trace2.h but the stub implementation of those functions\ncan simply define its own stub for struct repository. There is also no\nneed to use different headers when compiling against the stub\nimplementation of trace2.\n\nThis means we can simplify the stub implementation by removing\nstubs/{repository.[ch],trace2.h} and simplify the Makefile by removing\nthe code that replaces header files when compiling against the trace2\nstub. git-std-lib.a can now be built by running\n\n make git-std-lib.a GIT_STD_LIB=YesPlease STUB_TRACE2=YesPlease\n\nThere is one other small fixup in this commit:\n\n - `wrapper.c` includes `repository.h` but does not use any of the\n declarations.\n\nSigned-off-by: Phillip Wood \n---\n Makefile | 29 +-------------------\n stubs/repository.c | 4 ---\n stubs/repository.h | 8 ------\n stubs/trace2.c | 5 ++++\n stubs/trace2.h | 68 ----------------------------------------------\n wrapper.c | 1 -\n 6 files changed, 6 insertions(+), 109 deletions(-)\n delete mode 100644 stubs/repository.c\n delete mode 100644 stubs/repository.h\n delete mode 100644 stubs/trace2.h\n\ndiff --git a/Makefile b/Makefile\nindex a821d73c9d0..8eff4021025 100644\n--- a/Makefile\n+++ b/Makefile\n@@ -1209,10 +1209,6 @@ LIB_OBJS += usage.o\n LIB_OBJS += utf8.o\n LIB_OBJS += wrapper.o\n \n-ifdef STUB_REPOSITORY\n-STUB_OBJS += stubs/repository.o\n-endif\n-\n ifdef STUB_TRACE2\n STUB_OBJS += stubs/trace2.o\n endif\n@@ -3866,31 +3862,8 @@ fuzz-all: $(FUZZ_PROGRAMS)\n ### Libified Git rules\n \n # git-std-lib\n-# `make git-std-lib GIT_STD_LIB=YesPlease STUB_REPOSITORY=YesPlease STUB_TRACE2=YesPlease`\n+# `make git-std-lib.a GIT_STD_LIB=YesPlease STUB_TRACE2=YesPlease`\n STD_LIB = git-std-lib.a\n \n $(STD_LIB): $(LIB_OBJS) $(COMPAT_OBJS) $(STUB_OBJS)\n \t$(QUIET_AR)$(RM) $@ && $(AR) $(ARFLAGS) $@ $^\n-\n-TEMP_HEADERS = temp_headers/\n-\n-git-std-lib:\n-# Move headers to temporary folder and replace them with stubbed headers.\n-# After building, move headers and stubbed headers back.\n-ifneq ($(STUB_OBJS),)\n-\tmkdir -p $(TEMP_HEADERS); \\\n-\tfor d in $(STUB_OBJS); do \\\n-\t\tBASE=$${d%.*}; \\\n-\t\tmv $${BASE##*/}.h $(TEMP_HEADERS)$${BASE##*/}.h; \\\n-\t\tmv $${BASE}.h $${BASE##*/}.h; \\\n-\tdone; \\\n-\t$(MAKE) $(STD_LIB); \\\n-\tfor d in $(STUB_OBJS); do \\\n-\t\tBASE=$${d%.*}; \\\n-\t\tmv $${BASE##*/}.h $${BASE}.h; \\\n-\t\tmv $(TEMP_HEADERS)$${BASE##*/}.h $${BASE##*/}.h; \\\n-\tdone; \\\n-\trm -rf temp_headers\n-else\n-\t$(MAKE) $(STD_LIB)\n-endif\ndiff --git a/stubs/repository.c b/stubs/repository.c\ndeleted file mode 100644\nindex f81520d083a..00000000000\n--- a/stubs/repository.c\n+++ /dev/null\n@@ -1,4 +0,0 @@\n-#include \"git-compat-util.h\"\n-#include \"repository.h\"\n-\n-struct repository *the_repository;\ndiff --git a/stubs/repository.h b/stubs/repository.h\ndeleted file mode 100644\nindex 18262d748e5..00000000000\n--- a/stubs/repository.h\n+++ /dev/null\n@@ -1,8 +0,0 @@\n-#ifndef REPOSITORY_H\n-#define REPOSITORY_H\n-\n-struct repository { int stub; };\n-\n-extern struct repository *the_repository;\n-\n-#endif /* REPOSITORY_H */\ndiff --git a/stubs/trace2.c b/stubs/trace2.c\nindex efc3f9c1f39..7d894822288 100644\n--- a/stubs/trace2.c\n+++ b/stubs/trace2.c\n@@ -1,6 +1,10 @@\n #include \"git-compat-util.h\"\n #include \"trace2.h\"\n \n+struct child_process { int stub; };\n+struct repository { int stub; };\n+struct json_writer { int stub; };\n+\n void trace2_region_enter_fl(const char *file, int line, const char *category,\n \t\t\t const char *label, const struct repository *repo, ...) { }\n void trace2_region_leave_fl(const char *file, int line, const char *category,\n@@ -19,4 +23,5 @@ void trace2_data_intmax_fl(const char *file, int line, const char *category,\n \t\t\t const struct repository *repo, const char *key,\n \t\t\t intmax_t value) { }\n int trace2_is_enabled(void) { return 0; }\n+void trace2_counter_add(enum trace2_counter_id cid, uint64_t value) { }\n void trace2_collect_process_info(enum trace2_process_info_reason reason) { }\ndiff --git a/stubs/trace2.h b/stubs/trace2.h\ndeleted file mode 100644\nindex 836a14797cc..00000000000\n--- a/stubs/trace2.h\n+++ /dev/null\n@@ -1,68 +0,0 @@\n-#ifndef TRACE2_H\n-#define TRACE2_H\n-\n-struct child_process { int stub; };\n-struct repository;\n-struct json_writer { int stub; };\n-\n-void trace2_region_enter_fl(const char *file, int line, const char *category,\n-\t\t\t const char *label, const struct repository *repo, ...);\n-\n-#define trace2_region_enter(category, label, repo) \\\n-\ttrace2_region_enter_fl(__FILE__, __LINE__, (category), (label), (repo))\n-\n-void trace2_region_leave_fl(const char *file, int line, const char *category,\n-\t\t\t const char *label, const struct repository *repo, ...);\n-\n-#define trace2_region_leave(category, label, repo) \\\n-\ttrace2_region_leave_fl(__FILE__, __LINE__, (category), (label), (repo))\n-\n-void trace2_data_string_fl(const char *file, int line, const char *category,\n-\t\t\t const struct repository *repo, const char *key,\n-\t\t\t const char *value);\n-\n-#define trace2_data_string(category, repo, key, value) \\\n-\ttrace2_data_string_fl(__FILE__, __LINE__, (category), (repo), (key), \\\n-\t\t\t (value))\n-\n-void trace2_cmd_ancestry_fl(const char *file, int line, const char **parent_names);\n-\n-#define trace2_cmd_ancestry(v) trace2_cmd_ancestry_fl(__FILE__, __LINE__, (v))\n-\n-void trace2_cmd_error_va_fl(const char *file, int line, const char *fmt,\n-\t\t\t va_list ap);\n-\n-#define trace2_cmd_error_va(fmt, ap) \\\n-\ttrace2_cmd_error_va_fl(__FILE__, __LINE__, (fmt), (ap))\n-\n-\n-void trace2_cmd_name_fl(const char *file, int line, const char *name);\n-\n-#define trace2_cmd_name(v) trace2_cmd_name_fl(__FILE__, __LINE__, (v))\n-\n-void trace2_thread_start_fl(const char *file, int line,\n-\t\t\t const char *thread_base_name);\n-\n-#define trace2_thread_start(thread_base_name) \\\n-\ttrace2_thread_start_fl(__FILE__, __LINE__, (thread_base_name))\n-\n-void trace2_thread_exit_fl(const char *file, int line);\n-\n-#define trace2_thread_exit() trace2_thread_exit_fl(__FILE__, __LINE__)\n-\n-void trace2_data_intmax_fl(const char *file, int line, const char *category,\n-\t\t\t const struct repository *repo, const char *key,\n-\t\t\t intmax_t value);\n-\n-#define trace2_data_intmax(category, repo, key, value) \\\n-\ttrace2_data_intmax_fl(__FILE__, __LINE__, (category), (repo), (key), \\\n-\t\t\t (value))\n-\n-enum trace2_process_info_reason {\n-\tTRACE2_PROCESS_INFO_STARTUP,\n-\tTRACE2_PROCESS_INFO_EXIT,\n-};\n-int trace2_is_enabled(void);\n-void trace2_collect_process_info(enum trace2_process_info_reason reason);\n-\n-#endif /* TRACE2_H */\ndiff --git a/wrapper.c b/wrapper.c\nindex 9eae4a8b3a0..e6facc5ff0c 100644\n--- a/wrapper.c\n+++ b/wrapper.c\n@@ -5,7 +5,6 @@\n #include \"abspath.h\"\n #include \"parse.h\"\n #include \"gettext.h\"\n-#include \"repository.h\"\n #include \"strbuf.h\"\n #include \"trace2.h\"\n \n-- \n2.40.1.850.ge5e148ffb7d\n\n\n"},{"id":"480721","messageId":"CAFySSZBN+-HVPekCjkshepKZLB5uP-m9A=1bWm5Z_OBUVtgQYQ@mail.gmail.com","threadId":"59921","inReplyTo":"56d0838d-12a5-8a8a-3c3e-7d473f2977e8@gmail.com","subject":"Re: [RFC PATCH v2 0/7] Introduce Git Standard Library","fromName":"Calvin Wan","fromEmail":"calvinwan@google.com","sentAt":"2023-08-16T17:17:39Z","receivedAt":"2023-08-16T17:18:48Z","isPatch":true,"sender":{"key":"calvinwan@google.com","avatar":"https://avatars.githubusercontent.com/u/92547554?v=4"},"body":"Thanks for resolving the conflicts on master. I should've rebased\nbefore sending out this v2 since it's built off of 2.41 with some of\nmy other patch cleanup series.\n"},{"id":"480728","messageId":"xmqqr0o2eldl.fsf@gitster.g","threadId":"59921","inReplyTo":"CAFySSZBN+-HVPekCjkshepKZLB5uP-m9A=1bWm5Z_OBUVtgQYQ@mail.gmail.com","subject":"Re: [RFC PATCH v2 0/7] Introduce Git Standard Library","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2023-08-16T21:19:02Z","receivedAt":"2023-08-16T21:19:53Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Calvin Wan writes:\n\n> Thanks for resolving the conflicts on master. I should've rebased\n> before sending out this v2 since it's built off of 2.41 with some of\n> my other patch cleanup series.\n\nI think the freeze period before the release would be a good time to\nrebuild on an updated base to prepare v3 for posting.\n\nThanks.\n"},{"id":"481565","messageId":"20230908174134.1026823-1-calvinwan@google.com","threadId":"59921","inReplyTo":"a0f04bd7-3a1e-b303-fd52-eee2af4d38b3@gmail.com","subject":"[PATCH v3 0/6] Introduce Git Standard Library","fromName":"Calvin Wan","fromEmail":"calvinwan@google.com","sentAt":"2023-09-08T17:41:34Z","receivedAt":"2023-09-08T17:41:48Z","isPatch":true,"sender":{"key":"calvinwan@google.com","avatar":"https://avatars.githubusercontent.com/u/92547554?v=4"},"body":"Original cover letter:\nhttps://lore.kernel.org/git/20230627195251.1973421-1-calvinwan@google.com/\n\nI have taken this series out of RFC since there weren't any significant\nconcerns with the overall concept and design of this series. This reroll\nincorporates some smaller changes such as dropping the \"push pager\ndependency\" patch in favor of stubbing it out. The main change this\nreroll cleans up the Makefile rules and stubs, as suggested by\nPhillip Wood (appreciate the help on this one)!\n\nThis series has been rebased onto 1fc548b2d6a: The sixth batch\n\nOriginally this series was built on other patches that have since been\nmerged, which is why the range-diff is shown removing many of them.\n\nCalvin Wan (6):\n hex-ll: split out functionality from hex\n wrapper: remove dependency to Git-specific internal file\n config: correct bad boolean env value error message\n parse: create new library for parsing strings and env values\n git-std-lib: introduce git standard library\n git-std-lib: add test file to call git-std-lib.a functions\n\n Documentation/technical/git-std-lib.txt | 191 ++++++++++++++++++++\n Makefile | 41 ++++-\n attr.c | 2 +-\n color.c | 2 +-\n config.c | 173 +-----------------\n config.h | 14 +-\n entry.c | 5 +\n entry.h | 6 +\n git-compat-util.h | 7 +-\n hex-ll.c | 49 +++++\n hex-ll.h | 27 +++\n hex.c | 47 -----\n hex.h | 24 +--\n mailinfo.c | 2 +-\n pack-objects.c | 2 +-\n pack-revindex.c | 2 +-\n parse-options.c | 3 +-\n parse.c | 182 +++++++++++++++++++\n parse.h | 20 ++\n pathspec.c | 2 +-\n preload-index.c | 2 +-\n progress.c | 2 +-\n prompt.c | 2 +-\n rebase.c | 2 +-\n strbuf.c | 2 +-\n stubs/pager.c | 6 +\n stubs/pager.h | 6 +\n stubs/trace2.c | 27 +++\n symlinks.c | 2 +\n t/Makefile | 4 +\n t/helper/test-env-helper.c | 2 +-\n t/stdlib-test.c | 231 ++++++++++++++++++++++++\n unpack-trees.c | 2 +-\n url.c | 2 +-\n urlmatch.c | 2 +-\n wrapper.c | 9 +-\n wrapper.h | 5 -\n write-or-die.c | 2 +-\n 38 files changed, 824 insertions(+), 287 deletions(-)\n create mode 100644 Documentation/technical/git-std-lib.txt\n create mode 100644 hex-ll.c\n create mode 100644 hex-ll.h\n create mode 100644 parse.c\n create mode 100644 parse.h\n create mode 100644 stubs/pager.c\n create mode 100644 stubs/pager.h\n create mode 100644 stubs/trace2.c\n create mode 100644 t/stdlib-test.c\n\nRange-diff against v2:\n 1: 121788f263 < -: ---------- strbuf: clarify API boundary\n 2: 5e91404ecd < -: ---------- strbuf: clarify dependency\n 3: 5c05f40181 < -: ---------- abspath: move related functions to abspath\n 4: e1addc77e5 < -: ---------- credential-store: move related functions to credential-store file\n 5: 62e8c42f59 < -: ---------- object-name: move related functions to object-name\n 6: 0abba57acb < -: ---------- path: move related function to path\n 7: d33267a390 < -: ---------- strbuf: remove global variable\n 8: 665d2c2089 < -: ---------- init-db: document existing bug with core.bare in template config\n 9: 68d0a8ff16 < -: ---------- init-db: remove unnecessary global variable\n10: 8c8ec85507 < -: ---------- init-db, clone: change unnecessary global into passed parameter\n11: d555e2b365 < -: ---------- setup: adopt shared init-db & clone code\n12: 689a7bc8aa < -: ---------- read-cache: move shared commit and ls-files code\n13: 392f8e75b7 < -: ---------- add: modify add_files_to_cache() to avoid globals\n14: 49ce237013 < -: ---------- read-cache: move shared add/checkout/commit code\n15: c5d8370d40 < -: ---------- statinfo: move stat_{data,validity} functions from cache/read-cache\n16: 90a72b6f86 < -: ---------- run-command.h: move declarations for run-command.c from cache.h\n17: f27516c780 < -: ---------- name-hash.h: move declarations for name-hash.c from cache.h\n18: 895c38a050 < -: ---------- sparse-index.h: move declarations for sparse-index.c from cache.h\n19: 8678d4ad20 < -: ---------- preload-index.h: move declarations for preload-index.c from elsewhere\n20: 4a463abaae < -: ---------- diff.h: move declaration for global in diff.c from cache.h\n21: 3440e762c7 < -: ---------- merge.h: move declarations for merge.c from cache.h\n22: e70853e398 < -: ---------- repository.h: move declaration of the_index from cache.h\n23: ccd2014d73 < -: ---------- read-cache*.h: move declarations for read-cache.c functions from cache.h\n24: d3a482afa9 < -: ---------- cache.h: remove this no-longer-used header\n25: eaa087f446 < -: ---------- log-tree: replace include of revision.h with simple forward declaration\n26: 5d2b0a9c75 < -: ---------- repository: remove unnecessary include of path.h\n27: 250f83014e < -: ---------- diff.h: remove unnecessary include of oidset.h\n28: d0f9913958 < -: ---------- list-objects-filter-options.h: remove unneccessary include\n29: 03a2b2a515 < -: ---------- builtin.h: remove unneccessary includes\n30: 15edc22d00 < -: ---------- git-compat-util.h: remove unneccessary include of wildmatch.h\n31: e4e1bec8bd < -: ---------- merge-ll: rename from ll-merge\n32: 9185495fd0 < -: ---------- khash: name the structs that khash declares\n33: 15fb05e453 < -: ---------- object-store-ll.h: split this header out of object-store.h\n34: 2608fe4b23 < -: ---------- hash-ll, hashmap: move oidhash() to hash-ll\n35: 5e8dc5b574 < -: ---------- fsmonitor-ll.h: split this header out of fsmonitor.h\n36: 37d32fc3fd < -: ---------- git-compat-util: move strbuf.c funcs to its header\n37: 6ed19d5fe2 < -: ---------- git-compat-util: move wrapper.c funcs to its header\n38: 555d1b8942 < -: ---------- sane-ctype.h: create header for sane-ctype macros\n39: 72d591e282 < -: ---------- kwset: move translation table from ctype\n40: 5d1dc2a118 < -: ---------- common.h: move non-compat specific macros and functions\n41: 33e07e552e < -: ---------- git-compat-util: move usage.c funcs to its header\n42: 417a8aa733 < -: ---------- treewide: remove unnecessary includes for wrapper.h\n43: 65e35d00c1 < -: ---------- common: move alloc macros to common.h\n44: 78634bc406 ! 1: 2f99eb2ca4 hex-ll: split out functionality from hex\n @@ hex.h\n +#include \"hex-ll.h\"\n \n /*\n - * Try to read a SHA1 in hexadecimal format from the 40 characters\n -@@ hex.h: int get_oid_hex(const char *hex, struct object_id *sha1);\n + * Try to read a hash (specified by the_hash_algo) in hexadecimal\n +@@ hex.h: int get_oid_hex(const char *hex, struct object_id *oid);\n /* Like get_oid_hex, but for an arbitrary hash algorithm. */\n int get_oid_hex_algop(const char *hex, struct object_id *oid, const struct git_hash_algo *algop);\n \n45: 21ec1d276e ! 2: 7b2d123628 object: move function to object.c\n @@ Metadata\n Author: Calvin Wan \n \n ## Commit message ##\n - object: move function to object.c\n + wrapper: remove dependency to Git-specific internal file\n \n - While remove_or_warn() is a simple ternary operator to call two other\n - wrapper functions, it creates an unnecessary dependency to object.h in\n - wrapper.c. Therefore move the function to object.[ch] where the concept\n - of GITLINKs is first defined.\n + In order for wrapper.c to be built independently as part of a smaller\n + library, it cannot have dependencies to other Git specific\n + internals. remove_or_warn() creates an unnecessary dependency to\n + object.h in wrapper.c. Therefore move the function to entry.[ch] which\n + performs changes on the worktree based on the Git-specific file modes in\n + the index.\n \n - ## object.c ##\n -@@ object.c: void parsed_object_pool_clear(struct parsed_object_pool *o)\n - \tFREE_AND_NULL(o->object_state);\n - \tFREE_AND_NULL(o->shallow_stat);\n + ## entry.c ##\n +@@ entry.c: void unlink_entry(const struct cache_entry *ce, const char *super_prefix)\n + \t\treturn;\n + \tschedule_dir_for_removal(ce->name, ce_namelen(ce));\n }\n +\n +int remove_or_warn(unsigned int mode, const char *file)\n @@ object.c: void parsed_object_pool_clear(struct parsed_object_pool *o)\n +\treturn S_ISGITLINK(mode) ? rmdir_or_warn(file) : unlink_or_warn(file);\n +}\n \n - ## object.h ##\n -@@ object.h: void clear_object_flags(unsigned flags);\n - */\n - void repo_clear_commit_marks(struct repository *r, unsigned int flags);\n + ## entry.h ##\n +@@ entry.h: int fstat_checkout_output(int fd, const struct checkout *state, struct stat *st)\n + void update_ce_after_write(const struct checkout *state, struct cache_entry *ce,\n + \t\t\t struct stat *st);\n \n +/*\n + * Calls the correct function out of {unlink,rmdir}_or_warn based on\n @@ object.h: void clear_object_flags(unsigned flags);\n + */\n +int remove_or_warn(unsigned int mode, const char *path);\n +\n - #endif /* OBJECT_H */\n + #endif /* ENTRY_H */\n \n ## wrapper.c ##\n @@\n46: 41dcf8107c = 3: b37beb206a config: correct bad boolean env value error message\n47: 3e800a41c4 ! 4: 3a827cf45c parse: create new library for parsing strings and env values\n @@ Commit message\n config.c, there are other files that only need parsing functionality and\n not config functionality. By separating out string and environment value\n parsing from config, those files can instead be dependent on parse,\n - which has a much smaller dependency chain than config.\n + which has a much smaller dependency chain than config. This ultimately\n + allows us to inclue parse.[ch] in an independent library since it\n + doesn't have dependencies to Git-specific internals unlike in\n + config.[ch].\n \n Move general string and env parsing functions from config.[ch] to\n parse.[ch].\n @@ config.c: static int git_parse_source(struct config_source *cs, config_fn_t fn,\n -\treturn 1;\n -}\n -\n - static int reader_config_name(struct config_reader *reader, const char **out);\n - static int reader_origin_type(struct config_reader *reader,\n - \t\t\t enum config_origin_type *type);\n -@@ config.c: ssize_t git_config_ssize_t(const char *name, const char *value)\n + NORETURN\n + static void die_bad_number(const char *name, const char *value,\n + \t\t\t const struct key_value_info *kvi)\n +@@ config.c: ssize_t git_config_ssize_t(const char *name, const char *value,\n \treturn ret;\n }\n \n @@ config.c: static enum fsync_component parse_fsync_components(const char *var, co\n -\treturn -1;\n -}\n -\n - int git_config_bool_or_int(const char *name, const char *value, int *is_bool)\n + int git_config_bool_or_int(const char *name, const char *value,\n + \t\t\t const struct key_value_info *kvi, int *is_bool)\n {\n - \tint v = git_parse_maybe_bool_text(value);\n @@ config.c: void git_global_config(char **user_out, char **xdg_out)\n \t*xdg_out = xdg_config;\n }\n @@ config.c: void git_global_config(char **user_out, char **xdg_out)\n \n ## config.h ##\n @@\n - \n #include \"hashmap.h\"\n #include \"string-list.h\"\n + #include \"repository.h\"\n -\n +#include \"parse.h\"\n \n48: 7a4a088bc3 < -: ---------- date: push pager.h dependency up\n49: c9002734d0 ! 5: f8e4ac50a0 git-std-lib: introduce git standard library\n @@ Documentation/technical/git-std-lib.txt (new)\n +Rationale behind Git Standard Library\n +================\n +\n -+The rationale behind Git Standard Library essentially is the result of\n -+two observations within the Git codebase: every file includes\n -+git-compat-util.h which defines functions in a couple of different\n -+files, and wrapper.c + usage.c have difficult-to-separate circular\n -+dependencies with each other and other files.\n ++The rationale behind what's in and what's not in the Git Standard\n ++Library essentially is the result of two observations within the Git\n ++codebase: every file includes git-compat-util.h which defines functions\n ++in a couple of different files, and wrapper.c + usage.c have\n ++difficult-to-separate circular dependencies with each other and other\n ++files.\n +\n +Ubiquity of git-compat-util.h and circular dependencies\n +========\n @@ Documentation/technical/git-std-lib.txt (new)\n + - low-level git/* files with functions defined in git-compat-util.h\n + (ctype.c)\n + - compat/*\n -+ - stubbed out dependencies in stubs/ (stubs/repository.c, stubs/trace2.c)\n ++ - stubbed out dependencies in stubs/ (stubs/pager.c, stubs/trace2.c)\n +\n +There are other files that might fit this definition, but that does not\n +mean it should belong in git-std-lib.a. Those files should start as\n +their own separate library since any file added to git-std-lib.a loses\n +its flexibility of being easily swappable.\n +\n -+Wrapper.c and usage.c have dependencies on repository and trace2 that are\n ++Wrapper.c and usage.c have dependencies on pager and trace2 that are\n +possible to remove at the cost of sacrificing the ability for standard Git\n +to be able to trace functions in those files and other files in git-std-lib.a.\n +In order for git-std-lib.a to compile with those dependencies, stubbed out\n @@ Documentation/technical/git-std-lib.txt (new)\n +usage.c\n +utf8.c\n +wrapper.c\n -+stubs/repository.c\n -+stubs/trace2.c\n +relevant compat/ files\n +\n ++When these files are compiled together with the following files (or\n ++user-provided files that provide the same functions), they form a\n ++complete library:\n ++stubs/pager.c\n ++stubs/trace2.c\n ++\n +Pitfalls\n +================\n +\n @@ Makefile: LIB_OBJS += write-or-die.o\n +LIB_OBJS += utf8.o\n +LIB_OBJS += wrapper.o\n +\n -+ifdef STUB_REPOSITORY\n -+STUB_OBJS += stubs/repository.o\n -+endif\n -+\n +ifdef STUB_TRACE2\n +STUB_OBJS += stubs/trace2.o\n +endif\n +\n ++ifdef STUB_PAGER\n ++STUB_OBJS += stubs/pager.o\n ++endif\n ++\n +LIB_OBJS += $(STUB_OBJS)\n +endif\n \n @@ Makefile: ifdef FSMONITOR_OS_SETTINGS\n NO_TCLTK = NoThanks\n endif\n @@ Makefile: clean: profile-clean coverage-clean cocciclean\n - \t$(RM) po/git.pot po/git-core.pot\n \t$(RM) git.res\n \t$(RM) $(OBJECTS)\n + \t$(RM) headless-git.o\n -\t$(RM) $(LIB_FILE) $(XDIFF_LIB) $(REFTABLE_LIB) $(REFTABLE_TEST_LIB)\n +\t$(RM) $(LIB_FILE) $(XDIFF_LIB) $(REFTABLE_LIB) $(REFTABLE_TEST_LIB) $(STD_LIB_FILE)\n \t$(RM) $(ALL_PROGRAMS) $(SCRIPT_LIB) $(BUILT_INS) $(OTHER_PROGRAMS)\n @@ Makefile: $(FUZZ_PROGRAMS): all\n +### Libified Git rules\n +\n +# git-std-lib\n -+# `make git-std-lib GIT_STD_LIB=YesPlease STUB_REPOSITORY=YesPlease STUB_TRACE2=YesPlease`\n ++# `make git-std-lib.a GIT_STD_LIB=YesPlease STUB_TRACE2=YesPlease STUB_PAGER=YesPlease`\n +STD_LIB = git-std-lib.a\n +\n +$(STD_LIB): $(LIB_OBJS) $(COMPAT_OBJS) $(STUB_OBJS)\n +\t$(QUIET_AR)$(RM) $@ && $(AR) $(ARFLAGS) $@ $^\n -+\n -+TEMP_HEADERS = temp_headers/\n -+\n -+git-std-lib:\n -+# Move headers to temporary folder and replace them with stubbed headers.\n -+# After building, move headers and stubbed headers back.\n -+ifneq ($(STUB_OBJS),)\n -+\tmkdir -p $(TEMP_HEADERS); \\\n -+\tfor d in $(STUB_OBJS); do \\\n -+\t\tBASE=$${d%.*}; \\\n -+\t\tmv $${BASE##*/}.h $(TEMP_HEADERS)$${BASE##*/}.h; \\\n -+\t\tmv $${BASE}.h $${BASE##*/}.h; \\\n -+\tdone; \\\n -+\t$(MAKE) $(STD_LIB); \\\n -+\tfor d in $(STUB_OBJS); do \\\n -+\t\tBASE=$${d%.*}; \\\n -+\t\tmv $${BASE##*/}.h $${BASE}.h; \\\n -+\t\tmv $(TEMP_HEADERS)$${BASE##*/}.h $${BASE##*/}.h; \\\n -+\tdone; \\\n -+\trm -rf temp_headers\n -+else\n -+\t$(MAKE) $(STD_LIB)\n -+endif\n \n ## git-compat-util.h ##\n @@ git-compat-util.h: static inline int noop_core_config(const char *var UNUSED,\n @@ git-compat-util.h: const char *inet_ntop(int af, const void *src, char *dst, siz\n #endif\n +#endif\n \n - /*\n - * Limit size of IO chunks, because huge chunks only cause pain. OS X\n -@@ git-compat-util.h: int git_access(const char *path, int mode);\n - # endif\n - #endif\n + static inline size_t st_add(size_t a, size_t b)\n + {\n +@@ git-compat-util.h: static inline int is_missing_file_error(int errno_)\n + \treturn (errno_ == ENOENT || errno_ == ENOTDIR);\n + }\n \n +#ifndef GIT_STD_LIB\n int cmd_main(int, const char **);\n @@ git-compat-util.h: int git_access(const char *path, int mode);\n /*\n * You can mark a stack variable with UNLEAK(var) to avoid it being\n \n - ## stubs/repository.c (new) ##\n + ## stubs/pager.c (new) ##\n @@\n -+#include \"git-compat-util.h\"\n -+#include \"repository.h\"\n ++#include \"pager.h\"\n +\n -+struct repository *the_repository;\n ++int pager_in_use(void)\n ++{\n ++\treturn 0;\n ++}\n \n - ## stubs/repository.h (new) ##\n + ## stubs/pager.h (new) ##\n @@\n -+#ifndef REPOSITORY_H\n -+#define REPOSITORY_H\n ++#ifndef PAGER_H\n ++#define PAGER_H\n +\n -+struct repository { int stub; };\n ++int pager_in_use(void);\n +\n -+extern struct repository *the_repository;\n -+\n -+#endif /* REPOSITORY_H */\n ++#endif /* PAGER_H */\n \n ## stubs/trace2.c (new) ##\n @@\n +#include \"git-compat-util.h\"\n +#include \"trace2.h\"\n +\n ++struct child_process { int stub; };\n ++struct repository { int stub; };\n ++struct json_writer { int stub; };\n ++\n +void trace2_region_enter_fl(const char *file, int line, const char *category,\n +\t\t\t const char *label, const struct repository *repo, ...) { }\n +void trace2_region_leave_fl(const char *file, int line, const char *category,\n @@ stubs/trace2.c (new)\n +\t\t\t const struct repository *repo, const char *key,\n +\t\t\t intmax_t value) { }\n +int trace2_is_enabled(void) { return 0; }\n ++void trace2_counter_add(enum trace2_counter_id cid, uint64_t value) { }\n +void trace2_collect_process_info(enum trace2_process_info_reason reason) { }\n \n - ## stubs/trace2.h (new) ##\n -@@\n -+#ifndef TRACE2_H\n -+#define TRACE2_H\n -+\n -+struct child_process { int stub; };\n -+struct repository;\n -+struct json_writer { int stub; };\n -+\n -+void trace2_region_enter_fl(const char *file, int line, const char *category,\n -+\t\t\t const char *label, const struct repository *repo, ...);\n -+\n -+#define trace2_region_enter(category, label, repo) \\\n -+\ttrace2_region_enter_fl(__FILE__, __LINE__, (category), (label), (repo))\n -+\n -+void trace2_region_leave_fl(const char *file, int line, const char *category,\n -+\t\t\t const char *label, const struct repository *repo, ...);\n -+\n -+#define trace2_region_leave(category, label, repo) \\\n -+\ttrace2_region_leave_fl(__FILE__, __LINE__, (category), (label), (repo))\n -+\n -+void trace2_data_string_fl(const char *file, int line, const char *category,\n -+\t\t\t const struct repository *repo, const char *key,\n -+\t\t\t const char *value);\n -+\n -+#define trace2_data_string(category, repo, key, value) \\\n -+\ttrace2_data_string_fl(__FILE__, __LINE__, (category), (repo), (key), \\\n -+\t\t\t (value))\n -+\n -+void trace2_cmd_ancestry_fl(const char *file, int line, const char **parent_names);\n -+\n -+#define trace2_cmd_ancestry(v) trace2_cmd_ancestry_fl(__FILE__, __LINE__, (v))\n -+\n -+void trace2_cmd_error_va_fl(const char *file, int line, const char *fmt,\n -+\t\t\t va_list ap);\n -+\n -+#define trace2_cmd_error_va(fmt, ap) \\\n -+\ttrace2_cmd_error_va_fl(__FILE__, __LINE__, (fmt), (ap))\n -+\n -+\n -+void trace2_cmd_name_fl(const char *file, int line, const char *name);\n -+\n -+#define trace2_cmd_name(v) trace2_cmd_name_fl(__FILE__, __LINE__, (v))\n -+\n -+void trace2_thread_start_fl(const char *file, int line,\n -+\t\t\t const char *thread_base_name);\n -+\n -+#define trace2_thread_start(thread_base_name) \\\n -+\ttrace2_thread_start_fl(__FILE__, __LINE__, (thread_base_name))\n -+\n -+void trace2_thread_exit_fl(const char *file, int line);\n -+\n -+#define trace2_thread_exit() trace2_thread_exit_fl(__FILE__, __LINE__)\n -+\n -+void trace2_data_intmax_fl(const char *file, int line, const char *category,\n -+\t\t\t const struct repository *repo, const char *key,\n -+\t\t\t intmax_t value);\n -+\n -+#define trace2_data_intmax(category, repo, key, value) \\\n -+\ttrace2_data_intmax_fl(__FILE__, __LINE__, (category), (repo), (key), \\\n -+\t\t\t (value))\n -+\n -+enum trace2_process_info_reason {\n -+\tTRACE2_PROCESS_INFO_STARTUP,\n -+\tTRACE2_PROCESS_INFO_EXIT,\n -+};\n -+int trace2_is_enabled(void);\n -+void trace2_collect_process_info(enum trace2_process_info_reason reason);\n -+\n -+#endif /* TRACE2_H */\n -+\n -\n ## symlinks.c ##\n @@ symlinks.c: void invalidate_lstat_cache(void)\n \treset_lstat_cache(&default_cache);\n @@ symlinks.c: int lstat_cache_aware_rmdir(const char *path)\n \treturn ret;\n }\n +#endif\n +\n + ## wrapper.c ##\n +@@\n + #include \"abspath.h\"\n + #include \"parse.h\"\n + #include \"gettext.h\"\n +-#include \"repository.h\"\n + #include \"strbuf.h\"\n + #include \"trace2.h\"\n + \n50: 0bead8f980 ! 6: 7840e1830a git-std-lib: add test file to call git-std-lib.a functions\n @@ t/stdlib-test.c (new)\n +\tstruct strbuf sb3 = STRBUF_INIT;\n +\tstruct string_list list = STRING_LIST_INIT_NODUP;\n +\tchar *buf = \"foo\";\n -+\tstruct strbuf_expand_dict_entry dict[] = {\n -+\t\t{ \"foo\", NULL, },\n -+\t\t{ \"bar\", NULL, },\n -+\t};\n +\tint fd = open(\"/dev/null\", O_RDONLY);\n +\n +\tfprintf(stderr, \"calling strbuf functions\\n\");\n @@ t/stdlib-test.c (new)\n +\tstrbuf_add_commented_lines(sb, \"foo\", 3, '#');\n +\tstrbuf_commented_addf(sb, '#', \"%s\", \"foo\");\n +\t// strbuf_vaddf() called by strbuf_addf()\n -+\tstrbuf_expand(sb, \"%s\", strbuf_expand_literal_cb, NULL);\n -+\tstrbuf_expand(sb, \"%s\", strbuf_expand_dict_cb, &dict);\n -+\t// strbuf_expand_literal_cb() called by strbuf_expand()\n -+\t// strbuf_expand_dict_cb() called by strbuf_expand()\n +\tstrbuf_addbuf_percentquote(sb, &sb3);\n +\tstrbuf_add_percentencode(sb, \"foo\", STRBUF_ENCODE_SLASH);\n +\tstrbuf_fread(sb, 0, stdin);\n-- \n2.42.0.283.g2d96d420d3-goog\n\n"},{"id":"481567","messageId":"20230908174443.1027716-2-calvinwan@google.com","threadId":"59921","inReplyTo":"20230908174134.1026823-1-calvinwan@google.com","subject":"[PATCH v3 2/6] wrapper: remove dependency to Git-specific internal file","fromName":"Calvin Wan","fromEmail":"calvinwan@google.com","sentAt":"2023-09-08T17:44:39Z","receivedAt":"2023-09-08T17:45:04Z","isPatch":true,"sender":{"key":"calvinwan@google.com","avatar":"https://avatars.githubusercontent.com/u/92547554?v=4"},"body":"In order for wrapper.c to be built independently as part of a smaller\nlibrary, it cannot have dependencies to other Git specific\ninternals. remove_or_warn() creates an unnecessary dependency to\nobject.h in wrapper.c. Therefore move the function to entry.[ch] which\nperforms changes on the worktree based on the Git-specific file modes in\nthe index.\n\nSigned-off-by: Calvin Wan \n---\n entry.c | 5 +++++\n entry.h | 6 ++++++\n wrapper.c | 6 ------\n wrapper.h | 5 -----\n 4 files changed, 11 insertions(+), 11 deletions(-)\n\ndiff --git a/entry.c b/entry.c\nindex 43767f9043..076e97eb89 100644\n--- a/entry.c\n+++ b/entry.c\n@@ -581,3 +581,8 @@ void unlink_entry(const struct cache_entry *ce, const char *super_prefix)\n \t\treturn;\n \tschedule_dir_for_removal(ce->name, ce_namelen(ce));\n }\n+\n+int remove_or_warn(unsigned int mode, const char *file)\n+{\n+\treturn S_ISGITLINK(mode) ? rmdir_or_warn(file) : unlink_or_warn(file);\n+}\ndiff --git a/entry.h b/entry.h\nindex 7329f918a9..ca3ed35bc0 100644\n--- a/entry.h\n+++ b/entry.h\n@@ -62,4 +62,10 @@ int fstat_checkout_output(int fd, const struct checkout *state, struct stat *st)\n void update_ce_after_write(const struct checkout *state, struct cache_entry *ce,\n \t\t\t struct stat *st);\n \n+/*\n+ * Calls the correct function out of {unlink,rmdir}_or_warn based on\n+ * the supplied file mode.\n+ */\n+int remove_or_warn(unsigned int mode, const char *path);\n+\n #endif /* ENTRY_H */\ndiff --git a/wrapper.c b/wrapper.c\nindex 48065c4f53..453a20ed99 100644\n--- a/wrapper.c\n+++ b/wrapper.c\n@@ -5,7 +5,6 @@\n #include \"abspath.h\"\n #include \"config.h\"\n #include \"gettext.h\"\n-#include \"object.h\"\n #include \"repository.h\"\n #include \"strbuf.h\"\n #include \"trace2.h\"\n@@ -632,11 +631,6 @@ int rmdir_or_warn(const char *file)\n \treturn warn_if_unremovable(\"rmdir\", file, rmdir(file));\n }\n \n-int remove_or_warn(unsigned int mode, const char *file)\n-{\n-\treturn S_ISGITLINK(mode) ? rmdir_or_warn(file) : unlink_or_warn(file);\n-}\n-\n static int access_error_is_ok(int err, unsigned flag)\n {\n \treturn (is_missing_file_error(err) ||\ndiff --git a/wrapper.h b/wrapper.h\nindex 79c7321bb3..1b2b047ea0 100644\n--- a/wrapper.h\n+++ b/wrapper.h\n@@ -106,11 +106,6 @@ int unlink_or_msg(const char *file, struct strbuf *err);\n * not exist.\n */\n int rmdir_or_warn(const char *path);\n-/*\n- * Calls the correct function out of {unlink,rmdir}_or_warn based on\n- * the supplied file mode.\n- */\n-int remove_or_warn(unsigned int mode, const char *path);\n \n /*\n * Call access(2), but warn for any error except \"missing file\"\n-- \n2.42.0.283.g2d96d420d3-goog\n\n"},{"id":"481568","messageId":"20230908174443.1027716-1-calvinwan@google.com","threadId":"59921","inReplyTo":"20230908174134.1026823-1-calvinwan@google.com","subject":"[PATCH v3 1/6] hex-ll: split out functionality from hex","fromName":"Calvin Wan","fromEmail":"calvinwan@google.com","sentAt":"2023-09-08T17:44:38Z","receivedAt":"2023-09-08T17:45:10Z","isPatch":true,"sender":{"key":"calvinwan@google.com","avatar":"https://avatars.githubusercontent.com/u/92547554?v=4"},"body":"Separate out hex functionality that doesn't require a hash algo into\nhex-ll.[ch]. Since the hash algo is currently a global that sits in\nrepository, this separation removes that dependency for files that only\nneed basic hex manipulation functions.\n\nSigned-off-by: Calvin Wan \n---\n Makefile | 1 +\n color.c | 2 +-\n hex-ll.c | 49 +++++++++++++++++++++++++++++++++++++++++++++++++\n hex-ll.h | 27 +++++++++++++++++++++++++++\n hex.c | 47 -----------------------------------------------\n hex.h | 24 +-----------------------\n mailinfo.c | 2 +-\n strbuf.c | 2 +-\n url.c | 2 +-\n urlmatch.c | 2 +-\n 10 files changed, 83 insertions(+), 75 deletions(-)\n create mode 100644 hex-ll.c\n create mode 100644 hex-ll.h\n\ndiff --git a/Makefile b/Makefile\nindex 5776309365..861e643708 100644\n--- a/Makefile\n+++ b/Makefile\n@@ -1040,6 +1040,7 @@ LIB_OBJS += hash-lookup.o\n LIB_OBJS += hashmap.o\n LIB_OBJS += help.o\n LIB_OBJS += hex.o\n+LIB_OBJS += hex-ll.o\n LIB_OBJS += hook.o\n LIB_OBJS += ident.o\n LIB_OBJS += json-writer.o\ndiff --git a/color.c b/color.c\nindex b24b19566b..f663c06ac4 100644\n--- a/color.c\n+++ b/color.c\n@@ -3,7 +3,7 @@\n #include \"color.h\"\n #include \"editor.h\"\n #include \"gettext.h\"\n-#include \"hex.h\"\n+#include \"hex-ll.h\"\n #include \"pager.h\"\n #include \"strbuf.h\"\n \ndiff --git a/hex-ll.c b/hex-ll.c\nnew file mode 100644\nindex 0000000000..4d7ece1de5\n--- /dev/null\n+++ b/hex-ll.c\n@@ -0,0 +1,49 @@\n+#include \"git-compat-util.h\"\n+#include \"hex-ll.h\"\n+\n+const signed char hexval_table[256] = {\n+\t -1, -1, -1, -1, -1, -1, -1, -1,\t\t/* 00-07 */\n+\t -1, -1, -1, -1, -1, -1, -1, -1,\t\t/* 08-0f */\n+\t -1, -1, -1, -1, -1, -1, -1, -1,\t\t/* 10-17 */\n+\t -1, -1, -1, -1, -1, -1, -1, -1,\t\t/* 18-1f */\n+\t -1, -1, -1, -1, -1, -1, -1, -1,\t\t/* 20-27 */\n+\t -1, -1, -1, -1, -1, -1, -1, -1,\t\t/* 28-2f */\n+\t 0, 1, 2, 3, 4, 5, 6, 7,\t\t/* 30-37 */\n+\t 8, 9, -1, -1, -1, -1, -1, -1,\t\t/* 38-3f */\n+\t -1, 10, 11, 12, 13, 14, 15, -1,\t\t/* 40-47 */\n+\t -1, -1, -1, -1, -1, -1, -1, -1,\t\t/* 48-4f */\n+\t -1, -1, -1, -1, -1, -1, -1, -1,\t\t/* 50-57 */\n+\t -1, -1, -1, -1, -1, -1, -1, -1,\t\t/* 58-5f */\n+\t -1, 10, 11, 12, 13, 14, 15, -1,\t\t/* 60-67 */\n+\t -1, -1, -1, -1, -1, -1, -1, -1,\t\t/* 68-67 */\n+\t -1, -1, -1, -1, -1, -1, -1, -1,\t\t/* 70-77 */\n+\t -1, -1, -1, -1, -1, -1, -1, -1,\t\t/* 78-7f */\n+\t -1, -1, -1, -1, -1, -1, -1, -1,\t\t/* 80-87 */\n+\t -1, -1, -1, -1, -1, -1, -1, -1,\t\t/* 88-8f */\n+\t -1, -1, -1, -1, -1, -1, -1, -1,\t\t/* 90-97 */\n+\t -1, -1, -1, -1, -1, -1, -1, -1,\t\t/* 98-9f */\n+\t -1, -1, -1, -1, -1, -1, -1, -1,\t\t/* a0-a7 */\n+\t -1, -1, -1, -1, -1, -1, -1, -1,\t\t/* a8-af */\n+\t -1, -1, -1, -1, -1, -1, -1, -1,\t\t/* b0-b7 */\n+\t -1, -1, -1, -1, -1, -1, -1, -1,\t\t/* b8-bf */\n+\t -1, -1, -1, -1, -1, -1, -1, -1,\t\t/* c0-c7 */\n+\t -1, -1, -1, -1, -1, -1, -1, -1,\t\t/* c8-cf */\n+\t -1, -1, -1, -1, -1, -1, -1, -1,\t\t/* d0-d7 */\n+\t -1, -1, -1, -1, -1, -1, -1, -1,\t\t/* d8-df */\n+\t -1, -1, -1, -1, -1, -1, -1, -1,\t\t/* e0-e7 */\n+\t -1, -1, -1, -1, -1, -1, -1, -1,\t\t/* e8-ef */\n+\t -1, -1, -1, -1, -1, -1, -1, -1,\t\t/* f0-f7 */\n+\t -1, -1, -1, -1, -1, -1, -1, -1,\t\t/* f8-ff */\n+};\n+\n+int hex_to_bytes(unsigned char *binary, const char *hex, size_t len)\n+{\n+\tfor (; len; len--, hex += 2) {\n+\t\tunsigned int val = (hexval(hex[0]) << 4) | hexval(hex[1]);\n+\n+\t\tif (val & ~0xff)\n+\t\t\treturn -1;\n+\t\t*binary++ = val;\n+\t}\n+\treturn 0;\n+}\ndiff --git a/hex-ll.h b/hex-ll.h\nnew file mode 100644\nindex 0000000000..a381fa8556\n--- /dev/null\n+++ b/hex-ll.h\n@@ -0,0 +1,27 @@\n+#ifndef HEX_LL_H\n+#define HEX_LL_H\n+\n+extern const signed char hexval_table[256];\n+static inline unsigned int hexval(unsigned char c)\n+{\n+\treturn hexval_table[c];\n+}\n+\n+/*\n+ * Convert two consecutive hexadecimal digits into a char. Return a\n+ * negative value on error. Don't run over the end of short strings.\n+ */\n+static inline int hex2chr(const char *s)\n+{\n+\tunsigned int val = hexval(s[0]);\n+\treturn (val & ~0xf) ? val : (val << 4) | hexval(s[1]);\n+}\n+\n+/*\n+ * Read `len` pairs of hexadecimal digits from `hex` and write the\n+ * values to `binary` as `len` bytes. Return 0 on success, or -1 if\n+ * the input does not consist of hex digits).\n+ */\n+int hex_to_bytes(unsigned char *binary, const char *hex, size_t len);\n+\n+#endif\ndiff --git a/hex.c b/hex.c\nindex 01f17fe5c9..d42262bdca 100644\n--- a/hex.c\n+++ b/hex.c\n@@ -2,53 +2,6 @@\n #include \"hash.h\"\n #include \"hex.h\"\n \n-const signed char hexval_table[256] = {\n-\t -1, -1, -1, -1, -1, -1, -1, -1,\t\t/* 00-07 */\n-\t -1, -1, -1, -1, -1, -1, -1, -1,\t\t/* 08-0f */\n-\t -1, -1, -1, -1, -1, -1, -1, -1,\t\t/* 10-17 */\n-\t -1, -1, -1, -1, -1, -1, -1, -1,\t\t/* 18-1f */\n-\t -1, -1, -1, -1, -1, -1, -1, -1,\t\t/* 20-27 */\n-\t -1, -1, -1, -1, -1, -1, -1, -1,\t\t/* 28-2f */\n-\t 0, 1, 2, 3, 4, 5, 6, 7,\t\t/* 30-37 */\n-\t 8, 9, -1, -1, -1, -1, -1, -1,\t\t/* 38-3f */\n-\t -1, 10, 11, 12, 13, 14, 15, -1,\t\t/* 40-47 */\n-\t -1, -1, -1, -1, -1, -1, -1, -1,\t\t/* 48-4f */\n-\t -1, -1, -1, -1, -1, -1, -1, -1,\t\t/* 50-57 */\n-\t -1, -1, -1, -1, -1, -1, -1, -1,\t\t/* 58-5f */\n-\t -1, 10, 11, 12, 13, 14, 15, -1,\t\t/* 60-67 */\n-\t -1, -1, -1, -1, -1, -1, -1, -1,\t\t/* 68-67 */\n-\t -1, -1, -1, -1, -1, -1, -1, -1,\t\t/* 70-77 */\n-\t -1, -1, -1, -1, -1, -1, -1, -1,\t\t/* 78-7f */\n-\t -1, -1, -1, -1, -1, -1, -1, -1,\t\t/* 80-87 */\n-\t -1, -1, -1, -1, -1, -1, -1, -1,\t\t/* 88-8f */\n-\t -1, -1, -1, -1, -1, -1, -1, -1,\t\t/* 90-97 */\n-\t -1, -1, -1, -1, -1, -1, -1, -1,\t\t/* 98-9f */\n-\t -1, -1, -1, -1, -1, -1, -1, -1,\t\t/* a0-a7 */\n-\t -1, -1, -1, -1, -1, -1, -1, -1,\t\t/* a8-af */\n-\t -1, -1, -1, -1, -1, -1, -1, -1,\t\t/* b0-b7 */\n-\t -1, -1, -1, -1, -1, -1, -1, -1,\t\t/* b8-bf */\n-\t -1, -1, -1, -1, -1, -1, -1, -1,\t\t/* c0-c7 */\n-\t -1, -1, -1, -1, -1, -1, -1, -1,\t\t/* c8-cf */\n-\t -1, -1, -1, -1, -1, -1, -1, -1,\t\t/* d0-d7 */\n-\t -1, -1, -1, -1, -1, -1, -1, -1,\t\t/* d8-df */\n-\t -1, -1, -1, -1, -1, -1, -1, -1,\t\t/* e0-e7 */\n-\t -1, -1, -1, -1, -1, -1, -1, -1,\t\t/* e8-ef */\n-\t -1, -1, -1, -1, -1, -1, -1, -1,\t\t/* f0-f7 */\n-\t -1, -1, -1, -1, -1, -1, -1, -1,\t\t/* f8-ff */\n-};\n-\n-int hex_to_bytes(unsigned char *binary, const char *hex, size_t len)\n-{\n-\tfor (; len; len--, hex += 2) {\n-\t\tunsigned int val = (hexval(hex[0]) << 4) | hexval(hex[1]);\n-\n-\t\tif (val & ~0xff)\n-\t\t\treturn -1;\n-\t\t*binary++ = val;\n-\t}\n-\treturn 0;\n-}\n-\n static int get_hash_hex_algop(const char *hex, unsigned char *hash,\n \t\t\t const struct git_hash_algo *algop)\n {\ndiff --git a/hex.h b/hex.h\nindex 87abf66602..e0b83f776f 100644\n--- a/hex.h\n+++ b/hex.h\n@@ -2,22 +2,7 @@\n #define HEX_H\n \n #include \"hash-ll.h\"\n-\n-extern const signed char hexval_table[256];\n-static inline unsigned int hexval(unsigned char c)\n-{\n-\treturn hexval_table[c];\n-}\n-\n-/*\n- * Convert two consecutive hexadecimal digits into a char. Return a\n- * negative value on error. Don't run over the end of short strings.\n- */\n-static inline int hex2chr(const char *s)\n-{\n-\tunsigned int val = hexval(s[0]);\n-\treturn (val & ~0xf) ? val : (val << 4) | hexval(s[1]);\n-}\n+#include \"hex-ll.h\"\n \n /*\n * Try to read a hash (specified by the_hash_algo) in hexadecimal\n@@ -34,13 +19,6 @@ int get_oid_hex(const char *hex, struct object_id *oid);\n /* Like get_oid_hex, but for an arbitrary hash algorithm. */\n int get_oid_hex_algop(const char *hex, struct object_id *oid, const struct git_hash_algo *algop);\n \n-/*\n- * Read `len` pairs of hexadecimal digits from `hex` and write the\n- * values to `binary` as `len` bytes. Return 0 on success, or -1 if\n- * the input does not consist of hex digits).\n- */\n-int hex_to_bytes(unsigned char *binary, const char *hex, size_t len);\n-\n /*\n * Convert a binary hash in \"unsigned char []\" or an object name in\n * \"struct object_id *\" to its hex equivalent. The `_r` variant is reentrant,\ndiff --git a/mailinfo.c b/mailinfo.c\nindex 931505363c..a07d2da16d 100644\n--- a/mailinfo.c\n+++ b/mailinfo.c\n@@ -1,7 +1,7 @@\n #include \"git-compat-util.h\"\n #include \"config.h\"\n #include \"gettext.h\"\n-#include \"hex.h\"\n+#include \"hex-ll.h\"\n #include \"utf8.h\"\n #include \"strbuf.h\"\n #include \"mailinfo.h\"\ndiff --git a/strbuf.c b/strbuf.c\nindex 4c9ac6dc5e..7827178d8e 100644\n--- a/strbuf.c\n+++ b/strbuf.c\n@@ -1,6 +1,6 @@\n #include \"git-compat-util.h\"\n #include \"gettext.h\"\n-#include \"hex.h\"\n+#include \"hex-ll.h\"\n #include \"strbuf.h\"\n #include \"string-list.h\"\n #include \"utf8.h\"\ndiff --git a/url.c b/url.c\nindex 2e1a9f6fee..282b12495a 100644\n--- a/url.c\n+++ b/url.c\n@@ -1,5 +1,5 @@\n #include \"git-compat-util.h\"\n-#include \"hex.h\"\n+#include \"hex-ll.h\"\n #include \"strbuf.h\"\n #include \"url.h\"\n \ndiff --git a/urlmatch.c b/urlmatch.c\nindex 1c45f23adf..1d0254abac 100644\n--- a/urlmatch.c\n+++ b/urlmatch.c\n@@ -1,6 +1,6 @@\n #include \"git-compat-util.h\"\n #include \"gettext.h\"\n-#include \"hex.h\"\n+#include \"hex-ll.h\"\n #include \"strbuf.h\"\n #include \"urlmatch.h\"\n \n-- \n2.42.0.283.g2d96d420d3-goog\n\n"},{"id":"481569","messageId":"20230908174443.1027716-3-calvinwan@google.com","threadId":"59921","inReplyTo":"20230908174134.1026823-1-calvinwan@google.com","subject":"[PATCH v3 3/6] config: correct bad boolean env value error message","fromName":"Calvin Wan","fromEmail":"calvinwan@google.com","sentAt":"2023-09-08T17:44:40Z","receivedAt":"2023-09-08T17:45:12Z","isPatch":true,"sender":{"key":"calvinwan@google.com","avatar":"https://avatars.githubusercontent.com/u/92547554?v=4"},"body":"An incorrectly defined boolean environment value would result in the\nfollowing error message:\n\nbad boolean config value '%s' for '%s'\n\nThis is a misnomer since environment value != config value. Instead of\ncalling git_config_bool() to parse the environment value, mimic the\nfunctionality inside of git_config_bool() but with the correct error\nmessage.\n\nSigned-off-by: Calvin Wan \n---\n config.c | 9 ++++++++-\n 1 file changed, 8 insertions(+), 1 deletion(-)\n\ndiff --git a/config.c b/config.c\nindex 3846a37be9..7dde0aaa02 100644\n--- a/config.c\n+++ b/config.c\n@@ -2133,7 +2133,14 @@ void git_global_config(char **user_out, char **xdg_out)\n int git_env_bool(const char *k, int def)\n {\n \tconst char *v = getenv(k);\n-\treturn v ? git_config_bool(k, v) : def;\n+\tint val;\n+\tif (!v)\n+\t\treturn def;\n+\tval = git_parse_maybe_bool(v);\n+\tif (val < 0)\n+\t\tdie(_(\"bad boolean environment value '%s' for '%s'\"),\n+\t\t v, k);\n+\treturn val;\n }\n \n /*\n-- \n2.42.0.283.g2d96d420d3-goog\n\n"},{"id":"481570","messageId":"20230908174443.1027716-4-calvinwan@google.com","threadId":"59921","inReplyTo":"20230908174134.1026823-1-calvinwan@google.com","subject":"[PATCH v3 4/6] parse: create new library for parsing strings and env values","fromName":"Calvin Wan","fromEmail":"calvinwan@google.com","sentAt":"2023-09-08T17:44:41Z","receivedAt":"2023-09-08T17:45:13Z","isPatch":true,"sender":{"key":"calvinwan@google.com","avatar":"https://avatars.githubusercontent.com/u/92547554?v=4"},"body":"While string and environment value parsing is mainly consumed by\nconfig.c, there are other files that only need parsing functionality and\nnot config functionality. By separating out string and environment value\nparsing from config, those files can instead be dependent on parse,\nwhich has a much smaller dependency chain than config. This ultimately\nallows us to inclue parse.[ch] in an independent library since it\ndoesn't have dependencies to Git-specific internals unlike in\nconfig.[ch].\n\nMove general string and env parsing functions from config.[ch] to\nparse.[ch].\n\nSigned-off-by: Calvin Wan \n---\n Makefile | 1 +\n attr.c | 2 +-\n config.c | 180 +-----------------------------------\n config.h | 14 +--\n pack-objects.c | 2 +-\n pack-revindex.c | 2 +-\n parse-options.c | 3 +-\n parse.c | 182 +++++++++++++++++++++++++++++++++++++\n parse.h | 20 ++++\n pathspec.c | 2 +-\n preload-index.c | 2 +-\n progress.c | 2 +-\n prompt.c | 2 +-\n rebase.c | 2 +-\n t/helper/test-env-helper.c | 2 +-\n unpack-trees.c | 2 +-\n wrapper.c | 2 +-\n write-or-die.c | 2 +-\n 18 files changed, 219 insertions(+), 205 deletions(-)\n create mode 100644 parse.c\n create mode 100644 parse.h\n\ndiff --git a/Makefile b/Makefile\nindex 861e643708..9226c719a0 100644\n--- a/Makefile\n+++ b/Makefile\n@@ -1091,6 +1091,7 @@ LIB_OBJS += pack-write.o\n LIB_OBJS += packfile.o\n LIB_OBJS += pager.o\n LIB_OBJS += parallel-checkout.o\n+LIB_OBJS += parse.o\n LIB_OBJS += parse-options-cb.o\n LIB_OBJS += parse-options.o\n LIB_OBJS += patch-delta.o\ndiff --git a/attr.c b/attr.c\nindex 71c84fbcf8..3c0b4fb3d9 100644\n--- a/attr.c\n+++ b/attr.c\n@@ -7,7 +7,7 @@\n */\n \n #include \"git-compat-util.h\"\n-#include \"config.h\"\n+#include \"parse.h\"\n #include \"environment.h\"\n #include \"exec-cmd.h\"\n #include \"attr.h\"\ndiff --git a/config.c b/config.c\nindex 7dde0aaa02..c7bc21a25d 100644\n--- a/config.c\n+++ b/config.c\n@@ -11,6 +11,7 @@\n #include \"date.h\"\n #include \"branch.h\"\n #include \"config.h\"\n+#include \"parse.h\"\n #include \"convert.h\"\n #include \"environment.h\"\n #include \"gettext.h\"\n@@ -1165,129 +1166,6 @@ static int git_parse_source(struct config_source *cs, config_fn_t fn,\n \treturn error_return;\n }\n \n-static uintmax_t get_unit_factor(const char *end)\n-{\n-\tif (!*end)\n-\t\treturn 1;\n-\telse if (!strcasecmp(end, \"k\"))\n-\t\treturn 1024;\n-\telse if (!strcasecmp(end, \"m\"))\n-\t\treturn 1024 * 1024;\n-\telse if (!strcasecmp(end, \"g\"))\n-\t\treturn 1024 * 1024 * 1024;\n-\treturn 0;\n-}\n-\n-static int git_parse_signed(const char *value, intmax_t *ret, intmax_t max)\n-{\n-\tif (value && *value) {\n-\t\tchar *end;\n-\t\tintmax_t val;\n-\t\tintmax_t factor;\n-\n-\t\tif (max < 0)\n-\t\t\tBUG(\"max must be a positive integer\");\n-\n-\t\terrno = 0;\n-\t\tval = strtoimax(value, &end, 0);\n-\t\tif (errno == ERANGE)\n-\t\t\treturn 0;\n-\t\tif (end == value) {\n-\t\t\terrno = EINVAL;\n-\t\t\treturn 0;\n-\t\t}\n-\t\tfactor = get_unit_factor(end);\n-\t\tif (!factor) {\n-\t\t\terrno = EINVAL;\n-\t\t\treturn 0;\n-\t\t}\n-\t\tif ((val < 0 && -max / factor > val) ||\n-\t\t (val > 0 && max / factor < val)) {\n-\t\t\terrno = ERANGE;\n-\t\t\treturn 0;\n-\t\t}\n-\t\tval *= factor;\n-\t\t*ret = val;\n-\t\treturn 1;\n-\t}\n-\terrno = EINVAL;\n-\treturn 0;\n-}\n-\n-static int git_parse_unsigned(const char *value, uintmax_t *ret, uintmax_t max)\n-{\n-\tif (value && *value) {\n-\t\tchar *end;\n-\t\tuintmax_t val;\n-\t\tuintmax_t factor;\n-\n-\t\t/* negative values would be accepted by strtoumax */\n-\t\tif (strchr(value, '-')) {\n-\t\t\terrno = EINVAL;\n-\t\t\treturn 0;\n-\t\t}\n-\t\terrno = 0;\n-\t\tval = strtoumax(value, &end, 0);\n-\t\tif (errno == ERANGE)\n-\t\t\treturn 0;\n-\t\tif (end == value) {\n-\t\t\terrno = EINVAL;\n-\t\t\treturn 0;\n-\t\t}\n-\t\tfactor = get_unit_factor(end);\n-\t\tif (!factor) {\n-\t\t\terrno = EINVAL;\n-\t\t\treturn 0;\n-\t\t}\n-\t\tif (unsigned_mult_overflows(factor, val) ||\n-\t\t factor * val > max) {\n-\t\t\terrno = ERANGE;\n-\t\t\treturn 0;\n-\t\t}\n-\t\tval *= factor;\n-\t\t*ret = val;\n-\t\treturn 1;\n-\t}\n-\terrno = EINVAL;\n-\treturn 0;\n-}\n-\n-int git_parse_int(const char *value, int *ret)\n-{\n-\tintmax_t tmp;\n-\tif (!git_parse_signed(value, &tmp, maximum_signed_value_of_type(int)))\n-\t\treturn 0;\n-\t*ret = tmp;\n-\treturn 1;\n-}\n-\n-static int git_parse_int64(const char *value, int64_t *ret)\n-{\n-\tintmax_t tmp;\n-\tif (!git_parse_signed(value, &tmp, maximum_signed_value_of_type(int64_t)))\n-\t\treturn 0;\n-\t*ret = tmp;\n-\treturn 1;\n-}\n-\n-int git_parse_ulong(const char *value, unsigned long *ret)\n-{\n-\tuintmax_t tmp;\n-\tif (!git_parse_unsigned(value, &tmp, maximum_unsigned_value_of_type(long)))\n-\t\treturn 0;\n-\t*ret = tmp;\n-\treturn 1;\n-}\n-\n-int git_parse_ssize_t(const char *value, ssize_t *ret)\n-{\n-\tintmax_t tmp;\n-\tif (!git_parse_signed(value, &tmp, maximum_signed_value_of_type(ssize_t)))\n-\t\treturn 0;\n-\t*ret = tmp;\n-\treturn 1;\n-}\n-\n NORETURN\n static void die_bad_number(const char *name, const char *value,\n \t\t\t const struct key_value_info *kvi)\n@@ -1363,23 +1241,6 @@ ssize_t git_config_ssize_t(const char *name, const char *value,\n \treturn ret;\n }\n \n-static int git_parse_maybe_bool_text(const char *value)\n-{\n-\tif (!value)\n-\t\treturn 1;\n-\tif (!*value)\n-\t\treturn 0;\n-\tif (!strcasecmp(value, \"true\")\n-\t || !strcasecmp(value, \"yes\")\n-\t || !strcasecmp(value, \"on\"))\n-\t\treturn 1;\n-\tif (!strcasecmp(value, \"false\")\n-\t || !strcasecmp(value, \"no\")\n-\t || !strcasecmp(value, \"off\"))\n-\t\treturn 0;\n-\treturn -1;\n-}\n-\n static const struct fsync_component_name {\n \tconst char *name;\n \tenum fsync_component component_bits;\n@@ -1454,16 +1315,6 @@ static enum fsync_component parse_fsync_components(const char *var, const char *\n \treturn (current & ~negative) | positive;\n }\n \n-int git_parse_maybe_bool(const char *value)\n-{\n-\tint v = git_parse_maybe_bool_text(value);\n-\tif (0 <= v)\n-\t\treturn v;\n-\tif (git_parse_int(value, &v))\n-\t\treturn !!v;\n-\treturn -1;\n-}\n-\n int git_config_bool_or_int(const char *name, const char *value,\n \t\t\t const struct key_value_info *kvi, int *is_bool)\n {\n@@ -2126,35 +1977,6 @@ void git_global_config(char **user_out, char **xdg_out)\n \t*xdg_out = xdg_config;\n }\n \n-/*\n- * Parse environment variable 'k' as a boolean (in various\n- * possible spellings); if missing, use the default value 'def'.\n- */\n-int git_env_bool(const char *k, int def)\n-{\n-\tconst char *v = getenv(k);\n-\tint val;\n-\tif (!v)\n-\t\treturn def;\n-\tval = git_parse_maybe_bool(v);\n-\tif (val < 0)\n-\t\tdie(_(\"bad boolean environment value '%s' for '%s'\"),\n-\t\t v, k);\n-\treturn val;\n-}\n-\n-/*\n- * Parse environment variable 'k' as ulong with possibly a unit\n- * suffix; if missing, use the default value 'val'.\n- */\n-unsigned long git_env_ulong(const char *k, unsigned long val)\n-{\n-\tconst char *v = getenv(k);\n-\tif (v && !git_parse_ulong(v, &val))\n-\t\tdie(_(\"failed to parse %s\"), k);\n-\treturn val;\n-}\n-\n int git_config_system(void)\n {\n \treturn !git_env_bool(\"GIT_CONFIG_NOSYSTEM\", 0);\ndiff --git a/config.h b/config.h\nindex 6332d74904..14f881ecfa 100644\n--- a/config.h\n+++ b/config.h\n@@ -4,7 +4,7 @@\n #include \"hashmap.h\"\n #include \"string-list.h\"\n #include \"repository.h\"\n-\n+#include \"parse.h\"\n \n /**\n * The config API gives callers a way to access Git configuration files\n@@ -243,16 +243,6 @@ int config_with_options(config_fn_t fn, void *,\n * The following helper functions aid in parsing string values\n */\n \n-int git_parse_ssize_t(const char *, ssize_t *);\n-int git_parse_ulong(const char *, unsigned long *);\n-int git_parse_int(const char *value, int *ret);\n-\n-/**\n- * Same as `git_config_bool`, except that it returns -1 on error rather\n- * than dying.\n- */\n-int git_parse_maybe_bool(const char *);\n-\n /**\n * Parse the string to an integer, including unit factors. Dies on error;\n * otherwise, returns the parsed result.\n@@ -385,8 +375,6 @@ int git_config_rename_section(const char *, const char *);\n int git_config_rename_section_in_file(const char *, const char *, const char *);\n int git_config_copy_section(const char *, const char *);\n int git_config_copy_section_in_file(const char *, const char *, const char *);\n-int git_env_bool(const char *, int);\n-unsigned long git_env_ulong(const char *, unsigned long);\n int git_config_system(void);\n int config_error_nonbool(const char *);\n #if defined(__GNUC__)\ndiff --git a/pack-objects.c b/pack-objects.c\nindex 1b8052bece..f403ca6986 100644\n--- a/pack-objects.c\n+++ b/pack-objects.c\n@@ -3,7 +3,7 @@\n #include \"pack.h\"\n #include \"pack-objects.h\"\n #include \"packfile.h\"\n-#include \"config.h\"\n+#include \"parse.h\"\n \n static uint32_t locate_object_entry_hash(struct packing_data *pdata,\n \t\t\t\t\t const struct object_id *oid,\ndiff --git a/pack-revindex.c b/pack-revindex.c\nindex 7fffcad912..a01a2a4640 100644\n--- a/pack-revindex.c\n+++ b/pack-revindex.c\n@@ -6,7 +6,7 @@\n #include \"packfile.h\"\n #include \"strbuf.h\"\n #include \"trace2.h\"\n-#include \"config.h\"\n+#include \"parse.h\"\n #include \"midx.h\"\n #include \"csum-file.h\"\n \ndiff --git a/parse-options.c b/parse-options.c\nindex e8e076c3a6..093eaf2db8 100644\n--- a/parse-options.c\n+++ b/parse-options.c\n@@ -1,11 +1,12 @@\n #include \"git-compat-util.h\"\n #include \"parse-options.h\"\n #include \"abspath.h\"\n-#include \"config.h\"\n+#include \"parse.h\"\n #include \"commit.h\"\n #include \"color.h\"\n #include \"gettext.h\"\n #include \"strbuf.h\"\n+#include \"string-list.h\"\n #include \"utf8.h\"\n \n static int disallow_abbreviated_options;\ndiff --git a/parse.c b/parse.c\nnew file mode 100644\nindex 0000000000..42d691a0fb\n--- /dev/null\n+++ b/parse.c\n@@ -0,0 +1,182 @@\n+#include \"git-compat-util.h\"\n+#include \"gettext.h\"\n+#include \"parse.h\"\n+\n+static uintmax_t get_unit_factor(const char *end)\n+{\n+\tif (!*end)\n+\t\treturn 1;\n+\telse if (!strcasecmp(end, \"k\"))\n+\t\treturn 1024;\n+\telse if (!strcasecmp(end, \"m\"))\n+\t\treturn 1024 * 1024;\n+\telse if (!strcasecmp(end, \"g\"))\n+\t\treturn 1024 * 1024 * 1024;\n+\treturn 0;\n+}\n+\n+int git_parse_signed(const char *value, intmax_t *ret, intmax_t max)\n+{\n+\tif (value && *value) {\n+\t\tchar *end;\n+\t\tintmax_t val;\n+\t\tintmax_t factor;\n+\n+\t\tif (max < 0)\n+\t\t\tBUG(\"max must be a positive integer\");\n+\n+\t\terrno = 0;\n+\t\tval = strtoimax(value, &end, 0);\n+\t\tif (errno == ERANGE)\n+\t\t\treturn 0;\n+\t\tif (end == value) {\n+\t\t\terrno = EINVAL;\n+\t\t\treturn 0;\n+\t\t}\n+\t\tfactor = get_unit_factor(end);\n+\t\tif (!factor) {\n+\t\t\terrno = EINVAL;\n+\t\t\treturn 0;\n+\t\t}\n+\t\tif ((val < 0 && -max / factor > val) ||\n+\t\t (val > 0 && max / factor < val)) {\n+\t\t\terrno = ERANGE;\n+\t\t\treturn 0;\n+\t\t}\n+\t\tval *= factor;\n+\t\t*ret = val;\n+\t\treturn 1;\n+\t}\n+\terrno = EINVAL;\n+\treturn 0;\n+}\n+\n+static int git_parse_unsigned(const char *value, uintmax_t *ret, uintmax_t max)\n+{\n+\tif (value && *value) {\n+\t\tchar *end;\n+\t\tuintmax_t val;\n+\t\tuintmax_t factor;\n+\n+\t\t/* negative values would be accepted by strtoumax */\n+\t\tif (strchr(value, '-')) {\n+\t\t\terrno = EINVAL;\n+\t\t\treturn 0;\n+\t\t}\n+\t\terrno = 0;\n+\t\tval = strtoumax(value, &end, 0);\n+\t\tif (errno == ERANGE)\n+\t\t\treturn 0;\n+\t\tif (end == value) {\n+\t\t\terrno = EINVAL;\n+\t\t\treturn 0;\n+\t\t}\n+\t\tfactor = get_unit_factor(end);\n+\t\tif (!factor) {\n+\t\t\terrno = EINVAL;\n+\t\t\treturn 0;\n+\t\t}\n+\t\tif (unsigned_mult_overflows(factor, val) ||\n+\t\t factor * val > max) {\n+\t\t\terrno = ERANGE;\n+\t\t\treturn 0;\n+\t\t}\n+\t\tval *= factor;\n+\t\t*ret = val;\n+\t\treturn 1;\n+\t}\n+\terrno = EINVAL;\n+\treturn 0;\n+}\n+\n+int git_parse_int(const char *value, int *ret)\n+{\n+\tintmax_t tmp;\n+\tif (!git_parse_signed(value, &tmp, maximum_signed_value_of_type(int)))\n+\t\treturn 0;\n+\t*ret = tmp;\n+\treturn 1;\n+}\n+\n+int git_parse_int64(const char *value, int64_t *ret)\n+{\n+\tintmax_t tmp;\n+\tif (!git_parse_signed(value, &tmp, maximum_signed_value_of_type(int64_t)))\n+\t\treturn 0;\n+\t*ret = tmp;\n+\treturn 1;\n+}\n+\n+int git_parse_ulong(const char *value, unsigned long *ret)\n+{\n+\tuintmax_t tmp;\n+\tif (!git_parse_unsigned(value, &tmp, maximum_unsigned_value_of_type(long)))\n+\t\treturn 0;\n+\t*ret = tmp;\n+\treturn 1;\n+}\n+\n+int git_parse_ssize_t(const char *value, ssize_t *ret)\n+{\n+\tintmax_t tmp;\n+\tif (!git_parse_signed(value, &tmp, maximum_signed_value_of_type(ssize_t)))\n+\t\treturn 0;\n+\t*ret = tmp;\n+\treturn 1;\n+}\n+\n+int git_parse_maybe_bool_text(const char *value)\n+{\n+\tif (!value)\n+\t\treturn 1;\n+\tif (!*value)\n+\t\treturn 0;\n+\tif (!strcasecmp(value, \"true\")\n+\t || !strcasecmp(value, \"yes\")\n+\t || !strcasecmp(value, \"on\"))\n+\t\treturn 1;\n+\tif (!strcasecmp(value, \"false\")\n+\t || !strcasecmp(value, \"no\")\n+\t || !strcasecmp(value, \"off\"))\n+\t\treturn 0;\n+\treturn -1;\n+}\n+\n+int git_parse_maybe_bool(const char *value)\n+{\n+\tint v = git_parse_maybe_bool_text(value);\n+\tif (0 <= v)\n+\t\treturn v;\n+\tif (git_parse_int(value, &v))\n+\t\treturn !!v;\n+\treturn -1;\n+}\n+\n+/*\n+ * Parse environment variable 'k' as a boolean (in various\n+ * possible spellings); if missing, use the default value 'def'.\n+ */\n+int git_env_bool(const char *k, int def)\n+{\n+\tconst char *v = getenv(k);\n+\tint val;\n+\tif (!v)\n+\t\treturn def;\n+\tval = git_parse_maybe_bool(v);\n+\tif (val < 0)\n+\t\tdie(_(\"bad boolean environment value '%s' for '%s'\"),\n+\t\t v, k);\n+\treturn val;\n+}\n+\n+/*\n+ * Parse environment variable 'k' as ulong with possibly a unit\n+ * suffix; if missing, use the default value 'val'.\n+ */\n+unsigned long git_env_ulong(const char *k, unsigned long val)\n+{\n+\tconst char *v = getenv(k);\n+\tif (v && !git_parse_ulong(v, &val))\n+\t\tdie(_(\"failed to parse %s\"), k);\n+\treturn val;\n+}\ndiff --git a/parse.h b/parse.h\nnew file mode 100644\nindex 0000000000..07d2193d69\n--- /dev/null\n+++ b/parse.h\n@@ -0,0 +1,20 @@\n+#ifndef PARSE_H\n+#define PARSE_H\n+\n+int git_parse_signed(const char *value, intmax_t *ret, intmax_t max);\n+int git_parse_ssize_t(const char *, ssize_t *);\n+int git_parse_ulong(const char *, unsigned long *);\n+int git_parse_int(const char *value, int *ret);\n+int git_parse_int64(const char *value, int64_t *ret);\n+\n+/**\n+ * Same as `git_config_bool`, except that it returns -1 on error rather\n+ * than dying.\n+ */\n+int git_parse_maybe_bool(const char *);\n+int git_parse_maybe_bool_text(const char *value);\n+\n+int git_env_bool(const char *, int);\n+unsigned long git_env_ulong(const char *, unsigned long);\n+\n+#endif /* PARSE_H */\ndiff --git a/pathspec.c b/pathspec.c\nindex 3a3a5724c4..7f88f1c02b 100644\n--- a/pathspec.c\n+++ b/pathspec.c\n@@ -1,6 +1,6 @@\n #include \"git-compat-util.h\"\n #include \"abspath.h\"\n-#include \"config.h\"\n+#include \"parse.h\"\n #include \"dir.h\"\n #include \"environment.h\"\n #include \"gettext.h\"\ndiff --git a/preload-index.c b/preload-index.c\nindex e44530c80c..63fd35d64b 100644\n--- a/preload-index.c\n+++ b/preload-index.c\n@@ -7,7 +7,7 @@\n #include \"environment.h\"\n #include \"fsmonitor.h\"\n #include \"gettext.h\"\n-#include \"config.h\"\n+#include \"parse.h\"\n #include \"preload-index.h\"\n #include \"progress.h\"\n #include \"read-cache.h\"\ndiff --git a/progress.c b/progress.c\nindex f695798aca..c83cb60bf1 100644\n--- a/progress.c\n+++ b/progress.c\n@@ -17,7 +17,7 @@\n #include \"trace.h\"\n #include \"trace2.h\"\n #include \"utf8.h\"\n-#include \"config.h\"\n+#include \"parse.h\"\n \n #define TP_IDX_MAX 8\n \ndiff --git a/prompt.c b/prompt.c\nindex 3baa33f63d..8935fe4dfb 100644\n--- a/prompt.c\n+++ b/prompt.c\n@@ -1,5 +1,5 @@\n #include \"git-compat-util.h\"\n-#include \"config.h\"\n+#include \"parse.h\"\n #include \"environment.h\"\n #include \"run-command.h\"\n #include \"strbuf.h\"\ndiff --git a/rebase.c b/rebase.c\nindex 17a570f1ff..69a1822da3 100644\n--- a/rebase.c\n+++ b/rebase.c\n@@ -1,6 +1,6 @@\n #include \"git-compat-util.h\"\n #include \"rebase.h\"\n-#include \"config.h\"\n+#include \"parse.h\"\n #include \"gettext.h\"\n \n /*\ndiff --git a/t/helper/test-env-helper.c b/t/helper/test-env-helper.c\nindex 66c88b8ff3..1c486888a4 100644\n--- a/t/helper/test-env-helper.c\n+++ b/t/helper/test-env-helper.c\n@@ -1,5 +1,5 @@\n #include \"test-tool.h\"\n-#include \"config.h\"\n+#include \"parse.h\"\n #include \"parse-options.h\"\n \n static char const * const env__helper_usage[] = {\ndiff --git a/unpack-trees.c b/unpack-trees.c\nindex 87517364dc..761562a96e 100644\n--- a/unpack-trees.c\n+++ b/unpack-trees.c\n@@ -2,7 +2,7 @@\n #include \"advice.h\"\n #include \"strvec.h\"\n #include \"repository.h\"\n-#include \"config.h\"\n+#include \"parse.h\"\n #include \"dir.h\"\n #include \"environment.h\"\n #include \"gettext.h\"\ndiff --git a/wrapper.c b/wrapper.c\nindex 453a20ed99..7da15a56da 100644\n--- a/wrapper.c\n+++ b/wrapper.c\n@@ -3,7 +3,7 @@\n */\n #include \"git-compat-util.h\"\n #include \"abspath.h\"\n-#include \"config.h\"\n+#include \"parse.h\"\n #include \"gettext.h\"\n #include \"repository.h\"\n #include \"strbuf.h\"\ndiff --git a/write-or-die.c b/write-or-die.c\nindex d8355c0c3e..42a2dc73cd 100644\n--- a/write-or-die.c\n+++ b/write-or-die.c\n@@ -1,5 +1,5 @@\n #include \"git-compat-util.h\"\n-#include \"config.h\"\n+#include \"parse.h\"\n #include \"run-command.h\"\n #include \"write-or-die.h\"\n \n-- \n2.42.0.283.g2d96d420d3-goog\n\n"},{"id":"481571","messageId":"20230908174443.1027716-5-calvinwan@google.com","threadId":"59921","inReplyTo":"20230908174134.1026823-1-calvinwan@google.com","subject":"[PATCH v3 5/6] git-std-lib: introduce git standard library","fromName":"Calvin Wan","fromEmail":"calvinwan@google.com","sentAt":"2023-09-08T17:44:42Z","receivedAt":"2023-09-08T17:45:14Z","isPatch":true,"sender":{"key":"calvinwan@google.com","avatar":"https://avatars.githubusercontent.com/u/92547554?v=4"},"body":"The Git Standard Library intends to serve as the foundational library\nand root dependency that other libraries in Git will be built off of.\nThat is to say, suppose we have libraries X and Y; a user that wants to\nuse X and Y would need to include X, Y, and this Git Standard Library.\n\nAdd Documentation/technical/git-std-lib.txt to further explain the\ndesign and rationale.\n\nSigned-off-by: Calvin Wan \nHelped-by: Phillip Wood \n---\n Documentation/technical/git-std-lib.txt | 191 ++++++++++++++++++++++++\n Makefile | 39 ++++-\n git-compat-util.h | 7 +-\n stubs/pager.c | 6 +\n stubs/pager.h | 6 +\n stubs/trace2.c | 27 ++++\n symlinks.c | 2 +\n wrapper.c | 1 -\n 8 files changed, 276 insertions(+), 3 deletions(-)\n create mode 100644 Documentation/technical/git-std-lib.txt\n create mode 100644 stubs/pager.c\n create mode 100644 stubs/pager.h\n create mode 100644 stubs/trace2.c\n\ndiff --git a/Documentation/technical/git-std-lib.txt b/Documentation/technical/git-std-lib.txt\nnew file mode 100644\nindex 0000000000..397c1da8c8\n--- /dev/null\n+++ b/Documentation/technical/git-std-lib.txt\n@@ -0,0 +1,191 @@\n+Git Standard Library\n+================\n+\n+The Git Standard Library intends to serve as the foundational library\n+and root dependency that other libraries in Git will be built off of.\n+That is to say, suppose we have libraries X and Y; a user that wants to\n+use X and Y would need to include X, Y, and this Git Standard Library.\n+This does not mean that the Git Standard Library will be the only\n+possible root dependency in the future, but rather the most significant\n+and widely used one.\n+\n+Dependency graph in libified Git\n+================\n+\n+If you look in the Git Makefile, all of the objects defined in the Git\n+library are compiled and archived into a singular file, libgit.a, which\n+is linked against by common-main.o with other external dependencies and\n+turned into the Git executable. In other words, the Git executable has\n+dependencies on libgit.a and a couple of external libraries. The\n+libfication of Git will not affect this current build flow, but instead\n+will provide an alternate method for building Git.\n+\n+With our current method of building Git, we can imagine the dependency\n+graph as such:\n+\n+ Git\n+ /\\\n+ / \\\n+ / \\\n+ libgit.a ext deps\n+\n+In libifying parts of Git, we want to shrink the dependency graph to\n+only the minimal set of dependencies, so libraries should not use\n+libgit.a. Instead, it would look like:\n+\n+ Git\n+ /\\\n+ / \\\n+ / \\\n+ libgit.a ext deps\n+ /\\\n+ / \\\n+ / \\\n+object-store.a (other lib)\n+ | /\n+ | /\n+ | /\n+ config.a / \n+ | /\n+ | /\n+ | /\n+git-std-lib.a\n+\n+Instead of containing all of the objects in Git, libgit.a would contain\n+objects that are not built by libraries it links against. Consequently,\n+if someone wanted their own custom build of Git with their own custom\n+implementation of the object store, they would only have to swap out\n+object-store.a rather than do a hard fork of Git.\n+\n+Rationale behind Git Standard Library\n+================\n+\n+The rationale behind what's in and what's not in the Git Standard\n+Library essentially is the result of two observations within the Git\n+codebase: every file includes git-compat-util.h which defines functions\n+in a couple of different files, and wrapper.c + usage.c have\n+difficult-to-separate circular dependencies with each other and other\n+files.\n+\n+Ubiquity of git-compat-util.h and circular dependencies\n+========\n+\n+Every file in the Git codebase includes git-compat-util.h. It serves as\n+\"a compatibility aid that isolates the knowledge of platform specific\n+inclusion order and what feature macros to define before including which\n+system header\" (Junio[1]). Since every file includes git-compat-util.h, and\n+git-compat-util.h includes wrapper.h and usage.h, it would make sense\n+for wrapper.c and usage.c to be a part of the root library. They have\n+difficult to separate circular dependencies with each other so they\n+can't be independent libraries. Wrapper.c has dependencies on parse.c,\n+abspath.c, strbuf.c, which in turn also have dependencies on usage.c and\n+wrapper.c -- more circular dependencies. \n+\n+Tradeoff between swappability and refactoring\n+========\n+\n+From the above dependency graph, we can see that git-std-lib.a could be\n+many smaller libraries rather than a singular library. So why choose a\n+singular library when multiple libraries can be individually easier to\n+swap and are more modular? A singular library requires less work to\n+separate out circular dependencies within itself so it becomes a\n+tradeoff question between work and reward. While there may be a point in\n+the future where a file like usage.c would want its own library so that\n+someone can have custom die() or error(), the work required to refactor\n+out the circular dependencies in some files would be enormous due to\n+their ubiquity so therefore I believe it is not worth the tradeoff\n+currently. Additionally, we can in the future choose to do this refactor\n+and change the API for the library if there becomes enough of a reason\n+to do so (remember we are avoiding promising stability of the interfaces\n+of those libraries).\n+\n+Reuse of compatibility functions in git-compat-util.h\n+========\n+\n+Most functions defined in git-compat-util.h are implemented in compat/\n+and have dependencies limited to strbuf.h and wrapper.h so they can be\n+easily included in git-std-lib.a, which as a root dependency means that\n+higher level libraries do not have to worry about compatibility files in\n+compat/. The rest of the functions defined in git-compat-util.h are\n+implemented in top level files and are hidden behind\n+an #ifdef if their implementation is not in git-std-lib.a.\n+\n+Rationale summary\n+========\n+\n+The Git Standard Library allows us to get the libification ball rolling\n+with other libraries in Git. By not spending many\n+more months attempting to refactor difficult circular dependencies and\n+instead spending that time getting to a state where we can test out\n+swapping a library out such as config or object store, we can prove the\n+viability of Git libification on a much faster time scale. Additionally\n+the code cleanups that have happened so far have been minor and\n+beneficial for the codebase. It is probable that making large movements\n+would negatively affect code clarity.\n+\n+Git Standard Library boundary\n+================\n+\n+While I have described above some useful heuristics for identifying\n+potential candidates for git-std-lib.a, a standard library should not\n+have a shaky definition for what belongs in it.\n+\n+ - Low-level files (aka operates only on other primitive types) that are\n+ used everywhere within the codebase (wrapper.c, usage.c, strbuf.c)\n+ - Dependencies that are low-level and widely used\n+ (abspath.c, date.c, hex-ll.c, parse.c, utf8.c)\n+ - low-level git/* files with functions defined in git-compat-util.h\n+ (ctype.c)\n+ - compat/*\n+ - stubbed out dependencies in stubs/ (stubs/pager.c, stubs/trace2.c)\n+\n+There are other files that might fit this definition, but that does not\n+mean it should belong in git-std-lib.a. Those files should start as\n+their own separate library since any file added to git-std-lib.a loses\n+its flexibility of being easily swappable.\n+\n+Wrapper.c and usage.c have dependencies on pager and trace2 that are\n+possible to remove at the cost of sacrificing the ability for standard Git\n+to be able to trace functions in those files and other files in git-std-lib.a.\n+In order for git-std-lib.a to compile with those dependencies, stubbed out\n+versions of those files are implemented and swapped in during compilation time.\n+\n+Files inside of Git Standard Library\n+================\n+\n+The initial set of files in git-std-lib.a are:\n+abspath.c\n+ctype.c\n+date.c\n+hex-ll.c\n+parse.c\n+strbuf.c\n+usage.c\n+utf8.c\n+wrapper.c\n+relevant compat/ files\n+\n+When these files are compiled together with the following files (or\n+user-provided files that provide the same functions), they form a\n+complete library:\n+stubs/pager.c\n+stubs/trace2.c\n+\n+Pitfalls\n+================\n+\n+There are a small amount of files under compat/* that have dependencies\n+not inside of git-std-lib.a. While those functions are not called on\n+Linux, other OSes might call those problematic functions. I don't see\n+this as a major problem, just moreso an observation that libification in\n+general may also require some minor compatibility work in the future.\n+\n+Testing\n+================\n+\n+Unit tests should catch any breakages caused by changes to files in\n+git-std-lib.a (i.e. introduction of a out of scope dependency) and new\n+functions introduced to git-std-lib.a will require unit tests written\n+for them.\n+\n+[1] https://lore.kernel.org/git/xmqqwn17sydw.fsf@gitster.g/\n\\ No newline at end of file\ndiff --git a/Makefile b/Makefile\nindex 9226c719a0..0a2d1ae3cc 100644\n--- a/Makefile\n+++ b/Makefile\n@@ -669,6 +669,7 @@ FUZZ_PROGRAMS =\n GIT_OBJS =\n LIB_OBJS =\n SCALAR_OBJS =\n+STUB_OBJS =\n OBJECTS =\n OTHER_PROGRAMS =\n PROGRAM_OBJS =\n@@ -956,6 +957,7 @@ COCCI_SOURCES = $(filter-out $(THIRD_PARTY_SOURCES),$(FOUND_C_SOURCES))\n \n LIB_H = $(FOUND_H_SOURCES)\n \n+ifndef GIT_STD_LIB\n LIB_OBJS += abspath.o\n LIB_OBJS += add-interactive.o\n LIB_OBJS += add-patch.o\n@@ -1196,6 +1198,27 @@ LIB_OBJS += write-or-die.o\n LIB_OBJS += ws.o\n LIB_OBJS += wt-status.o\n LIB_OBJS += xdiff-interface.o\n+else ifdef GIT_STD_LIB\n+LIB_OBJS += abspath.o\n+LIB_OBJS += ctype.o\n+LIB_OBJS += date.o\n+LIB_OBJS += hex-ll.o\n+LIB_OBJS += parse.o\n+LIB_OBJS += strbuf.o\n+LIB_OBJS += usage.o\n+LIB_OBJS += utf8.o\n+LIB_OBJS += wrapper.o\n+\n+ifdef STUB_TRACE2\n+STUB_OBJS += stubs/trace2.o\n+endif\n+\n+ifdef STUB_PAGER\n+STUB_OBJS += stubs/pager.o\n+endif\n+\n+LIB_OBJS += $(STUB_OBJS)\n+endif\n \n BUILTIN_OBJS += builtin/add.o\n BUILTIN_OBJS += builtin/am.o\n@@ -2162,6 +2185,11 @@ ifdef FSMONITOR_OS_SETTINGS\n \tCOMPAT_OBJS += compat/fsmonitor/fsm-path-utils-$(FSMONITOR_OS_SETTINGS).o\n endif\n \n+ifdef GIT_STD_LIB\n+\tBASIC_CFLAGS += -DGIT_STD_LIB\n+\tBASIC_CFLAGS += -DNO_GETTEXT\n+endif\n+\n ifeq ($(TCLTK_PATH),)\n NO_TCLTK = NoThanks\n endif\n@@ -3668,7 +3696,7 @@ clean: profile-clean coverage-clean cocciclean\n \t$(RM) git.res\n \t$(RM) $(OBJECTS)\n \t$(RM) headless-git.o\n-\t$(RM) $(LIB_FILE) $(XDIFF_LIB) $(REFTABLE_LIB) $(REFTABLE_TEST_LIB)\n+\t$(RM) $(LIB_FILE) $(XDIFF_LIB) $(REFTABLE_LIB) $(REFTABLE_TEST_LIB) $(STD_LIB_FILE)\n \t$(RM) $(ALL_PROGRAMS) $(SCRIPT_LIB) $(BUILT_INS) $(OTHER_PROGRAMS)\n \t$(RM) $(TEST_PROGRAMS)\n \t$(RM) $(FUZZ_PROGRAMS)\n@@ -3849,3 +3877,12 @@ $(FUZZ_PROGRAMS): all\n \t\t$(XDIFF_OBJS) $(EXTLIBS) git.o $@.o $(LIB_FUZZING_ENGINE) -o $@\n \n fuzz-all: $(FUZZ_PROGRAMS)\n+\n+### Libified Git rules\n+\n+# git-std-lib\n+# `make git-std-lib.a GIT_STD_LIB=YesPlease STUB_TRACE2=YesPlease STUB_PAGER=YesPlease`\n+STD_LIB = git-std-lib.a\n+\n+$(STD_LIB): $(LIB_OBJS) $(COMPAT_OBJS) $(STUB_OBJS)\n+\t$(QUIET_AR)$(RM) $@ && $(AR) $(ARFLAGS) $@ $^\ndiff --git a/git-compat-util.h b/git-compat-util.h\nindex 3e7a59b5ff..14bf71c530 100644\n--- a/git-compat-util.h\n+++ b/git-compat-util.h\n@@ -455,8 +455,8 @@ static inline int noop_core_config(const char *var UNUSED,\n #define platform_core_config noop_core_config\n #endif\n \n+#if !defined(__MINGW32__) && !defined(_MSC_VER) && !defined(GIT_STD_LIB)\n int lstat_cache_aware_rmdir(const char *path);\n-#if !defined(__MINGW32__) && !defined(_MSC_VER)\n #define rmdir lstat_cache_aware_rmdir\n #endif\n \n@@ -966,9 +966,11 @@ const char *inet_ntop(int af, const void *src, char *dst, size_t size);\n #endif\n \n #ifdef NO_PTHREADS\n+#ifdef GIT_STD_LIB\n #define atexit git_atexit\n int git_atexit(void (*handler)(void));\n #endif\n+#endif\n \n static inline size_t st_add(size_t a, size_t b)\n {\n@@ -1462,14 +1464,17 @@ static inline int is_missing_file_error(int errno_)\n \treturn (errno_ == ENOENT || errno_ == ENOTDIR);\n }\n \n+#ifndef GIT_STD_LIB\n int cmd_main(int, const char **);\n \n /*\n * Intercept all calls to exit() and route them to trace2 to\n * optionally emit a message before calling the real exit().\n */\n+\n int common_exit(const char *file, int line, int code);\n #define exit(code) exit(common_exit(__FILE__, __LINE__, (code)))\n+#endif\n \n /*\n * You can mark a stack variable with UNLEAK(var) to avoid it being\ndiff --git a/stubs/pager.c b/stubs/pager.c\nnew file mode 100644\nindex 0000000000..4f575cada7\n--- /dev/null\n+++ b/stubs/pager.c\n@@ -0,0 +1,6 @@\n+#include \"pager.h\"\n+\n+int pager_in_use(void)\n+{\n+\treturn 0;\n+}\ndiff --git a/stubs/pager.h b/stubs/pager.h\nnew file mode 100644\nindex 0000000000..b797910881\n--- /dev/null\n+++ b/stubs/pager.h\n@@ -0,0 +1,6 @@\n+#ifndef PAGER_H\n+#define PAGER_H\n+\n+int pager_in_use(void);\n+\n+#endif /* PAGER_H */\ndiff --git a/stubs/trace2.c b/stubs/trace2.c\nnew file mode 100644\nindex 0000000000..7d89482228\n--- /dev/null\n+++ b/stubs/trace2.c\n@@ -0,0 +1,27 @@\n+#include \"git-compat-util.h\"\n+#include \"trace2.h\"\n+\n+struct child_process { int stub; };\n+struct repository { int stub; };\n+struct json_writer { int stub; };\n+\n+void trace2_region_enter_fl(const char *file, int line, const char *category,\n+\t\t\t const char *label, const struct repository *repo, ...) { }\n+void trace2_region_leave_fl(const char *file, int line, const char *category,\n+\t\t\t const char *label, const struct repository *repo, ...) { }\n+void trace2_data_string_fl(const char *file, int line, const char *category,\n+\t\t\t const struct repository *repo, const char *key,\n+\t\t\t const char *value) { }\n+void trace2_cmd_ancestry_fl(const char *file, int line, const char **parent_names) { }\n+void trace2_cmd_error_va_fl(const char *file, int line, const char *fmt,\n+\t\t\t va_list ap) { }\n+void trace2_cmd_name_fl(const char *file, int line, const char *name) { }\n+void trace2_thread_start_fl(const char *file, int line,\n+\t\t\t const char *thread_base_name) { }\n+void trace2_thread_exit_fl(const char *file, int line) { }\n+void trace2_data_intmax_fl(const char *file, int line, const char *category,\n+\t\t\t const struct repository *repo, const char *key,\n+\t\t\t intmax_t value) { }\n+int trace2_is_enabled(void) { return 0; }\n+void trace2_counter_add(enum trace2_counter_id cid, uint64_t value) { }\n+void trace2_collect_process_info(enum trace2_process_info_reason reason) { }\ndiff --git a/symlinks.c b/symlinks.c\nindex b29e340c2d..bced721a0c 100644\n--- a/symlinks.c\n+++ b/symlinks.c\n@@ -337,6 +337,7 @@ void invalidate_lstat_cache(void)\n \treset_lstat_cache(&default_cache);\n }\n \n+#if !defined(__MINGW32__) && !defined(_MSC_VER) && !defined(GIT_STD_LIB)\n #undef rmdir\n int lstat_cache_aware_rmdir(const char *path)\n {\n@@ -348,3 +349,4 @@ int lstat_cache_aware_rmdir(const char *path)\n \n \treturn ret;\n }\n+#endif\ndiff --git a/wrapper.c b/wrapper.c\nindex 7da15a56da..eeac3741cf 100644\n--- a/wrapper.c\n+++ b/wrapper.c\n@@ -5,7 +5,6 @@\n #include \"abspath.h\"\n #include \"parse.h\"\n #include \"gettext.h\"\n-#include \"repository.h\"\n #include \"strbuf.h\"\n #include \"trace2.h\"\n \n-- \n2.42.0.283.g2d96d420d3-goog\n\n"},{"id":"481572","messageId":"20230908174443.1027716-6-calvinwan@google.com","threadId":"59921","inReplyTo":"20230908174134.1026823-1-calvinwan@google.com","subject":"[PATCH v3 6/6] git-std-lib: add test file to call git-std-lib.a functions","fromName":"Calvin Wan","fromEmail":"calvinwan@google.com","sentAt":"2023-09-08T17:44:43Z","receivedAt":"2023-09-08T17:45:16Z","isPatch":true,"sender":{"key":"calvinwan@google.com","avatar":"https://avatars.githubusercontent.com/u/92547554?v=4"},"body":"Add test file that directly or indirectly calls all functions defined in\ngit-std-lib.a object files to showcase that they do not reference\nmissing objects and that git-std-lib.a can stand on its own.\n\nCertain functions that cause the program to exit or are already called\nby other functions are commented out.\n\nTODO: replace with unit tests\nSigned-off-by: Calvin Wan \n---\n t/Makefile | 4 +\n t/stdlib-test.c | 231 ++++++++++++++++++++++++++++++++++++++++++++++++\n 2 files changed, 235 insertions(+)\n create mode 100644 t/stdlib-test.c\n\ndiff --git a/t/Makefile b/t/Makefile\nindex 3e00cdd801..b6d0bc9daa 100644\n--- a/t/Makefile\n+++ b/t/Makefile\n@@ -150,3 +150,7 @@ perf:\n \n .PHONY: pre-clean $(T) aggregate-results clean valgrind perf \\\n \tcheck-chainlint clean-chainlint test-chainlint\n+\n+test-git-std-lib:\n+\tcc -It -o stdlib-test stdlib-test.c -L. -l:../git-std-lib.a\n+\t./stdlib-test\ndiff --git a/t/stdlib-test.c b/t/stdlib-test.c\nnew file mode 100644\nindex 0000000000..76fed9ecbf\n--- /dev/null\n+++ b/t/stdlib-test.c\n@@ -0,0 +1,231 @@\n+#include \"../git-compat-util.h\"\n+#include \"../abspath.h\"\n+#include \"../hex-ll.h\"\n+#include \"../parse.h\"\n+#include \"../strbuf.h\"\n+#include \"../string-list.h\"\n+\n+/*\n+ * Calls all functions from git-std-lib\n+ * Some inline/trivial functions are skipped\n+ */\n+\n+void abspath_funcs(void) {\n+\tstruct strbuf sb = STRBUF_INIT;\n+\n+\tfprintf(stderr, \"calling abspath functions\\n\");\n+\tis_directory(\"foo\");\n+\tstrbuf_realpath(&sb, \"foo\", 0);\n+\tstrbuf_realpath_forgiving(&sb, \"foo\", 0);\n+\treal_pathdup(\"foo\", 0);\n+\tabsolute_path(\"foo\");\n+\tabsolute_pathdup(\"foo\");\n+\tprefix_filename(\"foo/\", \"bar\");\n+\tprefix_filename_except_for_dash(\"foo/\", \"bar\");\n+\tis_absolute_path(\"foo\");\n+\tstrbuf_add_absolute_path(&sb, \"foo\");\n+\tstrbuf_add_real_path(&sb, \"foo\");\n+}\n+\n+void hex_ll_funcs(void) {\n+\tunsigned char c;\n+\n+\tfprintf(stderr, \"calling hex-ll functions\\n\");\n+\n+\thexval('c');\n+\thex2chr(\"A1\");\n+\thex_to_bytes(&c, \"A1\", 2);\n+}\n+\n+void parse_funcs(void) {\n+\tintmax_t foo;\n+\tssize_t foo1 = -1;\n+\tunsigned long foo2;\n+\tint foo3;\n+\tint64_t foo4;\n+\n+\tfprintf(stderr, \"calling parse functions\\n\");\n+\n+\tgit_parse_signed(\"42\", &foo, maximum_signed_value_of_type(int));\n+\tgit_parse_ssize_t(\"42\", &foo1);\n+\tgit_parse_ulong(\"42\", &foo2);\n+\tgit_parse_int(\"42\", &foo3);\n+\tgit_parse_int64(\"42\", &foo4);\n+\tgit_parse_maybe_bool(\"foo\");\n+\tgit_parse_maybe_bool_text(\"foo\");\n+\tgit_env_bool(\"foo\", 1);\n+\tgit_env_ulong(\"foo\", 1);\n+}\n+\n+static int allow_unencoded_fn(char ch) {\n+\treturn 0;\n+}\n+\n+void strbuf_funcs(void) {\n+\tstruct strbuf *sb = xmalloc(sizeof(void*));\n+\tstruct strbuf *sb2 = xmalloc(sizeof(void*));\n+\tstruct strbuf sb3 = STRBUF_INIT;\n+\tstruct string_list list = STRING_LIST_INIT_NODUP;\n+\tchar *buf = \"foo\";\n+\tint fd = open(\"/dev/null\", O_RDONLY);\n+\n+\tfprintf(stderr, \"calling strbuf functions\\n\");\n+\n+\tstarts_with(\"foo\", \"bar\");\n+\tistarts_with(\"foo\", \"bar\");\n+\t// skip_to_optional_arg_default(const char *str, const char *prefix,\n+\t// \t\t\t const char **arg, const char *def)\n+\tstrbuf_init(sb, 0);\n+\tstrbuf_init(sb2, 0);\n+\tstrbuf_release(sb);\n+\tstrbuf_attach(sb, strbuf_detach(sb, NULL), 0, 0); // calls strbuf_grow\n+\tstrbuf_swap(sb, sb2);\n+\tstrbuf_setlen(sb, 0);\n+\tstrbuf_trim(sb); // calls strbuf_rtrim, strbuf_ltrim\n+\t// strbuf_rtrim() called by strbuf_trim()\n+\t// strbuf_ltrim() called by strbuf_trim()\n+\tstrbuf_trim_trailing_dir_sep(sb);\n+\tstrbuf_trim_trailing_newline(sb);\n+\tstrbuf_reencode(sb, \"foo\", \"bar\");\n+\tstrbuf_tolower(sb);\n+\tstrbuf_add_separated_string_list(sb, \" \", &list);\n+\tstrbuf_list_free(strbuf_split_buf(\"foo bar\", 8, ' ', -1));\n+\tstrbuf_cmp(sb, sb2);\n+\tstrbuf_addch(sb, 1);\n+\tstrbuf_splice(sb, 0, 1, \"foo\", 3);\n+\tstrbuf_insert(sb, 0, \"foo\", 3);\n+\t// strbuf_vinsertf() called by strbuf_insertf\n+\tstrbuf_insertf(sb, 0, \"%s\", \"foo\");\n+\tstrbuf_remove(sb, 0, 1);\n+\tstrbuf_add(sb, \"foo\", 3);\n+\tstrbuf_addbuf(sb, sb2);\n+\tstrbuf_join_argv(sb, 0, NULL, ' ');\n+\tstrbuf_addchars(sb, 1, 1);\n+\tstrbuf_addf(sb, \"%s\", \"foo\");\n+\tstrbuf_add_commented_lines(sb, \"foo\", 3, '#');\n+\tstrbuf_commented_addf(sb, '#', \"%s\", \"foo\");\n+\t// strbuf_vaddf() called by strbuf_addf()\n+\tstrbuf_addbuf_percentquote(sb, &sb3);\n+\tstrbuf_add_percentencode(sb, \"foo\", STRBUF_ENCODE_SLASH);\n+\tstrbuf_fread(sb, 0, stdin);\n+\tstrbuf_read(sb, fd, 0);\n+\tstrbuf_read_once(sb, fd, 0);\n+\tstrbuf_write(sb, stderr);\n+\tstrbuf_readlink(sb, \"/dev/null\", 0);\n+\tstrbuf_getcwd(sb);\n+\tstrbuf_getwholeline(sb, stderr, '\\n');\n+\tstrbuf_appendwholeline(sb, stderr, '\\n');\n+\tstrbuf_getline(sb, stderr);\n+\tstrbuf_getline_lf(sb, stderr);\n+\tstrbuf_getline_nul(sb, stderr);\n+\tstrbuf_getwholeline_fd(sb, fd, '\\n');\n+\tstrbuf_read_file(sb, \"/dev/null\", 0);\n+\tstrbuf_add_lines(sb, \"foo\", \"bar\", 0);\n+\tstrbuf_addstr_xml_quoted(sb, \"foo\");\n+\tstrbuf_addstr_urlencode(sb, \"foo\", allow_unencoded_fn);\n+\tstrbuf_humanise_bytes(sb, 42);\n+\tstrbuf_humanise_rate(sb, 42);\n+\tprintf_ln(\"%s\", sb);\n+\tfprintf_ln(stderr, \"%s\", sb);\n+\txstrdup_tolower(\"foo\");\n+\txstrdup_toupper(\"foo\");\n+\t// xstrvfmt() called by xstrfmt()\n+\txstrfmt(\"%s\", \"foo\");\n+\t// strbuf_addftime(struct strbuf *sb, const char *fmt, const struct tm *tm,\n+\t// \t int tz_offset, int suppress_tz_name)\n+\t// strbuf_stripspace(struct strbuf *sb, char comment_line_char)\n+\t// strbuf_strip_suffix(struct strbuf *sb, const char *suffix)\n+\t// strbuf_strip_file_from_path(struct strbuf *sb)\n+}\n+\n+static void error_builtin(const char *err, va_list params) {}\n+static void warn_builtin(const char *err, va_list params) {}\n+\n+static report_fn error_routine = error_builtin;\n+static report_fn warn_routine = warn_builtin;\n+\n+void usage_funcs(void) {\n+\tfprintf(stderr, \"calling usage functions\\n\");\n+\t// Functions that call exit() are commented out\n+\n+\t// usage()\n+\t// usagef()\n+\t// die()\n+\t// die_errno();\n+\terror(\"foo\");\n+\terror_errno(\"foo\");\n+\tdie_message(\"foo\");\n+\tdie_message_errno(\"foo\");\n+\twarning(\"foo\");\n+\twarning_errno(\"foo\");\n+\n+\t// set_die_routine();\n+\tget_die_message_routine();\n+\tset_error_routine(error_builtin);\n+\tget_error_routine();\n+\tset_warn_routine(warn_builtin);\n+\tget_warn_routine();\n+\t// set_die_is_recursing_routine();\n+}\n+\n+void wrapper_funcs(void) {\n+\tvoid *ptr = xmalloc(1);\n+\tint fd = open(\"/dev/null\", O_RDONLY);\n+\tstruct strbuf sb = STRBUF_INIT;\n+\tint mode = 0444;\n+\tchar host[PATH_MAX], path[PATH_MAX], path1[PATH_MAX];\n+\txsnprintf(path, sizeof(path), \"out-XXXXXX\");\n+\txsnprintf(path1, sizeof(path1), \"out-XXXXXX\");\n+\tint tmp;\n+\n+\tfprintf(stderr, \"calling wrapper functions\\n\");\n+\n+\txstrdup(\"foo\");\n+\txmalloc(1);\n+\txmallocz(1);\n+\txmallocz_gently(1);\n+\txmemdupz(\"foo\", 3);\n+\txstrndup(\"foo\", 3);\n+\txrealloc(ptr, 2);\n+\txcalloc(1, 1);\n+\txsetenv(\"foo\", \"bar\", 0);\n+\txopen(\"/dev/null\", O_RDONLY);\n+\txread(fd, &sb, 1);\n+\txwrite(fd, &sb, 1);\n+\txpread(fd, &sb, 1, 0);\n+\txdup(fd);\n+\txfopen(\"/dev/null\", \"r\");\n+\txfdopen(fd, \"r\");\n+\ttmp = xmkstemp(path);\n+\tclose(tmp);\n+\tunlink(path);\n+\ttmp = xmkstemp_mode(path1, mode);\n+\tclose(tmp);\n+\tunlink(path1);\n+\txgetcwd();\n+\tfopen_for_writing(path);\n+\tfopen_or_warn(path, \"r\");\n+\txstrncmpz(\"foo\", \"bar\", 3);\n+\t// xsnprintf() called above\n+\txgethostname(host, 3);\n+\ttmp = git_mkstemps_mode(path, 1, mode);\n+\tclose(tmp);\n+\tunlink(path);\n+\ttmp = git_mkstemp_mode(path, mode);\n+\tclose(tmp);\n+\tunlink(path);\n+\tread_in_full(fd, &sb, 1);\n+\twrite_in_full(fd, &sb, 1);\n+\tpread_in_full(fd, &sb, 1, 0);\n+}\n+\n+int main() {\n+\tabspath_funcs();\n+\thex_ll_funcs();\n+\tparse_funcs();\n+\tstrbuf_funcs();\n+\tusage_funcs();\n+\twrapper_funcs();\n+\tfprintf(stderr, \"all git-std-lib functions finished calling\\n\");\n+\treturn 0;\n+}\n-- \n2.42.0.283.g2d96d420d3-goog\n\n"},{"id":"481575","messageId":"xmqqr0n8s8ps.fsf@gitster.g","threadId":"59921","inReplyTo":"20230908174134.1026823-1-calvinwan@google.com","subject":"Re: [PATCH v3 0/6] Introduce Git Standard Library","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2023-09-08T20:36:47Z","receivedAt":"2023-09-08T20:37:40Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Calvin Wan writes:\n\n> I have taken this series out of RFC since there weren't any significant\n> concerns with the overall concept and design of this series. This reroll\n> incorporates some smaller changes such as dropping the \"push pager\n> dependency\" patch in favor of stubbing it out. The main change this\n> reroll cleans up the Makefile rules and stubs, as suggested by\n> Phillip Wood (appreciate the help on this one)!\n\nWhat is your plan for the \"config-parse\" stuff? The \"create new library\"\nstep in this series seem to aim for the same goal in a different ways.\n\n> This series has been rebased onto 1fc548b2d6a: The sixth batch\n>\n> Originally this series was built on other patches that have since been\n> merged, which is why the range-diff is shown removing many of them.\n\nGood. Previous rounds did not really attract much interest from the\npublic if I recall correctly. Let's see how well this round fares.\n\n> Documentation/technical/git-std-lib.txt | 191 ++++++++++++++++++++\n\nIt is interesting to see that there is no \"std.*lib\\.c\" in the set\nof source files, or \"std.*lib\\.a\" target in the Makefile.\n\n"},{"id":"481576","messageId":"xmqqjzt0s686.fsf@gitster.g","threadId":"59921","inReplyTo":"xmqqr0n8s8ps.fsf@gitster.g","subject":"Re: [PATCH v3 0/6] Introduce Git Standard Library","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2023-09-08T21:30:33Z","receivedAt":"2023-09-08T21:30:43Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Junio C Hamano writes:\n\n> Calvin Wan writes:\n>\n>> I have taken this series out of RFC since there weren't any significant\n>> concerns with the overall concept and design of this series. This reroll\n>> incorporates some smaller changes such as dropping the \"push pager\n>> dependency\" patch in favor of stubbing it out. The main change this\n>> reroll cleans up the Makefile rules and stubs, as suggested by\n>> Phillip Wood (appreciate the help on this one)!\n>\n> What is your plan for the \"config-parse\" stuff? The \"create new library\"\n> step in this series seem to aim for the same goal in a different ways.\n\nActually, this one is far less ambitious in touching \"config\"\nsubsystem, in that it only deals with parsing strings as values.\nThe other one knows how a config file is laid out, what key the\nvalue we are about to read is expected for, etc., and it will\nbenefit by having the \"parse\" code separated out by this series, but\nthey are more or less orthogonal.\n\nQueued. Thanks.\n\n"},{"id":"481615","messageId":"xmqqy1hfrk6p.fsf@gitster.g","threadId":"59921","inReplyTo":"20230908174443.1027716-6-calvinwan@google.com","subject":"Re: [PATCH v3 6/6] git-std-lib: add test file to call git-std-lib.a functions","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2023-09-09T05:26:38Z","receivedAt":"2023-09-09T05:26:45Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Calvin Wan writes:\n\n> +\n> +test-git-std-lib:\n> +\tcc -It -o stdlib-test stdlib-test.c -L. -l:../git-std-lib.a\n\nYuck, no. Try to share as much with the main Makefile one level up.\n\n> +\t./stdlib-test\n> diff --git a/t/stdlib-test.c b/t/stdlib-test.c\n> new file mode 100644\n> index 0000000000..76fed9ecbf\n> --- /dev/null\n> +++ b/t/stdlib-test.c\n> @@ -0,0 +1,231 @@\n> +#include \"../git-compat-util.h\"\n> +#include \"../abspath.h\"\n> +#include \"../hex-ll.h\"\n> +#include \"../parse.h\"\n> +#include \"../strbuf.h\"\n> +#include \"../string-list.h\"\n\nUse -I.. or something, to match what the main Makefile does, so that\nyou do not have to have these \"../\". With -I.., you could even say\n\n #include \n #include \n\netc.\n\n\n> +\t// skip_to_optional_arg_default(const char *str, const char *prefix,\n> +\t// \t\t\t const char **arg, const char *def)\n\nNo // comments in this codebase, please.\n\n> +\tstrbuf_addchars(sb, 1, 1);\n> +\tstrbuf_addf(sb, \"%s\", \"foo\");\n\nhttps://github.com/git/git/actions/runs/6126669144/job/16631124765#step:4:657\n\n"},{"id":"481681","messageId":"4cefe4f8-04ee-48fb-aee4-07342b7a062f@gmail.com","threadId":"59921","inReplyTo":"20230908174443.1027716-5-calvinwan@google.com","subject":"Re: [PATCH v3 5/6] git-std-lib: introduce git standard library","fromName":"Phillip Wood","fromEmail":"phillip.wood123@gmail.com","sentAt":"2023-09-11T13:22:55Z","receivedAt":"2023-09-11T21:38:36Z","isPatch":true,"sender":{"key":"phillip.wood@dunelm.org.uk","avatar":null},"body":"Hi Calvin\n\nOn 08/09/2023 18:44, Calvin Wan wrote:\n> +ifndef GIT_STD_LIB\n> LIB_OBJS += abspath.o\n> LIB_OBJS += add-interactive.o\n> LIB_OBJS += add-patch.o\n> @@ -1196,6 +1198,27 @@ LIB_OBJS += write-or-die.o\n> LIB_OBJS += ws.o\n> LIB_OBJS += wt-status.o\n> LIB_OBJS += xdiff-interface.o\n> +else ifdef GIT_STD_LIB\n> +LIB_OBJS += abspath.o\n> +LIB_OBJS += ctype.o\n> +LIB_OBJS += date.o\n> +LIB_OBJS += hex-ll.o\n> +LIB_OBJS += parse.o\n> +LIB_OBJS += strbuf.o\n> +LIB_OBJS += usage.o\n> +LIB_OBJS += utf8.o\n> +LIB_OBJS += wrapper.o\n\nIt is still not clear to me how re-using LIB_OBJS like this is \ncompatible with building libgit.a and git-stb-lib.a in a single make \nprocess c.f. [1].\n\n> +ifdef GIT_STD_LIB\n> +\tBASIC_CFLAGS += -DGIT_STD_LIB\n> +\tBASIC_CFLAGS += -DNO_GETTEXT\n\nAs I've said before [2] I think that being able to built git-std-lib.a \nwith gettext support is a prerequisite for using it to build git (just \nlike trace2 support is). If we cannot build git using git-std-lib then \nthe latter is likely to bit rot and so I don't think git-std-lib should \nbe merged until there is a demonstration of building git using it.\n\n\n> +### Libified Git rules\n> +\n> +# git-std-lib\n> +# `make git-std-lib.a GIT_STD_LIB=YesPlease STUB_TRACE2=YesPlease STUB_PAGER=YesPlease`\n> +STD_LIB = git-std-lib.a\n> +\n> +$(STD_LIB): $(LIB_OBJS) $(COMPAT_OBJS) $(STUB_OBJS)\n> +\t$(QUIET_AR)$(RM) $@ && $(AR) $(ARFLAGS) $@ $^\n\nThis is much nicer that the previous version.\n\n> diff --git a/git-compat-util.h b/git-compat-util.h\n> index 3e7a59b5ff..14bf71c530 100644\n> --- a/git-compat-util.h\n> +++ b/git-compat-util.h\n> @@ -455,8 +455,8 @@ static inline int noop_core_config(const char *var UNUSED,\n> #define platform_core_config noop_core_config\n> #endif\n> \n> +#if !defined(__MINGW32__) && !defined(_MSC_VER) && !defined(GIT_STD_LIB)\n> int lstat_cache_aware_rmdir(const char *path);\n> -#if !defined(__MINGW32__) && !defined(_MSC_VER)\n> #define rmdir lstat_cache_aware_rmdir\n> #endif\n\nI thought we'd agreed that this represents a change in behavior that \nshould be fixed c.f. [2]\n\n> @@ -1462,14 +1464,17 @@ static inline int is_missing_file_error(int errno_)\n> \treturn (errno_ == ENOENT || errno_ == ENOTDIR);\n> }\n> \n> +#ifndef GIT_STD_LIB\n> int cmd_main(int, const char **);\n> \n> /*\n> * Intercept all calls to exit() and route them to trace2 to\n> * optionally emit a message before calling the real exit().\n> */\n> +\n\nNit: this blank line seems unnecessary\n\n> int common_exit(const char *file, int line, int code);\n> #define exit(code) exit(common_exit(__FILE__, __LINE__, (code)))\n> +#endif\n> \n> /*\n> * You can mark a stack variable with UNLEAK(var) to avoid it being\n> diff --git a/stubs/pager.c b/stubs/pager.c\n\n> diff --git a/stubs/pager.h b/stubs/pager.h\n> new file mode 100644\n> index 0000000000..b797910881\n> --- /dev/null\n> +++ b/stubs/pager.h\n> @@ -0,0 +1,6 @@\n> +#ifndef PAGER_H\n> +#define PAGER_H\n> +\n> +int pager_in_use(void);\n> +\n> +#endif /* PAGER_H */\n\nIs this file actually used for anything? pager_in_use() is already \ndeclared in pager.h in the project root directory.\n\n> diff --git a/wrapper.c b/wrapper.c\n> index 7da15a56da..eeac3741cf 100644\n> --- a/wrapper.c\n> +++ b/wrapper.c\n> @@ -5,7 +5,6 @@\n> #include \"abspath.h\"\n> #include \"parse.h\"\n> #include \"gettext.h\"\n> -#include \"repository.h\"\n\nIt is probably worth splitting this change out with a commit message \nexplaining why the include is unneeded.\n\nThis is looking good, it would be really nice to see a demonstration of \nbuilding git using git-std-lib (with gettext support) in the next iteration.\n\nBest Wishes\n\nPhillip\n\n\n[1] \nhttps://lore.kernel.org/git/a0f04bd7-3a1e-b303-fd52-eee2af4d38b3@gmail.com/\n[2] \nhttps://lore.kernel.org/git/CAFySSZBMng9nEdCkuT5+fc6rfFgaFfU2E0NP3=jUQC1yRcUE6Q@mail.gmail.com/\n"},{"id":"481883","messageId":"20230915175425.1585455-1-jonathantanmy@google.com","threadId":"59921","inReplyTo":"20230908174443.1027716-2-calvinwan@google.com","subject":"Re: [PATCH v3 2/6] wrapper: remove dependency to Git-specific internal file","fromName":"Jonathan Tan","fromEmail":"jonathantanmy@google.com","sentAt":"2023-09-15T17:54:25Z","receivedAt":"2023-09-15T17:55:20Z","isPatch":true,"sender":{"key":"jonathantanmy@fastmail.com","avatar":null},"body":"Calvin Wan writes:\n> In order for wrapper.c to be built independently as part of a smaller\n> library, it cannot have dependencies to other Git specific\n> internals. remove_or_warn() creates an unnecessary dependency to\n> object.h in wrapper.c. Therefore move the function to entry.[ch] which\n> performs changes on the worktree based on the Git-specific file modes in\n> the index.\n\nLooking at remove_or_warn(), it's only used from entry.c and apply.c\n(which already includes entry.h for another reason) so moving it to\nentry.c looks fine.\n"},{"id":"481890","messageId":"20230915183927.1597414-1-jonathantanmy@google.com","threadId":"59921","inReplyTo":"20230908174443.1027716-5-calvinwan@google.com","subject":"Re: [PATCH v3 5/6] git-std-lib: introduce git standard library","fromName":"Jonathan Tan","fromEmail":"jonathantanmy@google.com","sentAt":"2023-09-15T18:39:27Z","receivedAt":"2023-09-15T18:41:57Z","isPatch":true,"sender":{"key":"jonathantanmy@fastmail.com","avatar":null},"body":"Calvin Wan writes:\n> diff --git a/Makefile b/Makefile\n> index 9226c719a0..0a2d1ae3cc 100644\n> --- a/Makefile\n> +++ b/Makefile\n> @@ -669,6 +669,7 @@ FUZZ_PROGRAMS =\n> GIT_OBJS =\n> LIB_OBJS =\n> SCALAR_OBJS =\n> +STUB_OBJS =\n> OBJECTS =\n> OTHER_PROGRAMS =\n> PROGRAM_OBJS =\n\nI don't think stubs should be compiled into git-std-lib.a - I would\nexpect a consumer of this library to be able to specify their own\nimplementations if needed (e.g. their own trace2).\n\n> @@ -956,6 +957,7 @@ COCCI_SOURCES = $(filter-out $(THIRD_PARTY_SOURCES),$(FOUND_C_SOURCES))\n> \n> LIB_H = $(FOUND_H_SOURCES)\n> \n> +ifndef GIT_STD_LIB\n> LIB_OBJS += abspath.o\n> LIB_OBJS += add-interactive.o\n> LIB_OBJS += add-patch.o\n> @@ -1196,6 +1198,27 @@ LIB_OBJS += write-or-die.o\n> LIB_OBJS += ws.o\n> LIB_OBJS += wt-status.o\n> LIB_OBJS += xdiff-interface.o\n> +else ifdef GIT_STD_LIB\n> +LIB_OBJS += abspath.o\n> +LIB_OBJS += ctype.o\n> +LIB_OBJS += date.o\n> +LIB_OBJS += hex-ll.o\n> +LIB_OBJS += parse.o\n> +LIB_OBJS += strbuf.o\n> +LIB_OBJS += usage.o\n> +LIB_OBJS += utf8.o\n> +LIB_OBJS += wrapper.o\n\nThis means that LIB_OBJS (in this patch, used both by git-std-lib and\nas part of compiling the regular Git binary) can differ based on the\nGIT_STD_LIB variable. It does seem that we cannot avoid GIT_STD_LIB\nfor now, because the git-std-lib can only be compiled without GETTEXT\n(so we need a variable to make sure that none of these .o files are\ncompiled with GETTEXT), but we should still minimize the changes between\ncompiling with GIT_STD_LIB and without it, at least to minimize future\nwork. Could we have two separate lists? So, leave LIB_OBJS alone and\nmake a new STD_LIB_OBJS.\n\n> diff --git a/git-compat-util.h b/git-compat-util.h\n> index 3e7a59b5ff..14bf71c530 100644\n> --- a/git-compat-util.h\n> +++ b/git-compat-util.h\n> @@ -455,8 +455,8 @@ static inline int noop_core_config(const char *var UNUSED,\n> #define platform_core_config noop_core_config\n> #endif\n> \n> +#if !defined(__MINGW32__) && !defined(_MSC_VER) && !defined(GIT_STD_LIB)\n> int lstat_cache_aware_rmdir(const char *path);\n> -#if !defined(__MINGW32__) && !defined(_MSC_VER)\n> #define rmdir lstat_cache_aware_rmdir\n> #endif\n\nI think we still want to keep the idea of \"the code should still be good\neven if we have no use for git-std-lib\" as much as possible, so could we\nstub lstat_cache_aware_rmdir() instead? We could have a new git-compat-\nutil-stub.c (or whatever we want to call it).\n\n> @@ -966,9 +966,11 @@ const char *inet_ntop(int af, const void *src, char *dst, size_t size);\n> #endif\n> \n> #ifdef NO_PTHREADS\n> +#ifdef GIT_STD_LIB\n> #define atexit git_atexit\n> int git_atexit(void (*handler)(void));\n> #endif\n> +#endif\n> \n> static inline size_t st_add(size_t a, size_t b)\n> {\n\nSame for git_atexit().\n\n> @@ -1462,14 +1464,17 @@ static inline int is_missing_file_error(int errno_)\n> \treturn (errno_ == ENOENT || errno_ == ENOTDIR);\n> }\n> \n> +#ifndef GIT_STD_LIB\n> int cmd_main(int, const char **);\n> \n> /*\n> * Intercept all calls to exit() and route them to trace2 to\n> * optionally emit a message before calling the real exit().\n> */\n> +\n> int common_exit(const char *file, int line, int code);\n> #define exit(code) exit(common_exit(__FILE__, __LINE__, (code)))\n> +#endif\n> \n> /*\n> * You can mark a stack variable with UNLEAK(var) to avoid it being\n\nAnd for common_exit().\n\nAs for cmd_main(), that seems to be a convenience so that we can link\ncommon_main.o with various other files (e.g. http-backend.c). I think\nthe right thing to do is to define a new cmd-main.h that declares only\ncmd_main(), and then have only the files that need it (common_main.c and\nall the files that define cmd_main()) include it. This cleanup patch can\nbe done before this patch. I think this is a good change that we would\nwant even without libification.\n \n"},{"id":"481891","messageId":"20230915184321.1598611-1-jonathantanmy@google.com","threadId":"59921","inReplyTo":"20230908174443.1027716-6-calvinwan@google.com","subject":"Re: [PATCH v3 6/6] git-std-lib: add test file to call git-std-lib.a functions","fromName":"Jonathan Tan","fromEmail":"jonathantanmy@google.com","sentAt":"2023-09-15T18:43:21Z","receivedAt":"2023-09-15T18:45:41Z","isPatch":true,"sender":{"key":"jonathantanmy@fastmail.com","avatar":null},"body":"Calvin Wan writes:\n> Add test file that directly or indirectly calls all functions defined in\n> git-std-lib.a object files to showcase that they do not reference\n> missing objects and that git-std-lib.a can stand on its own.\n> \n> Certain functions that cause the program to exit or are already called\n> by other functions are commented out.\n> \n> TODO: replace with unit tests\n> Signed-off-by: Calvin Wan \n\nI think the TODO should go into the code, so that when we add a unit\ntest that also deletes stdlib-test.c, we can see what's happening just\nfrom the diff. The TODO should also explain what stdlib-test.c is hoping\nto do, and why replacing it is OK. (Also, do we need to invoke all the\nfunctions? I thought that missing functions are checked at link time, or\nat the very latest, when the executable is run. No need to change this,\nthough - invoking all the functions we can is fine.)\n \n"},{"id":"481893","messageId":"xmqqbke3dw4l.fsf@gitster.g","threadId":"59921","inReplyTo":"20230915184321.1598611-1-jonathantanmy@google.com","subject":"Re: [PATCH v3 6/6] git-std-lib: add test file to call git-std-lib.a functions","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2023-09-15T20:22:50Z","receivedAt":"2023-09-15T20:23:37Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Jonathan Tan writes:\n\n> Calvin Wan writes:\n>> Add test file that directly or indirectly calls all functions defined in\n>> git-std-lib.a object files to showcase that they do not reference\n>> missing objects and that git-std-lib.a can stand on its own.\n>> \n>> Certain functions that cause the program to exit or are already called\n>> by other functions are commented out.\n>> \n>> TODO: replace with unit tests\n>> Signed-off-by: Calvin Wan \n>\n> I think the TODO should go into the code, so that when we add a unit\n> test that also deletes stdlib-test.c, we can see what's happening just\n> from the diff. The TODO should also explain what stdlib-test.c is hoping\n> to do, and why replacing it is OK. (Also, do we need to invoke all the\n> functions? I thought that missing functions are checked at link time, or\n> at the very latest, when the executable is run. No need to change this,\n> though - invoking all the functions we can is fine.)\n> \n\nThanks for excellent reviews (not just against this 6/6 but others,\ntoo).\n\n"},{"id":"482339","messageId":"bd88d3ab-b7fd-4a77-851a-d012fec08523@gmail.com","threadId":"59921","inReplyTo":"20230908174443.1027716-5-calvinwan@google.com","subject":"Re: [PATCH v3 5/6] git-std-lib: introduce git standard library","fromName":"","fromEmail":"phillip.wood123@gmail.com","sentAt":"2023-09-26T14:23:33Z","receivedAt":"2023-09-26T14:23:47Z","isPatch":true,"sender":{"key":"phillip.wood@dunelm.org.uk","avatar":null},"body":"Hi Calvin\n\nOn 08/09/2023 18:44, Calvin Wan wrote:\n> The Git Standard Library intends to serve as the foundational library\n> and root dependency that other libraries in Git will be built off of.\n> That is to say, suppose we have libraries X and Y; a user that wants to\n> use X and Y would need to include X, Y, and this Git Standard Library.\n> \n> Add Documentation/technical/git-std-lib.txt to further explain the\n> design and rationale.\n> \n> Signed-off-by: Calvin Wan \n> Helped-by: Phillip Wood \n> ---\n> Documentation/technical/git-std-lib.txt | 191 ++++++++++++++++++++++++\n\nI need the following diff to build the html documentation.\n\nBest Wishes\n\nPhillip\n\ndiff --git a/Documentation/Makefile b/Documentation/Makefile\nindex 3f2383a12c..f1dc673838 100644\n--- a/Documentation/Makefile\n+++ b/Documentation/Makefile\n@@ -110,6 +110,7 @@ TECH_DOCS += SubmittingPatches\n TECH_DOCS += ToolsForGit\n TECH_DOCS += technical/bitmap-format\n TECH_DOCS += technical/bundle-uri\n+TECH_DOCS += technical/git-std-lib\n TECH_DOCS += technical/hash-function-transition\n TECH_DOCS += technical/long-running-process-protocol\n TECH_DOCS += technical/multi-pack-index\ndiff --git a/Documentation/technical/git-std-lib.txt b/Documentation/technical/git-std-lib.txt\nindex d03b8565b4..28e6cdac2a 100644\n--- a/Documentation/technical/git-std-lib.txt\n+++ b/Documentation/technical/git-std-lib.txt\n@@ -1,5 +1,4 @@\n-Git Standard Library\n-================\n+= Git Standard Library\n \n The Git Standard Library intends to serve as the foundational library\n and root dependency that other libraries in Git will be built off of.\n@@ -9,8 +8,7 @@ This does not mean that the Git Standard Library will be the only\n possible root dependency in the future, but rather the most significant\n and widely used one.\n \n-Dependency graph in libified Git\n-================\n+== Dependency graph in libified Git\n \n If you look in the Git Makefile, all of the objects defined in the Git\n library are compiled and archived into a singular file, libgit.a, which\n@@ -57,8 +55,7 @@ if someone wanted their own custom build of Git with their own custom\n implementation of the object store, they would only have to swap out\n object-store.a rather than do a hard fork of Git.\n \n-Rationale behind Git Standard Library\n-================\n+== Rationale behind Git Standard Library\n \n The rationale behind what's in and what's not in the Git Standard\n Library essentially is the result of two observations within the Git\n@@ -67,8 +64,7 @@ in a couple of different files, and wrapper.c + usage.c have\n difficult-to-separate circular dependencies with each other and other\n files.\n \n-Ubiquity of git-compat-util.h and circular dependencies\n-========\n+=== Ubiquity of git-compat-util.h and circular dependencies\n \n Every file in the Git codebase includes git-compat-util.h. It serves as\n \"a compatibility aid that isolates the knowledge of platform specific\n@@ -79,10 +75,9 @@ for wrapper.c and usage.c to be a part of the root library. They have\n difficult to separate circular dependencies with each other so they\n can't be independent libraries. Wrapper.c has dependencies on parse.c,\n abspath.c, strbuf.c, which in turn also have dependencies on usage.c and\n-wrapper.c -- more circular dependencies.\n+wrapper.c - more circular dependencies.\n \n-Tradeoff between swappability and refactoring\n-========\n+=== Tradeoff between swappability and refactoring\n \n From the above dependency graph, we can see that git-std-lib.a could be\n many smaller libraries rather than a singular library. So why choose a\n@@ -99,8 +94,7 @@ and change the API for the library if there becomes enough of a reason\n to do so (remember we are avoiding promising stability of the interfaces\n of those libraries).\n \n-Reuse of compatibility functions in git-compat-util.h\n-========\n+=== Reuse of compatibility functions in git-compat-util.h\n \n Most functions defined in git-compat-util.h are implemented in compat/\n and have dependencies limited to strbuf.h and wrapper.h so they can be\n@@ -110,8 +104,7 @@ compat/. The rest of the functions defined in git-compat-util.h are\n implemented in top level files and are hidden behind\n an #ifdef if their implementation is not in git-std-lib.a.\n \n-Rationale summary\n-========\n+=== Rationale summary\n \n The Git Standard Library allows us to get the libification ball rolling\n with other libraries in Git. By not spending many\n@@ -123,8 +116,7 @@ the code cleanups that have happened so far have been minor and\n beneficial for the codebase. It is probable that making large movements\n would negatively affect code clarity.\n \n-Git Standard Library boundary\n-================\n+== Git Standard Library boundary\n \n While I have described above some useful heuristics for identifying\n potential candidates for git-std-lib.a, a standard library should not\n@@ -150,8 +142,7 @@ to be able to trace functions in those files and other files in git-std-lib.a.\n In order for git-std-lib.a to compile with those dependencies, stubbed out\n versions of those files are implemented and swapped in during compilation time.\n \n-Files inside of Git Standard Library\n-================\n+== Files inside of Git Standard Library\n \n The initial set of files in git-std-lib.a are:\n abspath.c\n@@ -171,21 +162,19 @@ complete library:\n stubs/pager.c\n stubs/trace2.c\n \n-Pitfalls\n-================\n+== Pitfalls\n \n There are a small amount of files under compat/* that have dependencies\n not inside of git-std-lib.a. While those functions are not called on\n Linux, other OSes might call those problematic functions. I don't see\n this as a major problem, just moreso an observation that libification in\n general may also require some minor compatibility work in the future.\n \n-Testing\n-================\n+== Testing\n \n Unit tests should catch any breakages caused by changes to files in\n git-std-lib.a (i.e. introduction of a out of scope dependency) and new\n functions introduced to git-std-lib.a will require unit tests written\n for them.\n \n-[1] https://lore.kernel.org/git/xmqqwn17sydw.fsf@gitster.g/\n\\ No newline at end of file\n+[1] https://lore.kernel.org/git/xmqqwn17sydw.fsf@gitster.g/\n"},{"id":"482375","messageId":"98f3edcf-7f37-45ff-abd2-c0038d4e0589@gmail.com","threadId":"59921","inReplyTo":"4cefe4f8-04ee-48fb-aee4-07342b7a062f@gmail.com","subject":"Re: [PATCH v3 5/6] git-std-lib: introduce git standard library","fromName":"Phillip Wood","fromEmail":"phillip.wood123@gmail.com","sentAt":"2023-09-27T14:14:49Z","receivedAt":"2023-09-27T14:14:55Z","isPatch":true,"sender":{"key":"phillip.wood@dunelm.org.uk","avatar":null},"body":"Hi Calvin\n\nOn 11/09/2023 14:22, Phillip Wood wrote:\n> This is looking good, it would be really nice to see a demonstration of \n> building git using git-std-lib (with gettext support) in the next \n> iteration.\n\nI've just pushed a couple of patches on top of this series to \nhttps://github.com/phillipwood/git/tree/cw/git-std-lib/v3 that show a \npossible way to do that.\n\nRather than re-using LIB_OBJS when building git-std-lib.a it uses a \nseparate variable to hold the list of sources. git is linked against \ngit-std-lib.a so we don't have to worry about having the same source \nfile in two different libraries. The stub objects are moved into their \nown library. This enables git to be built using git-std-lib.a with\n\n make git\n\nwhich will link git against git-std-lib.a built with gettext and tracing \nsupport. To use git-std-lib.a in other programs compile it and \ngit-stub-lib.a with\n\n make NO_GETTEXT=YesPlease git-std-lib.a git-stub-lib.a\n\nand link your program against both libraries.\n\nThe GIT_STD_LIB define is also removed in favor of more stubbing to \navoid the complications of conditional compilation.\n\nBest Wishes\n\nPhillip\n"},{"id":"482457","messageId":"cover.1696021277.git.jonathantanmy@google.com","threadId":"59921","inReplyTo":"20230627195251.1973421-1-calvinwan@google.com","subject":"[PATCH v4 0/4] Preliminary patches before git-std-lib","fromName":"Jonathan Tan","fromEmail":"jonathantanmy@google.com","sentAt":"2023-09-29T21:20:47Z","receivedAt":"2023-09-29T21:21:02Z","isPatch":true,"sender":{"key":"jonathantanmy@fastmail.com","avatar":null},"body":"Calvin will be away for a few weeks and I'll be handling the git-std-lib\neffort in the meantime. My goals will be:\n\n- Get the preliminary patches in Calvin's patch set (patches 1-4) merged\nfirst.\n\n- Updating patches 5-6 based on reviewer feedback (including my\nfeedback). I have several aims including reducing or eliminating the\nneed for the GIT_STD_LIB preprocessor variable, and making stubs a test-\nonly concern (I think Phillip has some similar ideas [1] but I haven't\nlooked at their repo on GitHub yet).\n\n[1] https://lore.kernel.org/git/98f3edcf-7f37-45ff-abd2-c0038d4e0589@gmail.com/\n\nThis patch set is in service of the first goal. Because the libification\npatches are no longer included in this patch set, I have rewritten the\ncommit messages to justify the patches in terms of code organization.\nThere are no changes in the code itself. Also, I have retained Calvin's\nname as the author.\n\nPutting on my reviewer hat, if I was reviewing hex.h and config.h from\nscratch, I would have not thought twice about requesting the changes\nin these patches. But since we are not creating them from scratch but\nmodifying existing files, a question does arise about whether it's\nworth the additional noise that someone looking through history needs\nto handle. In this case, I think it's worth it - I think the future code\ndelver would appreciate being able to see the evolution of hex, hash\nalgo, config, and parse functions as their own files rather than, when\nlooking at one of them, having to filter out unrelated changes.\n\nBesides that, as Calvin has described in his other emails, these patches\nare prerequisites to being able to independently compile and use a\ncertain subset of the .c files. With patches that solely refactor, there\nis sometimes a worry that the benefits are nebulous and that we would\nbe moving code around for nothing, but I don't think that that applies\nhere: there is still more work to be done on patches 5 and 6, but what\nwe have in patches 5 and 6 now shows that the benefits are concrete and\nwithin reach.\n\nCalvin Wan (4):\n hex-ll: separate out non-hash-algo functions\n wrapper: reduce scope of remove_or_warn()\n config: correct bad boolean env value error message\n parse: separate out parsing functions from config.h\n\n Makefile | 2 +\n attr.c | 2 +-\n color.c | 2 +-\n config.c | 173 +----------------------------------\n config.h | 14 +--\n entry.c | 5 +\n entry.h | 6 ++\n hex-ll.c | 49 ++++++++++\n hex-ll.h | 27 ++++++\n hex.c | 47 ----------\n hex.h | 24 +----\n mailinfo.c | 2 +-\n pack-objects.c | 2 +-\n pack-revindex.c | 2 +-\n parse-options.c | 3 +-\n parse.c | 182 +++++++++++++++++++++++++++++++++++++\n parse.h | 20 ++++\n pathspec.c | 2 +-\n preload-index.c | 2 +-\n progress.c | 2 +-\n prompt.c | 2 +-\n rebase.c | 2 +-\n strbuf.c | 2 +-\n t/helper/test-env-helper.c | 2 +-\n unpack-trees.c | 2 +-\n url.c | 2 +-\n urlmatch.c | 2 +-\n wrapper.c | 8 +-\n wrapper.h | 5 -\n write-or-die.c | 2 +-\n 30 files changed, 313 insertions(+), 284 deletions(-)\n create mode 100644 hex-ll.c\n create mode 100644 hex-ll.h\n create mode 100644 parse.c\n create mode 100644 parse.h\n\nRange-diff against v3:\n1: fcce01bc19 ! 1: 02ecc00e9c hex-ll: split out functionality from hex\n @@ Metadata\n Author: Calvin Wan \n \n ## Commit message ##\n - hex-ll: split out functionality from hex\n + hex-ll: separate out non-hash-algo functions\n \n - Separate out hex functionality that doesn't require a hash algo into\n - hex-ll.[ch]. Since the hash algo is currently a global that sits in\n - repository, this separation removes that dependency for files that only\n - need basic hex manipulation functions.\n + In order to further reduce all-in-one headers, separate out functions in\n + hex.h that do not operate on object hashes into its own file, hex-ll.h,\n + and update the include directives in the .c files that need only such\n + functions accordingly.\n \n Signed-off-by: Calvin Wan \n - Signed-off-by: Junio C Hamano \n + Signed-off-by: Jonathan Tan \n \n ## Makefile ##\n @@ Makefile: LIB_OBJS += hash-lookup.o\n2: 95a369d02b ! 2: c9e7cd7857 wrapper: remove dependency to Git-specific internal file\n @@ Metadata\n Author: Calvin Wan \n \n ## Commit message ##\n - wrapper: remove dependency to Git-specific internal file\n + wrapper: reduce scope of remove_or_warn()\n \n - In order for wrapper.c to be built independently as part of a smaller\n - library, it cannot have dependencies to other Git specific\n - internals. remove_or_warn() creates an unnecessary dependency to\n - object.h in wrapper.c. Therefore move the function to entry.[ch] which\n - performs changes on the worktree based on the Git-specific file modes in\n - the index.\n + remove_or_warn() is only used by entry.c and apply.c, but it is\n + currently declared and defined in wrapper.{h,c}, so it has a scope much\n + greater than it needs. This needlessly large scope also causes wrapper.c\n + to need to include object.h, when this file is largely unconcerned with\n + Git objects.\n +\n + Move remove_or_warn() to entry.{h,c}. The file apply.c still has access\n + to it, since it already includes entry.h for another reason.\n \n Signed-off-by: Calvin Wan \n - Signed-off-by: Junio C Hamano \n + Signed-off-by: Jonathan Tan \n \n ## entry.c ##\n @@ entry.c: void unlink_entry(const struct cache_entry *ce, const char *super_prefix)\n3: 5348528865 = 3: e4c20a81f9 config: correct bad boolean env value error message\n4: b5a8945c5c ! 4: 5d9f0b3de0 parse: create new library for parsing strings and env values\n @@ Metadata\n Author: Calvin Wan \n \n ## Commit message ##\n - parse: create new library for parsing strings and env values\n + parse: separate out parsing functions from config.h\n \n - While string and environment value parsing is mainly consumed by\n - config.c, there are other files that only need parsing functionality and\n - not config functionality. By separating out string and environment value\n - parsing from config, those files can instead be dependent on parse,\n - which has a much smaller dependency chain than config. This ultimately\n - allows us to inclue parse.[ch] in an independent library since it\n - doesn't have dependencies to Git-specific internals unlike in\n - config.[ch].\n + The files config.{h,c} contain functions that have to do with parsing,\n + but not config.\n \n - Move general string and env parsing functions from config.[ch] to\n - parse.[ch].\n + In order to further reduce all-in-one headers, separate out functions in\n + config.c that do not operate on config into its own file, parse.h,\n + and update the include directives in the .c files that need only such\n + functions accordingly.\n \n Signed-off-by: Calvin Wan \n - Signed-off-by: Junio C Hamano \n + Signed-off-by: Jonathan Tan \n \n ## Makefile ##\n @@ Makefile: LIB_OBJS += pack-write.o\n-- \n2.42.0.582.g8ccd20d70d-goog\n\n"},{"id":"482458","messageId":"02ecc00e9c7226c9eeb960cc49c8c03dcb182a38.1696021277.git.jonathantanmy@google.com","threadId":"59921","inReplyTo":"cover.1696021277.git.jonathantanmy@google.com","subject":"[PATCH v4 1/4] hex-ll: separate out non-hash-algo functions","fromName":"Jonathan Tan","fromEmail":"jonathantanmy@google.com","sentAt":"2023-09-29T21:20:48Z","receivedAt":"2023-09-29T21:21:09Z","isPatch":true,"sender":{"key":"jonathantanmy@fastmail.com","avatar":null},"body":"From: Calvin Wan \n\nIn order to further reduce all-in-one headers, separate out functions in\nhex.h that do not operate on object hashes into its own file, hex-ll.h,\nand update the include directives in the .c files that need only such\nfunctions accordingly.\n\nSigned-off-by: Calvin Wan \nSigned-off-by: Jonathan Tan \n---\n Makefile | 1 +\n color.c | 2 +-\n hex-ll.c | 49 +++++++++++++++++++++++++++++++++++++++++++++++++\n hex-ll.h | 27 +++++++++++++++++++++++++++\n hex.c | 47 -----------------------------------------------\n hex.h | 24 +-----------------------\n mailinfo.c | 2 +-\n strbuf.c | 2 +-\n url.c | 2 +-\n urlmatch.c | 2 +-\n 10 files changed, 83 insertions(+), 75 deletions(-)\n create mode 100644 hex-ll.c\n create mode 100644 hex-ll.h\n\ndiff --git a/Makefile b/Makefile\nindex 5776309365..861e643708 100644\n--- a/Makefile\n+++ b/Makefile\n@@ -1040,6 +1040,7 @@ LIB_OBJS += hash-lookup.o\n LIB_OBJS += hashmap.o\n LIB_OBJS += help.o\n LIB_OBJS += hex.o\n+LIB_OBJS += hex-ll.o\n LIB_OBJS += hook.o\n LIB_OBJS += ident.o\n LIB_OBJS += json-writer.o\ndiff --git a/color.c b/color.c\nindex b24b19566b..f663c06ac4 100644\n--- a/color.c\n+++ b/color.c\n@@ -3,7 +3,7 @@\n #include \"color.h\"\n #include \"editor.h\"\n #include \"gettext.h\"\n-#include \"hex.h\"\n+#include \"hex-ll.h\"\n #include \"pager.h\"\n #include \"strbuf.h\"\n \ndiff --git a/hex-ll.c b/hex-ll.c\nnew file mode 100644\nindex 0000000000..4d7ece1de5\n--- /dev/null\n+++ b/hex-ll.c\n@@ -0,0 +1,49 @@\n+#include \"git-compat-util.h\"\n+#include \"hex-ll.h\"\n+\n+const signed char hexval_table[256] = {\n+\t -1, -1, -1, -1, -1, -1, -1, -1,\t\t/* 00-07 */\n+\t -1, -1, -1, -1, -1, -1, -1, -1,\t\t/* 08-0f */\n+\t -1, -1, -1, -1, -1, -1, -1, -1,\t\t/* 10-17 */\n+\t -1, -1, -1, -1, -1, -1, -1, -1,\t\t/* 18-1f */\n+\t -1, -1, -1, -1, -1, -1, -1, -1,\t\t/* 20-27 */\n+\t -1, -1, -1, -1, -1, -1, -1, -1,\t\t/* 28-2f */\n+\t 0, 1, 2, 3, 4, 5, 6, 7,\t\t/* 30-37 */\n+\t 8, 9, -1, -1, -1, -1, -1, -1,\t\t/* 38-3f */\n+\t -1, 10, 11, 12, 13, 14, 15, -1,\t\t/* 40-47 */\n+\t -1, -1, -1, -1, -1, -1, -1, -1,\t\t/* 48-4f */\n+\t -1, -1, -1, -1, -1, -1, -1, -1,\t\t/* 50-57 */\n+\t -1, -1, -1, -1, -1, -1, -1, -1,\t\t/* 58-5f */\n+\t -1, 10, 11, 12, 13, 14, 15, -1,\t\t/* 60-67 */\n+\t -1, -1, -1, -1, -1, -1, -1, -1,\t\t/* 68-67 */\n+\t -1, -1, -1, -1, -1, -1, -1, -1,\t\t/* 70-77 */\n+\t -1, -1, -1, -1, -1, -1, -1, -1,\t\t/* 78-7f */\n+\t -1, -1, -1, -1, -1, -1, -1, -1,\t\t/* 80-87 */\n+\t -1, -1, -1, -1, -1, -1, -1, -1,\t\t/* 88-8f */\n+\t -1, -1, -1, -1, -1, -1, -1, -1,\t\t/* 90-97 */\n+\t -1, -1, -1, -1, -1, -1, -1, -1,\t\t/* 98-9f */\n+\t -1, -1, -1, -1, -1, -1, -1, -1,\t\t/* a0-a7 */\n+\t -1, -1, -1, -1, -1, -1, -1, -1,\t\t/* a8-af */\n+\t -1, -1, -1, -1, -1, -1, -1, -1,\t\t/* b0-b7 */\n+\t -1, -1, -1, -1, -1, -1, -1, -1,\t\t/* b8-bf */\n+\t -1, -1, -1, -1, -1, -1, -1, -1,\t\t/* c0-c7 */\n+\t -1, -1, -1, -1, -1, -1, -1, -1,\t\t/* c8-cf */\n+\t -1, -1, -1, -1, -1, -1, -1, -1,\t\t/* d0-d7 */\n+\t -1, -1, -1, -1, -1, -1, -1, -1,\t\t/* d8-df */\n+\t -1, -1, -1, -1, -1, -1, -1, -1,\t\t/* e0-e7 */\n+\t -1, -1, -1, -1, -1, -1, -1, -1,\t\t/* e8-ef */\n+\t -1, -1, -1, -1, -1, -1, -1, -1,\t\t/* f0-f7 */\n+\t -1, -1, -1, -1, -1, -1, -1, -1,\t\t/* f8-ff */\n+};\n+\n+int hex_to_bytes(unsigned char *binary, const char *hex, size_t len)\n+{\n+\tfor (; len; len--, hex += 2) {\n+\t\tunsigned int val = (hexval(hex[0]) << 4) | hexval(hex[1]);\n+\n+\t\tif (val & ~0xff)\n+\t\t\treturn -1;\n+\t\t*binary++ = val;\n+\t}\n+\treturn 0;\n+}\ndiff --git a/hex-ll.h b/hex-ll.h\nnew file mode 100644\nindex 0000000000..a381fa8556\n--- /dev/null\n+++ b/hex-ll.h\n@@ -0,0 +1,27 @@\n+#ifndef HEX_LL_H\n+#define HEX_LL_H\n+\n+extern const signed char hexval_table[256];\n+static inline unsigned int hexval(unsigned char c)\n+{\n+\treturn hexval_table[c];\n+}\n+\n+/*\n+ * Convert two consecutive hexadecimal digits into a char. Return a\n+ * negative value on error. Don't run over the end of short strings.\n+ */\n+static inline int hex2chr(const char *s)\n+{\n+\tunsigned int val = hexval(s[0]);\n+\treturn (val & ~0xf) ? val : (val << 4) | hexval(s[1]);\n+}\n+\n+/*\n+ * Read `len` pairs of hexadecimal digits from `hex` and write the\n+ * values to `binary` as `len` bytes. Return 0 on success, or -1 if\n+ * the input does not consist of hex digits).\n+ */\n+int hex_to_bytes(unsigned char *binary, const char *hex, size_t len);\n+\n+#endif\ndiff --git a/hex.c b/hex.c\nindex 01f17fe5c9..d42262bdca 100644\n--- a/hex.c\n+++ b/hex.c\n@@ -2,53 +2,6 @@\n #include \"hash.h\"\n #include \"hex.h\"\n \n-const signed char hexval_table[256] = {\n-\t -1, -1, -1, -1, -1, -1, -1, -1,\t\t/* 00-07 */\n-\t -1, -1, -1, -1, -1, -1, -1, -1,\t\t/* 08-0f */\n-\t -1, -1, -1, -1, -1, -1, -1, -1,\t\t/* 10-17 */\n-\t -1, -1, -1, -1, -1, -1, -1, -1,\t\t/* 18-1f */\n-\t -1, -1, -1, -1, -1, -1, -1, -1,\t\t/* 20-27 */\n-\t -1, -1, -1, -1, -1, -1, -1, -1,\t\t/* 28-2f */\n-\t 0, 1, 2, 3, 4, 5, 6, 7,\t\t/* 30-37 */\n-\t 8, 9, -1, -1, -1, -1, -1, -1,\t\t/* 38-3f */\n-\t -1, 10, 11, 12, 13, 14, 15, -1,\t\t/* 40-47 */\n-\t -1, -1, -1, -1, -1, -1, -1, -1,\t\t/* 48-4f */\n-\t -1, -1, -1, -1, -1, -1, -1, -1,\t\t/* 50-57 */\n-\t -1, -1, -1, -1, -1, -1, -1, -1,\t\t/* 58-5f */\n-\t -1, 10, 11, 12, 13, 14, 15, -1,\t\t/* 60-67 */\n-\t -1, -1, -1, -1, -1, -1, -1, -1,\t\t/* 68-67 */\n-\t -1, -1, -1, -1, -1, -1, -1, -1,\t\t/* 70-77 */\n-\t -1, -1, -1, -1, -1, -1, -1, -1,\t\t/* 78-7f */\n-\t -1, -1, -1, -1, -1, -1, -1, -1,\t\t/* 80-87 */\n-\t -1, -1, -1, -1, -1, -1, -1, -1,\t\t/* 88-8f */\n-\t -1, -1, -1, -1, -1, -1, -1, -1,\t\t/* 90-97 */\n-\t -1, -1, -1, -1, -1, -1, -1, -1,\t\t/* 98-9f */\n-\t -1, -1, -1, -1, -1, -1, -1, -1,\t\t/* a0-a7 */\n-\t -1, -1, -1, -1, -1, -1, -1, -1,\t\t/* a8-af */\n-\t -1, -1, -1, -1, -1, -1, -1, -1,\t\t/* b0-b7 */\n-\t -1, -1, -1, -1, -1, -1, -1, -1,\t\t/* b8-bf */\n-\t -1, -1, -1, -1, -1, -1, -1, -1,\t\t/* c0-c7 */\n-\t -1, -1, -1, -1, -1, -1, -1, -1,\t\t/* c8-cf */\n-\t -1, -1, -1, -1, -1, -1, -1, -1,\t\t/* d0-d7 */\n-\t -1, -1, -1, -1, -1, -1, -1, -1,\t\t/* d8-df */\n-\t -1, -1, -1, -1, -1, -1, -1, -1,\t\t/* e0-e7 */\n-\t -1, -1, -1, -1, -1, -1, -1, -1,\t\t/* e8-ef */\n-\t -1, -1, -1, -1, -1, -1, -1, -1,\t\t/* f0-f7 */\n-\t -1, -1, -1, -1, -1, -1, -1, -1,\t\t/* f8-ff */\n-};\n-\n-int hex_to_bytes(unsigned char *binary, const char *hex, size_t len)\n-{\n-\tfor (; len; len--, hex += 2) {\n-\t\tunsigned int val = (hexval(hex[0]) << 4) | hexval(hex[1]);\n-\n-\t\tif (val & ~0xff)\n-\t\t\treturn -1;\n-\t\t*binary++ = val;\n-\t}\n-\treturn 0;\n-}\n-\n static int get_hash_hex_algop(const char *hex, unsigned char *hash,\n \t\t\t const struct git_hash_algo *algop)\n {\ndiff --git a/hex.h b/hex.h\nindex 87abf66602..e0b83f776f 100644\n--- a/hex.h\n+++ b/hex.h\n@@ -2,22 +2,7 @@\n #define HEX_H\n \n #include \"hash-ll.h\"\n-\n-extern const signed char hexval_table[256];\n-static inline unsigned int hexval(unsigned char c)\n-{\n-\treturn hexval_table[c];\n-}\n-\n-/*\n- * Convert two consecutive hexadecimal digits into a char. Return a\n- * negative value on error. Don't run over the end of short strings.\n- */\n-static inline int hex2chr(const char *s)\n-{\n-\tunsigned int val = hexval(s[0]);\n-\treturn (val & ~0xf) ? val : (val << 4) | hexval(s[1]);\n-}\n+#include \"hex-ll.h\"\n \n /*\n * Try to read a hash (specified by the_hash_algo) in hexadecimal\n@@ -34,13 +19,6 @@ int get_oid_hex(const char *hex, struct object_id *oid);\n /* Like get_oid_hex, but for an arbitrary hash algorithm. */\n int get_oid_hex_algop(const char *hex, struct object_id *oid, const struct git_hash_algo *algop);\n \n-/*\n- * Read `len` pairs of hexadecimal digits from `hex` and write the\n- * values to `binary` as `len` bytes. Return 0 on success, or -1 if\n- * the input does not consist of hex digits).\n- */\n-int hex_to_bytes(unsigned char *binary, const char *hex, size_t len);\n-\n /*\n * Convert a binary hash in \"unsigned char []\" or an object name in\n * \"struct object_id *\" to its hex equivalent. The `_r` variant is reentrant,\ndiff --git a/mailinfo.c b/mailinfo.c\nindex 931505363c..a07d2da16d 100644\n--- a/mailinfo.c\n+++ b/mailinfo.c\n@@ -1,7 +1,7 @@\n #include \"git-compat-util.h\"\n #include \"config.h\"\n #include \"gettext.h\"\n-#include \"hex.h\"\n+#include \"hex-ll.h\"\n #include \"utf8.h\"\n #include \"strbuf.h\"\n #include \"mailinfo.h\"\ndiff --git a/strbuf.c b/strbuf.c\nindex 4c9ac6dc5e..7827178d8e 100644\n--- a/strbuf.c\n+++ b/strbuf.c\n@@ -1,6 +1,6 @@\n #include \"git-compat-util.h\"\n #include \"gettext.h\"\n-#include \"hex.h\"\n+#include \"hex-ll.h\"\n #include \"strbuf.h\"\n #include \"string-list.h\"\n #include \"utf8.h\"\ndiff --git a/url.c b/url.c\nindex 2e1a9f6fee..282b12495a 100644\n--- a/url.c\n+++ b/url.c\n@@ -1,5 +1,5 @@\n #include \"git-compat-util.h\"\n-#include \"hex.h\"\n+#include \"hex-ll.h\"\n #include \"strbuf.h\"\n #include \"url.h\"\n \ndiff --git a/urlmatch.c b/urlmatch.c\nindex 1c45f23adf..1d0254abac 100644\n--- a/urlmatch.c\n+++ b/urlmatch.c\n@@ -1,6 +1,6 @@\n #include \"git-compat-util.h\"\n #include \"gettext.h\"\n-#include \"hex.h\"\n+#include \"hex-ll.h\"\n #include \"strbuf.h\"\n #include \"urlmatch.h\"\n \n-- \n2.42.0.582.g8ccd20d70d-goog\n\n"},{"id":"482459","messageId":"c9e7cd78576527571fd70b953e340b5bdd196221.1696021277.git.jonathantanmy@google.com","threadId":"59921","inReplyTo":"cover.1696021277.git.jonathantanmy@google.com","subject":"[PATCH v4 2/4] wrapper: reduce scope of remove_or_warn()","fromName":"Jonathan Tan","fromEmail":"jonathantanmy@google.com","sentAt":"2023-09-29T21:20:49Z","receivedAt":"2023-09-29T21:21:10Z","isPatch":true,"sender":{"key":"jonathantanmy@fastmail.com","avatar":null},"body":"From: Calvin Wan \n\nremove_or_warn() is only used by entry.c and apply.c, but it is\ncurrently declared and defined in wrapper.{h,c}, so it has a scope much\ngreater than it needs. This needlessly large scope also causes wrapper.c\nto need to include object.h, when this file is largely unconcerned with\nGit objects.\n\nMove remove_or_warn() to entry.{h,c}. The file apply.c still has access\nto it, since it already includes entry.h for another reason.\n\nSigned-off-by: Calvin Wan \nSigned-off-by: Jonathan Tan \n---\n entry.c | 5 +++++\n entry.h | 6 ++++++\n wrapper.c | 6 ------\n wrapper.h | 5 -----\n 4 files changed, 11 insertions(+), 11 deletions(-)\n\ndiff --git a/entry.c b/entry.c\nindex 43767f9043..076e97eb89 100644\n--- a/entry.c\n+++ b/entry.c\n@@ -581,3 +581,8 @@ void unlink_entry(const struct cache_entry *ce, const char *super_prefix)\n \t\treturn;\n \tschedule_dir_for_removal(ce->name, ce_namelen(ce));\n }\n+\n+int remove_or_warn(unsigned int mode, const char *file)\n+{\n+\treturn S_ISGITLINK(mode) ? rmdir_or_warn(file) : unlink_or_warn(file);\n+}\ndiff --git a/entry.h b/entry.h\nindex 7329f918a9..ca3ed35bc0 100644\n--- a/entry.h\n+++ b/entry.h\n@@ -62,4 +62,10 @@ int fstat_checkout_output(int fd, const struct checkout *state, struct stat *st)\n void update_ce_after_write(const struct checkout *state, struct cache_entry *ce,\n \t\t\t struct stat *st);\n \n+/*\n+ * Calls the correct function out of {unlink,rmdir}_or_warn based on\n+ * the supplied file mode.\n+ */\n+int remove_or_warn(unsigned int mode, const char *path);\n+\n #endif /* ENTRY_H */\ndiff --git a/wrapper.c b/wrapper.c\nindex 48065c4f53..453a20ed99 100644\n--- a/wrapper.c\n+++ b/wrapper.c\n@@ -5,7 +5,6 @@\n #include \"abspath.h\"\n #include \"config.h\"\n #include \"gettext.h\"\n-#include \"object.h\"\n #include \"repository.h\"\n #include \"strbuf.h\"\n #include \"trace2.h\"\n@@ -632,11 +631,6 @@ int rmdir_or_warn(const char *file)\n \treturn warn_if_unremovable(\"rmdir\", file, rmdir(file));\n }\n \n-int remove_or_warn(unsigned int mode, const char *file)\n-{\n-\treturn S_ISGITLINK(mode) ? rmdir_or_warn(file) : unlink_or_warn(file);\n-}\n-\n static int access_error_is_ok(int err, unsigned flag)\n {\n \treturn (is_missing_file_error(err) ||\ndiff --git a/wrapper.h b/wrapper.h\nindex 79c7321bb3..1b2b047ea0 100644\n--- a/wrapper.h\n+++ b/wrapper.h\n@@ -106,11 +106,6 @@ int unlink_or_msg(const char *file, struct strbuf *err);\n * not exist.\n */\n int rmdir_or_warn(const char *path);\n-/*\n- * Calls the correct function out of {unlink,rmdir}_or_warn based on\n- * the supplied file mode.\n- */\n-int remove_or_warn(unsigned int mode, const char *path);\n \n /*\n * Call access(2), but warn for any error except \"missing file\"\n-- \n2.42.0.582.g8ccd20d70d-goog\n\n"},{"id":"482460","messageId":"e4c20a81f93e2a4e2ad65920920ead690b162e25.1696021277.git.jonathantanmy@google.com","threadId":"59921","inReplyTo":"cover.1696021277.git.jonathantanmy@google.com","subject":"[PATCH v4 3/4] config: correct bad boolean env value error message","fromName":"Jonathan Tan","fromEmail":"jonathantanmy@google.com","sentAt":"2023-09-29T21:20:50Z","receivedAt":"2023-09-29T21:21:12Z","isPatch":true,"sender":{"key":"jonathantanmy@fastmail.com","avatar":null},"body":"From: Calvin Wan \n\nAn incorrectly defined boolean environment value would result in the\nfollowing error message:\n\nbad boolean config value '%s' for '%s'\n\nThis is a misnomer since environment value != config value. Instead of\ncalling git_config_bool() to parse the environment value, mimic the\nfunctionality inside of git_config_bool() but with the correct error\nmessage.\n\nSigned-off-by: Calvin Wan \nSigned-off-by: Junio C Hamano \n---\n config.c | 9 ++++++++-\n 1 file changed, 8 insertions(+), 1 deletion(-)\n\ndiff --git a/config.c b/config.c\nindex 3846a37be9..7dde0aaa02 100644\n--- a/config.c\n+++ b/config.c\n@@ -2133,7 +2133,14 @@ void git_global_config(char **user_out, char **xdg_out)\n int git_env_bool(const char *k, int def)\n {\n \tconst char *v = getenv(k);\n-\treturn v ? git_config_bool(k, v) : def;\n+\tint val;\n+\tif (!v)\n+\t\treturn def;\n+\tval = git_parse_maybe_bool(v);\n+\tif (val < 0)\n+\t\tdie(_(\"bad boolean environment value '%s' for '%s'\"),\n+\t\t v, k);\n+\treturn val;\n }\n \n /*\n-- \n2.42.0.582.g8ccd20d70d-goog\n\n"},{"id":"482461","messageId":"5d9f0b3de08ab8541482b9b640db06b6d3000b86.1696021277.git.jonathantanmy@google.com","threadId":"59921","inReplyTo":"cover.1696021277.git.jonathantanmy@google.com","subject":"[PATCH v4 4/4] parse: separate out parsing functions from config.h","fromName":"Jonathan Tan","fromEmail":"jonathantanmy@google.com","sentAt":"2023-09-29T21:20:51Z","receivedAt":"2023-09-29T21:21:14Z","isPatch":true,"sender":{"key":"jonathantanmy@fastmail.com","avatar":null},"body":"From: Calvin Wan \n\nThe files config.{h,c} contain functions that have to do with parsing,\nbut not config.\n\nIn order to further reduce all-in-one headers, separate out functions in\nconfig.c that do not operate on config into its own file, parse.h,\nand update the include directives in the .c files that need only such\nfunctions accordingly.\n\nSigned-off-by: Calvin Wan \nSigned-off-by: Jonathan Tan \n---\n Makefile | 1 +\n attr.c | 2 +-\n config.c | 180 +-----------------------------------\n config.h | 14 +--\n pack-objects.c | 2 +-\n pack-revindex.c | 2 +-\n parse-options.c | 3 +-\n parse.c | 182 +++++++++++++++++++++++++++++++++++++\n parse.h | 20 ++++\n pathspec.c | 2 +-\n preload-index.c | 2 +-\n progress.c | 2 +-\n prompt.c | 2 +-\n rebase.c | 2 +-\n t/helper/test-env-helper.c | 2 +-\n unpack-trees.c | 2 +-\n wrapper.c | 2 +-\n write-or-die.c | 2 +-\n 18 files changed, 219 insertions(+), 205 deletions(-)\n create mode 100644 parse.c\n create mode 100644 parse.h\n\ndiff --git a/Makefile b/Makefile\nindex 861e643708..9226c719a0 100644\n--- a/Makefile\n+++ b/Makefile\n@@ -1091,6 +1091,7 @@ LIB_OBJS += pack-write.o\n LIB_OBJS += packfile.o\n LIB_OBJS += pager.o\n LIB_OBJS += parallel-checkout.o\n+LIB_OBJS += parse.o\n LIB_OBJS += parse-options-cb.o\n LIB_OBJS += parse-options.o\n LIB_OBJS += patch-delta.o\ndiff --git a/attr.c b/attr.c\nindex 71c84fbcf8..3c0b4fb3d9 100644\n--- a/attr.c\n+++ b/attr.c\n@@ -7,7 +7,7 @@\n */\n \n #include \"git-compat-util.h\"\n-#include \"config.h\"\n+#include \"parse.h\"\n #include \"environment.h\"\n #include \"exec-cmd.h\"\n #include \"attr.h\"\ndiff --git a/config.c b/config.c\nindex 7dde0aaa02..c7bc21a25d 100644\n--- a/config.c\n+++ b/config.c\n@@ -11,6 +11,7 @@\n #include \"date.h\"\n #include \"branch.h\"\n #include \"config.h\"\n+#include \"parse.h\"\n #include \"convert.h\"\n #include \"environment.h\"\n #include \"gettext.h\"\n@@ -1165,129 +1166,6 @@ static int git_parse_source(struct config_source *cs, config_fn_t fn,\n \treturn error_return;\n }\n \n-static uintmax_t get_unit_factor(const char *end)\n-{\n-\tif (!*end)\n-\t\treturn 1;\n-\telse if (!strcasecmp(end, \"k\"))\n-\t\treturn 1024;\n-\telse if (!strcasecmp(end, \"m\"))\n-\t\treturn 1024 * 1024;\n-\telse if (!strcasecmp(end, \"g\"))\n-\t\treturn 1024 * 1024 * 1024;\n-\treturn 0;\n-}\n-\n-static int git_parse_signed(const char *value, intmax_t *ret, intmax_t max)\n-{\n-\tif (value && *value) {\n-\t\tchar *end;\n-\t\tintmax_t val;\n-\t\tintmax_t factor;\n-\n-\t\tif (max < 0)\n-\t\t\tBUG(\"max must be a positive integer\");\n-\n-\t\terrno = 0;\n-\t\tval = strtoimax(value, &end, 0);\n-\t\tif (errno == ERANGE)\n-\t\t\treturn 0;\n-\t\tif (end == value) {\n-\t\t\terrno = EINVAL;\n-\t\t\treturn 0;\n-\t\t}\n-\t\tfactor = get_unit_factor(end);\n-\t\tif (!factor) {\n-\t\t\terrno = EINVAL;\n-\t\t\treturn 0;\n-\t\t}\n-\t\tif ((val < 0 && -max / factor > val) ||\n-\t\t (val > 0 && max / factor < val)) {\n-\t\t\terrno = ERANGE;\n-\t\t\treturn 0;\n-\t\t}\n-\t\tval *= factor;\n-\t\t*ret = val;\n-\t\treturn 1;\n-\t}\n-\terrno = EINVAL;\n-\treturn 0;\n-}\n-\n-static int git_parse_unsigned(const char *value, uintmax_t *ret, uintmax_t max)\n-{\n-\tif (value && *value) {\n-\t\tchar *end;\n-\t\tuintmax_t val;\n-\t\tuintmax_t factor;\n-\n-\t\t/* negative values would be accepted by strtoumax */\n-\t\tif (strchr(value, '-')) {\n-\t\t\terrno = EINVAL;\n-\t\t\treturn 0;\n-\t\t}\n-\t\terrno = 0;\n-\t\tval = strtoumax(value, &end, 0);\n-\t\tif (errno == ERANGE)\n-\t\t\treturn 0;\n-\t\tif (end == value) {\n-\t\t\terrno = EINVAL;\n-\t\t\treturn 0;\n-\t\t}\n-\t\tfactor = get_unit_factor(end);\n-\t\tif (!factor) {\n-\t\t\terrno = EINVAL;\n-\t\t\treturn 0;\n-\t\t}\n-\t\tif (unsigned_mult_overflows(factor, val) ||\n-\t\t factor * val > max) {\n-\t\t\terrno = ERANGE;\n-\t\t\treturn 0;\n-\t\t}\n-\t\tval *= factor;\n-\t\t*ret = val;\n-\t\treturn 1;\n-\t}\n-\terrno = EINVAL;\n-\treturn 0;\n-}\n-\n-int git_parse_int(const char *value, int *ret)\n-{\n-\tintmax_t tmp;\n-\tif (!git_parse_signed(value, &tmp, maximum_signed_value_of_type(int)))\n-\t\treturn 0;\n-\t*ret = tmp;\n-\treturn 1;\n-}\n-\n-static int git_parse_int64(const char *value, int64_t *ret)\n-{\n-\tintmax_t tmp;\n-\tif (!git_parse_signed(value, &tmp, maximum_signed_value_of_type(int64_t)))\n-\t\treturn 0;\n-\t*ret = tmp;\n-\treturn 1;\n-}\n-\n-int git_parse_ulong(const char *value, unsigned long *ret)\n-{\n-\tuintmax_t tmp;\n-\tif (!git_parse_unsigned(value, &tmp, maximum_unsigned_value_of_type(long)))\n-\t\treturn 0;\n-\t*ret = tmp;\n-\treturn 1;\n-}\n-\n-int git_parse_ssize_t(const char *value, ssize_t *ret)\n-{\n-\tintmax_t tmp;\n-\tif (!git_parse_signed(value, &tmp, maximum_signed_value_of_type(ssize_t)))\n-\t\treturn 0;\n-\t*ret = tmp;\n-\treturn 1;\n-}\n-\n NORETURN\n static void die_bad_number(const char *name, const char *value,\n \t\t\t const struct key_value_info *kvi)\n@@ -1363,23 +1241,6 @@ ssize_t git_config_ssize_t(const char *name, const char *value,\n \treturn ret;\n }\n \n-static int git_parse_maybe_bool_text(const char *value)\n-{\n-\tif (!value)\n-\t\treturn 1;\n-\tif (!*value)\n-\t\treturn 0;\n-\tif (!strcasecmp(value, \"true\")\n-\t || !strcasecmp(value, \"yes\")\n-\t || !strcasecmp(value, \"on\"))\n-\t\treturn 1;\n-\tif (!strcasecmp(value, \"false\")\n-\t || !strcasecmp(value, \"no\")\n-\t || !strcasecmp(value, \"off\"))\n-\t\treturn 0;\n-\treturn -1;\n-}\n-\n static const struct fsync_component_name {\n \tconst char *name;\n \tenum fsync_component component_bits;\n@@ -1454,16 +1315,6 @@ static enum fsync_component parse_fsync_components(const char *var, const char *\n \treturn (current & ~negative) | positive;\n }\n \n-int git_parse_maybe_bool(const char *value)\n-{\n-\tint v = git_parse_maybe_bool_text(value);\n-\tif (0 <= v)\n-\t\treturn v;\n-\tif (git_parse_int(value, &v))\n-\t\treturn !!v;\n-\treturn -1;\n-}\n-\n int git_config_bool_or_int(const char *name, const char *value,\n \t\t\t const struct key_value_info *kvi, int *is_bool)\n {\n@@ -2126,35 +1977,6 @@ void git_global_config(char **user_out, char **xdg_out)\n \t*xdg_out = xdg_config;\n }\n \n-/*\n- * Parse environment variable 'k' as a boolean (in various\n- * possible spellings); if missing, use the default value 'def'.\n- */\n-int git_env_bool(const char *k, int def)\n-{\n-\tconst char *v = getenv(k);\n-\tint val;\n-\tif (!v)\n-\t\treturn def;\n-\tval = git_parse_maybe_bool(v);\n-\tif (val < 0)\n-\t\tdie(_(\"bad boolean environment value '%s' for '%s'\"),\n-\t\t v, k);\n-\treturn val;\n-}\n-\n-/*\n- * Parse environment variable 'k' as ulong with possibly a unit\n- * suffix; if missing, use the default value 'val'.\n- */\n-unsigned long git_env_ulong(const char *k, unsigned long val)\n-{\n-\tconst char *v = getenv(k);\n-\tif (v && !git_parse_ulong(v, &val))\n-\t\tdie(_(\"failed to parse %s\"), k);\n-\treturn val;\n-}\n-\n int git_config_system(void)\n {\n \treturn !git_env_bool(\"GIT_CONFIG_NOSYSTEM\", 0);\ndiff --git a/config.h b/config.h\nindex 6332d74904..14f881ecfa 100644\n--- a/config.h\n+++ b/config.h\n@@ -4,7 +4,7 @@\n #include \"hashmap.h\"\n #include \"string-list.h\"\n #include \"repository.h\"\n-\n+#include \"parse.h\"\n \n /**\n * The config API gives callers a way to access Git configuration files\n@@ -243,16 +243,6 @@ int config_with_options(config_fn_t fn, void *,\n * The following helper functions aid in parsing string values\n */\n \n-int git_parse_ssize_t(const char *, ssize_t *);\n-int git_parse_ulong(const char *, unsigned long *);\n-int git_parse_int(const char *value, int *ret);\n-\n-/**\n- * Same as `git_config_bool`, except that it returns -1 on error rather\n- * than dying.\n- */\n-int git_parse_maybe_bool(const char *);\n-\n /**\n * Parse the string to an integer, including unit factors. Dies on error;\n * otherwise, returns the parsed result.\n@@ -385,8 +375,6 @@ int git_config_rename_section(const char *, const char *);\n int git_config_rename_section_in_file(const char *, const char *, const char *);\n int git_config_copy_section(const char *, const char *);\n int git_config_copy_section_in_file(const char *, const char *, const char *);\n-int git_env_bool(const char *, int);\n-unsigned long git_env_ulong(const char *, unsigned long);\n int git_config_system(void);\n int config_error_nonbool(const char *);\n #if defined(__GNUC__)\ndiff --git a/pack-objects.c b/pack-objects.c\nindex 1b8052bece..f403ca6986 100644\n--- a/pack-objects.c\n+++ b/pack-objects.c\n@@ -3,7 +3,7 @@\n #include \"pack.h\"\n #include \"pack-objects.h\"\n #include \"packfile.h\"\n-#include \"config.h\"\n+#include \"parse.h\"\n \n static uint32_t locate_object_entry_hash(struct packing_data *pdata,\n \t\t\t\t\t const struct object_id *oid,\ndiff --git a/pack-revindex.c b/pack-revindex.c\nindex 7fffcad912..a01a2a4640 100644\n--- a/pack-revindex.c\n+++ b/pack-revindex.c\n@@ -6,7 +6,7 @@\n #include \"packfile.h\"\n #include \"strbuf.h\"\n #include \"trace2.h\"\n-#include \"config.h\"\n+#include \"parse.h\"\n #include \"midx.h\"\n #include \"csum-file.h\"\n \ndiff --git a/parse-options.c b/parse-options.c\nindex e8e076c3a6..093eaf2db8 100644\n--- a/parse-options.c\n+++ b/parse-options.c\n@@ -1,11 +1,12 @@\n #include \"git-compat-util.h\"\n #include \"parse-options.h\"\n #include \"abspath.h\"\n-#include \"config.h\"\n+#include \"parse.h\"\n #include \"commit.h\"\n #include \"color.h\"\n #include \"gettext.h\"\n #include \"strbuf.h\"\n+#include \"string-list.h\"\n #include \"utf8.h\"\n \n static int disallow_abbreviated_options;\ndiff --git a/parse.c b/parse.c\nnew file mode 100644\nindex 0000000000..42d691a0fb\n--- /dev/null\n+++ b/parse.c\n@@ -0,0 +1,182 @@\n+#include \"git-compat-util.h\"\n+#include \"gettext.h\"\n+#include \"parse.h\"\n+\n+static uintmax_t get_unit_factor(const char *end)\n+{\n+\tif (!*end)\n+\t\treturn 1;\n+\telse if (!strcasecmp(end, \"k\"))\n+\t\treturn 1024;\n+\telse if (!strcasecmp(end, \"m\"))\n+\t\treturn 1024 * 1024;\n+\telse if (!strcasecmp(end, \"g\"))\n+\t\treturn 1024 * 1024 * 1024;\n+\treturn 0;\n+}\n+\n+int git_parse_signed(const char *value, intmax_t *ret, intmax_t max)\n+{\n+\tif (value && *value) {\n+\t\tchar *end;\n+\t\tintmax_t val;\n+\t\tintmax_t factor;\n+\n+\t\tif (max < 0)\n+\t\t\tBUG(\"max must be a positive integer\");\n+\n+\t\terrno = 0;\n+\t\tval = strtoimax(value, &end, 0);\n+\t\tif (errno == ERANGE)\n+\t\t\treturn 0;\n+\t\tif (end == value) {\n+\t\t\terrno = EINVAL;\n+\t\t\treturn 0;\n+\t\t}\n+\t\tfactor = get_unit_factor(end);\n+\t\tif (!factor) {\n+\t\t\terrno = EINVAL;\n+\t\t\treturn 0;\n+\t\t}\n+\t\tif ((val < 0 && -max / factor > val) ||\n+\t\t (val > 0 && max / factor < val)) {\n+\t\t\terrno = ERANGE;\n+\t\t\treturn 0;\n+\t\t}\n+\t\tval *= factor;\n+\t\t*ret = val;\n+\t\treturn 1;\n+\t}\n+\terrno = EINVAL;\n+\treturn 0;\n+}\n+\n+static int git_parse_unsigned(const char *value, uintmax_t *ret, uintmax_t max)\n+{\n+\tif (value && *value) {\n+\t\tchar *end;\n+\t\tuintmax_t val;\n+\t\tuintmax_t factor;\n+\n+\t\t/* negative values would be accepted by strtoumax */\n+\t\tif (strchr(value, '-')) {\n+\t\t\terrno = EINVAL;\n+\t\t\treturn 0;\n+\t\t}\n+\t\terrno = 0;\n+\t\tval = strtoumax(value, &end, 0);\n+\t\tif (errno == ERANGE)\n+\t\t\treturn 0;\n+\t\tif (end == value) {\n+\t\t\terrno = EINVAL;\n+\t\t\treturn 0;\n+\t\t}\n+\t\tfactor = get_unit_factor(end);\n+\t\tif (!factor) {\n+\t\t\terrno = EINVAL;\n+\t\t\treturn 0;\n+\t\t}\n+\t\tif (unsigned_mult_overflows(factor, val) ||\n+\t\t factor * val > max) {\n+\t\t\terrno = ERANGE;\n+\t\t\treturn 0;\n+\t\t}\n+\t\tval *= factor;\n+\t\t*ret = val;\n+\t\treturn 1;\n+\t}\n+\terrno = EINVAL;\n+\treturn 0;\n+}\n+\n+int git_parse_int(const char *value, int *ret)\n+{\n+\tintmax_t tmp;\n+\tif (!git_parse_signed(value, &tmp, maximum_signed_value_of_type(int)))\n+\t\treturn 0;\n+\t*ret = tmp;\n+\treturn 1;\n+}\n+\n+int git_parse_int64(const char *value, int64_t *ret)\n+{\n+\tintmax_t tmp;\n+\tif (!git_parse_signed(value, &tmp, maximum_signed_value_of_type(int64_t)))\n+\t\treturn 0;\n+\t*ret = tmp;\n+\treturn 1;\n+}\n+\n+int git_parse_ulong(const char *value, unsigned long *ret)\n+{\n+\tuintmax_t tmp;\n+\tif (!git_parse_unsigned(value, &tmp, maximum_unsigned_value_of_type(long)))\n+\t\treturn 0;\n+\t*ret = tmp;\n+\treturn 1;\n+}\n+\n+int git_parse_ssize_t(const char *value, ssize_t *ret)\n+{\n+\tintmax_t tmp;\n+\tif (!git_parse_signed(value, &tmp, maximum_signed_value_of_type(ssize_t)))\n+\t\treturn 0;\n+\t*ret = tmp;\n+\treturn 1;\n+}\n+\n+int git_parse_maybe_bool_text(const char *value)\n+{\n+\tif (!value)\n+\t\treturn 1;\n+\tif (!*value)\n+\t\treturn 0;\n+\tif (!strcasecmp(value, \"true\")\n+\t || !strcasecmp(value, \"yes\")\n+\t || !strcasecmp(value, \"on\"))\n+\t\treturn 1;\n+\tif (!strcasecmp(value, \"false\")\n+\t || !strcasecmp(value, \"no\")\n+\t || !strcasecmp(value, \"off\"))\n+\t\treturn 0;\n+\treturn -1;\n+}\n+\n+int git_parse_maybe_bool(const char *value)\n+{\n+\tint v = git_parse_maybe_bool_text(value);\n+\tif (0 <= v)\n+\t\treturn v;\n+\tif (git_parse_int(value, &v))\n+\t\treturn !!v;\n+\treturn -1;\n+}\n+\n+/*\n+ * Parse environment variable 'k' as a boolean (in various\n+ * possible spellings); if missing, use the default value 'def'.\n+ */\n+int git_env_bool(const char *k, int def)\n+{\n+\tconst char *v = getenv(k);\n+\tint val;\n+\tif (!v)\n+\t\treturn def;\n+\tval = git_parse_maybe_bool(v);\n+\tif (val < 0)\n+\t\tdie(_(\"bad boolean environment value '%s' for '%s'\"),\n+\t\t v, k);\n+\treturn val;\n+}\n+\n+/*\n+ * Parse environment variable 'k' as ulong with possibly a unit\n+ * suffix; if missing, use the default value 'val'.\n+ */\n+unsigned long git_env_ulong(const char *k, unsigned long val)\n+{\n+\tconst char *v = getenv(k);\n+\tif (v && !git_parse_ulong(v, &val))\n+\t\tdie(_(\"failed to parse %s\"), k);\n+\treturn val;\n+}\ndiff --git a/parse.h b/parse.h\nnew file mode 100644\nindex 0000000000..07d2193d69\n--- /dev/null\n+++ b/parse.h\n@@ -0,0 +1,20 @@\n+#ifndef PARSE_H\n+#define PARSE_H\n+\n+int git_parse_signed(const char *value, intmax_t *ret, intmax_t max);\n+int git_parse_ssize_t(const char *, ssize_t *);\n+int git_parse_ulong(const char *, unsigned long *);\n+int git_parse_int(const char *value, int *ret);\n+int git_parse_int64(const char *value, int64_t *ret);\n+\n+/**\n+ * Same as `git_config_bool`, except that it returns -1 on error rather\n+ * than dying.\n+ */\n+int git_parse_maybe_bool(const char *);\n+int git_parse_maybe_bool_text(const char *value);\n+\n+int git_env_bool(const char *, int);\n+unsigned long git_env_ulong(const char *, unsigned long);\n+\n+#endif /* PARSE_H */\ndiff --git a/pathspec.c b/pathspec.c\nindex 3a3a5724c4..7f88f1c02b 100644\n--- a/pathspec.c\n+++ b/pathspec.c\n@@ -1,6 +1,6 @@\n #include \"git-compat-util.h\"\n #include \"abspath.h\"\n-#include \"config.h\"\n+#include \"parse.h\"\n #include \"dir.h\"\n #include \"environment.h\"\n #include \"gettext.h\"\ndiff --git a/preload-index.c b/preload-index.c\nindex e44530c80c..63fd35d64b 100644\n--- a/preload-index.c\n+++ b/preload-index.c\n@@ -7,7 +7,7 @@\n #include \"environment.h\"\n #include \"fsmonitor.h\"\n #include \"gettext.h\"\n-#include \"config.h\"\n+#include \"parse.h\"\n #include \"preload-index.h\"\n #include \"progress.h\"\n #include \"read-cache.h\"\ndiff --git a/progress.c b/progress.c\nindex f695798aca..c83cb60bf1 100644\n--- a/progress.c\n+++ b/progress.c\n@@ -17,7 +17,7 @@\n #include \"trace.h\"\n #include \"trace2.h\"\n #include \"utf8.h\"\n-#include \"config.h\"\n+#include \"parse.h\"\n \n #define TP_IDX_MAX 8\n \ndiff --git a/prompt.c b/prompt.c\nindex 3baa33f63d..8935fe4dfb 100644\n--- a/prompt.c\n+++ b/prompt.c\n@@ -1,5 +1,5 @@\n #include \"git-compat-util.h\"\n-#include \"config.h\"\n+#include \"parse.h\"\n #include \"environment.h\"\n #include \"run-command.h\"\n #include \"strbuf.h\"\ndiff --git a/rebase.c b/rebase.c\nindex 17a570f1ff..69a1822da3 100644\n--- a/rebase.c\n+++ b/rebase.c\n@@ -1,6 +1,6 @@\n #include \"git-compat-util.h\"\n #include \"rebase.h\"\n-#include \"config.h\"\n+#include \"parse.h\"\n #include \"gettext.h\"\n \n /*\ndiff --git a/t/helper/test-env-helper.c b/t/helper/test-env-helper.c\nindex 66c88b8ff3..1c486888a4 100644\n--- a/t/helper/test-env-helper.c\n+++ b/t/helper/test-env-helper.c\n@@ -1,5 +1,5 @@\n #include \"test-tool.h\"\n-#include \"config.h\"\n+#include \"parse.h\"\n #include \"parse-options.h\"\n \n static char const * const env__helper_usage[] = {\ndiff --git a/unpack-trees.c b/unpack-trees.c\nindex 87517364dc..761562a96e 100644\n--- a/unpack-trees.c\n+++ b/unpack-trees.c\n@@ -2,7 +2,7 @@\n #include \"advice.h\"\n #include \"strvec.h\"\n #include \"repository.h\"\n-#include \"config.h\"\n+#include \"parse.h\"\n #include \"dir.h\"\n #include \"environment.h\"\n #include \"gettext.h\"\ndiff --git a/wrapper.c b/wrapper.c\nindex 453a20ed99..7da15a56da 100644\n--- a/wrapper.c\n+++ b/wrapper.c\n@@ -3,7 +3,7 @@\n */\n #include \"git-compat-util.h\"\n #include \"abspath.h\"\n-#include \"config.h\"\n+#include \"parse.h\"\n #include \"gettext.h\"\n #include \"repository.h\"\n #include \"strbuf.h\"\ndiff --git a/write-or-die.c b/write-or-die.c\nindex d8355c0c3e..42a2dc73cd 100644\n--- a/write-or-die.c\n+++ b/write-or-die.c\n@@ -1,5 +1,5 @@\n #include \"git-compat-util.h\"\n-#include \"config.h\"\n+#include \"parse.h\"\n #include \"run-command.h\"\n #include \"write-or-die.h\"\n \n-- \n2.42.0.582.g8ccd20d70d-goog\n\n"},{"id":"482462","messageId":"xmqqa5t4inu7.fsf@gitster.g","threadId":"59921","inReplyTo":"e4c20a81f93e2a4e2ad65920920ead690b162e25.1696021277.git.jonathantanmy@google.com","subject":"Re: [PATCH v4 3/4] config: correct bad boolean env value error message","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2023-09-29T23:03:44Z","receivedAt":"2023-09-29T23:03:57Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Jonathan Tan writes:\n\n> From: Calvin Wan \n>\n> An incorrectly defined boolean environment value would result in the\n> following error message:\n>\n> bad boolean config value '%s' for '%s'\n>\n> This is a misnomer since environment value != config value. Instead of\n> calling git_config_bool() to parse the environment value, mimic the\n> functionality inside of git_config_bool() but with the correct error\n> message.\n>\n> Signed-off-by: Calvin Wan \n> Signed-off-by: Junio C Hamano \n> ---\n> config.c | 9 ++++++++-\n> 1 file changed, 8 insertions(+), 1 deletion(-)\n\nMakes sense.\n\nI briefly wondered if there are ways to share more code, but this\nseems to be the best we can do. The duplication is not too bad to\nbegin with anyway.\n\nLooking good. Will queue.\n\n\n> diff --git a/config.c b/config.c\n> index 3846a37be9..7dde0aaa02 100644\n> --- a/config.c\n> +++ b/config.c\n> @@ -2133,7 +2133,14 @@ void git_global_config(char **user_out, char **xdg_out)\n> int git_env_bool(const char *k, int def)\n> {\n> \tconst char *v = getenv(k);\n> -\treturn v ? git_config_bool(k, v) : def;\n> +\tint val;\n> +\tif (!v)\n> +\t\treturn def;\n> +\tval = git_parse_maybe_bool(v);\n> +\tif (val < 0)\n> +\t\tdie(_(\"bad boolean environment value '%s' for '%s'\"),\n> +\t\t v, k);\n> +\treturn val;\n> }\n> \n> /*\n"},{"id":"482954","messageId":"066b3162-6a81-45d7-b164-17b74e6c92dc@gmail.com","threadId":"59921","inReplyTo":"c9e7cd78576527571fd70b953e340b5bdd196221.1696021277.git.jonathantanmy@google.com","subject":"Re: [PATCH v4 2/4] wrapper: reduce scope of remove_or_warn()","fromName":"","fromEmail":"phillip.wood123@gmail.com","sentAt":"2023-10-10T09:59:38Z","receivedAt":"2023-10-10T09:59:48Z","isPatch":true,"sender":{"key":"phillip.wood@dunelm.org.uk","avatar":null},"body":"Hi Jonathan\n\nOn 29/09/2023 22:20, Jonathan Tan wrote:\n> From: Calvin Wan \n> \n> remove_or_warn() is only used by entry.c and apply.c, but it is\n> currently declared and defined in wrapper.{h,c}, so it has a scope much\n> greater than it needs. This needlessly large scope also causes wrapper.c\n> to need to include object.h, when this file is largely unconcerned with\n> Git objects.\n> \n> Move remove_or_warn() to entry.{h,c}. The file apply.c still has access\n> to it, since it already includes entry.h for another reason.\n\nThis looks good. On a related note wrapper.c includes repository.h but \ndoes use anything declared in that header.\n\nBest Wishes\n\nPhillip\n"},{"id":"482955","messageId":"1de0a6f3-e223-4e84-a6d2-51d9b51a02f6@gmail.com","threadId":"59921","inReplyTo":"5d9f0b3de08ab8541482b9b640db06b6d3000b86.1696021277.git.jonathantanmy@google.com","subject":"Re: [PATCH v4 4/4] parse: separate out parsing functions from config.h","fromName":"","fromEmail":"phillip.wood123@gmail.com","sentAt":"2023-10-10T10:00:15Z","receivedAt":"2023-10-10T10:00:30Z","isPatch":true,"sender":{"key":"phillip.wood@dunelm.org.uk","avatar":null},"body":"Hi Jonathan\n\nOn 29/09/2023 22:20, Jonathan Tan wrote:\n> diff --git a/parse.h b/parse.h\n> new file mode 100644\n> index 0000000000..07d2193d69\n> --- /dev/null\n> +++ b/parse.h\n> @@ -0,0 +1,20 @@\n> +#ifndef PARSE_H\n> +#define PARSE_H\n> +\n> +int git_parse_signed(const char *value, intmax_t *ret, intmax_t max);\n\nPreviously this function was private to config.c, now it needs to be \npublic because it is still called by \ngit_config_get_expiry_date_in_days(). As this is essentially an internal \nhelper for git_parse_int() and friends it is a bit unfortunate that it \nis now public. Perhaps we should change \ngit_config_get_expiry_date_in_days() to call git_parse_int() instead.\nThen we can keep git_parse_signed() and git_parse_unsigned() private to \nparse.c.\n\n> +int git_parse_ssize_t(const char *, ssize_t *);\n> +int git_parse_ulong(const char *, unsigned long *);\n> +int git_parse_int(const char *value, int *ret);\n> +int git_parse_int64(const char *value, int64_t *ret);\n\nThis was previously private but I think it makes sense for it to be \npublicly available.\n\n> +/**\n> + * Same as `git_config_bool`, except that it returns -1 on error rather\n> + * than dying.\n> + */\n> +int git_parse_maybe_bool(const char *);\n> +int git_parse_maybe_bool_text(const char *value);\n\nThis used to be private to config.c and now has callers in parse.c and \nconfig.c. We should make it clear that non-config code is likely to want \ngit_parse_maybe_bool() rather than this function.\n\nBest Wishes\n\nPhillip\n"},{"id":"482956","messageId":"4670774d-a899-492c-9b36-98ee243c8d4d@gmail.com","threadId":"59921","inReplyTo":"cover.1696021277.git.jonathantanmy@google.com","subject":"Re: [PATCH v4 0/4] Preliminary patches before git-std-lib","fromName":"","fromEmail":"phillip.wood123@gmail.com","sentAt":"2023-10-10T10:05:58Z","receivedAt":"2023-10-10T10:06:05Z","isPatch":true,"sender":{"key":"phillip.wood@dunelm.org.uk","avatar":null},"body":"Hi Jonathan\n\nOn 29/09/2023 22:20, Jonathan Tan wrote:\n> Calvin will be away for a few weeks and I'll be handling the git-std-lib\n> effort in the meantime. My goals will be:\n> \n> - Get the preliminary patches in Calvin's patch set (patches 1-4) merged\n> first.\n> \n> - Updating patches 5-6 based on reviewer feedback (including my\n> feedback). I have several aims including reducing or eliminating the\n> need for the GIT_STD_LIB preprocessor variable, and making stubs a test-\n> only concern (I think Phillip has some similar ideas [1] but I haven't\n> looked at their repo on GitHub yet).\n\nIt sounds like we're thinking along similar lines, do feel free get in \ntouch on or off the list if you want to ask anything about those patches \nI pushed to github.\n\n> [1] https://lore.kernel.org/git/98f3edcf-7f37-45ff-abd2-c0038d4e0589@gmail.com/\n> \n> This patch set is in service of the first goal. Because the libification\n> patches are no longer included in this patch set, I have rewritten the\n> commit messages to justify the patches in terms of code organization.\n> There are no changes in the code itself. Also, I have retained Calvin's\n> name as the author.\n\nI agree it makes sense to get the preliminary patches merged on their \nown. I think the argument that they reduce the scope of includes is a \nreasonable justification on its own. I've left a couple of comments but \nthey're looking pretty good.\n\nBest Wishes\n\nPhillip\n"},{"id":"482988","messageId":"xmqqjzruv4k1.fsf@gitster.g","threadId":"59921","inReplyTo":"066b3162-6a81-45d7-b164-17b74e6c92dc@gmail.com","subject":"Re: [PATCH v4 2/4] wrapper: reduce scope of remove_or_warn()","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2023-10-10T16:13:34Z","receivedAt":"2023-10-10T16:13:46Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"phillip.wood123@gmail.com writes:\n\n> Hi Jonathan\n>\n> On 29/09/2023 22:20, Jonathan Tan wrote:\n>> From: Calvin Wan \n>> remove_or_warn() is only used by entry.c and apply.c, but it is\n>> currently declared and defined in wrapper.{h,c}, so it has a scope much\n>> greater than it needs. This needlessly large scope also causes wrapper.c\n>> to need to include object.h, when this file is largely unconcerned with\n>> Git objects.\n>> Move remove_or_warn() to entry.{h,c}. The file apply.c still has\n>> access\n>> to it, since it already includes entry.h for another reason.\n>\n> This looks good. On a related note wrapper.c includes repository.h but\n> does use anything declared in that header.\n>\n> Best Wishes\n>\n> Phillip\n\nThanks for a review. I just checked 'master', 'next', and 'seen'\nand in all '#include ' can safely be dropped from\nthere, it seems. It may be too trivial even for a microproject,\nbut nevertheless a nice clean-up.\n\n"},{"id":"482989","messageId":"20231010162141.2133714-1-jonathantanmy@google.com","threadId":"59921","inReplyTo":"4670774d-a899-492c-9b36-98ee243c8d4d@gmail.com","subject":"Re: [PATCH v4 0/4] Preliminary patches before git-std-lib","fromName":"Jonathan Tan","fromEmail":"jonathantanmy@google.com","sentAt":"2023-10-10T16:21:41Z","receivedAt":"2023-10-10T16:21:48Z","isPatch":true,"sender":{"key":"jonathantanmy@fastmail.com","avatar":null},"body":"phillip.wood123@gmail.com writes:\n> Hi Jonathan\n> \n> On 29/09/2023 22:20, Jonathan Tan wrote:\n> > Calvin will be away for a few weeks and I'll be handling the git-std-lib\n> > effort in the meantime. My goals will be:\n> > \n> > - Get the preliminary patches in Calvin's patch set (patches 1-4) merged\n> > first.\n> > \n> > - Updating patches 5-6 based on reviewer feedback (including my\n> > feedback). I have several aims including reducing or eliminating the\n> > need for the GIT_STD_LIB preprocessor variable, and making stubs a test-\n> > only concern (I think Phillip has some similar ideas [1] but I haven't\n> > looked at their repo on GitHub yet).\n> \n> It sounds like we're thinking along similar lines, do feel free get in \n> touch on or off the list if you want to ask anything about those patches \n> I pushed to github.\n\nThanks. I'm updating patches 5-6 now and basing on your work, in fact.\n\n> > [1] https://lore.kernel.org/git/98f3edcf-7f37-45ff-abd2-c0038d4e0589@gmail.com/\n> > \n> > This patch set is in service of the first goal. Because the libification\n> > patches are no longer included in this patch set, I have rewritten the\n> > commit messages to justify the patches in terms of code organization.\n> > There are no changes in the code itself. Also, I have retained Calvin's\n> > name as the author.\n> \n> I agree it makes sense to get the preliminary patches merged on their \n> own. I think the argument that they reduce the scope of includes is a \n> reasonable justification on its own. I've left a couple of comments but \n> they're looking pretty good.\n> \n> Best Wishes\n> \n> Phillip\n\nThanks.\n"},{"id":"482993","messageId":"20231010173853.2147774-1-jonathantanmy@google.com","threadId":"59921","inReplyTo":"xmqqjzruv4k1.fsf@gitster.g","subject":"Re: [PATCH v4 2/4] wrapper: reduce scope of remove_or_warn()","fromName":"Jonathan Tan","fromEmail":"jonathantanmy@google.com","sentAt":"2023-10-10T17:38:53Z","receivedAt":"2023-10-10T17:39:00Z","isPatch":true,"sender":{"key":"jonathantanmy@fastmail.com","avatar":null},"body":"Junio C Hamano writes:\n> phillip.wood123@gmail.com writes:\n> \n> > Hi Jonathan\n> >\n> > On 29/09/2023 22:20, Jonathan Tan wrote:\n> >> From: Calvin Wan \n> >> remove_or_warn() is only used by entry.c and apply.c, but it is\n> >> currently declared and defined in wrapper.{h,c}, so it has a scope much\n> >> greater than it needs. This needlessly large scope also causes wrapper.c\n> >> to need to include object.h, when this file is largely unconcerned with\n> >> Git objects.\n> >> Move remove_or_warn() to entry.{h,c}. The file apply.c still has\n> >> access\n> >> to it, since it already includes entry.h for another reason.\n> >\n> > This looks good. On a related note wrapper.c includes repository.h but\n> > does use anything declared in that header.\n> >\n> > Best Wishes\n> >\n> > Phillip\n> \n> Thanks for a review. I just checked 'master', 'next', and 'seen'\n> and in all '#include ' can safely be dropped from\n> there, it seems. It may be too trivial even for a microproject,\n> but nevertheless a nice clean-up.\n\nAh, Calvin fixed this in one of the subsequent patches, but I'll put it\ninto its own patch in my updated version.\n"},{"id":"482994","messageId":"20231010174348.2150150-1-jonathantanmy@google.com","threadId":"59921","inReplyTo":"1de0a6f3-e223-4e84-a6d2-51d9b51a02f6@gmail.com","subject":"Re: [PATCH v4 4/4] parse: separate out parsing functions from config.h","fromName":"Jonathan Tan","fromEmail":"jonathantanmy@google.com","sentAt":"2023-10-10T17:43:47Z","receivedAt":"2023-10-10T17:43:56Z","isPatch":true,"sender":{"key":"jonathantanmy@fastmail.com","avatar":null},"body":"phillip.wood123@gmail.com writes:\n> Hi Jonathan\n> \n> On 29/09/2023 22:20, Jonathan Tan wrote:\n> > diff --git a/parse.h b/parse.h\n> > new file mode 100644\n> > index 0000000000..07d2193d69\n> > --- /dev/null\n> > +++ b/parse.h\n> > @@ -0,0 +1,20 @@\n> > +#ifndef PARSE_H\n> > +#define PARSE_H\n> > +\n> > +int git_parse_signed(const char *value, intmax_t *ret, intmax_t max);\n> \n> Previously this function was private to config.c, now it needs to be \n> public because it is still called by \n> git_config_get_expiry_date_in_days(). As this is essentially an internal \n> helper for git_parse_int() and friends it is a bit unfortunate that it \n> is now public. Perhaps we should change \n> git_config_get_expiry_date_in_days() to call git_parse_int() instead.\n> Then we can keep git_parse_signed() and git_parse_unsigned() private to \n> parse.c.\n\nIt could be argued also that it fits in with the rest of\nthe parsing functions - this one parses intmax, and we have\nothers of various signedness and size. I'm open to changing\ngit_config_get_expiry_date_in_days() too, though...we probably don't\nneed so many days.\n\n> > +/**\n> > + * Same as `git_config_bool`, except that it returns -1 on error rather\n> > + * than dying.\n> > + */\n> > +int git_parse_maybe_bool(const char *);\n> > +int git_parse_maybe_bool_text(const char *value);\n> \n> This used to be private to config.c and now has callers in parse.c and \n> config.c. We should make it clear that non-config code is likely to want \n> git_parse_maybe_bool() rather than this function.\n> \n> Best Wishes\n> \n> Phillip\n\nThe difference between these 2 functions here is that bool_text supports\nonly the textual forms (used, for example, in git_config_bool_or_int()\nwhich accepts both boolean strings and integers), which might be useful\nelsewhere too. But it could be better documented, yes.\n\nLooking at \"What's Cooking\", this series is about to be merged to\nmaster. We could hold off merging that, but I think we don't need to\n- it could be argued that git_parse_maybe_bool_text() could be better\ndocumented, but even if we wrote it from scratch, I would probably put\nthe extra documentation in its own patch anyway (so one patch for moving\nthe code, and another for adding documentation).\n"},{"id":"482996","messageId":"87c16c09-784f-4818-9231-8023fc2c5a02@gmail.com","threadId":"59921","inReplyTo":"20231010174348.2150150-1-jonathantanmy@google.com","subject":"Re: [PATCH v4 4/4] parse: separate out parsing functions from config.h","fromName":"Phillip Wood","fromEmail":"phillip.wood123@gmail.com","sentAt":"2023-10-10T17:58:43Z","receivedAt":"2023-10-10T17:58:49Z","isPatch":true,"sender":{"key":"phillip.wood@dunelm.org.uk","avatar":null},"body":"On 10/10/2023 18:43, Jonathan Tan wrote:\n> phillip.wood123@gmail.com writes:\n>> Hi Jonathan\n>>\n>> On 29/09/2023 22:20, Jonathan Tan wrote:\n>>> diff --git a/parse.h b/parse.h\n>>> new file mode 100644\n>>> index 0000000000..07d2193d69\n>>> --- /dev/null\n>>> +++ b/parse.h\n>>> @@ -0,0 +1,20 @@\n>>> +#ifndef PARSE_H\n>>> +#define PARSE_H\n>>> +\n>>> +int git_parse_signed(const char *value, intmax_t *ret, intmax_t max);\n>>\n>> Previously this function was private to config.c, now it needs to be\n>> public because it is still called by\n>> git_config_get_expiry_date_in_days(). As this is essentially an internal\n>> helper for git_parse_int() and friends it is a bit unfortunate that it\n>> is now public. Perhaps we should change\n>> git_config_get_expiry_date_in_days() to call git_parse_int() instead.\n>> Then we can keep git_parse_signed() and git_parse_unsigned() private to\n>> parse.c.\n> \n> It could be argued also that it fits in with the rest of\n> the parsing functions - this one parses intmax, and we have\n> others of various signedness and size.\n\nThis one differs from the others because it expects the caller to pass a \nmaximum value, the intmax_t equivalent to git_parse_int() would be\n\nint git_parse_intmax(const char*, intmax_t*);\n\nWe now expose git_parse_int64() which covers a similar case.\n\n> I'm open to changing\n> git_config_get_expiry_date_in_days() too, though...we probably don't\n> need so many days.\n\nIndeed, the existing code passes maximum_signed_value_of_type(int) as \nthe third argument to limit it to INT_MAX already.\n\n>>> +/**\n>>> + * Same as `git_config_bool`, except that it returns -1 on error rather\n>>> + * than dying.\n>>> + */\n>>> +int git_parse_maybe_bool(const char *);\n>>> +int git_parse_maybe_bool_text(const char *value);\n>>\n>> This used to be private to config.c and now has callers in parse.c and\n>> config.c. We should make it clear that non-config code is likely to want\n>> git_parse_maybe_bool() rather than this function.\n>>\n>> Best Wishes\n>>\n>> Phillip\n> \n> The difference between these 2 functions here is that bool_text supports\n> only the textual forms (used, for example, in git_config_bool_or_int()\n> which accepts both boolean strings and integers), which might be useful\n> elsewhere too. But it could be better documented, yes.\n> \n> Looking at \"What's Cooking\", this series is about to be merged to\n> master. We could hold off merging that, but I think we don't need to\n> - it could be argued that git_parse_maybe_bool_text() could be better\n> documented, but even if we wrote it from scratch, I would probably put\n> the extra documentation in its own patch anyway (so one patch for moving\n> the code, and another for adding documentation).\n\nI agree it's not worth re-rolling just to add some documentation here.\n\nBest Wishes\n\nPhillip\n"},{"id":"483024","messageId":"xmqqpm1mrya2.fsf@gitster.g","threadId":"59921","inReplyTo":"87c16c09-784f-4818-9231-8023fc2c5a02@gmail.com","subject":"Re: [PATCH v4 4/4] parse: separate out parsing functions from config.h","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2023-10-10T20:57:25Z","receivedAt":"2023-10-10T20:57:36Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Phillip Wood writes:\n\n>> I'm open to changing\n>> git_config_get_expiry_date_in_days() too, though...we probably don't\n>> need so many days.\n>\n> Indeed, the existing code passes maximum_signed_value_of_type(int) as\n> the third argument to limit it to INT_MAX already.\n\nYeah, in other words, the current implementation does not allow us\nto express the days more than an int can, so it is no brainer to\nswitch to use git_parse_int(). Allowing longer expiration period\nis obviously outside the scope of this series.\n\nThanks.\n"},{"id":"483615","messageId":"owlyh6mkr4qu.fsf@fine.c.googlers.com","threadId":"59921","inReplyTo":"02ecc00e9c7226c9eeb960cc49c8c03dcb182a38.1696021277.git.jonathantanmy@google.com","subject":"Re: [PATCH v4 1/4] hex-ll: separate out non-hash-algo functions","fromName":"Linus Arver","fromEmail":"linusa@google.com","sentAt":"2023-10-21T04:14:49Z","receivedAt":"2023-10-21T04:14:53Z","isPatch":true,"sender":{"key":"linus@ucla.edu","avatar":null},"body":"Jonathan Tan writes:\n\n> From: Calvin Wan \n>\n> In order to further reduce all-in-one headers, separate out functions in\n> hex.h that do not operate on object hashes into its own file, hex-ll.h,\n\nNit: I was wondering what the \"-ll\" in \"hex-ll.h\" meant, then found\nd1cbe1e6d8 (hash-ll.h: split out of hash.h to remove dependency on\nrepository.h, 2023-04-22) which seems to have set the precedent for this\nnaming style. Might be worth including here.\n"},{"id":"489175","messageId":"20240222175033.1489723-1-calvinwan@google.com","threadId":"59921","inReplyTo":"cover.1696021277.git.jonathantanmy@google.com","subject":"[PATCH v5 0/3] Introduce Git Standard Library","fromName":"Calvin Wan","fromEmail":"calvinwan@google.com","sentAt":"2024-02-22T17:50:30Z","receivedAt":"2024-02-22T17:50:51Z","isPatch":true,"sender":{"key":"calvinwan@google.com","avatar":"https://avatars.githubusercontent.com/u/92547554?v=4"},"body":"While it has been a while since the last reroll of this series[1], the\ncontents and boundaries of git-std-lib have not changed. The focus for\nthis reroll are improvements to the Makefile, test file, and\ndocumentation. Patch 1 contains a small fix for a missing include\ndiscovered by Jonathan Tan. Patch 2 introduces the Git Standard Library.\nAnd patch 3 introduces preliminary testing and usage examples for the\nGit Standard Library.\n\nOne important piece of feedback I received from the previous series is\nthat Git should be the first consumer of its own libraries. Objects in\ngit-std-lib.a are no longer contained in LIB_OBJS, but rather directly\nbuilt into git-std-lib.a and then linked into git.a. There is some\nfunctionality that is used by git-std-lib.a that's not included in\ngit-std-lib.a, such as tracing support. These have been stubbed out into\ngit-stub-lib.a and can be built with git-std-lib.a to be used\nexternally. Thank you to Philip Wood for these suggestions[2]!\n\nThe test file and Makefile have been updated to include cleanups\nsuggested by Junio. Since git-std-lib.a is now a dependency of Git, the\ntest file has also been included as part of the test suite. The series\nhas been rebased onto a recent version of `next` and function calls that\nhave been added/changed since the last reroll have also been included\ninto the test file.\n\nFinally, through our libification syncs, there have been various topics\nand questions brought up that would be better clarified with additional\ndocumentation in technical/git-std-lib.txt.\n\n[1]\nhttps://lore.kernel.org/git/20230908174134.1026823-1-calvinwan@google.com/\n[2]\nhttps://lore.kernel.org/git/98f3edcf-7f37-45ff-abd2-c0038d4e0589@gmail.com/\n\n\nCalvin Wan (2):\n git-std-lib: introduce Git Standard Library\n test-stdlib: show that git-std-lib is independent\n\nJonathan Tan (1):\n pager: include stdint.h because uintmax_t is used\n\n Documentation/Makefile | 1 +\n Documentation/technical/git-std-lib.txt | 170 +++++++++++++++\n Makefile | 71 +++++--\n pager.h | 2 +\n strbuf.h | 2 +\n stubs/misc.c | 34 +++\n stubs/pager.c | 6 +\n stubs/trace2.c | 27 +++\n t/helper/.gitignore | 1 +\n t/helper/test-stdlib.c | 266 ++++++++++++++++++++++++\n t/t0082-std-lib.sh | 11 +\n 11 files changed, 575 insertions(+), 16 deletions(-)\n create mode 100644 Documentation/technical/git-std-lib.txt\n create mode 100644 stubs/misc.c\n create mode 100644 stubs/pager.c\n create mode 100644 stubs/trace2.c\n create mode 100644 t/helper/test-stdlib.c\n create mode 100755 t/t0082-std-lib.sh\n\nRange-diff against v4:\n1: 2f99eb2ca4 < -: ---------- hex-ll: split out functionality from hex\n2: 7b2d123628 < -: ---------- wrapper: remove dependency to Git-specific internal file\n3: b37beb206a < -: ---------- config: correct bad boolean env value error message\n4: 3a827cf45c < -: ---------- parse: create new library for parsing strings and env values\n5: f8e4ac50a0 < -: ---------- git-std-lib: introduce git standard library\n6: 7840e1830a < -: ---------- git-std-lib: add test file to call git-std-lib.a functions\n-: ---------- > 1: 57b751a497 pager: include stdint.h because uintmax_t is used\n-: ---------- > 2: e64f3c73c2 git-std-lib: introduce Git Standard Library\n-: ---------- > 3: e2d930f729 test-stdlib: show that git-std-lib is independent\n-- \n2.44.0.rc0.258.g7320e95886-goog\n\n"},{"id":"489176","messageId":"20240222175033.1489723-2-calvinwan@google.com","threadId":"59921","inReplyTo":"cover.1696021277.git.jonathantanmy@google.com","subject":"[PATCH v5 1/3] pager: include stdint.h because uintmax_t is used","fromName":"Calvin Wan","fromEmail":"calvinwan@google.com","sentAt":"2024-02-22T17:50:31Z","receivedAt":"2024-02-22T17:50:56Z","isPatch":true,"sender":{"key":"calvinwan@google.com","avatar":"https://avatars.githubusercontent.com/u/92547554?v=4"},"body":"From: Jonathan Tan \n\npager.h uses uintmax_t but does not include stdint.h. Therefore, add\nthis include statement.\n\nThis was discovered when writing a stub pager.c file.\n\nSigned-off-by: Jonathan Tan \nSigned-off-by: Calvin Wan \n---\n pager.h | 2 ++\n 1 file changed, 2 insertions(+)\n\ndiff --git a/pager.h b/pager.h\nindex b77433026d..015bca95e3 100644\n--- a/pager.h\n+++ b/pager.h\n@@ -1,6 +1,8 @@\n #ifndef PAGER_H\n #define PAGER_H\n \n+#include \n+\n struct child_process;\n \n const char *git_pager(int stdout_is_tty);\n-- \n2.44.0.rc0.258.g7320e95886-goog\n\n"},{"id":"489177","messageId":"20240222175033.1489723-3-calvinwan@google.com","threadId":"59921","inReplyTo":"cover.1696021277.git.jonathantanmy@google.com","subject":"[PATCH v5 2/3] git-std-lib: introduce Git Standard Library","fromName":"Calvin Wan","fromEmail":"calvinwan@google.com","sentAt":"2024-02-22T17:50:32Z","receivedAt":"2024-02-22T17:50:58Z","isPatch":true,"sender":{"key":"calvinwan@google.com","avatar":"https://avatars.githubusercontent.com/u/92547554?v=4"},"body":"This commit contains:\n- Makefile rules for git-std-lib.a\n- code and Makefile rules for git-stub-lib.a\n- description and rationale of the above in Documentation/\n\nQuoting from documentation introduced in this commit:\n\n The Git Standard Library intends to serve as the foundational library\n and root dependency that other libraries in Git will be built off\n of. That is to say, suppose we have libraries X and Y; a user that\n wants to use X and Y would need to include X, Y, and this Git Standard\n Library.\n\nCode demonstrating the use of git-std-lib.a and git-stub-lib.a will be\nin a subsequent commit.\n\nSigned-off-by: Calvin Wan \nHelped-by: Phillip Wood \nSigned-off-by: Jonathan Tan \n---\n Documentation/Makefile | 1 +\n Documentation/technical/git-std-lib.txt | 170 ++++++++++++++++++++++++\n Makefile | 48 +++++--\n stubs/misc.c | 33 +++++\n stubs/pager.c | 6 +\n stubs/trace2.c | 27 ++++\n 6 files changed, 274 insertions(+), 11 deletions(-)\n create mode 100644 Documentation/technical/git-std-lib.txt\n create mode 100644 stubs/misc.c\n create mode 100644 stubs/pager.c\n create mode 100644 stubs/trace2.c\n\ndiff --git a/Documentation/Makefile b/Documentation/Makefile\nindex 3f2383a12c..f1dc673838 100644\n--- a/Documentation/Makefile\n+++ b/Documentation/Makefile\n@@ -110,6 +110,7 @@ TECH_DOCS += SubmittingPatches\n TECH_DOCS += ToolsForGit\n TECH_DOCS += technical/bitmap-format\n TECH_DOCS += technical/bundle-uri\n+TECH_DOCS += technical/git-std-lib\n TECH_DOCS += technical/hash-function-transition\n TECH_DOCS += technical/long-running-process-protocol\n TECH_DOCS += technical/multi-pack-index\ndiff --git a/Documentation/technical/git-std-lib.txt b/Documentation/technical/git-std-lib.txt\nnew file mode 100644\nindex 0000000000..3d9aa121ac\n--- /dev/null\n+++ b/Documentation/technical/git-std-lib.txt\n@@ -0,0 +1,170 @@\n+= Git Standard Library\n+\n+The Git Standard Library intends to serve as the foundational library\n+and root dependency that other libraries in Git will be built off of.\n+That is to say, suppose we have libraries X and Y; a user that wants to\n+use X and Y would need to include X, Y, and this Git Standard Library.\n+This does not mean that the Git Standard Library will be the only\n+possible root dependency in the future, but rather the most significant\n+and widely used one. Git itself is also built off of the Git Standard\n+Library.\n+\n+== Dependency graph in libified Git\n+\n+Before the introduction of the Git Standard Library, all objects defined\n+in the Git library are compiled and archived into a singular file,\n+libgit.a, which is then linked against by common-main.o with other\n+external dependencies and turned into the Git executable. In other\n+words, the Git executable has dependencies on libgit.a and a couple of\n+external libraries. The libfication of Git slightly alters this build\n+flow by separating out libgit.a into libgit.a and git-std-lib.a. \n+\n+With our current method of building Git, we can imagine the dependency\n+graph as such:\n+\n+\tGit\n+\t /\\\n+\t/ \\\n+ / \\\n+ libgit.a ext deps\n+\n+We want to separate out potential libraries from libgit.a and have\n+libgit.a depend on them, which would possibly look like:\n+\n+\t\tGit\n+\t\t/\\\n+\t / \\\n+\t / \\\n+\t libgit.a ext deps\n+\t /\\\n+\t / \\\n+\t / \\\n+object-store.a (other lib)\n+ | /\n+ | /\n+ | /\n+ | /\n+ | /\n+ | /\n+ | /\n+git-std-lib.a\n+\n+Instead of containing all objects in Git, libgit.a would contain objects\n+that are not built by libraries it links against. Consequently, if\n+someone wanted a custom build of Git with a custom implementation of the\n+object store, they would only have to swap out object-store.a rather\n+than do a hard fork of Git.\n+\n+== Rationale behind Git Standard Library\n+\n+The rationale behind the selected object files in the Git Standard\n+Library is the result of two observations within the Git\n+codebase:\n+ 1. every file includes git-compat-util.h which defines functions\n+ in a couple of different files\n+ 2. wrapper.c + usage.c have difficult-to-separate circular\n+ dependencies with each other and other files.\n+\n+=== Ubiquity of git-compat-util.h and circular dependencies\n+\n+Every file in the Git codebase includes git-compat-util.h. It serves as\n+\"a compatibility aid that isolates the knowledge of platform specific\n+inclusion order and what feature macros to define before including which\n+system header\" (Junio[1]). Since every file includes git-compat-util.h,\n+and git-compat-util.h includes wrapper.h and usage.h, it would make\n+sense for wrapper.c and usage.c to be a part of the root library. They\n+have difficult to separate circular dependencies with each other so it\n+would impractical for them to be independent libraries. Wrapper.c has\n+dependencies on parse.c, abspath.c, strbuf.c, which in turn also have\n+dependencies on usage.c and wrapper.c - more circular dependencies.\n+\n+=== Tradeoff between swappability and refactoring\n+\n+From the above dependency graph, we can see that git-std-lib.a could be\n+many smaller libraries rather than a singular library. So why choose a\n+singular library when multiple libraries can be individually easier to\n+swap and are more modular? A singular library requires less work to\n+separate out circular dependencies within itself so it becomes a\n+tradeoff question between work and reward. While there may be a point in\n+the future where a file like usage.c would want its own library so that\n+someone can have custom die() or error(), the work required to refactor\n+out the circular dependencies in some files would be enormous due to\n+their ubiquity so therefore I believe it is not worth the tradeoff\n+currently. Additionally, we can in the future choose to do this refactor\n+and change the API for the library if there becomes enough of a reason\n+to do so (remember we are avoiding promising stability of the interfaces\n+of those libraries).\n+\n+=== Reuse of compatibility functions in git-compat-util.h\n+\n+Most functions defined in git-compat-util.h are implemented in compat/\n+and have dependencies limited to strbuf.h and wrapper.h so they can be\n+easily included in git-std-lib.a, which as a root dependency means that\n+higher level libraries do not have to worry about compatibility files in\n+compat/. The rest of the functions defined in git-compat-util.h are\n+implemented in top level files and are hidden behind\n+an #ifdef if their implementation is not in git-std-lib.a.\n+\n+=== Rationale summary\n+\n+The Git Standard Library allows us to get the libification ball rolling\n+with other libraries in Git. By not spending many more months attempting\n+to refactor difficult circular dependencies and instead spending that\n+time getting to a state where we can test out swapping a library out\n+such as config or object store, we can prove the viability of Git\n+libification on a much faster time scale. Additionally the code cleanups\n+that have happened so far have been minor and beneficial for the\n+codebase. It is probable that making large movements would negatively\n+affect code clarity.\n+\n+== Git Standard Library boundary\n+\n+While I have described above some useful heuristics for identifying\n+potential candidates for git-std-lib.a, a standard library should not\n+have a shaky definition for what belongs in it.\n+\n+ - Low-level files (aka operates only on other primitive types) that are\n+ used everywhere within the codebase (wrapper.c, usage.c, strbuf.c)\n+ - Dependencies that are low-level and widely used\n+ (abspath.c, date.c, hex-ll.c, parse.c, utf8.c)\n+ - low-level git/* files with functions defined in git-compat-util.h\n+ (ctype.c)\n+ - compat/*\n+\n+There are other files that might fit this definition, but that does not\n+mean it should belong in git-std-lib.a. Those files should start as\n+their own separate library since any file added to git-std-lib.a loses\n+its flexibility of being easily swappable.\n+\n+Wrapper.c and usage.c have dependencies on pager and trace2 that are\n+possible to remove at the cost of sacrificing the ability for standard Git\n+to be able to trace functions in those files and other files in git-std-lib.a.\n+In order for git-std-lib.a to compile with those dependencies, stubbed out\n+versions of those files are implemented and swapped in during compilation time\n+(see STUB_LIB_OBJS in the Makefile).\n+\n+== Files inside of Git Standard Library\n+\n+The set of files in git-std-lib.a can be found in STD_LIB_OBJS and COMPAT_OBJS\n+in the Makefile.\n+\n+When these files are compiled together with the files in STUB_LIB_OBJS (or\n+user-provided files that provide the same functions), they form a complete\n+library.\n+\n+== Pitfalls\n+\n+There are a small amount of files under compat/* that have dependencies\n+not inside of git-std-lib.a. While those functions are not called on\n+Linux, other OSes might call those problematic functions. I don't see\n+this as a major problem, just moreso an observation that libification in\n+general may also require some minor compatibility work in the future.\n+\n+== Testing\n+\n+Unit tests should catch any breakages caused by changes to files in\n+git-std-lib.a (i.e. introduction of a out of scope dependency) and new\n+functions introduced to git-std-lib.a will require unit tests written\n+for them.\n+\n+[1] https://lore.kernel.org/git/xmqqwn17sydw.fsf@gitster.g/\ndiff --git a/Makefile b/Makefile\nindex 4e255c81f2..d37ea9d34b 100644\n--- a/Makefile\n+++ b/Makefile\n@@ -669,6 +669,8 @@ FUZZ_PROGRAMS =\n GIT_OBJS =\n LIB_OBJS =\n SCALAR_OBJS =\n+STD_LIB_OBJS =\n+STUB_LIB_OBJS =\n OBJECTS =\n OTHER_PROGRAMS =\n PROGRAM_OBJS =\n@@ -923,6 +925,8 @@ TEST_SHELL_PATH = $(SHELL_PATH)\n \n LIB_FILE = libgit.a\n XDIFF_LIB = xdiff/lib.a\n+STD_LIB_FILE = git-std-lib.a\n+STUB_LIB_FILE = git-stub-lib.a\n REFTABLE_LIB = reftable/libreftable.a\n REFTABLE_TEST_LIB = reftable/libreftable_test.a\n \n@@ -962,7 +966,6 @@ COCCI_SOURCES = $(filter-out $(THIRD_PARTY_SOURCES),$(FOUND_C_SOURCES))\n \n LIB_H = $(FOUND_H_SOURCES)\n \n-LIB_OBJS += abspath.o\n LIB_OBJS += add-interactive.o\n LIB_OBJS += add-patch.o\n LIB_OBJS += advice.o\n@@ -1004,8 +1007,6 @@ LIB_OBJS += convert.o\n LIB_OBJS += copy.o\n LIB_OBJS += credential.o\n LIB_OBJS += csum-file.o\n-LIB_OBJS += ctype.o\n-LIB_OBJS += date.o\n LIB_OBJS += decorate.o\n LIB_OBJS += delta-islands.o\n LIB_OBJS += diagnose.o\n@@ -1046,7 +1047,6 @@ LIB_OBJS += hash-lookup.o\n LIB_OBJS += hashmap.o\n LIB_OBJS += help.o\n LIB_OBJS += hex.o\n-LIB_OBJS += hex-ll.o\n LIB_OBJS += hook.o\n LIB_OBJS += ident.o\n LIB_OBJS += json-writer.o\n@@ -1097,7 +1097,6 @@ LIB_OBJS += pack-write.o\n LIB_OBJS += packfile.o\n LIB_OBJS += pager.o\n LIB_OBJS += parallel-checkout.o\n-LIB_OBJS += parse.o\n LIB_OBJS += parse-options-cb.o\n LIB_OBJS += parse-options.o\n LIB_OBJS += patch-delta.o\n@@ -1152,7 +1151,6 @@ LIB_OBJS += sparse-index.o\n LIB_OBJS += split-index.o\n LIB_OBJS += stable-qsort.o\n LIB_OBJS += statinfo.o\n-LIB_OBJS += strbuf.o\n LIB_OBJS += streaming.o\n LIB_OBJS += string-list.o\n LIB_OBJS += strmap.o\n@@ -1189,21 +1187,32 @@ LIB_OBJS += unpack-trees.o\n LIB_OBJS += upload-pack.o\n LIB_OBJS += url.o\n LIB_OBJS += urlmatch.o\n-LIB_OBJS += usage.o\n LIB_OBJS += userdiff.o\n-LIB_OBJS += utf8.o\n LIB_OBJS += varint.o\n LIB_OBJS += version.o\n LIB_OBJS += versioncmp.o\n LIB_OBJS += walker.o\n LIB_OBJS += wildmatch.o\n LIB_OBJS += worktree.o\n-LIB_OBJS += wrapper.o\n LIB_OBJS += write-or-die.o\n LIB_OBJS += ws.o\n LIB_OBJS += wt-status.o\n LIB_OBJS += xdiff-interface.o\n \n+STD_LIB_OBJS += abspath.o\n+STD_LIB_OBJS += ctype.o\n+STD_LIB_OBJS += date.o\n+STD_LIB_OBJS += hex-ll.o\n+STD_LIB_OBJS += parse.o\n+STD_LIB_OBJS += strbuf.o\n+STD_LIB_OBJS += usage.o\n+STD_LIB_OBJS += utf8.o\n+STD_LIB_OBJS += wrapper.o\n+\n+STUB_LIB_OBJS += stubs/trace2.o\n+STUB_LIB_OBJS += stubs/pager.o\n+STUB_LIB_OBJS += stubs/misc.o\n+\n BUILTIN_OBJS += builtin/add.o\n BUILTIN_OBJS += builtin/am.o\n BUILTIN_OBJS += builtin/annotate.o\n@@ -1352,7 +1361,7 @@ UNIT_TEST_OBJS = $(patsubst %,$(UNIT_TEST_DIR)/%.o,$(UNIT_TEST_PROGRAMS))\n UNIT_TEST_OBJS += $(UNIT_TEST_DIR)/test-lib.o\n \n # xdiff and reftable libs may in turn depend on what is in libgit.a\n-GITLIBS = common-main.o $(LIB_FILE) $(XDIFF_LIB) $(REFTABLE_LIB) $(LIB_FILE)\n+GITLIBS = common-main.o $(STD_LIB_FILE) $(LIB_FILE) $(XDIFF_LIB) $(REFTABLE_LIB) $(LIB_FILE)\n EXTLIBS =\n \n GIT_USER_AGENT = git/$(GIT_VERSION)\n@@ -2693,6 +2702,8 @@ OBJECTS += $(XDIFF_OBJS)\n OBJECTS += $(FUZZ_OBJS)\n OBJECTS += $(REFTABLE_OBJS) $(REFTABLE_TEST_OBJS)\n OBJECTS += $(UNIT_TEST_OBJS)\n+OBJECTS += $(STD_LIB_OBJS)\n+OBJECTS += $(STUB_LIB_OBJS)\n \n ifndef NO_CURL\n \tOBJECTS += http.o http-walker.o remote-curl.o\n@@ -3686,7 +3697,7 @@ clean: profile-clean coverage-clean cocciclean\n \t$(RM) git.res\n \t$(RM) $(OBJECTS)\n \t$(RM) headless-git.o\n-\t$(RM) $(LIB_FILE) $(XDIFF_LIB) $(REFTABLE_LIB) $(REFTABLE_TEST_LIB)\n+\t$(RM) $(LIB_FILE) $(XDIFF_LIB) $(REFTABLE_LIB) $(REFTABLE_TEST_LIB) $(STD_LIB_FILE) $(STUB_LIB_FILE)\n \t$(RM) $(ALL_PROGRAMS) $(SCRIPT_LIB) $(BUILT_INS) $(OTHER_PROGRAMS)\n \t$(RM) $(TEST_PROGRAMS)\n \t$(RM) $(FUZZ_PROGRAMS)\n@@ -3878,3 +3889,18 @@ $(UNIT_TEST_PROGS): $(UNIT_TEST_BIN)/%$X: $(UNIT_TEST_DIR)/%.o $(UNIT_TEST_DIR)/\n build-unit-tests: $(UNIT_TEST_PROGS)\n unit-tests: $(UNIT_TEST_PROGS)\n \t$(MAKE) -C t/ unit-tests\n+\n+### Libified Git rules\n+\n+# git-std-lib.a\n+# Programs other than git should compile this with\n+# make NO_GETTEXT=YesPlease git-std-lib.a\n+# and link against git-stub-lib.a (if the default no-op functionality is fine)\n+# or a custom .a file with the same interface as git-stub-lib.a (if custom\n+# functionality is needed) as well.\n+$(STD_LIB_FILE): $(STD_LIB_OBJS) $(COMPAT_OBJS)\n+\t$(QUIET_AR)$(RM) $@ && $(AR) $(ARFLAGS) $@ $^\n+\n+# git-stub-lib.a\n+$(STUB_LIB_FILE): $(STUB_LIB_OBJS)\n+\t$(QUIET_AR)$(RM) $@ && $(AR) $(ARFLAGS) $@ $^\ndiff --git a/stubs/misc.c b/stubs/misc.c\nnew file mode 100644\nindex 0000000000..92da76fd46\n--- /dev/null\n+++ b/stubs/misc.c\n@@ -0,0 +1,33 @@\n+#include \n+#include \n+\n+#ifndef NO_GETTEXT\n+/*\n+ * NEEDSWORK: This is enough to link our unit tests against\n+ * git-std-lib.a built with gettext support. We don't really support\n+ * programs other than git using git-std-lib.a with gettext support\n+ * yet. To do that we need to start using dgettext() rather than\n+ * gettext() in our code.\n+ */\n+int git_gettext_enabled = 0;\n+#endif\n+\n+int common_exit(const char *file, int line, int code);\n+\n+int common_exit(const char *file, int line, int code)\n+{\n+\texit(code);\n+}\n+\n+#if !defined(__MINGW32__) && !defined(_MSC_VER)\n+int lstat_cache_aware_rmdir(const char *path);\n+\n+int lstat_cache_aware_rmdir(const char *path)\n+{\n+\t/*\n+\t * This function should not be called by programs linked\n+\t * against git-stub-lib.a\n+\t */\n+\tassert(0);\n+}\n+#endif\ndiff --git a/stubs/pager.c b/stubs/pager.c\nnew file mode 100644\nindex 0000000000..4f575cada7\n--- /dev/null\n+++ b/stubs/pager.c\n@@ -0,0 +1,6 @@\n+#include \"pager.h\"\n+\n+int pager_in_use(void)\n+{\n+\treturn 0;\n+}\ndiff --git a/stubs/trace2.c b/stubs/trace2.c\nnew file mode 100644\nindex 0000000000..7d89482228\n--- /dev/null\n+++ b/stubs/trace2.c\n@@ -0,0 +1,27 @@\n+#include \"git-compat-util.h\"\n+#include \"trace2.h\"\n+\n+struct child_process { int stub; };\n+struct repository { int stub; };\n+struct json_writer { int stub; };\n+\n+void trace2_region_enter_fl(const char *file, int line, const char *category,\n+\t\t\t const char *label, const struct repository *repo, ...) { }\n+void trace2_region_leave_fl(const char *file, int line, const char *category,\n+\t\t\t const char *label, const struct repository *repo, ...) { }\n+void trace2_data_string_fl(const char *file, int line, const char *category,\n+\t\t\t const struct repository *repo, const char *key,\n+\t\t\t const char *value) { }\n+void trace2_cmd_ancestry_fl(const char *file, int line, const char **parent_names) { }\n+void trace2_cmd_error_va_fl(const char *file, int line, const char *fmt,\n+\t\t\t va_list ap) { }\n+void trace2_cmd_name_fl(const char *file, int line, const char *name) { }\n+void trace2_thread_start_fl(const char *file, int line,\n+\t\t\t const char *thread_base_name) { }\n+void trace2_thread_exit_fl(const char *file, int line) { }\n+void trace2_data_intmax_fl(const char *file, int line, const char *category,\n+\t\t\t const struct repository *repo, const char *key,\n+\t\t\t intmax_t value) { }\n+int trace2_is_enabled(void) { return 0; }\n+void trace2_counter_add(enum trace2_counter_id cid, uint64_t value) { }\n+void trace2_collect_process_info(enum trace2_process_info_reason reason) { }\n-- \n2.44.0.rc0.258.g7320e95886-goog\n\n"},{"id":"489178","messageId":"20240222175033.1489723-4-calvinwan@google.com","threadId":"59921","inReplyTo":"cover.1696021277.git.jonathantanmy@google.com","subject":"[PATCH v5 3/3] test-stdlib: show that git-std-lib is independent","fromName":"Calvin Wan","fromEmail":"calvinwan@google.com","sentAt":"2024-02-22T17:50:33Z","receivedAt":"2024-02-22T17:51:01Z","isPatch":true,"sender":{"key":"calvinwan@google.com","avatar":"https://avatars.githubusercontent.com/u/92547554?v=4"},"body":"Add a test file that calls some functions defined in git-std-lib.a\nobject files to showcase that they do not reference missing objects and\nthat, together with git-stub-lib.a, git-std-lib.a can stand on its own.\n\nAs described in test-stdlib.c, this can probably be removed once we have\nunit tests.\n\nThe variable TEST_PROGRAMS is moved lower in the Makefile after\nNO_POSIX_GOODIES is defined in config.make.uname. TEST_PROGRAMS isn't\nused earlier than that so this change should be safe.\n\nSigned-off-by: Calvin Wan \nHelped-by: Phillip Wood \nSigned-off-by: Jonathan Tan \n---\n Makefile | 23 +++-\n strbuf.h | 2 +\n stubs/misc.c | 1 +\n t/helper/.gitignore | 1 +\n t/helper/test-stdlib.c | 266 +++++++++++++++++++++++++++++++++++++++++\n t/t0082-std-lib.sh | 11 ++\n 6 files changed, 299 insertions(+), 5 deletions(-)\n create mode 100644 t/helper/test-stdlib.c\n create mode 100755 t/t0082-std-lib.sh\n\ndiff --git a/Makefile b/Makefile\nindex d37ea9d34b..1d762ce13a 100644\n--- a/Makefile\n+++ b/Makefile\n@@ -870,9 +870,7 @@ TEST_BUILTINS_OBJS += test-xml-encode.o\n # Do not add more tests here unless they have extra dependencies. Add\n # them in TEST_BUILTINS_OBJS above.\n TEST_PROGRAMS_NEED_X += test-fake-ssh\n-TEST_PROGRAMS_NEED_X += test-tool\n-\n-TEST_PROGRAMS = $(patsubst %,t/helper/%$X,$(TEST_PROGRAMS_NEED_X))\n+TEST_PROGRAMS_NEED_X += $(info tpnxnpg=$(NO_POSIX_GOODIES))test-tool\n \n # List built-in command $C whose implementation cmd_$C() is not in\n # builtin/$C.o but is linked in as part of some other command.\n@@ -2678,6 +2676,16 @@ REFTABLE_TEST_OBJS += reftable/stack_test.o\n REFTABLE_TEST_OBJS += reftable/test_framework.o\n REFTABLE_TEST_OBJS += reftable/tree_test.o\n \n+ifndef NO_POSIX_GOODIES\n+TEST_PROGRAMS_NEED_X += test-stdlib\n+MY_VAR = not_else\n+$(info insideifndefnpg=$(NO_POSIX_GOODIES))\n+else\n+MY_VAR = else\n+endif\n+\n+TEST_PROGRAMS = $(info tptpnx=$(TEST_PROGRAMS_NEED_X) myvar=$(MY_VAR))$(patsubst %,t/helper/%$X,$(TEST_PROGRAMS_NEED_X))\n+\n TEST_OBJS := $(patsubst %$X,%.o,$(TEST_PROGRAMS)) $(patsubst %,t/helper/%,$(TEST_BUILTINS_OBJS))\n \n .PHONY: test-objs\n@@ -3204,7 +3212,11 @@ GIT-PYTHON-VARS: FORCE\n fi\n endif\n \n-test_bindir_programs := $(patsubst %,bin-wrappers/%,$(BINDIR_PROGRAMS_NEED_X) $(BINDIR_PROGRAMS_NO_X) $(TEST_PROGRAMS_NEED_X))\n+test_bindir_programs := $(info tbptpnx=$(TEST_PROGRAMS_NEED_X))$(patsubst %,bin-wrappers/%,$(BINDIR_PROGRAMS_NEED_X) $(BINDIR_PROGRAMS_NO_X) $(TEST_PROGRAMS_NEED_X))\n+\n+t/helper/test-stdlib$X: t/helper/test-stdlib.o GIT-LDFLAGS $(STD_LIB_FILE) $(STUB_LIB_FILE) $(GITLIBS)\n+\t$(QUIET_LINK)$(CC) $(ALL_CFLAGS) -o $@ $(ALL_LDFLAGS) \\\n+\t\t$< $(STD_LIB_FILE) $(STUB_LIB_FILE) $(EXTLIBS)\n \n all:: $(TEST_PROGRAMS) $(test_bindir_programs) $(UNIT_TEST_PROGS)\n \n@@ -3635,7 +3647,8 @@ ifneq ($(INCLUDE_DLLS_IN_ARTIFACTS),)\n OTHER_PROGRAMS += $(shell echo *.dll t/helper/*.dll t/unit-tests/bin/*.dll)\n endif\n \n-artifacts-tar:: $(ALL_COMMANDS_TO_INSTALL) $(SCRIPT_LIB) $(OTHER_PROGRAMS) \\\n+# Added an info for debugging\n+artifacts-tar:: $(info npg=$(NO_POSIX_GOODIES) cc=$(COMPAT_CFLAGS) tp=$(TEST_PROGRAMS))$(ALL_COMMANDS_TO_INSTALL) $(SCRIPT_LIB) $(OTHER_PROGRAMS) \\\n \t\tGIT-BUILD-OPTIONS $(TEST_PROGRAMS) $(test_bindir_programs) \\\n \t\t$(UNIT_TEST_PROGS) $(MOFILES)\n \t$(QUIET_SUBDIR0)templates $(QUIET_SUBDIR1) \\\ndiff --git a/strbuf.h b/strbuf.h\nindex e959caca87..f775416307 100644\n--- a/strbuf.h\n+++ b/strbuf.h\n@@ -1,6 +1,8 @@\n #ifndef STRBUF_H\n #define STRBUF_H\n \n+#include \"git-compat-util.h\"\n+\n /*\n * NOTE FOR STRBUF DEVELOPERS\n *\ndiff --git a/stubs/misc.c b/stubs/misc.c\nindex 92da76fd46..8d80581e39 100644\n--- a/stubs/misc.c\n+++ b/stubs/misc.c\n@@ -9,6 +9,7 @@\n * yet. To do that we need to start using dgettext() rather than\n * gettext() in our code.\n */\n+#include \"gettext.h\"\n int git_gettext_enabled = 0;\n #endif\n \ndiff --git a/t/helper/.gitignore b/t/helper/.gitignore\nindex 8c2ddcce95..5cec3b357f 100644\n--- a/t/helper/.gitignore\n+++ b/t/helper/.gitignore\n@@ -1,2 +1,3 @@\n /test-tool\n /test-fake-ssh\n+/test-stdlib\ndiff --git a/t/helper/test-stdlib.c b/t/helper/test-stdlib.c\nnew file mode 100644\nindex 0000000000..460b472fb4\n--- /dev/null\n+++ b/t/helper/test-stdlib.c\n@@ -0,0 +1,266 @@\n+#include \"git-compat-util.h\"\n+#include \"abspath.h\"\n+#include \"hex-ll.h\"\n+#include \"parse.h\"\n+#include \"strbuf.h\"\n+#include \"string-list.h\"\n+\n+/*\n+ * Calls all functions from git-std-lib\n+ * Some inline/trivial functions are skipped\n+ *\n+ * NEEDSWORK: The purpose of this file is to show that an executable can be\n+ * built with git-std-lib.a and git-stub-lib.a, and then executed. If there\n+ * is another executable that demonstrates this (for example, a unit test that\n+ * takes the form of an executable compiled with git-std-lib.a and git-stub-\n+ * lib.a), this file can be removed.\n+ */\n+\n+static void abspath_funcs(void) {\n+\tstruct strbuf sb = STRBUF_INIT;\n+\n+\tfprintf(stderr, \"calling abspath functions\\n\");\n+\tis_directory(\"foo\");\n+\tstrbuf_realpath(&sb, \"foo\", 0);\n+\tstrbuf_realpath_forgiving(&sb, \"foo\", 0);\n+\treal_pathdup(\"foo\", 0);\n+\tabsolute_path(\"foo\");\n+\tabsolute_pathdup(\"foo\");\n+\tprefix_filename(\"foo/\", \"bar\");\n+\tprefix_filename_except_for_dash(\"foo/\", \"bar\");\n+\tis_absolute_path(\"foo\");\n+\tstrbuf_add_absolute_path(&sb, \"foo\");\n+\tstrbuf_add_real_path(&sb, \"foo\");\n+}\n+\n+static void hex_ll_funcs(void) {\n+\tunsigned char c;\n+\n+\tfprintf(stderr, \"calling hex-ll functions\\n\");\n+\n+\thexval('c');\n+\thex2chr(\"A1\");\n+\thex_to_bytes(&c, \"A1\", 1);\n+}\n+\n+static void parse_funcs(void) {\n+\tintmax_t foo;\n+\tssize_t foo1 = -1;\n+\tunsigned long foo2;\n+\tint foo3;\n+\tint64_t foo4;\n+\n+\tfprintf(stderr, \"calling parse functions\\n\");\n+\n+\tgit_parse_signed(\"42\", &foo, maximum_signed_value_of_type(int));\n+\tgit_parse_ssize_t(\"42\", &foo1);\n+\tgit_parse_ulong(\"42\", &foo2);\n+\tgit_parse_int(\"42\", &foo3);\n+\tgit_parse_int64(\"42\", &foo4);\n+\tgit_parse_maybe_bool(\"foo\");\n+\tgit_parse_maybe_bool_text(\"foo\");\n+\tgit_env_bool(\"foo\", 1);\n+\tgit_env_ulong(\"foo\", 1);\n+}\n+\n+static int allow_unencoded_fn(char ch) {\n+\treturn 0;\n+}\n+\n+static void strbuf_funcs(void) {\n+\tstruct strbuf *sb = xmalloc(sizeof(*sb));\n+\tstruct strbuf *sb2 = xmalloc(sizeof(*sb2));\n+\tstruct strbuf sb3 = STRBUF_INIT;\n+\tstruct string_list list = STRING_LIST_INIT_NODUP;\n+\tint fd = open(\"/dev/null\", O_RDONLY);\n+\n+\tfprintf(stderr, \"calling strbuf functions\\n\");\n+\n+\tfprintf(stderr, \"at line %d\\n\", __LINE__);\n+\tstarts_with(\"foo\", \"bar\");\n+\tfprintf(stderr, \"at line %d\\n\", __LINE__);\n+\tistarts_with(\"foo\", \"bar\");\n+\tfprintf(stderr, \"at line %d\\n\", __LINE__);\n+\tstrbuf_init(sb, 0);\n+\tfprintf(stderr, \"at line %d\\n\", __LINE__);\n+\tstrbuf_init(sb2, 0);\n+\tfprintf(stderr, \"at line %d\\n\", __LINE__);\n+\tstrbuf_release(sb);\n+\tfprintf(stderr, \"at line %d\\n\", __LINE__);\n+\tstrbuf_attach(sb, strbuf_detach(sb, NULL), 0, 0);\n+\tfprintf(stderr, \"at line %d\\n\", __LINE__);\n+\tstrbuf_swap(sb, sb2);\n+\tfprintf(stderr, \"at line %d\\n\", __LINE__);\n+\tstrbuf_setlen(sb, 0);\n+\tfprintf(stderr, \"at line %d\\n\", __LINE__);\n+\tstrbuf_trim(sb);\n+\tfprintf(stderr, \"at line %d\\n\", __LINE__);\n+\tstrbuf_trim_trailing_dir_sep(sb);\n+\tfprintf(stderr, \"at line %d\\n\", __LINE__);\n+\tstrbuf_trim_trailing_newline(sb);\n+\tfprintf(stderr, \"at line %d\\n\", __LINE__);\n+\tstrbuf_reencode(sb, \"foo\", \"bar\");\n+\tfprintf(stderr, \"at line %d\\n\", __LINE__);\n+\tstrbuf_tolower(sb);\n+\tfprintf(stderr, \"at line %d\\n\", __LINE__);\n+\tstrbuf_add_separated_string_list(sb, \" \", &list);\n+\tfprintf(stderr, \"at line %d\\n\", __LINE__);\n+\tstrbuf_list_free(strbuf_split_buf(\"foo bar\", 8, ' ', -1));\n+\tfprintf(stderr, \"at line %d\\n\", __LINE__);\n+\tstrbuf_cmp(sb, sb2);\n+\tfprintf(stderr, \"at line %d\\n\", __LINE__);\n+\tstrbuf_addch(sb, 1);\n+\tfprintf(stderr, \"at line %d\\n\", __LINE__);\n+\tstrbuf_splice(sb, 0, 1, \"foo\", 3);\n+\tfprintf(stderr, \"at line %d\\n\", __LINE__);\n+\tstrbuf_insert(sb, 0, \"foo\", 3);\n+\tfprintf(stderr, \"at line %d\\n\", __LINE__);\n+\tstrbuf_insertf(sb, 0, \"%s\", \"foo\");\n+\tfprintf(stderr, \"at line %d\\n\", __LINE__);\n+\tstrbuf_remove(sb, 0, 1);\n+\tfprintf(stderr, \"at line %d\\n\", __LINE__);\n+\tstrbuf_add(sb, \"foo\", 3);\n+\tfprintf(stderr, \"at line %d\\n\", __LINE__);\n+\tstrbuf_addbuf(sb, sb2);\n+\tfprintf(stderr, \"at line %d\\n\", __LINE__);\n+\tstrbuf_join_argv(sb, 0, NULL, ' ');\n+\tfprintf(stderr, \"at line %d\\n\", __LINE__);\n+\tstrbuf_addchars(sb, 1, 1);\n+\tfprintf(stderr, \"at line %d\\n\", __LINE__);\n+\tstrbuf_addstr(sb, \"foo\");\n+\tfprintf(stderr, \"at line %d\\n\", __LINE__);\n+\tstrbuf_add_commented_lines(sb, \"foo\", 3, '#');\n+\tfprintf(stderr, \"at line %d\\n\", __LINE__);\n+\tstrbuf_commented_addf(sb, '#', \"%s\", \"foo\");\n+\tfprintf(stderr, \"at line %d\\n\", __LINE__);\n+\tstrbuf_addbuf_percentquote(sb, &sb3);\n+\tfprintf(stderr, \"at line %d\\n\", __LINE__);\n+\tstrbuf_add_percentencode(sb, \"foo\", STRBUF_ENCODE_SLASH);\n+\tfprintf(stderr, \"at line %d\\n\", __LINE__);\n+\tstrbuf_fread(sb, 0, stdin);\n+\tfprintf(stderr, \"at line %d\\n\", __LINE__);\n+\tstrbuf_read(sb, fd, 0);\n+\tfprintf(stderr, \"at line %d\\n\", __LINE__);\n+\tstrbuf_read_once(sb, fd, 0);\n+\tfprintf(stderr, \"at line %d\\n\", __LINE__);\n+\tstrbuf_write(sb, stderr);\n+\tfprintf(stderr, \"at line %d\\n\", __LINE__);\n+\tstrbuf_readlink(sb, \"/dev/null\", 0);\n+\tfprintf(stderr, \"at line %d\\n\", __LINE__);\n+\tstrbuf_getcwd(sb);\n+\tfprintf(stderr, \"at line %d\\n\", __LINE__);\n+\tstrbuf_getwholeline(sb, stderr, '\\n');\n+\tfprintf(stderr, \"at line %d\\n\", __LINE__);\n+\tstrbuf_appendwholeline(sb, stderr, '\\n');\n+\tfprintf(stderr, \"at line %d\\n\", __LINE__);\n+\tstrbuf_getline(sb, stderr);\n+\tfprintf(stderr, \"at line %d\\n\", __LINE__);\n+\tstrbuf_getline_lf(sb, stderr);\n+\tfprintf(stderr, \"at line %d\\n\", __LINE__);\n+\tstrbuf_getline_nul(sb, stderr);\n+\tfprintf(stderr, \"at line %d\\n\", __LINE__);\n+\tstrbuf_getwholeline_fd(sb, fd, '\\n');\n+\tfprintf(stderr, \"at line %d\\n\", __LINE__);\n+\tstrbuf_read_file(sb, \"/dev/null\", 0);\n+\tfprintf(stderr, \"at line %d\\n\", __LINE__);\n+\tstrbuf_add_lines(sb, \"foo\", \"bar\", 0);\n+\tfprintf(stderr, \"at line %d\\n\", __LINE__);\n+\tstrbuf_addstr_xml_quoted(sb, \"foo\");\n+\tfprintf(stderr, \"at line %d\\n\", __LINE__);\n+\tstrbuf_addstr_urlencode(sb, \"foo\", allow_unencoded_fn);\n+\tfprintf(stderr, \"at line %d\\n\", __LINE__);\n+\tstrbuf_humanise_bytes(sb, 42);\n+\tfprintf(stderr, \"at line %d\\n\", __LINE__);\n+\tstrbuf_humanise_rate(sb, 42);\n+\tfprintf(stderr, \"at line %d\\n\", __LINE__);\n+\tprintf_ln(\"%s\", sb->buf);\n+\tfprintf(stderr, \"at line %d\\n\", __LINE__);\n+\tfprintf_ln(stderr, \"%s\", sb->buf);\n+\tfprintf(stderr, \"at line %d\\n\", __LINE__);\n+\txstrdup_tolower(\"foo\");\n+\tfprintf(stderr, \"at line %d\\n\", __LINE__);\n+\txstrdup_toupper(\"foo\");\n+\tfprintf(stderr, \"at line %d\\n\", __LINE__);\n+\txstrfmt(\"%s\", \"foo\");\n+\tfprintf(stderr, \"at line %d\\n\", __LINE__);\n+}\n+\n+static void error_builtin(const char *err, va_list params) {}\n+static void warn_builtin(const char *err, va_list params) {}\n+\n+static void usage_funcs(void) {\n+\tfprintf(stderr, \"calling usage functions\\n\");\n+\terror(\"foo\");\n+\terror_errno(\"foo\");\n+\tdie_message(\"foo\");\n+\tdie_message_errno(\"foo\");\n+\twarning(\"foo\");\n+\twarning_errno(\"foo\");\n+\n+\tget_die_message_routine();\n+\tset_error_routine(error_builtin);\n+\tget_error_routine();\n+\tset_warn_routine(warn_builtin);\n+\tget_warn_routine();\n+}\n+\n+static void wrapper_funcs(void) {\n+\tint tmp;\n+\tvoid *ptr = xmalloc(1);\n+\tint fd = open(\"/dev/null\", O_RDONLY);\n+\tstruct strbuf sb = STRBUF_INIT;\n+\tint mode = 0444;\n+\tchar host[PATH_MAX], path[PATH_MAX], path1[PATH_MAX];\n+\txsnprintf(path, sizeof(path), \"out-XXXXXX\");\n+\txsnprintf(path1, sizeof(path1), \"out-XXXXXX\");\n+\n+\tfprintf(stderr, \"calling wrapper functions\\n\");\n+\n+\txstrdup(\"foo\");\n+\txmalloc(1);\n+\txmallocz(1);\n+\txmallocz_gently(1);\n+\txmemdupz(\"foo\", 3);\n+\txstrndup(\"foo\", 3);\n+\txrealloc(ptr, 2);\n+\txcalloc(1, 1);\n+\txsetenv(\"foo\", \"bar\", 0);\n+\txopen(\"/dev/null\", O_RDONLY);\n+\txread(fd, &sb, 1);\n+\txwrite(fd, &sb, 1);\n+\txpread(fd, &sb, 1, 0);\n+\txdup(fd);\n+\txfopen(\"/dev/null\", \"r\");\n+\txfdopen(fd, \"r\");\n+\ttmp = xmkstemp(path);\n+\tclose(tmp);\n+\tunlink(path);\n+\ttmp = xmkstemp_mode(path1, mode);\n+\tclose(tmp);\n+\tunlink(path1);\n+\txgetcwd();\n+\tfopen_for_writing(path);\n+\tfopen_or_warn(path, \"r\");\n+\txstrncmpz(\"foo\", \"bar\", 3);\n+\txgethostname(host, 3);\n+\ttmp = git_mkstemps_mode(path, 1, mode);\n+\tclose(tmp);\n+\tunlink(path);\n+\ttmp = git_mkstemp_mode(path, mode);\n+\tclose(tmp);\n+\tunlink(path);\n+\tread_in_full(fd, &sb, 1);\n+\twrite_in_full(fd, &sb, 1);\n+\tpread_in_full(fd, &sb, 1, 0);\n+}\n+\n+int main(int argc, const char **argv) {\n+\tabspath_funcs();\n+\thex_ll_funcs();\n+\tparse_funcs();\n+\tstrbuf_funcs();\n+\tusage_funcs();\n+\twrapper_funcs();\n+\tfprintf(stderr, \"all git-std-lib functions finished calling\\n\");\n+\treturn 0;\n+}\ndiff --git a/t/t0082-std-lib.sh b/t/t0082-std-lib.sh\nnew file mode 100755\nindex 0000000000..0d5a024deb\n--- /dev/null\n+++ b/t/t0082-std-lib.sh\n@@ -0,0 +1,11 @@\n+#!/bin/sh\n+\n+test_description='Test git-std-lib compilation'\n+\n+. ./test-lib.sh\n+\n+test_expect_success !WINDOWS 'stdlib-test compiles and runs' '\n+\ttest-stdlib\n+'\n+\n+test_done\n-- \n2.44.0.rc0.258.g7320e95886-goog\n\n"},{"id":"489185","messageId":"xmqqh6i0cgyo.fsf@gitster.g","threadId":"59921","inReplyTo":"20240222175033.1489723-2-calvinwan@google.com","subject":"Re: [PATCH v5 1/3] pager: include stdint.h because uintmax_t is used","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2024-02-22T21:43:59Z","receivedAt":"2024-02-22T21:44:05Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Calvin Wan writes:\n\n> From: Jonathan Tan \n>\n> pager.h uses uintmax_t but does not include stdint.h. Therefore, add\n> this include statement.\n>\n> This was discovered when writing a stub pager.c file.\n>\n> Signed-off-by: Jonathan Tan \n> Signed-off-by: Calvin Wan \n> ---\n> pager.h | 2 ++\n> 1 file changed, 2 insertions(+)\n>\n> diff --git a/pager.h b/pager.h\n> index b77433026d..015bca95e3 100644\n> --- a/pager.h\n> +++ b/pager.h\n> @@ -1,6 +1,8 @@\n> #ifndef PAGER_H\n> #define PAGER_H\n> \n> +#include \n> +\n> struct child_process;\n> \n> const char *git_pager(int stdout_is_tty);\n\nThis is not going in a sensible direction from our portability\nstandard's point of view.\n\nThe reason why we do not include these system headers directly to\nour source files, and instead make it a rule to include\n as the first header instead, is exactly because\nthere are curiosities in various platforms that Git wants to run on\nwhich system include headers give us the declarations for types and\nfunctions we rely on, in what order they must be included, and after\nwhat feature macros (the ones that give adjustment to what the\nsystem headers do, like _POSIX_C_SOURCE) are defined, etc.\n\nGiven that in , inclusion of is\nconditional behind some #ifdef's, it does not look like a sensible\nchange. It is not very likely for and to\ndeclare uintmax_t in an incompatible way, but on a platform where\n decides to include and use its\ndefinition of what uintmax_t is, we should follow the same choice\nand be consistent.\n\nIf there is a feature macro that affects sizes of the integer on a\nplatform, this patch will break it even more badly. Perhaps there\nis a platform whose C-library header requires you to define a\nfeature macro to use 64-bit, and we may define that feature macro\nin before including either or\n, but by including directly like the above\npatch does, only this file and the sources that include only this\nfile, refusing to include as everybody in the\nGit source tree should, will end up using different notion of what\nthe integral type with maximum width is from everybody else.\n\nWhat this patch _wants_ to do is of course sympathizable, and we\nhave \"make hdr-check\" rule to enforce \"a header must include the\nheaders that declare what it uses\", except that it lets the header\nfiles being tested assume that the things made available by\nincluding are always available.\n\nI think a sensible direction to go for libification purposes is to\nalso make sure that sources that are compiled into gitstdlib.a, and\nthe headers that makes what is in gitstdlib.a available, include the\n header file. There may be things declared in\nthe header that are _too_ specific to what ought\nto be linked into the final \"git\" binary and unwanted by library\nclients that are not \"git\" binary, and the right way to deal with it\nis to split into two parts, i.e. what makes\nsystem services available like its conditional inclusion of\n vs , definition of feature macros, order in\nwhich the current includes system headers, etc.,\nexcluding those that made you write this patch to avoid assuming\nthat the client code would have included before\n, would be the new . And everything\nelse will remain in , which will include the\n. The header for library clients would\ninclude instead, to still allow them to use the\nsame types as \"git\" binary itself that way.\n\n\n\n\n\n"},{"id":"489186","messageId":"xmqqr0h4b0ic.fsf@gitster.g","threadId":"59921","inReplyTo":"20240222175033.1489723-4-calvinwan@google.com","subject":"Re: [PATCH v5 3/3] test-stdlib: show that git-std-lib is independent","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2024-02-22T22:24:43Z","receivedAt":"2024-02-22T22:24:46Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Calvin Wan writes:\n\n> diff --git a/Makefile b/Makefile\n> index d37ea9d34b..1d762ce13a 100644\n> --- a/Makefile\n> +++ b/Makefile\n> @@ -870,9 +870,7 @@ TEST_BUILTINS_OBJS += test-xml-encode.o\n> # Do not add more tests here unless they have extra dependencies. Add\n> # them in TEST_BUILTINS_OBJS above.\n> TEST_PROGRAMS_NEED_X += test-fake-ssh\n> -TEST_PROGRAMS_NEED_X += test-tool\n> -\n> -TEST_PROGRAMS = $(patsubst %,t/helper/%$X,$(TEST_PROGRAMS_NEED_X))\n> +TEST_PROGRAMS_NEED_X += $(info tpnxnpg=$(NO_POSIX_GOODIES))test-tool\n\nIs this version meant to be ready for reviewing? $(info) used like\nthis does not look like a good fit for production code.\n\n> diff --git a/strbuf.h b/strbuf.h\n> index e959caca87..f775416307 100644\n> --- a/strbuf.h\n> +++ b/strbuf.h\n> @@ -1,6 +1,8 @@\n> #ifndef STRBUF_H\n> #define STRBUF_H\n> \n> +#include \"git-compat-util.h\"\n> +\n> /*\n> * NOTE FOR STRBUF DEVELOPERS\n> *\n\nThe same comment about header inclusion I made on [1/3] applies\nhere, too. I am open to hearing better ideas to handle system\nheaders, but my preference is to allow any and all headers assume\n (or its moral equivalent that may be stripped\ndown by moving non-essential things out) is already included, which\nin turn means those *.c files (like t/helper/test-stdlib.c we see\nbelow) would include (or its trimmed down\nversion) as the first header, before including .\n\nIn any case, this change, if we were to make it (and I do not think\nwe should), should be treated like the change to pager.h in [1/3],\ni.e. part of making the existing headers ready to be shared with the\n\"stdlib\" effort. It does not belong to this [3/3] step, where we\nare supposed to be demonstrating the use of \"stdlib\", which has\nbecome (minimally) usable with the steps before this one.\n\n> diff --git a/stubs/misc.c b/stubs/misc.c\n> index 92da76fd46..8d80581e39 100644\n> --- a/stubs/misc.c\n> +++ b/stubs/misc.c\n> @@ -9,6 +9,7 @@\n> * yet. To do that we need to start using dgettext() rather than\n> * gettext() in our code.\n> */\n> +#include \"gettext.h\"\n> int git_gettext_enabled = 0;\n> #endif\n\nThis change should have happened before this [3/3] step, whose point\nis to demonstrate \"stdlib\" that has already been made (minimally)\nusable with steps before this one.\n\n> diff --git a/t/helper/.gitignore b/t/helper/.gitignore\n> index 8c2ddcce95..5cec3b357f 100644\n> --- a/t/helper/.gitignore\n> +++ b/t/helper/.gitignore\n> @@ -1,2 +1,3 @@\n> /test-tool\n> /test-fake-ssh\n> +/test-stdlib\n> diff --git a/t/helper/test-stdlib.c b/t/helper/test-stdlib.c\n> new file mode 100644\n> index 0000000000..460b472fb4\n> --- /dev/null\n> +++ b/t/helper/test-stdlib.c\n> @@ -0,0 +1,266 @@\n> +#include \"git-compat-util.h\"\n> +#include \"abspath.h\"\n> +#include \"hex-ll.h\"\n> +#include \"parse.h\"\n> +#include \"strbuf.h\"\n> +#include \"string-list.h\"\n> +\n> +/*\n> + * Calls all functions from git-std-lib\n> + * Some inline/trivial functions are skipped\n> + *\n> + * NEEDSWORK: The purpose of this file is to show that an executable can be\n> + * built with git-std-lib.a and git-stub-lib.a, and then executed. If there\n> + * is another executable that demonstrates this (for example, a unit test that\n> + * takes the form of an executable compiled with git-std-lib.a and git-stub-\n> + * lib.a), this file can be removed.\n> + */\n\nOr alternatively, these \"random list of function calls\" can be\nturned into a more realistic test helpers in place. \"stdlib\"\nwill hopefully gain more coverage of the features of low level\nhelpers \"git\" binary proper uses, and I do not think it is\nfar-fetched to migrate the \"test-tool date\" subcommands all to not\nlink directly with \"libgit.a\" but with gitstdlib instead and the\nthings should work, right? Right now, the \"random list of function\ncalls\" do not do anything useful, but that does not have to be the\ncase. It should offer us more value to us than \"It links!\" ;-).\n\nHaving said that, the most valuable part in this [3/3] step is how\nthis t/helper/test-stdlib is linked, i.e. this part from the\nMakefile:\n\n> +t/helper/test-stdlib$X: t/helper/test-stdlib.o GIT-LDFLAGS $(STD_LIB_FILE) $(STUB_LIB_FILE) $(GITLIBS)\n> +\t$(QUIET_LINK)$(CC) $(ALL_CFLAGS) -o $@ $(ALL_LDFLAGS) \\\n> +\t\t$< $(STD_LIB_FILE) $(STUB_LIB_FILE) $(EXTLIBS)\n\nwhere we have no $(LIB_FILE) (aka libgit.a). Especially if we can\ngrow the capability in $(STD_LIB_FILE) without adding too much stuff\nto $(STUB_LIB_FILE), this is a major achievement. Very nice.\n\n"},{"id":"489271","messageId":"CAO_smVh6PyxbnXfo0K1aDjEFPc3jTF4X_grerkxNZJdQe8V3sg@mail.gmail.com","threadId":"59921","inReplyTo":"20240222175033.1489723-2-calvinwan@google.com","subject":"Re: [PATCH v5 1/3] pager: include stdint.h because uintmax_t is used","fromName":"Kyle Lippincott","fromEmail":"spectral@google.com","sentAt":"2024-02-24T01:33:02Z","receivedAt":"2024-02-24T01:33:19Z","isPatch":true,"sender":{"key":"spectral@google.com","avatar":"https://avatars.githubusercontent.com/u/6371650?v=4"},"body":"On Thu, Feb 22, 2024 at 9:51 AM Calvin Wan wrote:\n>\n> From: Jonathan Tan \n>\n> pager.h uses uintmax_t but does not include stdint.h. Therefore, add\n> this include statement.\n>\n> This was discovered when writing a stub pager.c file.\n>\n> Signed-off-by: Jonathan Tan \n> Signed-off-by: Calvin Wan \n> ---\n> pager.h | 2 ++\n> 1 file changed, 2 insertions(+)\n>\n> diff --git a/pager.h b/pager.h\n> index b77433026d..015bca95e3 100644\n> --- a/pager.h\n> +++ b/pager.h\n> @@ -1,6 +1,8 @@\n> #ifndef PAGER_H\n> #define PAGER_H\n>\n> +#include \n> +\n> struct child_process;\n>\n> const char *git_pager(int stdout_is_tty);\n> --\n> 2.44.0.rc0.258.g7320e95886-goog\n>\n>\n\nAs far as I can tell, we need pager.h because of the `pager_in_use`\nsymbol. We need that symbol because of its use in date.c's\n`parse_date_format`. I wonder if we can side step the `#include\n` concerns by splitting pager.h into pager.h and\npager_in_use.h, and have pager.h include pager_in_use.h instead. This\nway pager.h (and its [unused] forward declarations) aren't part of\ngit-std-lib at all. I believe this was done for things like hex-ll.h,\nso maybe we call it pager-ll.h. The goal being to (a) not need the\n`#include ` because that's currently contentious, but also\n(b) to identify the minimum set of symbols needed for the stubs\nlibrary, and not declare things that we don't have any intention of\nactually providing / stubbing out.\n\nI have some more thoughts on this, but they're much more appropriate\nfor the next patch in the series, so I'll leave them there.\n"},{"id":"489274","messageId":"xmqqv86eqonh.fsf@gitster.g","threadId":"59921","inReplyTo":"CAO_smVh6PyxbnXfo0K1aDjEFPc3jTF4X_grerkxNZJdQe8V3sg@mail.gmail.com","subject":"Re: [PATCH v5 1/3] pager: include stdint.h because uintmax_t is used","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2024-02-24T07:58:42Z","receivedAt":"2024-02-24T07:58:53Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Kyle Lippincott writes:\n\n> As far as I can tell, we need pager.h because of the `pager_in_use`\n> symbol. We need that symbol because of its use in date.c's\n> `parse_date_format`. I wonder if we can side step the `#include\n> ` concerns by splitting pager.h into pager.h and\n> pager_in_use.h, and have pager.h include pager_in_use.h instead. This\n> way pager.h (and its [unused] forward declarations) aren't part of\n> git-std-lib at all.\n\nStep back a bit. Why do you even need to touch pager.h in the first\nplace? Whatever thing that needs to define a mock version of\npager_in_use() would need to be able to find out that it is supposed\nto take nothing as arguments and return an integer, and it can\ninclude without modification. Just like everybody else,\nit has to include so that the system header that\ngives us uintmax_t gets include appropriately in platform-dependent\nway, no? Why do we even need to butcher pager.h into two pieces in\nthe first place?\n\nIf you just include and then in\nstubs/pager.c and you're OK, no?\n\nIf anything, as I already said, I think it is more reasonable to\ntweak what does. For example, it might be\nunwieldy for gitstdlib's purpose that it unconditionally overrides\nexit(), in which case it may be OK to introduce some conditional\ncompilation macros to omit that override when building stub code.\nOr even split parts of the that both Git's use\nand gitstdlib's purpose are OK with into a separate header file\n, while leaving (hopefully a very minor) other\nparts in *and* include in\n. That way, the sources of Git can continue\nincluding while stub code can include\n, and we will get system library symbols and\nsystem defined types like uintmax_t in a consistent way, both in Git\nitself and in gitstdlib.\n\nBut once such a sanitization is done on the compat-util header,\nother \"ordinary\" header files that should not have to care about\nportability (because they can assume that inclusion of\ngit-compat-util.h will give them access to system types and symbols\nwithout having to worry about portability issues) and should not\nhave to include system header files themselves.\n\nAt least, that is the idea behind in the first\nplace. Including any system headers directly in ordinary headers,\nor splitting ordinary headers at an arbitrary and artificial\nboundary, should not be necessary. I'd have to say that such\nchanges are tail wagging the dog.\n\nI do not have sufficient cycles to spend actually splitting\ngit-compat-util.h into two myself, but as an illustration, here is\nhow I would tweak cw/git-std-lib topic to make it build without\nbreaking our headers and including system header files directly.\n\n git-compat-util.h | 2 ++\n pager.h | 2 --\n stubs/misc.c | 4 ++--\n stubs/pager.c | 1 +\n 4 files changed, 5 insertions(+), 4 deletions(-)\n\ndiff --git c/git-compat-util.h w/git-compat-util.h\nindex 7c2a6538e5..981d526d18 100644\n--- c/git-compat-util.h\n+++ w/git-compat-util.h\n@@ -1475,12 +1475,14 @@ static inline int is_missing_file_error(int errno_)\n \n int cmd_main(int, const char **);\n \n+#ifndef _GIT_NO_OVERRIDE_EXIT\n /*\n * Intercept all calls to exit() and route them to trace2 to\n * optionally emit a message before calling the real exit().\n */\n int common_exit(const char *file, int line, int code);\n #define exit(code) exit(common_exit(__FILE__, __LINE__, (code)))\n+#endif\n \n /*\n * You can mark a stack variable with UNLEAK(var) to avoid it being\ndiff --git c/pager.h w/pager.h\nindex 015bca95e3..b77433026d 100644\n--- c/pager.h\n+++ w/pager.h\n@@ -1,8 +1,6 @@\n #ifndef PAGER_H\n #define PAGER_H\n \n-#include \n-\n struct child_process;\n \n const char *git_pager(int stdout_is_tty);\ndiff --git c/stubs/misc.c w/stubs/misc.c\nindex 8d80581e39..d0379dcb69 100644\n--- c/stubs/misc.c\n+++ w/stubs/misc.c\n@@ -1,5 +1,5 @@\n-#include \n-#include \n+#define _GIT_NO_OVERRIDE_EXIT\n+#include \n \n #ifndef NO_GETTEXT\n /*\ndiff --git c/stubs/pager.c w/stubs/pager.c\nindex 4f575cada7..04517aad4c 100644\n--- c/stubs/pager.c\n+++ w/stubs/pager.c\n@@ -1,3 +1,4 @@\n+#include \n #include \"pager.h\"\n \n int pager_in_use(void)\n\n\n"},{"id":"489409","messageId":"CAO_smVi76TbmHd5w2rpBEEYbaw46SNTrekFHE-ohDC6-=dk6DA@mail.gmail.com","threadId":"59921","inReplyTo":"xmqqh6i0cgyo.fsf@gitster.g","subject":"Re: [PATCH v5 1/3] pager: include stdint.h because uintmax_t is used","fromName":"Kyle Lippincott","fromEmail":"spectral@google.com","sentAt":"2024-02-26T18:59:38Z","receivedAt":"2024-02-26T18:59:55Z","isPatch":true,"sender":{"key":"spectral@google.com","avatar":"https://avatars.githubusercontent.com/u/6371650?v=4"},"body":"On Thu, Feb 22, 2024 at 1:44 PM Junio C Hamano wrote:\n>\n> Calvin Wan writes:\n>\n> > From: Jonathan Tan \n> >\n> > pager.h uses uintmax_t but does not include stdint.h. Therefore, add\n> > this include statement.\n> >\n> > This was discovered when writing a stub pager.c file.\n> >\n> > Signed-off-by: Jonathan Tan \n> > Signed-off-by: Calvin Wan \n> > ---\n> > pager.h | 2 ++\n> > 1 file changed, 2 insertions(+)\n> >\n> > diff --git a/pager.h b/pager.h\n> > index b77433026d..015bca95e3 100644\n> > --- a/pager.h\n> > +++ b/pager.h\n> > @@ -1,6 +1,8 @@\n> > #ifndef PAGER_H\n> > #define PAGER_H\n> >\n> > +#include \n> > +\n> > struct child_process;\n> >\n> > const char *git_pager(int stdout_is_tty);\n>\n> This is not going in a sensible direction from our portability\n> standard's point of view.\n>\n> The reason why we do not include these system headers directly to\n> our source files, and instead make it a rule to include\n> as the first header instead, is exactly because\n> there are curiosities in various platforms that Git wants to run on\n> which system include headers give us the declarations for types and\n> functions we rely on, in what order they must be included, and after\n> what feature macros (the ones that give adjustment to what the\n> system headers do, like _POSIX_C_SOURCE) are defined, etc.\n>\n> Given that in , inclusion of is\n> conditional behind some #ifdef's, it does not look like a sensible\n> change. It is not very likely for and to\n> declare uintmax_t in an incompatible way, but on a platform where\n> decides to include and use its\n> definition of what uintmax_t is, we should follow the same choice\n> and be consistent.\n\nSpeaking of this specific header file inclusion and the oddities that\nhave gotten us to where we are:\n- Originally, it seems that we were including stdint.h\n- 17 years ago, to work around Solaris not providing stdint.h, but\nproviding inttypes.h, it was switched to being just inttypes.h, with\nthe explanation being that inttypes is a superset of stdint.\nhttps://github.com/git/git/commit/007e2ba65902b484fc65a313e54594a009841740\n- 13 years ago, to work around some platforms not having inttypes.h,\nit was made conditional.\n(https://github.com/git/git/commit/2844923d62a4c408bd59ddb2caacca4aa7eb86bc)\n\nThe condition added 13 years ago was, IMHO, backwards from what it\nshould have been. The intent is to have stdint.h included. We should\ninclude stdint.h. I suspect that 17 years is enough time for that\nplatform to start conforming to what is now a 25 year old standard,\nand I don't know how we can verify that and have this stop being a\nhaunted graveyard without just trying it and seeing if the build bots\nor maintainers identify it as a continuing issue. If it's still an\nissue (and only if), we should reintroduce a conditional, but invert\nit: if there's no stdint.h, THEN include inttypes.h.\n\nOh, no, that doesn't work. I tried that, and the build bots told me\nthat doesn't work, because we're using things from inttypes.h (PRIuMAX\nshowed up several times in the errors, there may be others). This\nmakes me wonder how the platforms with no inttypes.h work at all. I\nstill think we should do something here because it's a 13-year-old\ncompatibility fix that \"shouldn't\" be needed anymore, and causes\nconfusion/concerns like this thread. Maybe just see if we can get away\nwith always including inttypes.h in git-compat-util.h, or maybe _both_\ninttypes.h and stdint.h (in either order), just to be really obvious\nthat it's acceptable to include stdint.h.\n\n>\n> If there is a feature macro that affects sizes of the integer on a\n> platform, this patch will break it even more badly. Perhaps there\n> is a platform whose C-library header requires you to define a\n> feature macro to use 64-bit, and we may define that feature macro\n> in before including either or\n> , but by including directly like the above\n> patch does, only this file and the sources that include only this\n> file, refusing to include as everybody in the\n> Git source tree should, will end up using different notion of what\n> the integral type with maximum width is from everybody else.\n\nI agree that for pager.h, something like the patch in your next email\nwould resolve that particular problem. The stub library is of\nbasically the same stature as git-std-lib: it's code that is provided\nby the Git project, compiled by Makefile rules owned and maintained by\nthe Git project, and should conform to the Git coding standards. The\n.c files in the stubs library should include git-compat-util.h,\nthere's basically no reason not to.\n\nHowever, I believe that we'll need a good policy for what to do with\nlibified headers + sources in general. I can see many potential\ncategorizations of source; there's no need to formally define all of\nthem and assign files to each category, but the main categories are\nbasically:\n1. files that have code that is an internal part of Git, one of the\nhelper binaries, or one of its tests, whether it's a library or not.\nThese should include git-compat-util.h at the top of the .c files like\nthey do today. The .h files for these translation units are also\nconsidered \"internal\". These header files should assume that\ngit-compat-util.h has been included properly, and don't need to be\nself-contained, because they're _only_ included by things in this\ncategory.\n2. files that have code that define the \"library interface\", probably\nonly the ones defining the library interface _as used by external\nprojects_. I think that we'll likely need to be principled about\ndefining these, and having them be as minimal and compatible as\npossible.\n3. code in external projects that directly uses libraries from the Git\nproject, and thus includes Git headers from category 2\n4. the rest of the code in external projects (the code that does not\ndirectly use libraries from the Git project)\n\nA hypothetical git-compat-core.h being included at the top of the .c\nfiles in category 2 is feasible, but needs to be carefully handled due\nto potential symbol collision (which we're discussing in another\nthread and I may have a possible solution for, at least on some\nplatforms). On the other hand, a git-compat-core.h being included at\nthe top of the .h files belonging to category 2 doesn't work, because\nwhen these .h files are included by code in category 3, it's too late.\n\nIn this example, gnu-source-header.h below is a system header that\nchanges behavior depending on _GNU_SOURCE (effectively the same\nconcern as you were raising in the quoted paragraph):\n\nexternal-project-category3.c:\n#include \n#include \n#include \n\ngit/some-lib-interface.h:\n#include \n\nWe can't do anything in git/git-compat-core.h that relies on careful\ninclusion order, requiring that various things are #defined prior to\nthe first inclusion of certain headers, etc. stdint.h and\ngnu-source-header.h are already included, and so it's too late for us\nto #define things that change their behavior, because that won't have\nany effect. I don't think it's reasonable to expect the external\nproject to #include at the top of their files\nthat are in category3. It's definitely not reasonable to require the\nexternal project to do that for all their files (category 3 and\ncategory 4). It's slightly more reasonable to have them do some set of\n#defines for their binaries and libraries, but still quite awkward and\npotentially not feasible (for things like _FILE_OFFSET_BITS). This is\nwhy I split it into 4 categories.\n\nI believe that category 2 files need to be maximally compatible, both\nto platforms [where we provide support for libraries, and I think this\nprobably will end up being a subset of all the platforms, especially\nat first] and to the environment they're being #included in and\ninteracting with. So they need to be self-contained: they can't rely\non stdint.h having been included, but they also can't rely on it _not_\nhaving been included already. The category 2 .h files need to be\nminimal: just what we want in this external library interface, and\nideally nothing else. The category 2 .h and .c files need to be\ncompatible with common #defines being set OR not set. The point of a\ncategory 2 .c file is to bridge the gap between the category 1 and\ncategory 3 environments. This likely means that we need to be careful\nabout certain structs and typedefs defined by the system (vs. structs\ndefined by the category 2 headers themselves) being passed between the\ndifferent environments. For example, if we were to have a library\ninterface that included a `struct stat`, and the category 3 files\ndidn't have _FILE_OFFSET_BITS 64, but the category 1 files do? Instant\nbreakage.\n\nThis aspect of this discussion probably should happen on the next\npatch, or in a separate thread :) But since I'm here, I'll summarize\nthese thoughts: basically, the next patch, imho, doesn't go far\nenough, but is a very good first step that we can build on. We need to\ndefine what belongs to the \"external interface\" of the various\nlibraries (category 2 above) and what is considered category 1.\npager.h is pretty obviously category 1. strbuf.h, abspath.h, etc? I'm\nnot sure. git-std-lib is weird, because it's so low level and there's\nnot really much \"internal\" code to this library. So maybe we declare\nthose as category 2. But I don't know how that will actually work in\npractice. I'll try to find time to write up these thoughts on that\npatch.\n\n>\n> What this patch _wants_ to do is of course sympathizable, and we\n> have \"make hdr-check\" rule to enforce \"a header must include the\n> headers that declare what it uses\", except that it lets the header\n> files being tested assume that the things made available by\n> including are always available.\n>\n> I think a sensible direction to go for libification purposes is to\n> also make sure that sources that are compiled into gitstdlib.a, and\n> the headers that makes what is in gitstdlib.a available, include the\n> header file. There may be things declared in\n> the header that are _too_ specific to what ought\n> to be linked into the final \"git\" binary and unwanted by library\n> clients that are not \"git\" binary, and the right way to deal with it\n> is to split into two parts, i.e. what makes\n> system services available like its conditional inclusion of\n> vs , definition of feature macros, order in\n> which the current includes system headers, etc.,\n\n\n\n\n\n\n> excluding those that made you write this patch to avoid assuming\n> that the client code would have included before\n> , would be the new . And everything\n> else will remain in , which will include the\n> . The header for library clients would\n> include instead, to still allow them to use the\n> same types as \"git\" binary itself that way.\n>\n>\n>\n>\n>\n>\n"},{"id":"489453","messageId":"xmqqle76iwqw.fsf@gitster.g","threadId":"59921","inReplyTo":"CAO_smVi76TbmHd5w2rpBEEYbaw46SNTrekFHE-ohDC6-=dk6DA@mail.gmail.com","subject":"Re: [PATCH v5 1/3] pager: include stdint.h because uintmax_t is used","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2024-02-27T00:20:07Z","receivedAt":"2024-02-27T00:20:11Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Kyle Lippincott writes:\n\n> The condition added 13 years ago was, IMHO, backwards from what it\n> should have been. The intent is to have stdint.h included. We should\n> include stdint.h. I suspect that 17 years is enough time for that\n> platform to start conforming to what is now a 25 year old standard,\n> and I don't know how we can verify that and have this stop being a\n> haunted graveyard without just trying it and seeing if the build bots\n> or maintainers identify it as a continuing issue.\n\nThe nightmare of Solaris might be luckily behind us, but the world\ndoes not only run Linux and GNU libc, and it is not just \nvs . This is about general code hygiene.\n\n> If it's still an\n> issue (and only if), we should reintroduce a conditional, but invert\n> it: if there's no stdint.h, THEN include inttypes.h.\n\nBut it would give the wrong order in general in the modern world,\nwhere is supposed to include and extends it.\n\nWe use inttypes.h by default because the C standard already talks\nabout it, and fall back to stdint.h when the platform lacks it. But\nwhat I suspect is that nobody compiles with NO_INTTYPES_H and we\nwould unknowingly (but not \"unintentionally\") start using the\nextended types that are only available in but not in\n sometime in the future. It might already have happened,\nbut I do not know. I haven't compiled with NO_INTTYPES_H for some\ntime (to experiment), and I haven't met a platform that actually\nrequires NO_INTTYPES_H defined to build. Once after such a change\nis made without anybody being knowingly breaking some rare platform,\nif nobody complains, we can just drop the support to allow us to\nlimit ourselves to , but since we hear nobody complaining,\nwe should be OK with the current rule of including system header\nfiles that is embodied in header file.\n\nIn any case, your sources should not include a standard library\nheader directly yourself, period. Instead let \ntake care of the details of how we need to obtain what we need out\nof the system on various platforms.\n\nThanks.\n"},{"id":"489454","messageId":"CAO_smVjeNYFTgh4MZjRM9U1coY=UJxo-E6bD9OfdS1A1Xf6vcQ@mail.gmail.com","threadId":"59921","inReplyTo":"xmqqle76iwqw.fsf@gitster.g","subject":"Re: [PATCH v5 1/3] pager: include stdint.h because uintmax_t is used","fromName":"Kyle Lippincott","fromEmail":"spectral@google.com","sentAt":"2024-02-27T00:56:28Z","receivedAt":"2024-02-27T00:56:46Z","isPatch":true,"sender":{"key":"spectral@google.com","avatar":"https://avatars.githubusercontent.com/u/6371650?v=4"},"body":"On Mon, Feb 26, 2024 at 4:20 PM Junio C Hamano wrote:\n>\n> Kyle Lippincott writes:\n>\n> > The condition added 13 years ago was, IMHO, backwards from what it\n> > should have been. The intent is to have stdint.h included. We should\n> > include stdint.h. I suspect that 17 years is enough time for that\n> > platform to start conforming to what is now a 25 year old standard,\n> > and I don't know how we can verify that and have this stop being a\n> > haunted graveyard without just trying it and seeing if the build bots\n> > or maintainers identify it as a continuing issue.\n>\n> The nightmare of Solaris might be luckily behind us, but the world\n> does not only run Linux and GNU libc, and it is not just \n> vs . This is about general code hygiene.\n>\n> > If it's still an\n> > issue (and only if), we should reintroduce a conditional, but invert\n> > it: if there's no stdint.h, THEN include inttypes.h.\n>\n> But it would give the wrong order in general in the modern world,\n> where is supposed to include and extends it.\n>\n> We use inttypes.h by default because the C standard already talks\n> about it, and fall back to stdint.h when the platform lacks it. But\n> what I suspect is that nobody compiles with NO_INTTYPES_H and we\n> would unknowingly (but not \"unintentionally\") start using the\n> extended types that are only available in but not in\n> sometime in the future. It might already have happened,\n\nIt has. We use PRIuMAX, which is from inttypes.h. I think it's only\n\"accidentally\" working if anyone uses NO_INTTYPES_H. I changed my\nstance halfway through this investigation in my previous email, I\napologize for not going back and editing it to make it clear at the\nbeginning that I'd done so. My current stance is that\n should be either (a) including only inttypes.h\n(since it includes stdint.h), or (b) including both inttypes.h and\nstdint.h (in either order), just to demonstrate that we can.\n\n> but I do not know. I haven't compiled with NO_INTTYPES_H for some\n> time (to experiment), and I haven't met a platform that actually\n> requires NO_INTTYPES_H defined to build. Once after such a change\n> is made without anybody being knowingly breaking some rare platform,\n> if nobody complains, we can just drop the support to allow us to\n> limit ourselves to , but since we hear nobody complaining,\n> we should be OK with the current rule of including system header\n> files that is embodied in header file.\n>\n> In any case, your sources should not include a standard library\n> header directly yourself, period. Instead let \n> take care of the details of how we need to obtain what we need out\n> of the system on various platforms.\n\nI disagree with this statement. We _can't_ use a magic compatibility\nheader file in the library interfaces, for the reasons I outlined\nfurther below in my previous message. For those headers, the ones that\nmight be included by code that's not under the Git project's control,\nthey need to be self-contained, minimal, and maximally compatible.\n\n>\n> Thanks.\n"},{"id":"489456","messageId":"xmqqzfvmhbfs.fsf@gitster.g","threadId":"59921","inReplyTo":"CAO_smVjeNYFTgh4MZjRM9U1coY=UJxo-E6bD9OfdS1A1Xf6vcQ@mail.gmail.com","subject":"Re: [PATCH v5 1/3] pager: include stdint.h because uintmax_t is used","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2024-02-27T02:45:43Z","receivedAt":"2024-02-27T02:45:46Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Kyle Lippincott writes:\n\n>> In any case, your sources should not include a standard library\n>> header directly yourself, period. Instead let \n>> take care of the details of how we need to obtain what we need out\n>> of the system on various platforms.\n>\n> I disagree with this statement. We _can't_ use a magic compatibility\n> header file in the library interfaces, for the reasons I outlined\n> further below in my previous message. For those headers, the ones that\n> might be included by code that's not under the Git project's control,\n> they need to be self-contained, minimal, and maximally compatible.\n\nNote that I am not talking about your random outside program that\nhappens to link with gitstdlib.a; it would want to include a header\nfile that comes with the library.\n\nEarlier I suggested that you may want to take a subset of\n, because may have a lot more\nthan what is minimally necessary to allow our sources to be\ninsulated from details of platform dependence. You can think of\nthat subset as a good starting point to build the \nheader file to be given to the library customers.\n\nBut the sources that go to the library, as gitstdlib.a is supposed\nto serve as a subset of gitlib.a to our internal codebase when\nbuilding the git binary, should still follow our header inclusion\nrules.\n\nBecause we would want to make sure that the sources that are made\ninto gitstdlib.a, the sources to the rest of libgit.a, and the\nsources to the rest of git, all agree on what system features we ask\nfrom the system, feature macros that must be defined to certain\nvalues before we include system library files (like _XOPEN_SOURCE\nand _FILE_OFFSET_BITS) must be defined consistently across all of\nthese three pieces. One way to do so may be to ensure that the\ndefinition of them would be migrated to when we\nseparate a subset out of to it (and of course,\nwe make to include so that it\nwould be still sufficient for our in-tree users to include the\n)\n\n may have to expose an API function that uses some\nextended types only available by including system header files,\ne.g. some function may return ssize_t as its value or take an off_t\nvalue as its argument.\n\nIf our header should include system headers to make these types\navailable to our definitions is probably open to discussion. It is\nharder to do so portably, unless your world is limited to POSIX.1\nand ISO C, than making it the responsibility of library users. \n\nBut if the platform headers and libraries support feature macros\nthat allows you to tweak these sizes (e.g. the size of off_t may be\ncontrolled by setting the _FILE_OFFSET_BITS to an appropriate\nvalue), it may be irresponsible to leave that to the library users,\nas they MUST make sure to define such feature macros exactly the\nsame way as we define for our code, which currently is done in\n, before they include their system headers to\nobtain off_t so that they can use .\n\nSo the rules for library clients (random outside programs that\nhappen to link with gitstdlib.a) may not be that they must include\n as the first thing, but they probably still have\nto include fairly early before including any of their\nsystem headers, I would suspect, unless they are willing to accept\nsuch responsibility fully to ensure they compile the same way as the\ngitstdlib library, I would think.\n\n\n\n"},{"id":"489474","messageId":"20240227084529.GJ3263678@coredump.intra.peff.net","threadId":"59921","inReplyTo":"CAO_smVjeNYFTgh4MZjRM9U1coY=UJxo-E6bD9OfdS1A1Xf6vcQ@mail.gmail.com","subject":"Re: [PATCH v5 1/3] pager: include stdint.h because uintmax_t is used","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2024-02-27T08:45:29Z","receivedAt":"2024-02-27T08:45:30Z","isPatch":true,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Mon, Feb 26, 2024 at 04:56:28PM -0800, Kyle Lippincott wrote:\n\n> > We use inttypes.h by default because the C standard already talks\n> > about it, and fall back to stdint.h when the platform lacks it. But\n> > what I suspect is that nobody compiles with NO_INTTYPES_H and we\n> > would unknowingly (but not \"unintentionally\") start using the\n> > extended types that are only available in but not in\n> > sometime in the future. It might already have happened,\n> \n> It has. We use PRIuMAX, which is from inttypes.h.\n\nIs it always, though? That's what C99 says, but if you have a system\nthat does not have inttypes.h in the first place, but does have\nstdint.h, it seems possible that it provides conversion macros elsewhere\n(either via stdint.h, or possibly just as part of stdio.h).\n\nSo it might be that things have been horribly broken on NO_INTTYPES_H\nsystems for a while, and nobody is checking. But I don't think you can\nreally say so without looking at such a system.\n\nAnd looking at config.mak.uname, it looks like Windows is such a system.\nDoes it really have inttypes.h and it is getting included from somewhere\nelse, making format conversion macros work? Or does it provide those\nmacros elsewhere, and really needs stdint? It does look like\ncompat/mingw.h includes it, but I think we wouldn't use that for msvc\nbuilds.\n\n> I think it's only\n> \"accidentally\" working if anyone uses NO_INTTYPES_H. I changed my\n> stance halfway through this investigation in my previous email, I\n> apologize for not going back and editing it to make it clear at the\n> beginning that I'd done so. My current stance is that\n> should be either (a) including only inttypes.h\n> (since it includes stdint.h), or (b) including both inttypes.h and\n> stdint.h (in either order), just to demonstrate that we can.\n\nIt is good to clean up old conditionals if they are no longer\napplicable, as they are a burden to reason about later (as this\ndiscussion shows). But I am not sure about your \"just to demonstrate we\ncan\". It is good to try it out, but it looks like there is a non-zero\nchance that MSVC on Windows might break. It is probably better to try\nbuilding there or looping in folks who can, rather than just making a\nchange and seeing if anybody screams.\n\nI think the \"win+VS\" test in the GitHub Actions CI job might cover this\ncase. It is not run by default (because it was considered be mostly\nredundant with the mingw build), but it shouldn't be too hard to enable\nit for a one-off test.\n\n-Peff\n"},{"id":"489476","messageId":"20240227090547.GK3263678@coredump.intra.peff.net","threadId":"59921","inReplyTo":"20240227084529.GJ3263678@coredump.intra.peff.net","subject":"Re: [PATCH v5 1/3] pager: include stdint.h because uintmax_t is used","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2024-02-27T09:05:47Z","receivedAt":"2024-02-27T09:05:48Z","isPatch":true,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Tue, Feb 27, 2024 at 03:45:29AM -0500, Jeff King wrote:\n\n> It is good to clean up old conditionals if they are no longer\n> applicable, as they are a burden to reason about later (as this\n> discussion shows). But I am not sure about your \"just to demonstrate we\n> can\". It is good to try it out, but it looks like there is a non-zero\n> chance that MSVC on Windows might break. It is probably better to try\n> building there or looping in folks who can, rather than just making a\n> change and seeing if anybody screams.\n> \n> I think the \"win+VS\" test in the GitHub Actions CI job might cover this\n> case. It is not run by default (because it was considered be mostly\n> redundant with the mingw build), but it shouldn't be too hard to enable\n> it for a one-off test.\n\nHere's a successful run with the NO_INTTYPES_H line removed:\n\n https://github.com/peff/git/actions/runs/8062063219\n\nBlaming the code around the inttypes.h reference in compat/mingw.h\nturned up 0ef60afdd4 (MSVC: use shipped headers instead of fallback\ndefinitions, 2016-03-30), which claims that inttypes.h was added to\nVS2013.\n\nThat sounds old-ish in 2024, but I don't know how old is normal in the\nWindows world.\n\nAll of which to me says that cleaning this up is something that should\ninvolve Windows folks, who can make the judgement for their platform.\n\n-Peff\n"},{"id":"489571","messageId":"CAO_smVg9phJGv=fMcHXjqvY74pHuMdousqfLMpyBcQ+fVMY90g@mail.gmail.com","threadId":"59921","inReplyTo":"20240227084529.GJ3263678@coredump.intra.peff.net","subject":"Re: [PATCH v5 1/3] pager: include stdint.h because uintmax_t is used","fromName":"Kyle Lippincott","fromEmail":"spectral@google.com","sentAt":"2024-02-27T20:10:07Z","receivedAt":"2024-02-27T20:10:25Z","isPatch":true,"sender":{"key":"spectral@google.com","avatar":"https://avatars.githubusercontent.com/u/6371650?v=4"},"body":"On Tue, Feb 27, 2024 at 12:45 AM Jeff King wrote:\n>\n> On Mon, Feb 26, 2024 at 04:56:28PM -0800, Kyle Lippincott wrote:\n>\n> > > We use inttypes.h by default because the C standard already talks\n> > > about it, and fall back to stdint.h when the platform lacks it. But\n> > > what I suspect is that nobody compiles with NO_INTTYPES_H and we\n> > > would unknowingly (but not \"unintentionally\") start using the\n> > > extended types that are only available in but not in\n> > > sometime in the future. It might already have happened,\n> >\n> > It has. We use PRIuMAX, which is from inttypes.h.\n>\n> Is it always, though? That's what C99 says, but if you have a system\n> that does not have inttypes.h in the first place, but does have\n> stdint.h, it seems possible that it provides conversion macros elsewhere\n> (either via stdint.h, or possibly just as part of stdio.h).\n\nIt's of course possible that on some platforms, stdio.h or stdint.h\ndefines these types (or includes inttypes.h internally, which defines\nthese types). However, I think that to be \"correct\" and for a compiler\nto claim it supports C99 (and the compiler _must_ claim that because\nof the guard in ), inttypes.h must exist, and it\nmust cause these symbols to appear. If there are platforms that are\nclaiming to be C99 and inttypes.h doesn't exist or doesn't provide the\nsymbols it should, I don't think we should try to support them - they\ncan maintain platform-specific patches for whatever not-actually-C99\nlanguage the platform supports. Basically what git for windows is\nalready doing (presumably for other reasons), as far as I can tell.\n\n>\n> So it might be that things have been horribly broken on NO_INTTYPES_H\n> systems for a while, and nobody is checking. But I don't think you can\n> really say so without looking at such a system.\n>\n> And looking at config.mak.uname, it looks like Windows is such a system.\n> Does it really have inttypes.h and it is getting included from somewhere\n> else, making format conversion macros work? Or does it provide those\n> macros elsewhere, and really needs stdint? It does look like\n> compat/mingw.h includes it, but I think we wouldn't use that for msvc\n> builds.\n>\n> > I think it's only\n> > \"accidentally\" working if anyone uses NO_INTTYPES_H. I changed my\n> > stance halfway through this investigation in my previous email, I\n> > apologize for not going back and editing it to make it clear at the\n> > beginning that I'd done so. My current stance is that\n> > should be either (a) including only inttypes.h\n> > (since it includes stdint.h), or (b) including both inttypes.h and\n> > stdint.h (in either order), just to demonstrate that we can.\n>\n> It is good to clean up old conditionals if they are no longer\n> applicable, as they are a burden to reason about later (as this\n> discussion shows). But I am not sure about your \"just to demonstrate we\n> can\".\n\nYeah, I'm also not convinced the \"just to demonstrate we can\" has much\nvalue. I was trying to get ahead of future discussions where we claim\nit's important to never include stdint.h (because people remember this\ndiscussion and how contentious it was) and think it might misbehave,\nand instead just include it in to prove it\n_doesn't_ misbehave, and thus start to allow usage in self-contained\nheaders when necessary.\n\n> It is good to try it out, but it looks like there is a non-zero\n> chance that MSVC on Windows might break. It is probably better to try\n> building there or looping in folks who can, rather than just making a\n> change and seeing if anybody screams.\n\nI think I miscommunicated here, or had too many assumptions about the\ncurrent state of things that I didn't actually verify. When I wrote\n\"and seeing if the build bots or maintainers identify it as a\ncontinuing issue\", I was assuming that we had build bots for all major\nplatforms (including windows, with however it gets built: mingw or VC\nor whatever), and I included the \"maintainers\" part of it for the long\ntail of esoteric platforms that we either don't know about, or can't\nhave automated builds on for whatever reason. I agree that making\nchanges that have a high likelihood of breaking supported platforms\n(which gets back to that platform support thread that was started a\nfew weeks ago) should not be done lightly, and it's not reasonable to\nmake the change and wait for maintainers of these \"supported\nplatforms\" to complain. I was relying on the build bots covering the\n\"supported platforms\" and stopping me from even sending such a patch\nto the mailing list :)\n\n>\n> I think the \"win+VS\" test in the GitHub Actions CI job might cover this\n> case. It is not run by default (because it was considered be mostly\n> redundant with the mingw build), but it shouldn't be too hard to enable\n> it for a one-off test.\n>\n> -Peff\n"},{"id":"489580","messageId":"CAO_smVgmaXvyZZ7zp6RCFD_6kpL2pHKC9gMDeg+yXBb9R4rR5w@mail.gmail.com","threadId":"59921","inReplyTo":"xmqqzfvmhbfs.fsf@gitster.g","subject":"Re: [PATCH v5 1/3] pager: include stdint.h because uintmax_t is used","fromName":"Kyle Lippincott","fromEmail":"spectral@google.com","sentAt":"2024-02-27T22:29:23Z","receivedAt":"2024-02-27T22:29:41Z","isPatch":true,"sender":{"key":"spectral@google.com","avatar":"https://avatars.githubusercontent.com/u/6371650?v=4"},"body":"On Mon, Feb 26, 2024 at 6:45 PM Junio C Hamano wrote:\n>\n> Kyle Lippincott writes:\n>\n> >> In any case, your sources should not include a standard library\n> >> header directly yourself, period. Instead let \n> >> take care of the details of how we need to obtain what we need out\n> >> of the system on various platforms.\n> >\n> > I disagree with this statement. We _can't_ use a magic compatibility\n> > header file in the library interfaces, for the reasons I outlined\n> > further below in my previous message. For those headers, the ones that\n> > might be included by code that's not under the Git project's control,\n> > they need to be self-contained, minimal, and maximally compatible.\n>\n> Note that I am not talking about your random outside program that\n> happens to link with gitstdlib.a; it would want to include a header\n> file that comes with the library.\n\nI agree with this.\n\n>\n> Earlier I suggested that you may want to take a subset of\n> , because may have a lot more\n> than what is minimally necessary to allow our sources to be\n> insulated from details of platform dependence. You can think of\n> that subset as a good starting point to build the \n> header file to be given to the library customers.\n>\n> But the sources that go to the library, as gitstdlib.a is supposed\n> to serve as a subset of gitlib.a to our internal codebase when\n> building the git binary, should still follow our header inclusion\n> rules.\n\nIf I'm understanding this correctly, I agree with it. The .c files\nstill include , and don't change. The internal-only\n.h files (ones that a pre-built-library consumer doesn't need to even\nhave in the filesystem) still assume that was\nincluded, and don't change. falls into this category.\n\n>\n> Because we would want to make sure that the sources that are made\n> into gitstdlib.a, the sources to the rest of libgit.a, and the\n> sources to the rest of git, all agree on what system features we ask\n> from the system, feature macros that must be defined to certain\n> values before we include system library files (like _XOPEN_SOURCE\n> and _FILE_OFFSET_BITS) must be defined consistently across all of\n> these three pieces. One way to do so may be to ensure that the\n> definition of them would be migrated to when we\n> separate a subset out of to it (and of course,\n> we make to include so that it\n> would be still sufficient for our in-tree users to include the\n> )\n>\n> may have to expose an API function that uses some\n> extended types only available by including system header files,\n> e.g. some function may return ssize_t as its value or take an off_t\n> value as its argument.\n\nI agree that these types will be necessary (specifically ssize_t and\nint##_t, but less so off_t) in the \"external\" (used by projects other\nthan Git) library interfaces.\n\n>\n> If our header should include system headers to make these types\n> available to our definitions is probably open to discussion. It is\n> harder to do so portably, unless your world is limited to POSIX.1\n> and ISO C, than making it the responsibility of library users.\n\nI think I'm probably missing the nuance here, and may be making this\ndiscussion much harder because of it. My understanding is that Git is\nusing C99; is that different from ISO C? There's something at the top\nof that enforces that we're using C99. Therefore,\nI'm assuming that any compiler that claims to be C99 and passes that\ncheck at the top of will support inttypes.h,\nstdint.h, stdbool.h, and other files defined by the C99 standard to\ninclude types that we need in our .h files are able to be included\nwithout reservation. To flip it around: any compiler/platform that's\nmissing inttypes.h, or is missing stdint.h, or raises errors if both\nare included, or requires other headers to be included before them\n_isn't a C99 compiler_, and _isn't supported_. I'm picking on these\nfiles because I think they will be necessary for the external library\ninterfaces. I'm intentionally ignoring any file not mentioned in the\nC99 standard, because those are platform specific. I acknowledge that\nthere may be some functionality in these files that's only enabled if\ncertain #defines are set. Our external interfaces should strive to not\nuse that functionality, and only do so if we are able to test for this\nfunctionality and refuse to compile if it's not available. I have an\nexample with uintmax_t below.\n\n>\n> But if the platform headers and libraries support feature macros\n> that allows you to tweak these sizes (e.g. the size of off_t may be\n> controlled by setting the _FILE_OFFSET_BITS to an appropriate\n> value), it may be irresponsible to leave that to the library users,\n> as they MUST make sure to define such feature macros exactly the\n> same way as we define for our code, which currently is done in\n> , before they include their system headers to\n> obtain off_t so that they can use .\n\nI think the only viable solution to this is to not use these types\nthat depend on #defines in the interface available to non-git\nprojects. We can't set _FILE_OFFSET_BITS in the library's external\n(used by non-Git projects) interface header, as there's a high\nlikelihood that it's either too late (external project #included\nsomething that relies on _FILE_OFFSET_BITS already), or, if not, we\ncreate the \"off_t is a different size\" problem for their code.\n\nThis means that we can't use off_t in these external interface headers\n(and in the .c files that support them, if any). We can't use `struct\nstat`. We likely need to limit ourselves to just the typedefs from\nstdint.h, and probably will need some additional checks that enforce\nthat we have the types and sizes we expect (ex: I could imagine that\nsome platforms define uintmax_t as 32-bit. or 128-bit. Either we can't\nuse it in these external interfaces, or we have to enforce somehow\nthat the simplest file we can imagine (#include ) gets a\ndefinition of uintmax_t that is the exact same as the one we'd get if\nwe included ). The external interface headers don't\nneed to be as platform-compatible as the rest of the git code base,\nbecause not every platform is going to be a supported target for using\nthe library in non-git projects, especially at first. The external\ninterface headers _do_ need to be as tolerant and well behaved as\npossible when being included by external projects, which I'm asserting\nmeans they need to be self-contained and minimal. If that means these\nexternal interfaces don't get to use off_t at all, so be it. If it\nmeans they can only be included if sizeof(off_t) == 64, and we have a\nway of enforcing that at compile time, that's fine with me too. But we\ncan't #define _FILE_OFFSET_BITS ourselves in this external interface\nto get that behavior, because it just doesn't work.\n\nI'm making some assumptions here. I'm assuming that the git binary\nuses a different interface to a hypothetical libgitobjstore.a than an\nexternal project would (i.e. that there'd be some\ngit-obj-store-interface.h that gets included by non-Git projects, but\nnot by git itself). Is git-std-lib an obvious counterexample to this\nassumption? Yes and no. No one (besides Git itself) is going to\ninclude libgitstdlib.a in their project any time soon, so there's no\nreal \"external interface\" to define right now. Eventually, having\ngit-std-lib types in the hypothetical git-obj-store-interface.h _may_\nhappen, or it may not. I don't know.\n\n...\n\nBut I think we're in agreement that pager.h isn't part of\ngit-std-lib's (currently undefined/non-existent) external interface,\nand so doesn't need to be self-contained, and this patch should\nprobably be dropped?\n>\n> So the rules for library clients (random outside programs that\n> happen to link with gitstdlib.a) may not be that they must include\n> as the first thing, but they probably still have\n> to include fairly early before including any of their\n> system headers, I would suspect, unless they are willing to accept\n> such responsibility fully to ensure they compile the same way as the\n> gitstdlib library, I would think.\n>\n>\n>\n"},{"id":"489584","messageId":"xmqq1q8xo5fu.fsf@gitster.g","threadId":"59921","inReplyTo":"CAO_smVgmaXvyZZ7zp6RCFD_6kpL2pHKC9gMDeg+yXBb9R4rR5w@mail.gmail.com","subject":"Re: [PATCH v5 1/3] pager: include stdint.h because uintmax_t is used","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2024-02-27T23:25:41Z","receivedAt":"2024-02-27T23:25:46Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Kyle Lippincott writes:\n\n> of that enforces that we're using C99. Therefore,\n> I'm assuming that any compiler that claims to be C99 and passes that\n> check at the top of will support inttypes.h,\n> stdint.h, stdbool.h, and other files defined by the C99 standard to\n> include types that we need in our .h files are able to be included\n> without reservation.\n\nWe at the git project is much more conservative than trusting the\ncompilers and take their \"claim\" to support a standard at the face\nvalue, though ;-). As our CodingGuidelines say, we honor real world\nconstraints more than what's written on paper as \"standard\", and\nwould ...\n\n> To flip it around: any compiler/platform that's\n> missing inttypes.h, or is missing stdint.h, or raises errors if both\n> are included, or requires other headers to be included before them\n> _isn't a C99 compiler_, and _isn't supported_.\n\n... refrain from taking such a position as much as possible.\n\n> I think the only viable solution to this is to not use these types\n> that depend on #defines in the interface available to non-git\n> projects.\n\nOK. Now we have that behind us, can we start outlining what kind of\nthings _are_ exported out in the library to the outside world? The\nonly example of the C source file that is a client of the library we\nhave is t/helper/test-stdlib.c but it includes \n\n \n \n \n \n \n\nafter including . Are the services offered by\nthese five header files the initial set of the library, minimally\nviable demonstration? Has somebody already analyzed and enumerated\nwhat kind of system definitions we need to support the interface\nthese five header files offer?\n\nOnce we know that, perhaps the next task would be to create a\n header by taking a\nsubset of , and have test-stdlib.c include that\ninstead of . will of course\ninclude that header to replace the parts it lost to the new header.\n\nIt does *not* make it a requirement for client programs to include\nthe , though. They\ncan include the system headers in the way they like, as long as they\ndo not let them define symbols in ways contradicting to what our\nheaders expect. is\nmerely a way for us to tell those who write these client programs\nwhat the system symbols we rely on are.\n\nSo, that's one (or two? first to analyse and enumerate, second to\nsplit a new header file out of compat-util) actionable task, I\nguess.\n\n"},{"id":"489658","messageId":"db8b2398-c6d6-47a1-a4d4-7e4726d13312@gmail.com","threadId":"59921","inReplyTo":"20240222175033.1489723-3-calvinwan@google.com","subject":"Re: [PATCH v5 2/3] git-std-lib: introduce Git Standard Library","fromName":"Phillip Wood","fromEmail":"phillip.wood123@gmail.com","sentAt":"2024-02-29T11:16:41Z","receivedAt":"2024-02-29T11:16:44Z","isPatch":true,"sender":{"key":"phillip.wood@dunelm.org.uk","avatar":null},"body":"Hi Calvin\n\nOn 22/02/2024 17:50, Calvin Wan wrote:\n> This commit contains:\n> - Makefile rules for git-std-lib.a\n> - code and Makefile rules for git-stub-lib.a\n> - description and rationale of the above in Documentation/\n\nWe tend to avoid lists like this in our commit messages. Starting with \nthe motivation for adding git-std-lib would be more helpful to the \nreader I think.\n\n> Quoting from documentation introduced in this commit:\n> \n> The Git Standard Library intends to serve as the foundational library\n> and root dependency that other libraries in Git will be built off\n> of. That is to say, suppose we have libraries X and Y; a user that\n> wants to use X and Y would need to include X, Y, and this Git Standard\n> Library.\n> \n> Code demonstrating the use of git-std-lib.a and git-stub-lib.a will be\n> in a subsequent commit.\n> \n> Signed-off-by: Calvin Wan \n> Helped-by: Phillip Wood \n> Signed-off-by: Jonathan Tan \n\nI think virtually all the changes since the last version apart from \nrewording the documentation come from my fixup at [1] I'm happy to offer \nmy Signed-off-by: for those. I cannot offer a review of the changes from \nthat fixup though I'm still happy with the approach. I do however think \nI should have included git-compat-util.h in the stub implementations and \nused BUG() instead of assert(). I've left some comments on the \ndocumentation below.\n\n[1] \nhttps://github.com/phillipwood/git/commit/0f8393d2189a4c73d3f00f5ae74d3972677309d0\n\n> diff --git a/Documentation/technical/git-std-lib.txt b/Documentation/technical/git-std-lib.txt\n> new file mode 100644\n> index 0000000000..3d9aa121ac\n> --- /dev/null\n> +++ b/Documentation/technical/git-std-lib.txt\n> @@ -0,0 +1,170 @@\n> += Git Standard Library\n> +\n> +The Git Standard Library intends to serve as the foundational library\n> +and root dependency that other libraries in Git will be built off of.\n> +That is to say, suppose we have libraries X and Y; a user that wants to\n> +use X and Y would need to include X, Y, and this Git Standard Library.\n> +This does not mean that the Git Standard Library will be the only\n> +possible root dependency in the future, but rather the most significant\n> +and widely used one. Git itself is also built off of the Git Standard\n> +Library.\n> +\n> +== Dependency graph in libified Git\n> +\n> +Before the introduction of the Git Standard Library, all objects defined\n> +in the Git library are compiled and archived into a singular file,\n\ns/singular/single/ there are some instances of \"singular\" later on as \nwell all of which would be better as \"single\" I think.\n\n> +libgit.a, which is then linked against by common-main.o with other\n> +external dependencies and turned into the Git executable.\n\nI found this description a bit confusing. As I understand it to build \ngit we link git.o against common-main.o, libgit.a, xdiff/lib.a, \nreftable/libreftable.a and libpcre etc.\n\n> In other\n> +words, the Git executable has dependencies on libgit.a and a couple of\n> +external libraries. The libfication of Git slightly alters this build\n> +flow by separating out libgit.a into libgit.a and git-std-lib.a.\n> +\n> +With our current method of building Git, we can imagine the dependency\n\ns/imagine/visualize/\n\n> +graph as such:\n> +\n> +\tGit\n> +\t /\\\n> +\t/ \\\n> + / \\\n> + libgit.a ext deps\n> +\n> +We want to separate out potential libraries from libgit.a and have\n> +libgit.a depend on them, which would possibly look like:\n> +\n> +\t\tGit\n> +\t\t/\\\n> +\t / \\\n> +\t / \\\n> +\t libgit.a ext deps\n> +\t /\\\n> +\t / \\\n> +\t / \\\n> +object-store.a (other lib)\n> + | /\n> + | /\n> + | /\n> + | /\n> + | /\n> + | /\n> + | /\n> +git-std-lib.a\n> +\n> +Instead of containing all objects in Git, libgit.a would contain objects\n> +that are not built by libraries it links against. Consequently, if\n\ns/by libraries/by the libraries/\n\n> +someone wanted a custom build of Git with a custom implementation of the\n> +object store, they would only have to swap out object-store.a rather\n> +than do a hard fork of Git.\n> +\n> +== Rationale behind Git Standard Library\n> +\n> +The rationale behind the selected object files in the Git Standard\n> +Library is the result of two observations within the Git\n> +codebase:\n> + 1. every file includes git-compat-util.h which defines functions\n> + in a couple of different files\n> + 2. wrapper.c + usage.c have difficult-to-separate circular\n\ns/+/and/\n\n> + dependencies with each other and other files.\n> +\n> +=== Ubiquity of git-compat-util.h and circular dependencies\n> +\n> +Every file in the Git codebase includes git-compat-util.h. It serves as\n> +\"a compatibility aid that isolates the knowledge of platform specific\n> +inclusion order and what feature macros to define before including which\n> +system header\" (Junio[1]). Since every file includes git-compat-util.h,\n> +and git-compat-util.h includes wrapper.h and usage.h, it would make\n> +sense for wrapper.c and usage.c to be a part of the root library. They\n> +have difficult to separate circular dependencies with each other so it\n> +would impractical for them to be independent libraries. Wrapper.c has\n> +dependencies on parse.c, abspath.c, strbuf.c, which in turn also have\n> +dependencies on usage.c and wrapper.c - more circular dependencies.\n> +\n> +=== Tradeoff between swappability and refactoring\n> +\n> +From the above dependency graph, we can see that git-std-lib.a could be\n> +many smaller libraries rather than a singular library. So why choose a\n> +singular library when multiple libraries can be individually easier to\n> +swap and are more modular? A singular library requires less work to\n> +separate out circular dependencies within itself so it becomes a\n> +tradeoff question between work and reward. While there may be a point in\n> +the future where a file like usage.c would want its own library so that\n> +someone can have custom die() or error(), the work required to refactor\n> +out the circular dependencies in some files would be enormous due to\n> +their ubiquity so therefore I believe it is not worth the tradeoff\n\nI'm not sure if we want to use the first person in our technical \ndocumentation, unlike the cover letter to a patch series it is not \nimmediately obvious to the reader who \"I\" is. This applies the passages \nin the first person below as well.\n\n> +currently. Additionally, we can in the future choose to do this refactor\n> +and change the API for the library if there becomes enough of a reason\n> +to do so (remember we are avoiding promising stability of the interfaces\n> +of those libraries).\n> +\n> +=== Reuse of compatibility functions in git-compat-util.h\n> +\n> +Most functions defined in git-compat-util.h are implemented in compat/\n> +and have dependencies limited to strbuf.h and wrapper.h so they can be\n> +easily included in git-std-lib.a, which as a root dependency means that\n> +higher level libraries do not have to worry about compatibility files in\n> +compat/. The rest of the functions defined in git-compat-util.h are\n> +implemented in top level files and are hidden behind\n> +an #ifdef if their implementation is not in git-std-lib.a.\n\nI think the reference to #ifdef is out of date now we've moved to more stubs\n\n> +=== Rationale summary\n> +\n> +The Git Standard Library allows us to get the libification ball rolling\n> +with other libraries in Git. By not spending many more months attempting\n> +to refactor difficult circular dependencies and instead spending that\n> +time getting to a state where we can test out swapping a library out\n> +such as config or object store, we can prove the viability of Git\n> +libification on a much faster time scale. Additionally the code cleanups\n> +that have happened so far have been minor and beneficial for the\n> +codebase. It is probable that making large movements would negatively\n> +affect code clarity.\n> +\n> +== Git Standard Library boundary\n> +\n> +While I have described above some useful heuristics for identifying\n> +potential candidates for git-std-lib.a, a standard library should not\n> +have a shaky definition for what belongs in it.\n\nMaybe \"we need a more precise definition\" rather than the \"shaky \ndefinition\" bit\n\n> + - Low-level files (aka operates only on other primitive types) that are\n> + used everywhere within the codebase (wrapper.c, usage.c, strbuf.c)\n> + - Dependencies that are low-level and widely used\n> + (abspath.c, date.c, hex-ll.c, parse.c, utf8.c)\n> + - low-level git/* files with functions defined in git-compat-util.h\n> + (ctype.c)\n> + - compat/*\n> +\n> +There are other files that might fit this definition, but that does not\n> +mean it should belong in git-std-lib.a. Those files should start as\n> +their own separate library since any file added to git-std-lib.a loses\n> +its flexibility of being easily swappable.\n> +\n> +Wrapper.c and usage.c have dependencies on pager and trace2 that are\n> +possible to remove at the cost of sacrificing the ability for standard Git\n> +to be able to trace functions in those files and other files in git-std-lib.a.\n> +In order for git-std-lib.a to compile with those dependencies, stubbed out\n> +versions of those files are implemented and swapped in during compilation time\n> +(see STUB_LIB_OBJS in the Makefile).\n> +\n> +== Files inside of Git Standard Library\n> +\n> +The set of files in git-std-lib.a can be found in STD_LIB_OBJS and COMPAT_OBJS\n> +in the Makefile.\n> +\n> +When these files are compiled together with the files in STUB_LIB_OBJS (or\n> +user-provided files that provide the same functions), they form a complete\n> +library.\n> +\n> +== Pitfalls\n> +\n> +There are a small amount of files under compat/* that have dependencies\n\ns/amount/number/ as files are countable\n\n> +not inside of git-std-lib.a. While those functions are not called on\n> +Linux, other OSes might call those problematic functions. I don't see\n> +this as a major problem, just moreso an observation that libification in\n> +general may also require some minor compatibility work in the future.\n> +\n> +== Testing\n> +\n> +Unit tests should catch any breakages caused by changes to files in\n> +git-std-lib.a (i.e. introduction of a out of scope dependency) and new\n> +functions introduced to git-std-lib.a will require unit tests written\n\ns/test written/tests to be written/\n\n> +for them.\n> +\n> +[1] https://lore.kernel.org/git/xmqqwn17sydw.fsf@gitster.g/\n\nIt is nice to see us compiling git using git-std-lib.\n\nBest Wishes\n\nPhillip\n\n> diff --git a/Makefile b/Makefile\n> index 4e255c81f2..d37ea9d34b 100644\n> --- a/Makefile\n> +++ b/Makefile\n> @@ -669,6 +669,8 @@ FUZZ_PROGRAMS =\n> GIT_OBJS =\n> LIB_OBJS =\n> SCALAR_OBJS =\n> +STD_LIB_OBJS =\n> +STUB_LIB_OBJS =\n> OBJECTS =\n> OTHER_PROGRAMS =\n> PROGRAM_OBJS =\n> @@ -923,6 +925,8 @@ TEST_SHELL_PATH = $(SHELL_PATH)\n> \n> LIB_FILE = libgit.a\n> XDIFF_LIB = xdiff/lib.a\n> +STD_LIB_FILE = git-std-lib.a\n> +STUB_LIB_FILE = git-stub-lib.a\n> REFTABLE_LIB = reftable/libreftable.a\n> REFTABLE_TEST_LIB = reftable/libreftable_test.a\n> \n> @@ -962,7 +966,6 @@ COCCI_SOURCES = $(filter-out $(THIRD_PARTY_SOURCES),$(FOUND_C_SOURCES))\n> \n> LIB_H = $(FOUND_H_SOURCES)\n> \n> -LIB_OBJS += abspath.o\n> LIB_OBJS += add-interactive.o\n> LIB_OBJS += add-patch.o\n> LIB_OBJS += advice.o\n> @@ -1004,8 +1007,6 @@ LIB_OBJS += convert.o\n> LIB_OBJS += copy.o\n> LIB_OBJS += credential.o\n> LIB_OBJS += csum-file.o\n> -LIB_OBJS += ctype.o\n> -LIB_OBJS += date.o\n> LIB_OBJS += decorate.o\n> LIB_OBJS += delta-islands.o\n> LIB_OBJS += diagnose.o\n> @@ -1046,7 +1047,6 @@ LIB_OBJS += hash-lookup.o\n> LIB_OBJS += hashmap.o\n> LIB_OBJS += help.o\n> LIB_OBJS += hex.o\n> -LIB_OBJS += hex-ll.o\n> LIB_OBJS += hook.o\n> LIB_OBJS += ident.o\n> LIB_OBJS += json-writer.o\n> @@ -1097,7 +1097,6 @@ LIB_OBJS += pack-write.o\n> LIB_OBJS += packfile.o\n> LIB_OBJS += pager.o\n> LIB_OBJS += parallel-checkout.o\n> -LIB_OBJS += parse.o\n> LIB_OBJS += parse-options-cb.o\n> LIB_OBJS += parse-options.o\n> LIB_OBJS += patch-delta.o\n> @@ -1152,7 +1151,6 @@ LIB_OBJS += sparse-index.o\n> LIB_OBJS += split-index.o\n> LIB_OBJS += stable-qsort.o\n> LIB_OBJS += statinfo.o\n> -LIB_OBJS += strbuf.o\n> LIB_OBJS += streaming.o\n> LIB_OBJS += string-list.o\n> LIB_OBJS += strmap.o\n> @@ -1189,21 +1187,32 @@ LIB_OBJS += unpack-trees.o\n> LIB_OBJS += upload-pack.o\n> LIB_OBJS += url.o\n> LIB_OBJS += urlmatch.o\n> -LIB_OBJS += usage.o\n> LIB_OBJS += userdiff.o\n> -LIB_OBJS += utf8.o\n> LIB_OBJS += varint.o\n> LIB_OBJS += version.o\n> LIB_OBJS += versioncmp.o\n> LIB_OBJS += walker.o\n> LIB_OBJS += wildmatch.o\n> LIB_OBJS += worktree.o\n> -LIB_OBJS += wrapper.o\n> LIB_OBJS += write-or-die.o\n> LIB_OBJS += ws.o\n> LIB_OBJS += wt-status.o\n> LIB_OBJS += xdiff-interface.o\n> \n> +STD_LIB_OBJS += abspath.o\n> +STD_LIB_OBJS += ctype.o\n> +STD_LIB_OBJS += date.o\n> +STD_LIB_OBJS += hex-ll.o\n> +STD_LIB_OBJS += parse.o\n> +STD_LIB_OBJS += strbuf.o\n> +STD_LIB_OBJS += usage.o\n> +STD_LIB_OBJS += utf8.o\n> +STD_LIB_OBJS += wrapper.o\n> +\n> +STUB_LIB_OBJS += stubs/trace2.o\n> +STUB_LIB_OBJS += stubs/pager.o\n> +STUB_LIB_OBJS += stubs/misc.o\n> +\n> BUILTIN_OBJS += builtin/add.o\n> BUILTIN_OBJS += builtin/am.o\n> BUILTIN_OBJS += builtin/annotate.o\n> @@ -1352,7 +1361,7 @@ UNIT_TEST_OBJS = $(patsubst %,$(UNIT_TEST_DIR)/%.o,$(UNIT_TEST_PROGRAMS))\n> UNIT_TEST_OBJS += $(UNIT_TEST_DIR)/test-lib.o\n> \n> # xdiff and reftable libs may in turn depend on what is in libgit.a\n> -GITLIBS = common-main.o $(LIB_FILE) $(XDIFF_LIB) $(REFTABLE_LIB) $(LIB_FILE)\n> +GITLIBS = common-main.o $(STD_LIB_FILE) $(LIB_FILE) $(XDIFF_LIB) $(REFTABLE_LIB) $(LIB_FILE)\n> EXTLIBS =\n> \n> GIT_USER_AGENT = git/$(GIT_VERSION)\n> @@ -2693,6 +2702,8 @@ OBJECTS += $(XDIFF_OBJS)\n> OBJECTS += $(FUZZ_OBJS)\n> OBJECTS += $(REFTABLE_OBJS) $(REFTABLE_TEST_OBJS)\n> OBJECTS += $(UNIT_TEST_OBJS)\n> +OBJECTS += $(STD_LIB_OBJS)\n> +OBJECTS += $(STUB_LIB_OBJS)\n> \n> ifndef NO_CURL\n> \tOBJECTS += http.o http-walker.o remote-curl.o\n> @@ -3686,7 +3697,7 @@ clean: profile-clean coverage-clean cocciclean\n> \t$(RM) git.res\n> \t$(RM) $(OBJECTS)\n> \t$(RM) headless-git.o\n> -\t$(RM) $(LIB_FILE) $(XDIFF_LIB) $(REFTABLE_LIB) $(REFTABLE_TEST_LIB)\n> +\t$(RM) $(LIB_FILE) $(XDIFF_LIB) $(REFTABLE_LIB) $(REFTABLE_TEST_LIB) $(STD_LIB_FILE) $(STUB_LIB_FILE)\n> \t$(RM) $(ALL_PROGRAMS) $(SCRIPT_LIB) $(BUILT_INS) $(OTHER_PROGRAMS)\n> \t$(RM) $(TEST_PROGRAMS)\n> \t$(RM) $(FUZZ_PROGRAMS)\n> @@ -3878,3 +3889,18 @@ $(UNIT_TEST_PROGS): $(UNIT_TEST_BIN)/%$X: $(UNIT_TEST_DIR)/%.o $(UNIT_TEST_DIR)/\n> build-unit-tests: $(UNIT_TEST_PROGS)\n> unit-tests: $(UNIT_TEST_PROGS)\n> \t$(MAKE) -C t/ unit-tests\n> +\n> +### Libified Git rules\n> +\n> +# git-std-lib.a\n> +# Programs other than git should compile this with\n> +# make NO_GETTEXT=YesPlease git-std-lib.a\n> +# and link against git-stub-lib.a (if the default no-op functionality is fine)\n> +# or a custom .a file with the same interface as git-stub-lib.a (if custom\n> +# functionality is needed) as well.\n> +$(STD_LIB_FILE): $(STD_LIB_OBJS) $(COMPAT_OBJS)\n> +\t$(QUIET_AR)$(RM) $@ && $(AR) $(ARFLAGS) $@ $^\n> +\n> +# git-stub-lib.a\n> +$(STUB_LIB_FILE): $(STUB_LIB_OBJS)\n> +\t$(QUIET_AR)$(RM) $@ && $(AR) $(ARFLAGS) $@ $^\n> diff --git a/stubs/misc.c b/stubs/misc.c\n> new file mode 100644\n> index 0000000000..92da76fd46\n> --- /dev/null\n> +++ b/stubs/misc.c\n> @@ -0,0 +1,33 @@\n> +#include \n> +#include \n> +\n> +#ifndef NO_GETTEXT\n> +/*\n> + * NEEDSWORK: This is enough to link our unit tests against\n> + * git-std-lib.a built with gettext support. We don't really support\n> + * programs other than git using git-std-lib.a with gettext support\n> + * yet. To do that we need to start using dgettext() rather than\n> + * gettext() in our code.\n> + */\n> +int git_gettext_enabled = 0;\n> +#endif\n> +\n> +int common_exit(const char *file, int line, int code);\n> +\n> +int common_exit(const char *file, int line, int code)\n> +{\n> +\texit(code);\n> +}\n> +\n> +#if !defined(__MINGW32__) && !defined(_MSC_VER)\n> +int lstat_cache_aware_rmdir(const char *path);\n> +\n> +int lstat_cache_aware_rmdir(const char *path)\n> +{\n> +\t/*\n> +\t * This function should not be called by programs linked\n> +\t * against git-stub-lib.a\n> +\t */\n> +\tassert(0);\n> +}\n> +#endif\n> diff --git a/stubs/pager.c b/stubs/pager.c\n> new file mode 100644\n> index 0000000000..4f575cada7\n> --- /dev/null\n> +++ b/stubs/pager.c\n> @@ -0,0 +1,6 @@\n> +#include \"pager.h\"\n> +\n> +int pager_in_use(void)\n> +{\n> +\treturn 0;\n> +}\n> diff --git a/stubs/trace2.c b/stubs/trace2.c\n> new file mode 100644\n> index 0000000000..7d89482228\n> --- /dev/null\n> +++ b/stubs/trace2.c\n> @@ -0,0 +1,27 @@\n> +#include \"git-compat-util.h\"\n> +#include \"trace2.h\"\n> +\n> +struct child_process { int stub; };\n> +struct repository { int stub; };\n> +struct json_writer { int stub; };\n> +\n> +void trace2_region_enter_fl(const char *file, int line, const char *category,\n> +\t\t\t const char *label, const struct repository *repo, ...) { }\n> +void trace2_region_leave_fl(const char *file, int line, const char *category,\n> +\t\t\t const char *label, const struct repository *repo, ...) { }\n> +void trace2_data_string_fl(const char *file, int line, const char *category,\n> +\t\t\t const struct repository *repo, const char *key,\n> +\t\t\t const char *value) { }\n> +void trace2_cmd_ancestry_fl(const char *file, int line, const char **parent_names) { }\n> +void trace2_cmd_error_va_fl(const char *file, int line, const char *fmt,\n> +\t\t\t va_list ap) { }\n> +void trace2_cmd_name_fl(const char *file, int line, const char *name) { }\n> +void trace2_thread_start_fl(const char *file, int line,\n> +\t\t\t const char *thread_base_name) { }\n> +void trace2_thread_exit_fl(const char *file, int line) { }\n> +void trace2_data_intmax_fl(const char *file, int line, const char *category,\n> +\t\t\t const struct repository *repo, const char *key,\n> +\t\t\t intmax_t value) { }\n> +int trace2_is_enabled(void) { return 0; }\n> +void trace2_counter_add(enum trace2_counter_id cid, uint64_t value) { }\n> +void trace2_collect_process_info(enum trace2_process_info_reason reason) { }\n"},{"id":"489669","messageId":"xmqq1q8vi3pz.fsf@gitster.g","threadId":"59921","inReplyTo":"db8b2398-c6d6-47a1-a4d4-7e4726d13312@gmail.com","subject":"Re: [PATCH v5 2/3] git-std-lib: introduce Git Standard Library","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2024-02-29T17:23:52Z","receivedAt":"2024-02-29T17:23:55Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Phillip Wood writes:\n\n> Hi Calvin\n> On 22/02/2024 17:50, Calvin Wan wrote:\n\nThanks for reviewing -- I held mine back as I expected this will see\nanother reroll soonish, but you already have raised many points I\nalso had trouble with, so I do not have to ;-) \n\nBelow, I'll liberally omit everything you wrote that I agree with.\n\n>> +libgit.a, which is then linked against by common-main.o with other\n>> +external dependencies and turned into the Git executable.\n>\n> I found this description a bit confusing. As I understand it to build\n> git we link git.o against common-main.o, libgit.a, xdiff/lib.a,\n> reftable/libreftable.a and libpcre etc.\n\nIn addition, there is no single \"the Git executable\", simply because\nnot everything is builtin command. The purpose of using libgit.a is\nbecause we are too lazy to list and maintain all the internal\ndependencies to link final executables like 'git' (which has all the\nbuilt-in command implementations) and 'git-remote-curl' (which is a\nstandalone program). Instead of feeding exact list of object files\nto \"$(CC) -o git\" command line, we throw everything into libgit.a\nand let the linker pick what is needed. To link \"git\", we may\ninclude all builtin/*.o, git.o, common-main.o, libgit.a and the\nexternal [*] library dependencies they have. To link \"git-daemon\",\nwe may not link builtin/*.o and git.o and link daemon.o instead.\n\n\tSide note: here I am counting xdiff/lib.a as an external\n\tlibrary as it is mostly a borrowed code.\n\nIn other words, libgit.a is not a true library in the sense that it\nwas designed to be _used_ as a library. It was merely a detail of\nhow we implemented lazy dependency management in our build process,\nwhich happend with 0a02ce72 (Clean up the Makefile a bit.,\n2005-04-18) whose commit log message uses air-quotes around the word\n\"library\".\n\n>> +From the above dependency graph, we can see that git-std-lib.a could be\n>> +many smaller libraries rather than a singular library. So why choose a\n>> +singular library when multiple libraries can be individually easier to\n>> +swap and are more modular? A singular library requires less work to\n>> +separate out circular dependencies within itself so it becomes a\n>> +tradeoff question between work and reward. While there may be a point in\n>> +the future where a file like usage.c would want its own library so that\n>> +someone can have custom die() or error(), the work required to refactor\n>> +out the circular dependencies in some files would be enormous due to\n>> +their ubiquity so therefore I believe it is not worth the tradeoff\n>\n> I'm not sure if we want to use the first person in our technical\n> documentation, unlike the cover letter to a patch series it is not\n> immediately obvious to the reader who \"I\" is. This applies the\n> passages in the first person below as well.\n\nI found it highly annoying while reading it, too. If the document\n(not the commit that introduced the document) were signed and\nwritten to state the position of one author, as opposed to spell out\nthe position the project will collectively take, it would have been\nOK, but this document is meant to set a course of the project (and\ndiscussion on it is the process to decide which course to take), the\nfirst person singular \"I\" did not sit well for me.\n\nAnother thing to consider that I do not think you covered is the\nname of the resulting .a archive. By starting it with \"lib\", a\ncustomer can find your libstdgit.a with -lstdgit on the command\nline, once libstdgit.a is installed at an appropriate location (or\nthe build-time library path is configured to point the location you\nhave libstdgit.a at). libgit.a that wasn't really designed to be\nused as such a library did not have to follow the naming convention,\nbut if the thing being proposed is meant to be eventually used as a\nlibrary by external entities, git-std-lib.a is a rather poor name\nfor it.\n\n\nPS.\n\nI seem to have been hit by a power outage and am on UPS, so I'll\nprobably be offline until the power comes back.\n\n\n\n"},{"id":"489672","messageId":"owly34tb5dop.fsf@fine.c.googlers.com","threadId":"59921","inReplyTo":"xmqq1q8vi3pz.fsf@gitster.g","subject":"Re: [PATCH v5 2/3] git-std-lib: introduce Git Standard Library","fromName":"Linus Arver","fromEmail":"linusa@google.com","sentAt":"2024-02-29T18:27:02Z","receivedAt":"2024-02-29T18:27:05Z","isPatch":true,"sender":{"key":"linus@ucla.edu","avatar":null},"body":"Junio C Hamano writes:\n\n> Phillip Wood writes:\n>\n>> On 22/02/2024 17:50, Calvin Wan wrote:\n>\n>>> +libgit.a, which is then linked against by common-main.o with other\n>>> +external dependencies and turned into the Git executable.\n>>\n>> I found this description a bit confusing. As I understand it to build\n>> git we link git.o against common-main.o, libgit.a, xdiff/lib.a,\n>> reftable/libreftable.a and libpcre etc.\n>\n> In addition, there is no single \"the Git executable\", simply because\n> not everything is builtin command. The purpose of using libgit.a is\n> because we are too lazy to list and maintain all the internal\n> dependencies to link final executables like 'git' (which has all the\n> built-in command implementations) and 'git-remote-curl' (which is a\n> standalone program). Instead of feeding exact list of object files\n> to \"$(CC) -o git\" command line, we throw everything into libgit.a\n> and let the linker pick what is needed. To link \"git\", we may\n> include all builtin/*.o, git.o, common-main.o, libgit.a and the\n> external [*] library dependencies they have. To link \"git-daemon\",\n> we may not link builtin/*.o and git.o and link daemon.o instead.\n>\n> \tSide note: here I am counting xdiff/lib.a as an external\n> \tlibrary as it is mostly a borrowed code.\n>\n> In other words, libgit.a is not a true library in the sense that it\n> was designed to be _used_ as a library. It was merely a detail of\n> how we implemented lazy dependency management in our build process,\n> which happend with 0a02ce72 (Clean up the Makefile a bit.,\n> 2005-04-18) whose commit log message uses air-quotes around the word\n> \"library\".\n\nSomehow I did not realize that this was going on. Thank you for pointing\nthis out!\n\nIt does make me wonder if we should stop being lazy and do the work that\nthe linker has been doing for us \"for free\" ourselves. IOW, stop linking\nagainst a monolithic libgit.a. That way we would replace implicit\ndependencies with explicit ones, which might help us understand which\nthings need what.\n\nBut maybe doing that is super painful, so, maybe it's not worth it. IDK.\n"},{"id":"489673","messageId":"xmqq8r33gkxu.fsf@gitster.g","threadId":"59921","inReplyTo":"owly34tb5dop.fsf@fine.c.googlers.com","subject":"Re: [PATCH v5 2/3] git-std-lib: introduce Git Standard Library","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2024-02-29T18:54:53Z","receivedAt":"2024-02-29T18:55:04Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Linus Arver writes:\n\n> It does make me wonder if we should stop being lazy and do the\n> work that the linker has been doing for us \"for free\"\n> ourselves. IOW, stop linking against a monolithic libgit.a.\n> ... which might help us understand which things need what.\n\nSorry, but I fail see a point in such an exercise. If a tool is\navailable to help us and if there is no downside of using the tool,\nwe should keep using it. If you are proposing to move away from the\ncurrent build practice because you have a concrete downside of the\napproach and avoid that, then it might be a good proposal, though.\n\nAnd \"we do not learn otherwise\" is not a downside of the approach;\n\"we do not learn\" comes from your not learning, the tools do not\nforce you to be ignorant. We do not propose to use more __asm__ in\nour C sources only because compilers were doing that for us \"for\nfree\" and because the compilers were somehow robbing us the\nopportunity to learn micro-optimization techniques, do we?\n\nA small downside I can immediately think of is possible in a\nsituation where we have been throwing an object file into libgit.a\narchive that is no longer used by any final executable. In such a\nscenario, if you change the source file that is compiled into such\nan unused object file, your next \"make\" will update libgit.a to\nreplace the unused object file with its new version with your\nupdates, and that would cause the final build product to be linked\nagain with objects needed from libgit.a, but there shouldn't be any\nchange because we are talking about an object that is *not* used by\nthem but still is in libgit.a due to be listed on LIB_OBJS variable.\n\nBut that is a purely theoretical downside. It may be the case that\nwe haven't done our spring cleaning recently and we haven't noticed\nthat a source file or two are now unused but are still listed on\nLIB_OBJS to be included in the libgit.a archive. But even if that\nwere the case, it is implausible that you are touching such an\nunused source file in the first place.\n"},{"id":"489688","messageId":"owlyzfvj3unf.fsf@fine.c.googlers.com","threadId":"59921","inReplyTo":"xmqq8r33gkxu.fsf@gitster.g","subject":"Re: [PATCH v5 2/3] git-std-lib: introduce Git Standard Library","fromName":"Linus Arver","fromEmail":"linusa@google.com","sentAt":"2024-02-29T20:03:32Z","receivedAt":"2024-02-29T20:03:34Z","isPatch":true,"sender":{"key":"linus@ucla.edu","avatar":null},"body":"Junio C Hamano writes:\n\n> Linus Arver writes:\n>\n>> It does make me wonder if we should stop being lazy and do the\n>> work that the linker has been doing for us \"for free\"\n>> ourselves. IOW, stop linking against a monolithic libgit.a.\n>> ... which might help us understand which things need what.\n>\n> [...] If a tool is\n> available to help us and if there is no downside of using the tool,\n> we should keep using it.\n\nOf course, if there is no downside, we should use the tool as is.\n\n> If you are proposing to move away from the\n> current build practice because you have a concrete downside of the\n> approach and avoid that, then it might be a good proposal, though.\n\nRight. I was just wondering if the \"explicit dependencies declared in\nthe Makefile\" would provide some value WRT libification. Currently IDK\nthe answer to that.\n\n> And \"we do not learn otherwise\" is not a downside of the approach;\n> \"we do not learn\" comes from your not learning, the tools do not\n> force you to be ignorant. We do not propose to use more __asm__ in\n> our C sources only because compilers were doing that for us \"for\n> free\" and because the compilers were somehow robbing us the\n> opportunity to learn micro-optimization techniques, do we?\n\nTrue.\n\n> A small downside I can immediately think of is possible in a\n> situation where we have been throwing an object file into libgit.a\n> archive that is no longer used by any final executable. In such a\n> scenario, if you change the source file that is compiled into such\n> an unused object file, your next \"make\" will update libgit.a to\n> replace the unused object file with its new version with your\n> updates, and that would cause the final build product to be linked\n> again with objects needed from libgit.a, but there shouldn't be any\n> change because we are talking about an object that is *not* used by\n> them but still is in libgit.a due to be listed on LIB_OBJS variable.\n\nIIUC, this (theoretical) downside will result in Make thinking that it\nneeds to rebuild libgit.a when it actually doesn't need to (because the\nupdated change is for an unused object). So it could slow down the build\nunnecessarily. Makes sense.\n\n> But that is a purely theoretical downside. It may be the case that\n> we haven't done our spring cleaning recently and we haven't noticed\n> that a source file or two are now unused but are still listed on\n> LIB_OBJS to be included in the libgit.a archive. But even if that\n> were the case, it is implausible that you are touching such an\n> unused source file in the first place.\n\nAs you noted, libgit.a is not a true library; it's just a big archive of\neverything and we let the linker figure out what the executables need\nout of it. But I was under the impression that with Git libification, we\nwould want to create a real library in the fullest sense of the word ---\nsuch that our executables also need this library (\"-lgit\") to be built\nin the exact same way that external programs would need this same\n(single) Git library. For example, I believe this is how curl is built\n(it first builds libcurl, and then links against it as an internal user\nfor generating the \"curl\" executable).\n\nGoing back to libgit.a, I was just wondering if the exercise of breaking\nit up into smaller pieces would have been helpful in figuring out this\n\"-lgit\" library (or what a smaller version of it would look like).\n\nI sense that I may be missing large pieces of context around the\ngit-std-lib discussions though, so I apologize if my points above are\nnot new and moot. Thanks.\n"},{"id":"490222","messageId":"xmqqsf11kaoq.fsf@gitster.g","threadId":"59921","inReplyTo":"20240222175033.1489723-4-calvinwan@google.com","subject":"Re: [PATCH v5 3/3] test-stdlib: show that git-std-lib is independent","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2024-03-07T21:13:09Z","receivedAt":"2024-03-07T21:13:12Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Calvin Wan writes:\n\n> +\tstrbuf_commented_addf(sb, '#', \"%s\", \"foo\");\n\nOf course, this will need to be adjusted when it meets the \"let's\nallow more than one byte for a comment character\" series by Peff.\nIt should now read\n\n\tstrbuf_commented_addf(sb, \"#\", \"%s\", \"foo\");\n\nof course.\n\nThis is a usual \"a function changes in one topic, while the other\ntopic adds more callers to it\" problem a maintainer is expected to\nhandle fine, so there is nothing special that needs to be done by\ncontributors, but just giving you a head's up when you yourself test\nyour updated version to ensure it works well with other topics in\nflight.\n\n"}]}