{"thread":{"id":"58383","subject":"[PATCH 0/8] scalar: integrate into core Git","startedAt":"2022-08-31T16:03:06Z","lastAt":"2022-11-01T01:55:44Z","messageCount":70,"participants":["Victoria Dye via GitGitGadget","Johannes Schindelin via GitGitGadget","Ævar Arnfjörð Bjarmason","Victoria Dye","Johannes Schindelin","Derrick Stolee","Junio C Hamano","Jeff King","Taylor Blau"],"isPatch":true,"patchVersion":1,"patchTotal":8},"messages":[{"id":"462281","messageId":"pull.1341.git.1661961746.gitgitgadget@gmail.com","threadId":"58383","inReplyTo":null,"subject":"[PATCH 0/8] scalar: integrate into core Git","fromName":"Victoria Dye via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2022-08-31T16:02:17Z","receivedAt":"2022-08-31T16:03:06Z","isPatch":true,"sender":{"key":"vdye@github.com","avatar":"https://avatars.githubusercontent.com/u/3619353?v=4"},"body":"This series completes the initial implementation of the Scalar command as a\ncore component of Git. For readers new to the topic of Scalar, the\nroadmap/design doc [1] provides some background information including how\nthe project started & evolved, as well as its current intent & high-level\ndesign.\n\n\nGoals\n=====\n\nThe goal of this series is, as noted in the roadmap linked earlier, to\nintegrate Scalar as a core user-facing component of Git. There's no\nindividual existing precedent that the integration is 100% based on; it's\nmore of an amalgam of the approaches used with builtins installed directly\nto the bindir (e.g. 'git-upload-pack'), contributed tools (e.g. 'gitk'), and\nthe 'git' executable itself. The resulting behavior is:\n\n * Scalar is built & installed unconditionally (like a builtin).\n * Scalar is included in 'bin-wrappers/' and is installed into the 'bin/'\n directory (like 'git-upload-pack').\n * Scalar's documentation is installed as 'man1' docs (like 'gitk').\n * Scalar is not invoked via the 'git' executable (it's called directly,\n like 'git' itself).\n\nThe move out of 'contrib/' is also intended to mean that the relationship\nbetween Git and Scalar should no longer be a strict one-way dependency. In\nthis series, this architectural change is what permits making a change to\n'help.c' specifically for the sake of Scalar. Future contributions can and\nshould (where appropriate!) share code between Scalar and non-Scalar parts\nof Git.\n\n\nOrganization\n============\n\nThe series is broken up as follows:\n\n 1. Fix a CI-breaking documentation error in 'scalar.txt' (patch 1).\n 2. Move Scalar code out of 'contrib/' and into core Git, including changes\n in build definition(s) to build & install Scalar and its resources\n (patch 2).\n 3. Make 'git help scalar' open the 'scalar' command documentation,\n introduce 'scalar help' as an alias. This is done after moving out of\n 'contrib/' due to it involving Scalar-related changes to the non-Scalar\n file 'help.c' (patches 3-4).\n 4. Expand test coverage, including perf testing (patches 5-6)\n 5. Add a perf test configuration allowing users to register standard perf\n test repos with Scalar (patch 7).\n 6. Remove the internally-focused Scalar roadmap from\n 'Documentation/technical/scalar.txt', include it as one of the HTML\n technical docs (patch 8).\n\n\nPrior series\n============\n\n * Add 'scalar' command in 'contrib/':\n https://lore.kernel.org/git/pull.1005.v10.git.1638538470.gitgitgadget@gmail.com/\n * Introduce 'scalar diagnose':\n https://lore.kernel.org/git/pull.1128.v6.git.1653145696.gitgitgadget@gmail.com/\n * Add '-c/-C' compatibility:\n https://lore.kernel.org/git/pull.1130.v2.git.1643380317358.gitgitgadget@gmail.com/\n * [DROPPED] Integrate Scalar into CI builds:\n https://lore.kernel.org/git/pull.1129.git.1654160735.gitgitgadget@gmail.com/\n * Document Scalar's role in Git & plan remaining work:\n https://lore.kernel.org/git/pull.1275.v2.git.1657584367.gitgitgadget@gmail.com/\n * Generalize 'scalar diagnose' into 'git diagnose' builtin & 'git bugreport\n --diagnose':\n https://lore.kernel.org/git/pull.1310.v4.git.1660335019.gitgitgadget@gmail.com/\n * Add FSMonitor support to Scalar, refactor enlistment search:\n https://lore.kernel.org/git/pull.1324.v3.git.1660858853.gitgitgadget@gmail.com/\n\nThanks!\n\n * Victoria\n\n[1]\nhttps://lore.kernel.org/git/65eda9755d2a29e222bf7e6cc719921b24377cc6.1657584367.git.gitgitgadget@gmail.com/\n\nJohannes Schindelin (2):\n git help: special-case `scalar`\n scalar: implement the `help` subcommand\n\nVictoria Dye (6):\n scalar: fix command documentation section header\n scalar: include in standard Git build & installation\n scalar-clone: add test coverage\n t/perf: add Scalar performance tests\n t/perf: add 'GIT_PERF_USE_SCALAR' run option\n Documentation/technical: include Scalar technical doc\n\n .gitignore | 1 +\n Documentation/Makefile | 2 +\n {contrib/scalar => Documentation}/scalar.txt | 2 +-\n Documentation/technical/scalar.txt | 61 --------\n Makefile | 30 ++--\n builtin/help.c | 2 +\n contrib/buildsystems/CMakeLists.txt | 9 +-\n contrib/scalar/.gitignore | 2 -\n contrib/scalar/Makefile | 35 -----\n contrib/scalar/t/Makefile | 81 -----------\n contrib/scalar/scalar.c => scalar.c | 20 +++\n t/perf/README | 4 +\n t/perf/p9210-scalar.sh | 43 ++++++\n t/perf/perf-lib.sh | 13 +-\n t/perf/run | 3 +\n .../t/t9099-scalar.sh => t/t9210-scalar.sh | 10 +-\n t/t9211-scalar-clone.sh | 135 ++++++++++++++++++\n 17 files changed, 249 insertions(+), 204 deletions(-)\n rename {contrib/scalar => Documentation}/scalar.txt (99%)\n delete mode 100644 contrib/scalar/.gitignore\n delete mode 100644 contrib/scalar/Makefile\n delete mode 100644 contrib/scalar/t/Makefile\n rename contrib/scalar/scalar.c => scalar.c (98%)\n create mode 100755 t/perf/p9210-scalar.sh\n rename contrib/scalar/t/t9099-scalar.sh => t/t9210-scalar.sh (96%)\n create mode 100755 t/t9211-scalar-clone.sh\n\n\nbase-commit: d42b38dfb5edf1a7fddd9542d722f91038407819\nPublished-As: https://github.com/gitgitgadget/git/releases/tag/pr-1341%2Fvdye%2Ffeature%2Fscalar-toplevel-v1\nFetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-1341/vdye/feature/scalar-toplevel-v1\nPull-Request: https://github.com/gitgitgadget/git/pull/1341\n-- \ngitgitgadget\n"},{"id":"462282","messageId":"fcf73e775746b102a4d5196c5ebbb06467e08cf6.1661961746.git.gitgitgadget@gmail.com","threadId":"58383","inReplyTo":"pull.1341.git.1661961746.gitgitgadget@gmail.com","subject":"[PATCH 1/8] scalar: fix command documentation section header","fromName":"Victoria Dye via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2022-08-31T16:02:18Z","receivedAt":"2022-08-31T16:03:09Z","isPatch":true,"sender":{"key":"vdye@github.com","avatar":"https://avatars.githubusercontent.com/u/3619353?v=4"},"body":"From: Victoria Dye \n\nRename the last section header in 'contrib/scalar/scalar.txt' from \"Scalar\"\nto \"GIT\". The linting rules of the 'documentation' CI build enforce the\nexistence of a \"GIT\" section in command documentation. Although 'scalar.txt'\nis not yet checked, it will be in a future patch.\n\nHere, changing the header name is more appropriate than making a\nScalar-specific exception to the linting rule. The existing \"Scalar\" section\ncontains only a link back to the main Git documentation, essentially the\nsame as the \"GIT\" section in builtin documentation. Changing the section\nname further clarifies the Scalar-Git association and maintains consistency\nwith the rest of Git.\n\nSigned-off-by: Victoria Dye \n---\n contrib/scalar/scalar.txt | 2 +-\n 1 file changed, 1 insertion(+), 1 deletion(-)\n\ndiff --git a/contrib/scalar/scalar.txt b/contrib/scalar/scalar.txt\nindex 1a12dc45077..505a1cea0fd 100644\n--- a/contrib/scalar/scalar.txt\n+++ b/contrib/scalar/scalar.txt\n@@ -161,6 +161,6 @@ SEE ALSO\n --------\n linkgit:git-clone[1], linkgit:git-maintenance[1].\n \n-Scalar\n+GIT\n ---\n Associated with the linkgit:git[1] suite\n-- \ngitgitgadget\n\n"},{"id":"462283","messageId":"969160650aa08f0cb6ef03cbdd3324f640a20a4e.1661961746.git.gitgitgadget@gmail.com","threadId":"58383","inReplyTo":"pull.1341.git.1661961746.gitgitgadget@gmail.com","subject":"[PATCH 3/8] git help: special-case `scalar`","fromName":"Johannes Schindelin via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2022-08-31T16:02:20Z","receivedAt":"2022-08-31T16:03:23Z","isPatch":true,"sender":{"key":"johannes.schindelin@gmx.de","avatar":"https://avatars.githubusercontent.com/u/127790?v=4"},"body":"From: Johannes Schindelin \n\nWith this commit, `git help scalar` will open the appropriate manual\nor HTML page (instead of looking for `gitscalar`).\n\nSigned-off-by: Johannes Schindelin \nSigned-off-by: Victoria Dye \n---\n builtin/help.c | 2 ++\n 1 file changed, 2 insertions(+)\n\ndiff --git a/builtin/help.c b/builtin/help.c\nindex 09ac4289f13..6f2796f211e 100644\n--- a/builtin/help.c\n+++ b/builtin/help.c\n@@ -440,6 +440,8 @@ static const char *cmd_to_page(const char *git_cmd)\n \t\treturn git_cmd;\n \telse if (is_git_command(git_cmd))\n \t\treturn xstrfmt(\"git-%s\", git_cmd);\n+\telse if (!strcmp(\"scalar\", git_cmd))\n+\t\treturn xstrdup(git_cmd);\n \telse\n \t\treturn xstrfmt(\"git%s\", git_cmd);\n }\n-- \ngitgitgadget\n\n"},{"id":"462284","messageId":"4d69e5eaccb8873eece666a3d2bb2b22abdde7ea.1661961746.git.gitgitgadget@gmail.com","threadId":"58383","inReplyTo":"pull.1341.git.1661961746.gitgitgadget@gmail.com","subject":"[PATCH 2/8] scalar: include in standard Git build & installation","fromName":"Victoria Dye via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2022-08-31T16:02:19Z","receivedAt":"2022-08-31T16:03:26Z","isPatch":true,"sender":{"key":"vdye@github.com","avatar":"https://avatars.githubusercontent.com/u/3619353?v=4"},"body":"From: Victoria Dye \n\nMove 'scalar' out of 'contrib/' and into the root of the Git tree. The goal\nof this change is to build 'scalar' as part of the standard Git build &\ninstall processes.\n\nThis patch includes both the physical move of Scalar's files out of\n'contrib/' ('scalar.c', 'scalar.txt', and 't9xxx-scalar.sh'), and the\nchanges to the build definitions in 'Makefile' and 'CMakelists.txt' to\naccommodate the new program.\n\nAt a high level, Scalar is built so that:\n- there is a 'scalar-objs' target (similar to those created in 029bac01a8\n (Makefile: add {program,xdiff,test,git,fuzz}-objs & objects targets,\n 2021-02-23)) for debugging purposes.\n- it appears in the root of the install directory (rather than the\n gitexecdir).\n- it is included in the 'bin-wrappers/' directory for use in tests.\n- it receives a platform-specific executable suffix (e.g., '.exe'), if\n applicable.\n- 'scalar.txt' is installed as 'man1' documentation.\n- the 'clean' target removes the 'scalar' executable.\n\nAdditionally, update the root level '.gitignore' file to ignore the Scalar\nexecutable.\n\nSigned-off-by: Victoria Dye \n---\n .gitignore | 1 +\n Documentation/Makefile | 1 +\n {contrib/scalar => Documentation}/scalar.txt | 0\n Makefile | 30 ++++---\n contrib/buildsystems/CMakeLists.txt | 9 ++-\n contrib/scalar/.gitignore | 2 -\n contrib/scalar/Makefile | 35 --------\n contrib/scalar/t/Makefile | 81 -------------------\n contrib/scalar/scalar.c => scalar.c | 0\n .../t/t9099-scalar.sh => t/t9210-scalar.sh | 10 +--\n 10 files changed, 28 insertions(+), 141 deletions(-)\n rename {contrib/scalar => Documentation}/scalar.txt (100%)\n delete mode 100644 contrib/scalar/.gitignore\n delete mode 100644 contrib/scalar/Makefile\n delete mode 100644 contrib/scalar/t/Makefile\n rename contrib/scalar/scalar.c => scalar.c (100%)\n rename contrib/scalar/t/t9099-scalar.sh => t/t9210-scalar.sh (96%)\n\ndiff --git a/.gitignore b/.gitignore\nindex 80b530bbed2..3d1b880101e 100644\n--- a/.gitignore\n+++ b/.gitignore\n@@ -185,6 +185,7 @@\n /git-whatchanged\n /git-worktree\n /git-write-tree\n+/scalar\n /git-core-*/?*\n /git.res\n /gitweb/GITWEB-BUILD-OPTIONS\ndiff --git a/Documentation/Makefile b/Documentation/Makefile\nindex bd6b6fcb930..16c9e062390 100644\n--- a/Documentation/Makefile\n+++ b/Documentation/Makefile\n@@ -21,6 +21,7 @@ MAN1_TXT += $(filter-out \\\n MAN1_TXT += git.txt\n MAN1_TXT += gitk.txt\n MAN1_TXT += gitweb.txt\n+MAN1_TXT += scalar.txt\n \n # man5 / man7 guides (note: new guides should also be added to command-list.txt)\n MAN5_TXT += gitattributes.txt\ndiff --git a/contrib/scalar/scalar.txt b/Documentation/scalar.txt\nsimilarity index 100%\nrename from contrib/scalar/scalar.txt\nrename to Documentation/scalar.txt\ndiff --git a/Makefile b/Makefile\nindex eac30126e29..66dd3321f57 100644\n--- a/Makefile\n+++ b/Makefile\n@@ -608,7 +608,9 @@ FUZZ_OBJS =\n FUZZ_PROGRAMS =\n GIT_OBJS =\n LIB_OBJS =\n+SCALAR_OBJS =\n OBJECTS =\n+OTHER_PROGRAMS =\n PROGRAM_OBJS =\n PROGRAMS =\n EXCLUDED_PROGRAMS =\n@@ -821,10 +823,12 @@ BUILT_INS += git-switch$X\n BUILT_INS += git-whatchanged$X\n \n # what 'all' will build but not install in gitexecdir\n-OTHER_PROGRAMS = git$X\n+OTHER_PROGRAMS += git$X\n+OTHER_PROGRAMS += scalar$X\n \n # what test wrappers are needed and 'install' will install, in bindir\n BINDIR_PROGRAMS_NEED_X += git\n+BINDIR_PROGRAMS_NEED_X += scalar\n BINDIR_PROGRAMS_NEED_X += git-receive-pack\n BINDIR_PROGRAMS_NEED_X += git-shell\n BINDIR_PROGRAMS_NEED_X += git-upload-archive\n@@ -2222,7 +2226,7 @@ profile-fast: profile-clean\n \n all:: $(ALL_COMMANDS_TO_INSTALL) $(SCRIPT_LIB) $(OTHER_PROGRAMS) GIT-BUILD-OPTIONS\n ifneq (,$X)\n-\t$(QUIET_BUILT_IN)$(foreach p,$(patsubst %$X,%,$(filter %$X,$(ALL_COMMANDS_TO_INSTALL) git$X)), test -d '$p' -o '$p' -ef '$p$X' || $(RM) '$p';)\n+\t$(QUIET_BUILT_IN)$(foreach p,$(patsubst %$X,%,$(filter %$X,$(ALL_COMMANDS_TO_INSTALL) $(OTHER_PROGRAMS))), test -d '$p' -o '$p' -ef '$p$X' || $(RM) '$p';)\n endif\n \n all::\n@@ -2545,7 +2549,12 @@ GIT_OBJS += git.o\n .PHONY: git-objs\n git-objs: $(GIT_OBJS)\n \n+SCALAR_OBJS += scalar.o\n+.PHONY: scalar-objs\n+scalar-objs: $(SCALAR_OBJS)\n+\n OBJECTS += $(GIT_OBJS)\n+OBJECTS += $(SCALAR_OBJS)\n OBJECTS += $(PROGRAM_OBJS)\n OBJECTS += $(TEST_OBJS)\n OBJECTS += $(XDIFF_OBJS)\n@@ -2556,10 +2565,6 @@ ifndef NO_CURL\n \tOBJECTS += http.o http-walker.o remote-curl.o\n endif\n \n-SCALAR_SOURCES := contrib/scalar/scalar.c\n-SCALAR_OBJECTS := $(SCALAR_SOURCES:c=o)\n-OBJECTS += $(SCALAR_OBJECTS)\n-\n .PHONY: objects\n objects: $(OBJECTS)\n \n@@ -2691,7 +2696,7 @@ $(REMOTE_CURL_PRIMARY): remote-curl.o http.o http-walker.o GIT-LDFLAGS $(GITLIBS\n \t$(QUIET_LINK)$(CC) $(ALL_CFLAGS) -o $@ $(ALL_LDFLAGS) $(filter %.o,$^) \\\n \t\t$(CURL_LIBCURL) $(EXPAT_LIBEXPAT) $(LIBS)\n \n-contrib/scalar/scalar$X: $(SCALAR_OBJECTS) GIT-LDFLAGS $(GITLIBS)\n+scalar$X: scalar.o GIT-LDFLAGS $(GITLIBS)\n \t$(QUIET_LINK)$(CC) $(ALL_CFLAGS) -o $@ $(ALL_LDFLAGS) \\\n \t\t$(filter %.o,$^) $(LIBS)\n \n@@ -2747,7 +2752,7 @@ XGETTEXT_FLAGS_SH = $(XGETTEXT_FLAGS) --language=Shell \\\n XGETTEXT_FLAGS_PERL = $(XGETTEXT_FLAGS) --language=Perl \\\n \t--keyword=__ --keyword=N__ --keyword=\"__n:1,2\"\n MSGMERGE_FLAGS = --add-location --backup=off --update\n-LOCALIZED_C = $(sort $(FOUND_C_SOURCES) $(FOUND_H_SOURCES) $(SCALAR_SOURCES) \\\n+LOCALIZED_C = $(sort $(FOUND_C_SOURCES) $(FOUND_H_SOURCES) \\\n \t $(GENERATED_H))\n LOCALIZED_SH = $(sort $(SCRIPT_SH) git-sh-setup.sh)\n LOCALIZED_PERL = $(sort $(SCRIPT_PERL))\n@@ -3062,7 +3067,7 @@ bin-wrappers/%: wrap-for-bin.sh\n \t$(call mkdir_p_parent_template)\n \t$(QUIET_GEN)sed -e '1s|#!.*/sh|#!$(SHELL_PATH_SQ)|' \\\n \t -e 's|@@BUILD_DIR@@|$(shell pwd)|' \\\n-\t -e 's|@@PROG@@|$(patsubst test-%,t/helper/test-%$(X),$(@F))$(patsubst git%,$(X),$(filter $(@F),$(BINDIR_PROGRAMS_NEED_X)))|' < $< > $@ && \\\n+\t -e 's|@@PROG@@|$(patsubst test-%,t/helper/test-%,$(@F))$(if $(filter-out $(BINDIR_PROGRAMS_NO_X),$(@F)),$(X),)|' < $< > $@ && \\\n \tchmod +x $@\n \n # GNU make supports exporting all variables by \"export\" without parameters.\n@@ -3276,14 +3281,14 @@ ifndef NO_TCLTK\n \t$(MAKE) -C git-gui gitexecdir='$(gitexec_instdir_SQ)' install\n endif\n ifneq (,$X)\n-\t$(foreach p,$(patsubst %$X,%,$(filter %$X,$(ALL_COMMANDS_TO_INSTALL) git$X)), test '$(DESTDIR_SQ)$(gitexec_instdir_SQ)/$p' -ef '$(DESTDIR_SQ)$(gitexec_instdir_SQ)/$p$X' || $(RM) '$(DESTDIR_SQ)$(gitexec_instdir_SQ)/$p';)\n+\t$(foreach p,$(patsubst %$X,%,$(filter %$X,$(ALL_COMMANDS_TO_INSTALL) $(OTHER_PROGRAMS))), test '$(DESTDIR_SQ)$(gitexec_instdir_SQ)/$p' -ef '$(DESTDIR_SQ)$(gitexec_instdir_SQ)/$p$X' || $(RM) '$(DESTDIR_SQ)$(gitexec_instdir_SQ)/$p';)\n endif\n \n \tbindir=$$(cd '$(DESTDIR_SQ)$(bindir_SQ)' && pwd) && \\\n \texecdir=$$(cd '$(DESTDIR_SQ)$(gitexec_instdir_SQ)' && pwd) && \\\n \tdestdir_from_execdir_SQ=$$(echo '$(gitexecdir_relative_SQ)' | sed -e 's|[^/][^/]*|..|g') && \\\n \t{ test \"$$bindir/\" = \"$$execdir/\" || \\\n-\t for p in git$X $(filter $(install_bindir_programs),$(ALL_PROGRAMS)); do \\\n+\t for p in $(OTHER_PROGRAMS) $(filter $(install_bindir_programs),$(ALL_PROGRAMS)); do \\\n \t\t$(RM) \"$$execdir/$$p\" && \\\n \t\ttest -n \"$(INSTALL_SYMLINKS)\" && \\\n \t\tln -s \"$$destdir_from_execdir_SQ/$(bindir_relative_SQ)/$$p\" \"$$execdir/$$p\" || \\\n@@ -3458,7 +3463,7 @@ clean: profile-clean coverage-clean cocciclean\n \t$(RM) git.res\n \t$(RM) $(OBJECTS)\n \t$(RM) $(LIB_FILE) $(XDIFF_LIB) $(REFTABLE_LIB) $(REFTABLE_TEST_LIB)\n-\t$(RM) $(ALL_PROGRAMS) $(SCRIPT_LIB) $(BUILT_INS) git$X\n+\t$(RM) $(ALL_PROGRAMS) $(SCRIPT_LIB) $(BUILT_INS) $(OTHER_PROGRAMS)\n \t$(RM) $(TEST_PROGRAMS)\n \t$(RM) $(FUZZ_PROGRAMS)\n \t$(RM) $(SP_OBJ)\n@@ -3509,6 +3514,7 @@ ALL_COMMANDS += git-citool\n ALL_COMMANDS += git-gui\n ALL_COMMANDS += gitk\n ALL_COMMANDS += gitweb\n+ALL_COMMANDS += scalar\n \n .PHONY: check-docs\n check-docs::\ndiff --git a/contrib/buildsystems/CMakeLists.txt b/contrib/buildsystems/CMakeLists.txt\nindex 2237109b57f..bae203c1fb9 100644\n--- a/contrib/buildsystems/CMakeLists.txt\n+++ b/contrib/buildsystems/CMakeLists.txt\n@@ -610,7 +610,7 @@ unset(CMAKE_REQUIRED_INCLUDES)\n #programs\n set(PROGRAMS_BUILT\n \tgit git-daemon git-http-backend git-sh-i18n--envsubst\n-\tgit-shell)\n+\tgit-shell scalar)\n \n if(NOT CURL_FOUND)\n \tlist(APPEND excluded_progs git-http-fetch git-http-push)\n@@ -757,6 +757,9 @@ target_link_libraries(git-sh-i18n--envsubst common-main)\n add_executable(git-shell ${CMAKE_SOURCE_DIR}/shell.c)\n target_link_libraries(git-shell common-main)\n \n+add_executable(scalar ${CMAKE_SOURCE_DIR}/scalar.c)\n+target_link_libraries(scalar common-main)\n+\n if(CURL_FOUND)\n \tadd_library(http_obj OBJECT ${CMAKE_SOURCE_DIR}/http.c)\n \n@@ -903,7 +906,7 @@ list(TRANSFORM git_perl_scripts PREPEND \"${CMAKE_BINARY_DIR}/\")\n \n #install\n foreach(program ${PROGRAMS_BUILT})\n-if(program STREQUAL \"git\" OR program STREQUAL \"git-shell\")\n+if(program MATCHES \"^(git|git-shell|scalar)$\")\n install(TARGETS ${program}\n \tRUNTIME DESTINATION bin)\n else()\n@@ -977,7 +980,7 @@ endif()\n \n #wrapper scripts\n set(wrapper_scripts\n-\tgit git-upload-pack git-receive-pack git-upload-archive git-shell git-remote-ext)\n+\tgit git-upload-pack git-receive-pack git-upload-archive git-shell git-remote-ext scalar)\n \n set(wrapper_test_scripts\n \ttest-fake-ssh test-tool)\ndiff --git a/contrib/scalar/.gitignore b/contrib/scalar/.gitignore\ndeleted file mode 100644\nindex ff3d47e84d0..00000000000\n--- a/contrib/scalar/.gitignore\n+++ /dev/null\n@@ -1,2 +0,0 @@\n-/*.exe\n-/scalar\ndiff --git a/contrib/scalar/Makefile b/contrib/scalar/Makefile\ndeleted file mode 100644\nindex 37f283f35d7..00000000000\n--- a/contrib/scalar/Makefile\n+++ /dev/null\n@@ -1,35 +0,0 @@\n-# The default target of this Makefile is...\n-all::\n-\n-# Import tree-wide shared Makefile behavior and libraries\n-include ../../shared.mak\n-\n-include ../../config.mak.uname\n--include ../../config.mak.autogen\n--include ../../config.mak\n-\n-TARGETS = scalar$(X) scalar.o\n-GITLIBS = ../../common-main.o ../../libgit.a ../../xdiff/lib.a\n-\n-all:: scalar$(X) ../../bin-wrappers/scalar\n-\n-$(GITLIBS):\n-\t$(QUIET_SUBDIR0)../.. $(QUIET_SUBDIR1) $(subst ../../,,$@)\n-\n-$(TARGETS): $(GITLIBS) scalar.c\n-\t$(QUIET_SUBDIR0)../.. $(QUIET_SUBDIR1) $(patsubst %,contrib/scalar/%,$@)\n-\n-clean:\n-\t$(RM) $(TARGETS) ../../bin-wrappers/scalar\n-\n-../../bin-wrappers/scalar: ../../wrap-for-bin.sh Makefile\n-\t@mkdir -p ../../bin-wrappers\n-\t$(QUIET_GEN)sed -e '1s|#!.*/sh|#!$(SHELL_PATH_SQ)|' \\\n-\t -e 's|@@BUILD_DIR@@|$(shell cd ../.. && pwd)|' \\\n-\t -e 's|@@PROG@@|contrib/scalar/scalar$(X)|' < $< > $@ && \\\n-\tchmod +x $@\n-\n-test: all\n-\t$(MAKE) -C t\n-\n-.PHONY: $(GITLIBS) all clean test FORCE\ndiff --git a/contrib/scalar/t/Makefile b/contrib/scalar/t/Makefile\ndeleted file mode 100644\nindex 1ed174a8cf3..00000000000\n--- a/contrib/scalar/t/Makefile\n+++ /dev/null\n@@ -1,81 +0,0 @@\n-# Import tree-wide shared Makefile behavior and libraries\n-include ../../../shared.mak\n-\n-# Run scalar tests\n-#\n-# Copyright (c) 2005,2021 Junio C Hamano, Johannes Schindelin\n-#\n-\n--include ../../../config.mak.autogen\n--include ../../../config.mak\n-\n-SHELL_PATH ?= $(SHELL)\n-PERL_PATH ?= /usr/bin/perl\n-RM ?= rm -f\n-PROVE ?= prove\n-DEFAULT_TEST_TARGET ?= test\n-TEST_LINT ?= test-lint\n-\n-ifdef TEST_OUTPUT_DIRECTORY\n-TEST_RESULTS_DIRECTORY = $(TEST_OUTPUT_DIRECTORY)/test-results\n-else\n-TEST_RESULTS_DIRECTORY = ../../../t/test-results\n-endif\n-\n-# Shell quote;\n-SHELL_PATH_SQ = $(subst ','\\'',$(SHELL_PATH))\n-PERL_PATH_SQ = $(subst ','\\'',$(PERL_PATH))\n-TEST_RESULTS_DIRECTORY_SQ = $(subst ','\\'',$(TEST_RESULTS_DIRECTORY))\n-\n-T = $(sort $(wildcard t[0-9][0-9][0-9][0-9]-*.sh))\n-\n-all: $(DEFAULT_TEST_TARGET)\n-\n-test: $(TEST_LINT)\n-\t$(MAKE) aggregate-results-and-cleanup\n-\n-prove: $(TEST_LINT)\n-\t@echo \"*** prove ***\"; GIT_CONFIG=.git/config $(PROVE) --exec '$(SHELL_PATH_SQ)' $(GIT_PROVE_OPTS) $(T) :: $(GIT_TEST_OPTS)\n-\t$(MAKE) clean-except-prove-cache\n-\n-$(T):\n-\t@echo \"*** $@ ***\"; GIT_CONFIG=.git/config '$(SHELL_PATH_SQ)' $@ $(GIT_TEST_OPTS)\n-\n-clean-except-prove-cache:\n-\t$(RM) -r 'trash directory'.*\n-\t$(RM) -r valgrind/bin\n-\n-clean: clean-except-prove-cache\n-\t$(RM) .prove\n-\n-test-lint: test-lint-duplicates test-lint-executable test-lint-shell-syntax\n-\n-test-lint-duplicates:\n-\t@dups=`echo $(T) | tr ' ' '\\n' | sed 's/-.*//' | sort | uniq -d` && \\\n-\t\ttest -z \"$$dups\" || { \\\n-\t\techo >&2 \"duplicate test numbers:\" $$dups; exit 1; }\n-\n-test-lint-executable:\n-\t@bad=`for i in $(T); do test -x \"$$i\" || echo $$i; done` && \\\n-\t\ttest -z \"$$bad\" || { \\\n-\t\techo >&2 \"non-executable tests:\" $$bad; exit 1; }\n-\n-test-lint-shell-syntax:\n-\t@'$(PERL_PATH_SQ)' ../../../t/check-non-portable-shell.pl $(T)\n-\n-aggregate-results-and-cleanup: $(T)\n-\t$(MAKE) aggregate-results\n-\t$(MAKE) clean\n-\n-aggregate-results:\n-\tfor f in '$(TEST_RESULTS_DIRECTORY_SQ)'/t*-*.counts; do \\\n-\t\techo \"$$f\"; \\\n-\tdone | '$(SHELL_PATH_SQ)' ../../../t/aggregate-results.sh\n-\n-valgrind:\n-\t$(MAKE) GIT_TEST_OPTS=\"$(GIT_TEST_OPTS) --valgrind\"\n-\n-test-results:\n-\tmkdir -p test-results\n-\n-.PHONY: $(T) aggregate-results clean valgrind\ndiff --git a/contrib/scalar/scalar.c b/scalar.c\nsimilarity index 100%\nrename from contrib/scalar/scalar.c\nrename to scalar.c\ndiff --git a/contrib/scalar/t/t9099-scalar.sh b/t/t9210-scalar.sh\nsimilarity index 96%\nrename from contrib/scalar/t/t9099-scalar.sh\nrename to t/t9210-scalar.sh\nindex dfb949f52ee..14ca575a214 100755\n--- a/contrib/scalar/t/t9099-scalar.sh\n+++ b/t/t9210-scalar.sh\n@@ -2,15 +2,9 @@\n \n test_description='test the `scalar` command'\n \n-TEST_DIRECTORY=$PWD/../../../t\n-export TEST_DIRECTORY\n+. ./test-lib.sh\n \n-# Make it work with --no-bin-wrappers\n-PATH=$PWD/..:$PATH\n-\n-. ../../../t/test-lib.sh\n-\n-GIT_TEST_MAINT_SCHEDULER=\"crontab:test-tool crontab ../cron.txt,launchctl:true,schtasks:true\"\n+GIT_TEST_MAINT_SCHEDULER=\"crontab:test-tool crontab cron.txt,launchctl:true,schtasks:true\"\n export GIT_TEST_MAINT_SCHEDULER\n \n test_expect_success 'scalar shows a usage' '\n-- \ngitgitgadget\n\n"},{"id":"462285","messageId":"46d0fddfe8fbc2c568cb5a3d14594276db2bc1a9.1661961746.git.gitgitgadget@gmail.com","threadId":"58383","inReplyTo":"pull.1341.git.1661961746.gitgitgadget@gmail.com","subject":"[PATCH 4/8] scalar: implement the `help` subcommand","fromName":"Johannes Schindelin via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2022-08-31T16:02:21Z","receivedAt":"2022-08-31T16:03:28Z","isPatch":true,"sender":{"key":"johannes.schindelin@gmx.de","avatar":"https://avatars.githubusercontent.com/u/127790?v=4"},"body":"From: Johannes Schindelin \n\nIt is merely handing off to `git help scalar`.\n\nSigned-off-by: Johannes Schindelin \nSigned-off-by: Victoria Dye \n---\n scalar.c | 20 ++++++++++++++++++++\n 1 file changed, 20 insertions(+)\n\ndiff --git a/scalar.c b/scalar.c\nindex 642d16124eb..675d7a6b0a9 100644\n--- a/scalar.c\n+++ b/scalar.c\n@@ -819,6 +819,25 @@ static int cmd_delete(int argc, const char **argv)\n \treturn res;\n }\n \n+static int cmd_help(int argc, const char **argv)\n+{\n+\tstruct option options[] = {\n+\t\tOPT_END(),\n+\t};\n+\tconst char * const usage[] = {\n+\t\tN_(\"scalar help\"),\n+\t\tNULL\n+\t};\n+\n+\targc = parse_options(argc, argv, NULL, options,\n+\t\t\t usage, 0);\n+\n+\tif (argc != 0)\n+\t\tusage_with_options(usage, options);\n+\n+\treturn run_git(\"help\", \"scalar\", NULL);\n+}\n+\n static int cmd_version(int argc, const char **argv)\n {\n \tint verbose = 0, build_options = 0;\n@@ -858,6 +877,7 @@ static struct {\n \t{ \"run\", cmd_run },\n \t{ \"reconfigure\", cmd_reconfigure },\n \t{ \"delete\", cmd_delete },\n+\t{ \"help\", cmd_help },\n \t{ \"version\", cmd_version },\n \t{ \"diagnose\", cmd_diagnose },\n \t{ NULL, NULL},\n-- \ngitgitgadget\n\n"},{"id":"462286","messageId":"a3b7cb0a3bd1f56172db8420b9e80a26be1fda5d.1661961746.git.gitgitgadget@gmail.com","threadId":"58383","inReplyTo":"pull.1341.git.1661961746.gitgitgadget@gmail.com","subject":"[PATCH 5/8] scalar-clone: add test coverage","fromName":"Victoria Dye via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2022-08-31T16:02:22Z","receivedAt":"2022-08-31T16:03:31Z","isPatch":true,"sender":{"key":"vdye@github.com","avatar":"https://avatars.githubusercontent.com/u/3619353?v=4"},"body":"From: Victoria Dye \n\nCreate a new test file ('t9211-scalar-clone.sh') to exercise the options and\nbehavior of the 'scalar clone' command.\n\nSigned-off-by: Victoria Dye \n---\n t/t9211-scalar-clone.sh | 135 ++++++++++++++++++++++++++++++++++++++++\n 1 file changed, 135 insertions(+)\n create mode 100755 t/t9211-scalar-clone.sh\n\ndiff --git a/t/t9211-scalar-clone.sh b/t/t9211-scalar-clone.sh\nnew file mode 100755\nindex 00000000000..9fbbc4de2c0\n--- /dev/null\n+++ b/t/t9211-scalar-clone.sh\n@@ -0,0 +1,135 @@\n+#!/bin/sh\n+\n+test_description='test the `scalar clone` subcommand'\n+\n+. ./test-lib.sh\n+\n+GIT_TEST_MAINT_SCHEDULER=\"crontab:test-tool crontab cron.txt,launchctl:true,schtasks:true\"\n+export GIT_TEST_MAINT_SCHEDULER\n+\n+test_expect_success 'set up repository to clone' '\n+\trm -rf .git &&\n+\tgit init to-clone &&\n+\t(\n+\t\tcd to-clone &&\n+\t\tgit branch -m base &&\n+\n+\t\ttest_commit first &&\n+\t\ttest_commit second &&\n+\t\ttest_commit third &&\n+\n+\t\tgit switch -c parallel first &&\n+\t\tmkdir -p 1/2 &&\n+\t\ttest_commit 1/2/3 &&\n+\n+\t\tgit switch base &&\n+\n+\t\t# By default, permit\n+\t\tgit config uploadpack.allowfilter true &&\n+\t\tgit config uploadpack.allowanysha1inwant true\n+\t)\n+'\n+\n+cleanup_clone () {\n+\trm -rf \"$1\"\n+}\n+\n+test_expect_success 'creates content in enlistment root' '\n+\ttest_when_finished cleanup_clone cloned &&\n+\n+\tscalar clone \"file://$(pwd)/to-clone\" cloned &&\n+\tls -A cloned >enlistment-root &&\n+\ttest_line_count = 1 enlistment-root &&\n+\ttest_path_is_dir cloned/src &&\n+\ttest_path_is_dir cloned/src/.git\n+'\n+\n+test_expect_success 'with spaces' '\n+\ttest_when_finished cleanup_clone \"cloned with space\" &&\n+\n+\tscalar clone \"file://$(pwd)/to-clone\" \"cloned with space\" &&\n+\ttest_path_is_dir \"cloned with space\" &&\n+\ttest_path_is_dir \"cloned with space/src\" &&\n+\ttest_path_is_dir \"cloned with space/src/.git\"\n+'\n+\n+test_expect_success 'partial clone if supported by server' '\n+\ttest_when_finished cleanup_clone cloned &&\n+\n+\tscalar clone \"file://$(pwd)/to-clone\" cloned &&\n+\n+\t(\n+\t\tcd cloned/src &&\n+\n+\t\t# Two promisor packs: one for refs, the other for blobs\n+\t\tls .git/objects/pack/pack-*.promisor >promisorlist &&\n+\t\ttest_line_count = 2 promisorlist\n+\t)\n+'\n+\n+test_expect_success 'fall back on full clone if partial unsupported' '\n+\ttest_when_finished cleanup_clone cloned &&\n+\n+\ttest_config -C to-clone uploadpack.allowfilter false &&\n+\ttest_config -C to-clone uploadpack.allowanysha1inwant false &&\n+\n+\tscalar clone \"file://$(pwd)/to-clone\" cloned 2>err &&\n+\tgrep \"filtering not recognized by server, ignoring\" err &&\n+\n+\t(\n+\t\tcd cloned/src &&\n+\n+\t\t# Still get a refs promisor file, but none for blobs\n+\t\tls .git/objects/pack/pack-*.promisor >promisorlist &&\n+\t\ttest_line_count = 1 promisorlist\n+\t)\n+'\n+\n+test_expect_success 'initializes sparse-checkout by default' '\n+\ttest_when_finished cleanup_clone cloned &&\n+\n+\tscalar clone \"file://$(pwd)/to-clone\" cloned &&\n+\t(\n+\t\tcd cloned/src &&\n+\t\ttest_cmp_config true core.sparseCheckout &&\n+\t\ttest_cmp_config true core.sparseCheckoutCone\n+\t)\n+'\n+\n+test_expect_success '--full-clone does not create sparse-checkout' '\n+\ttest_when_finished cleanup_clone cloned &&\n+\n+\tscalar clone --full-clone \"file://$(pwd)/to-clone\" cloned &&\n+\t(\n+\t\tcd cloned/src &&\n+\t\ttest_cmp_config \"\" --default \"\" core.sparseCheckout &&\n+\t\ttest_cmp_config \"\" --default \"\" core.sparseCheckoutCone\n+\t)\n+'\n+\n+test_expect_success '--single-branch clones HEAD only' '\n+\ttest_when_finished cleanup_clone cloned &&\n+\n+\tscalar clone --single-branch \"file://$(pwd)/to-clone\" cloned &&\n+\t(\n+\t\tcd cloned/src &&\n+\t\tgit for-each-ref refs/remotes/origin >out &&\n+\t\ttest_line_count = 1 out &&\n+\t\tgrep \"refs/remotes/origin/base\" out\n+\t)\n+'\n+\n+test_expect_success '--no-single-branch clones all branches' '\n+\ttest_when_finished cleanup_clone cloned &&\n+\n+\tscalar clone --no-single-branch \"file://$(pwd)/to-clone\" cloned &&\n+\t(\n+\t\tcd cloned/src &&\n+\t\tgit for-each-ref refs/remotes/origin >out &&\n+\t\ttest_line_count = 2 out &&\n+\t\tgrep \"refs/remotes/origin/base\" out &&\n+\t\tgrep \"refs/remotes/origin/parallel\" out\n+\t)\n+'\n+\n+test_done\n-- \ngitgitgadget\n\n"},{"id":"462287","messageId":"96e08a95265ea66839b439ce8abc50b34395aaa3.1661961746.git.gitgitgadget@gmail.com","threadId":"58383","inReplyTo":"pull.1341.git.1661961746.gitgitgadget@gmail.com","subject":"[PATCH 7/8] t/perf: add 'GIT_PERF_USE_SCALAR' run option","fromName":"Victoria Dye via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2022-08-31T16:02:24Z","receivedAt":"2022-08-31T16:03:32Z","isPatch":true,"sender":{"key":"vdye@github.com","avatar":"https://avatars.githubusercontent.com/u/3619353?v=4"},"body":"From: Victoria Dye \n\nAdd a 'GIT_PERF_USE_SCALAR' environment variable (and corresponding perf\nconfig 'useScalar') to register a repository created with any of:\n\n* test_perf_fresh_repo\n* test_perf_default_repo\n* test_perf_large_repo\n\nas a Scalar enlistment. This is intended to allow a developer to test the\nimpact of Scalar on already-defined performance scenarios.\n\nSuggested-by: Derrick Stolee \nSigned-off-by: Victoria Dye \n---\n t/perf/README | 4 ++++\n t/perf/perf-lib.sh | 13 ++++++++++++-\n t/perf/run | 3 +++\n 3 files changed, 19 insertions(+), 1 deletion(-)\n\ndiff --git a/t/perf/README b/t/perf/README\nindex fb9127a66f7..8f217d7be7d 100644\n--- a/t/perf/README\n+++ b/t/perf/README\n@@ -95,6 +95,10 @@ You can set the following variables (also in your config.mak):\n \tGit (e.g., performance of index-pack as the number of threads\n \tchanges). These can be enabled with GIT_PERF_EXTRA.\n \n+ GIT_PERF_USE_SCALAR\n+\tBoolean indicating whether to register test repo(s) with Scalar\n+\tbefore executing tests.\n+\n You can also pass the options taken by ordinary git tests; the most\n useful one is:\n \ndiff --git a/t/perf/perf-lib.sh b/t/perf/perf-lib.sh\nindex 27c28017921..b960b0f6301 100644\n--- a/t/perf/perf-lib.sh\n+++ b/t/perf/perf-lib.sh\n@@ -49,6 +49,9 @@ export TEST_DIRECTORY TRASH_DIRECTORY GIT_BUILD_DIR GIT_TEST_CMP\n MODERN_GIT=$GIT_BUILD_DIR/bin-wrappers/git\n export MODERN_GIT\n \n+MODERN_SCALAR=$GIT_BUILD_DIR/bin-wrappers/scalar\n+export MODERN_SCALAR\n+\n perf_results_dir=$TEST_RESULTS_DIR\n test -n \"$GIT_PERF_SUBSECTION\" && perf_results_dir=\"$perf_results_dir/$GIT_PERF_SUBSECTION\"\n mkdir -p \"$perf_results_dir\"\n@@ -120,6 +123,10 @@ test_perf_create_repo_from () {\n \t\t\t# status\" due to a locked index. Since we have\n \t\t\t# a copy it's fine to remove the lock.\n \t\t\trm .git/index.lock\n+\t\tfi &&\n+\t\tif test_bool_env \"$GIT_PERF_USE_SCALAR\" false\n+\t\tthen\n+\t\t\t\"$MODERN_SCALAR\" register\n \t\tfi\n \t) || error \"failed to copy repository '$source' to '$repo'\"\n }\n@@ -130,7 +137,11 @@ test_perf_fresh_repo () {\n \t\"$MODERN_GIT\" init -q \"$repo\" &&\n \t(\n \t\tcd \"$repo\" &&\n-\t\ttest_perf_do_repo_symlink_config_\n+\t\ttest_perf_do_repo_symlink_config_ &&\n+\t\tif test_bool_env \"$GIT_PERF_USE_SCALAR\" false\n+\t\tthen\n+\t\t\t\"$MODERN_SCALAR\" register\n+\t\tfi\n \t)\n }\n \ndiff --git a/t/perf/run b/t/perf/run\nindex 55219aa4056..33da4d2aba2 100755\n--- a/t/perf/run\n+++ b/t/perf/run\n@@ -171,6 +171,9 @@ run_subsection () {\n \tget_var_from_env_or_config \"GIT_PERF_MAKE_COMMAND\" \"perf\" \"makeCommand\"\n \tget_var_from_env_or_config \"GIT_PERF_MAKE_OPTS\" \"perf\" \"makeOpts\"\n \n+\tget_var_from_env_or_config \"GIT_PERF_USE_SCALAR\" \"perf\" \"useScalar\" \"--bool\"\n+\texport GIT_PERF_USE_SCALAR\n+\n \tget_var_from_env_or_config \"GIT_PERF_REPO_NAME\" \"perf\" \"repoName\"\n \texport GIT_PERF_REPO_NAME\n \n-- \ngitgitgadget\n\n"},{"id":"462288","messageId":"42ab39f21212d3da1af3546d3425aa790637056f.1661961746.git.gitgitgadget@gmail.com","threadId":"58383","inReplyTo":"pull.1341.git.1661961746.gitgitgadget@gmail.com","subject":"[PATCH 6/8] t/perf: add Scalar performance tests","fromName":"Victoria Dye via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2022-08-31T16:02:23Z","receivedAt":"2022-08-31T16:03:53Z","isPatch":true,"sender":{"key":"vdye@github.com","avatar":"https://avatars.githubusercontent.com/u/3619353?v=4"},"body":"From: Victoria Dye \n\nCreate 'p9210-scalar.sh' for testing Scalar performance and comparing\nperformance of Git operations in Scalar registrations and standard\nrepositories. Example results:\n\nTest this tree\n------------------------------------------------------------------------\n9210.2: scalar clone 14.82(18.00+3.63)\n9210.3: git clone 26.15(36.67+6.90)\n9210.4: git status (scalar) 0.04(0.01+0.01)\n9210.5: git status (non-scalar) 0.10(0.02+0.11)\n9210.6: test_commit --append --no-tag A (scalar) 0.08(0.02+0.03)\n9210.7: test_commit --append --no-tag A (non-scalar) 0.13(0.03+0.11)\n\nSigned-off-by: Victoria Dye \n---\n t/perf/p9210-scalar.sh | 43 ++++++++++++++++++++++++++++++++++++++++++\n 1 file changed, 43 insertions(+)\n create mode 100755 t/perf/p9210-scalar.sh\n\ndiff --git a/t/perf/p9210-scalar.sh b/t/perf/p9210-scalar.sh\nnew file mode 100755\nindex 00000000000..a68eb6b223d\n--- /dev/null\n+++ b/t/perf/p9210-scalar.sh\n@@ -0,0 +1,43 @@\n+#!/bin/sh\n+\n+test_description='test scalar performance'\n+. ./perf-lib.sh\n+\n+test_perf_large_repo \"$TRASH_DIRECTORY/to-clone\"\n+\n+test_expect_success 'enable server-side partial clone' '\n+\tgit -C to-clone config uploadpack.allowFilter true &&\n+\tgit -C to-clone config uploadpack.allowAnySHA1InWant true\n+'\n+\n+test_perf 'scalar clone' '\n+\trm -rf scalar-clone &&\n+\tscalar clone \"file://$(pwd)/to-clone\" scalar-clone\n+'\n+\n+test_perf 'git clone' '\n+\trm -rf git-clone &&\n+\tgit clone \"file://$(pwd)/to-clone\" git-clone\n+'\n+\n+test_compare_perf () {\n+\tcommand=\"$@\"\n+\ttest_perf \"$command (scalar)\" \"\n+\t\t(\n+\t\t\tcd scalar-clone/src &&\n+\t\t\t$command\n+\t\t)\n+\t\"\n+\n+\ttest_perf \"$command (non-scalar)\" \"\n+\t\t(\n+\t\t\tcd git-clone &&\n+\t\t\t$command\n+\t\t)\n+\t\"\n+}\n+\n+test_compare_perf git status\n+test_compare_perf test_commit --append --no-tag A\n+\n+test_done\n-- \ngitgitgadget\n\n"},{"id":"462289","messageId":"8bbe913dccf91bc4e6c76cbe85c585d223fc4aef.1661961746.git.gitgitgadget@gmail.com","threadId":"58383","inReplyTo":"pull.1341.git.1661961746.gitgitgadget@gmail.com","subject":"[PATCH 8/8] Documentation/technical: include Scalar technical doc","fromName":"Victoria Dye via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2022-08-31T16:02:25Z","receivedAt":"2022-08-31T16:03:56Z","isPatch":true,"sender":{"key":"vdye@github.com","avatar":"https://avatars.githubusercontent.com/u/3619353?v=4"},"body":"From: Victoria Dye \n\nInclude 'Documentation/technical/scalar.txt' alongside the other HTML\ntechnical docs when installing them.\n\nNow that the document is intended as a widely-accessible reference, remove\nthe internal work-in-progress roadmap from the document. Those details\nshould no longer be needed to guide Scalar's development and, if they were\nleft, they could fall out-of-date and be misleading to readers.\n\nSigned-off-by: Victoria Dye \n---\n Documentation/Makefile | 1 +\n Documentation/technical/scalar.txt | 61 ------------------------------\n 2 files changed, 1 insertion(+), 61 deletions(-)\n\ndiff --git a/Documentation/Makefile b/Documentation/Makefile\nindex 16c9e062390..9ec53afdf18 100644\n--- a/Documentation/Makefile\n+++ b/Documentation/Makefile\n@@ -116,6 +116,7 @@ TECH_DOCS += technical/parallel-checkout\n TECH_DOCS += technical/partial-clone\n TECH_DOCS += technical/racy-git\n TECH_DOCS += technical/reftable\n+TECH_DOCS += technical/scalar\n TECH_DOCS += technical/send-pack-pipeline\n TECH_DOCS += technical/shallow\n TECH_DOCS += technical/trivial-merge\ndiff --git a/Documentation/technical/scalar.txt b/Documentation/technical/scalar.txt\nindex 0600150b3ad..921cb104c3c 100644\n--- a/Documentation/technical/scalar.txt\n+++ b/Documentation/technical/scalar.txt\n@@ -64,64 +64,3 @@ some \"global\" `git` options (e.g., `-c` and `-C`).\n Because `scalar` is not invoked as a Git subcommand (like `git scalar`), it is\n built and installed as its own executable in the `bin/` directory, alongside\n `git`, `git-gui`, etc.\n-\n-Roadmap\n--------\n-\n-NOTE: this section will be removed once the remaining tasks outlined in this\n-roadmap are complete.\n-\n-Scalar is a large enough project that it is being upstreamed incrementally,\n-living in `contrib/` until it is feature-complete. So far, the following patch\n-series have been accepted:\n-\n-- `scalar-the-beginning`: The initial patch series which sets up\n- `contrib/scalar/` and populates it with a minimal `scalar` command that\n- demonstrates the fundamental ideas.\n-\n-- `scalar-c-and-C`: The `scalar` command learns about two options that can be\n- specified before the command, `-c =` and `-C `.\n-\n-- `scalar-diagnose`: The `scalar` command is taught the `diagnose` subcommand.\n-\n-- `scalar-generalize-diagnose`: Move the functionality of `scalar diagnose`\n- into `git diagnose` and `git bugreport --diagnose`.\n-\n-- 'scalar-add-fsmonitor: Enable the built-in FSMonitor in Scalar\n- enlistments. At the end of this series, Scalar should be feature-complete\n- from the perspective of a user.\n-\n-Roughly speaking (and subject to change), the following series are needed to\n-\"finish\" this initial version of Scalar:\n-\n-- Move Scalar to toplevel: Move Scalar out of `contrib/` and into the root of\n- `git`. This includes a variety of related updates, including:\n- - building & installing Scalar in the Git root-level 'make [install]'.\n- - builing & testing Scalar as part of CI.\n- - moving and expanding test coverage of Scalar (including perf tests).\n- - implementing 'scalar help'/'git help scalar' to display scalar\n- documentation.\n-\n-Finally, there are two additional patch series that exist in Microsoft's fork of\n-Git, but there is no current plan to upstream them. There are some interesting\n-ideas there, but the implementation is too specific to Azure Repos and/or VFS\n-for Git to be of much help in general.\n-\n-These still exist mainly because the GVFS protocol is what Azure Repos has\n-instead of partial clone, while Git is focused on improving partial clone:\n-\n-- `scalar-with-gvfs`: The primary purpose of this patch series is to support\n- existing Scalar users whose repositories are hosted in Azure Repos (which does\n- not support Git's partial clones, but supports its predecessor, the GVFS\n- protocol, which is used by Scalar to emulate the partial clone).\n-\n- Since the GVFS protocol will never be supported by core Git, this patch series\n- will remain in Microsoft's fork of Git.\n-\n-- `run-scalar-functional-tests`: The Scalar project developed a quite\n- comprehensive set of integration tests (or, \"Functional Tests\"). They are the\n- sole remaining part of the original C#-based Scalar project, and this patch\n- adds a GitHub workflow that runs them all.\n-\n- Since the tests partially depend on features that are only provided in the\n- `scalar-with-gvfs` patch series, this patch cannot be upstreamed.\n-- \ngitgitgadget\n"},{"id":"462296","messageId":"220831.8635dc9wzx.gmgdl@evledraar.gmail.com","threadId":"58383","inReplyTo":"46d0fddfe8fbc2c568cb5a3d14594276db2bc1a9.1661961746.git.gitgitgadget@gmail.com","subject":"Re: [PATCH 4/8] scalar: implement the `help` subcommand","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2022-08-31T16:48:58Z","receivedAt":"2022-08-31T16:52:27Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"\nOn Wed, Aug 31 2022, Johannes Schindelin via GitGitGadget wrote:\n\n> From: Johannes Schindelin \n>\n> It is merely handing off to `git help scalar`.\n>\n> Signed-off-by: Johannes Schindelin \n> Signed-off-by: Victoria Dye \n> ---\n> scalar.c | 20 ++++++++++++++++++++\n> 1 file changed, 20 insertions(+)\n>\n> diff --git a/scalar.c b/scalar.c\n> index 642d16124eb..675d7a6b0a9 100644\n> --- a/scalar.c\n> +++ b/scalar.c\n> @@ -819,6 +819,25 @@ static int cmd_delete(int argc, const char **argv)\n> \treturn res;\n> }\n> \n> +static int cmd_help(int argc, const char **argv)\n> +{\n> +\tstruct option options[] = {\n> +\t\tOPT_END(),\n> +\t};\n> +\tconst char * const usage[] = {\n> +\t\tN_(\"scalar help\"),\n\n\nThis should not have N_(), as it's a literal command.\n\n> +\t\tNULL\n> +\t};\n> +\n> +\targc = parse_options(argc, argv, NULL, options,\n> +\t\t\t usage, 0);\n> +\n> +\tif (argc != 0)\n\nIf we're re-rolling anyway we usually just do \"if (argc)\". We don't need\nto worry about argc < 0 (despite the signed type, which is a historical\nC wart).\n\n> +\t\tusage_with_options(usage, options);\n> +\n> +\treturn run_git(\"help\", \"scalar\", NULL);\n\nPerformance isn't sensitive here, but have you tried just calling\ncmd_help() instead with the appropriate arguments? It would avoid\nspawning another command..\n"},{"id":"462297","messageId":"220831.86y1v48h2x.gmgdl@evledraar.gmail.com","threadId":"58383","inReplyTo":"pull.1341.git.1661961746.gitgitgadget@gmail.com","subject":"Re: [PATCH 0/8] scalar: integrate into core Git","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2022-08-31T17:03:00Z","receivedAt":"2022-08-31T17:21:43Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"\nOn Wed, Aug 31 2022, Victoria Dye via GitGitGadget wrote:\n\n> This series completes the initial implementation of the Scalar command as a\n> core component of Git. For readers new to the topic of Scalar, the\n> roadmap/design doc [1] provides some background information including how\n> the project started & evolved, as well as its current intent & high-level\n> design.\n>\n> [...]\n>\n> Prior series\n> ============\n>\n> * Add 'scalar' command in 'contrib/':\n> https://lore.kernel.org/git/pull.1005.v10.git.1638538470.gitgitgadget@gmail.com/\n> * Introduce 'scalar diagnose':\n> https://lore.kernel.org/git/pull.1128.v6.git.1653145696.gitgitgadget@gmail.com/\n> * Add '-c/-C' compatibility:\n> https://lore.kernel.org/git/pull.1130.v2.git.1643380317358.gitgitgadget@gmail.com/\n> * [DROPPED] Integrate Scalar into CI builds:\n> https://lore.kernel.org/git/pull.1129.git.1654160735.gitgitgadget@gmail.com/\n> * Document Scalar's role in Git & plan remaining work:\n> https://lore.kernel.org/git/pull.1275.v2.git.1657584367.gitgitgadget@gmail.com/\n> * Generalize 'scalar diagnose' into 'git diagnose' builtin & 'git bugreport\n> --diagnose':\n> https://lore.kernel.org/git/pull.1310.v4.git.1660335019.gitgitgadget@gmail.com/\n> * Add FSMonitor support to Scalar, refactor enlistment search:\n> https://lore.kernel.org/git/pull.1324.v3.git.1660858853.gitgitgadget@gmail.com/\n>\n> Thanks!\n>\n> * Victoria\n\nI'm happy to see this finally coming. I can say I've thoroughly reviewed\nit & tested it for the better part of a year now. Since most of it is\nthe same or functionally the same as previous patches I sent at [1] and\n[2]. It's odd not to see any mention of that here:\n\n\t1. https://lore.kernel.org/git/cover-v2-0.1-00000000000-20220623T100554Z-avarab@gmail.com/\n\t2. https://lore.kernel.org/git/patch-1.1-86fb8d56307-20211028T185016Z-avarab@gmail.com/\n\nIn any case. I applied this & a rebased version I've kept of [1]\nlocally, and I'll be commenting below on the diff between the two, which\nI produced with:\n\n\tgit diff --stat -p avar/scalar-move-build-from-contrib-3 HEAD -- ':!t/t9211-scalar-clone.sh' ':!Documentation/technical/scalar.txt' ':!t/perf/'\n\nI.e. you can get my version at\nhttp://github.com/avar/git/tree/avar/scalar-move-build-from-contrib-3 if\nyou're interested, and I omitted the changes to paths unique to yours:\n\t\n\t .gitignore | 2 +-\n\t Documentation/Makefile | 4 +-\n\t Documentation/cmd-list.perl | 2 +-\n\t Documentation/git.txt | 13 ------\n\t Documentation/scalar.txt | 2 +-\n\t Makefile | 75 ++++++++++++----------------------\n\t builtin/help.c | 2 +\n\t command-list.txt | 2 -\n\t contrib/buildsystems/CMakeLists.txt | 20 +++------\n\t scalar.c | 20 +++++++++\n\t t/{t7990-scalar.sh => t9210-scalar.sh} | 2 +-\n\t 11 files changed, 60 insertions(+), 84 deletions(-)\n\t\n\tdiff --git a/.gitignore b/.gitignore\n\tindex b1db05a9207..3d1b880101e 100644\n\t--- a/.gitignore\n\t+++ b/.gitignore\n\t@@ -185,6 +185,7 @@\n\t /git-whatchanged\n\t /git-worktree\n\t /git-write-tree\n\t+/scalar\n\t /git-core-*/?*\n\t /git.res\n\t /gitweb/GITWEB-BUILD-OPTIONS\n\t@@ -220,7 +221,6 @@\n\t /config.mak.append\n\t /configure\n\t /.vscode/\n\t-/scalar\n\t /tags\n\t /TAGS\n\t /cscope*\n\nFunctionally the same.\n\n\tdiff --git a/Documentation/Makefile b/Documentation/Makefile\n\tindex 85b68ab3e9c..9ec53afdf18 100644\n\t--- a/Documentation/Makefile\n\t+++ b/Documentation/Makefile\n\t@@ -21,9 +21,7 @@ MAN1_TXT += $(filter-out \\\n\t MAN1_TXT += git.txt\n\t MAN1_TXT += gitk.txt\n\t MAN1_TXT += gitweb.txt\n\t-ifndef NO_INSTALL_SCALAR_DOC\n\t MAN1_TXT += scalar.txt\n\t-endif\n\nSo, this series goes the full way and install scalar unconditionally. I\ndon't mind it, but all previous discussions on the matter were IIRC of\nsome initial optional installation. I think doing it unconditionally\nmakes sense, but that explains this difference.\n\t \n\t # man5 / man7 guides (note: new guides should also be added to command-list.txt)\n\t MAN5_TXT += gitattributes.txt\n\t@@ -118,6 +116,7 @@ TECH_DOCS += technical/parallel-checkout\n\t TECH_DOCS += technical/partial-clone\n\t TECH_DOCS += technical/racy-git\n\t TECH_DOCS += technical/reftable\n\t+TECH_DOCS += technical/scalar\n\t TECH_DOCS += technical/send-pack-pipeline\n\t TECH_DOCS += technical/shallow\n\t TECH_DOCS += technical/trivial-merge\n\nMakes sense.\n\n\t@@ -290,7 +289,6 @@ endif\n\t cmds_txt = cmds-ancillaryinterrogators.txt \\\n\t \tcmds-ancillarymanipulators.txt \\\n\t \tcmds-mainporcelain.txt \\\n\t-\tcmds-optionalcontrib.txt \\\n\t \tcmds-plumbinginterrogators.txt \\\n\t \tcmds-plumbingmanipulators.txt \\\n\t \tcmds-synchingrepositories.txt \\\n\nThe \"optional install\" also explains this bit, i.e. this was the \"scalar\nstill in the proverbial contrib\", whatever \"contrib\" meant.\n\n\tdiff --git a/Documentation/cmd-list.perl b/Documentation/cmd-list.perl\n\tindex 0f4b1b23cfe..af5da45d287 100755\n\t--- a/Documentation/cmd-list.perl\n\t+++ b/Documentation/cmd-list.perl\n\t@@ -10,7 +10,7 @@ sub format_one {\n\t \t$state = 0;\n\t \topen I, '<', \"$name.txt\" or die \"No such file $name.txt\";\n\t \twhile () {\n\t-\t\tif (/^[a-z0-9-]*\\(([0-9])\\)$/) {\n\t+\t\tif (/^git[a-z0-9-]*\\(([0-9])\\)$/) {\n\t \t\t\t$mansection = $1;\n\t \t\t\tnext;\n\t \t\t}\n\nThis is an artifact of you not having added it to command-list.txt, more on that later.\n\n\tdiff --git a/Documentation/git.txt b/Documentation/git.txt\n\tindex 40ab70f1381..0ef7f5e4ece 100644\n\t--- a/Documentation/git.txt\n\t+++ b/Documentation/git.txt\n\t@@ -357,19 +357,6 @@ linkgit:git-help[1].\n\t \n\t include::cmds-developerinterfaces.txt[]\n\t \n\t-Optional contrib commands\n\t--------------------------\n\t-\n\t-The following commands are included with the git sources, but may not\n\t-be present in your installation.\n\t-\n\t-These should be considered \"contrib\"-level when it comes to\n\t-maintenance and stability promises. They might not even be included in\n\t-your installation, and may either drastically change in the future, or\n\t-go away entirely.\n\t-\n\t-include::cmds-optionalcontrib.txt[]\n\t-\n\t Configuration Mechanism\n\t -----------------------\n\nThe \"optional contrib\".\n\t \n\tdiff --git a/Documentation/scalar.txt b/Documentation/scalar.txt\n\tindex f33436c7f65..505a1cea0fd 100644\n\t--- a/Documentation/scalar.txt\n\t+++ b/Documentation/scalar.txt\n\t@@ -163,4 +163,4 @@ linkgit:git-clone[1], linkgit:git-maintenance[1].\n\t \n\t GIT\n\t ---\n\t-Part of the linkgit:git[1] suite\n\t+Associated with the linkgit:git[1] suite\n\nYou just kept this, but it really should be the former. The target\naudience of this bit of the documentation is some sysadmin that's\nwondering what this \"scalar\" thing is, and does \"man scalar\".\n\nLet's not be cute, we're shipping it as part of git, it's not\n\"associated\" anymore, it's part of the git suite.\n\n\tdiff --git a/Makefile b/Makefile\n\tindex c8108ded394..66dd3321f57 100644\n\t--- a/Makefile\n\t+++ b/Makefile\n\t@@ -476,11 +476,6 @@ include shared.mak\n\t # INSTALL_STRIP can be set to \"-s\" to strip binaries during installation,\n\t # if your $(INSTALL) command supports the option.\n\t #\n\t-# Define INSTALL_SCALAR if you would like to install the optional\n\t-# \"scalar\" command. This command is considered \"contrib\"-level, see\n\t-# 'Optional \"contrib\" commands' in the built (with \"make man\") git(1)\n\t-# manual page.\n\t-#\n\t # Define GENERATE_COMPILATION_DATABASE to \"yes\" to generate JSON compilation\n\t # database entries during compilation if your compiler supports it, using the\n\t # `-MJ` flag. The JSON entries will be placed in the `compile_commands/`\n\nMore \"optional install\" etc.\n\n\t@@ -613,9 +608,10 @@ FUZZ_OBJS =\n\t FUZZ_PROGRAMS =\n\t GIT_OBJS =\n\t LIB_OBJS =\n\t+SCALAR_OBJS =\n\t OBJECTS =\n\t-PROGRAM_OBJS =\n\t OTHER_PROGRAMS =\n\t+PROGRAM_OBJS =\n\t PROGRAMS =\n\t EXCLUDED_PROGRAMS =\n\t SCRIPT_PERL =\n\t@@ -832,23 +828,12 @@ OTHER_PROGRAMS += scalar$X\n\t \n\t # what test wrappers are needed and 'install' will install, in bindir\n\t BINDIR_PROGRAMS_NEED_X += git\n\t+BINDIR_PROGRAMS_NEED_X += scalar\n\t BINDIR_PROGRAMS_NEED_X += git-receive-pack\n\t BINDIR_PROGRAMS_NEED_X += git-shell\n\t BINDIR_PROGRAMS_NEED_X += git-upload-archive\n\t BINDIR_PROGRAMS_NEED_X += git-upload-pack\n\t \n\t-# Sometimes we only have a test wrapper, but not a program to\n\t-# install. This isn't so pretty, and we could refactor the\n\t-# bin-wrappers/% and install code to make it unnecessary.\n\t-ifdef INSTALL_SCALAR\n\t-PROGRAMS += scalar$X\n\t-BINDIR_PROGRAMS_NEED_X += scalar\n\t-endif\n\t-TEST_BINDIR_PROGRAMS_NEED_X = $(BINDIR_PROGRAMS_NEED_X)\n\t-ifndef INSTALL_SCALAR\n\t-TEST_BINDIR_PROGRAMS_NEED_X += scalar\n\t-endif\n\t-\n\t BINDIR_PROGRAMS_NO_X += git-cvsserver\n\t \n\t # Set paths to tools early so that they can be used for version tests.\n\t@@ -2241,7 +2226,7 @@ profile-fast: profile-clean\n\t \n\t all:: $(ALL_COMMANDS_TO_INSTALL) $(SCRIPT_LIB) $(OTHER_PROGRAMS) GIT-BUILD-OPTIONS\n\t ifneq (,$X)\n\t-\t$(QUIET_BUILT_IN)$(foreach p,$(patsubst %$X,%,$(filter %$X,$(ALL_COMMANDS_TO_INSTALL) git$X)), test -d '$p' -o '$p' -ef '$p$X' || $(RM) '$p';)\n\t+\t$(QUIET_BUILT_IN)$(foreach p,$(patsubst %$X,%,$(filter %$X,$(ALL_COMMANDS_TO_INSTALL) $(OTHER_PROGRAMS))), test -d '$p' -o '$p' -ef '$p$X' || $(RM) '$p';)\n\t endif\n\t \n\t all::\n\t@@ -2564,7 +2549,12 @@ GIT_OBJS += git.o\n\t .PHONY: git-objs\n\t git-objs: $(GIT_OBJS)\n\nditto.\n\t \n\t+SCALAR_OBJS += scalar.o\n\t+.PHONY: scalar-objs\n\t+scalar-objs: $(SCALAR_OBJS)\n\nThis part looks missing from yours. I.e. we do this with the rest of our\n\"objects\", just gravy of course...\n\n\t+\n\t OBJECTS += $(GIT_OBJS)\n\t+OBJECTS += $(SCALAR_OBJS)\n\t OBJECTS += $(PROGRAM_OBJS)\n\t OBJECTS += $(TEST_OBJS)\n\t OBJECTS += $(XDIFF_OBJS)\n\t@@ -2575,9 +2565,6 @@ ifndef NO_CURL\n\t \tOBJECTS += http.o http-walker.o remote-curl.o\n\t endif\n\t \n\t-SCALAR_OBJECTS := scalar.o\n\t-OBJECTS += $(SCALAR_OBJECTS)\n\t-\n\t .PHONY: objects\n\t objects: $(OBJECTS)\n\t \n\t@@ -2709,7 +2696,7 @@ $(REMOTE_CURL_PRIMARY): remote-curl.o http.o http-walker.o GIT-LDFLAGS $(GITLIBS\n\t \t$(QUIET_LINK)$(CC) $(ALL_CFLAGS) -o $@ $(ALL_LDFLAGS) $(filter %.o,$^) \\\n\t \t\t$(CURL_LIBCURL) $(EXPAT_LIBEXPAT) $(LIBS)\n\t \n\t-scalar$X: $(SCALAR_OBJECTS) GIT-LDFLAGS $(GITLIBS)\n\t+scalar$X: scalar.o GIT-LDFLAGS $(GITLIBS)\n\t \t$(QUIET_LINK)$(CC) $(ALL_CFLAGS) -o $@ $(ALL_LDFLAGS) \\\n\t \t\t$(filter %.o,$^) $(LIBS)\n\t \n\t@@ -2765,7 +2752,8 @@ XGETTEXT_FLAGS_SH = $(XGETTEXT_FLAGS) --language=Shell \\\n\t XGETTEXT_FLAGS_PERL = $(XGETTEXT_FLAGS) --language=Perl \\\n\t \t--keyword=__ --keyword=N__ --keyword=\"__n:1,2\"\n\t MSGMERGE_FLAGS = --add-location --backup=off --update\n\t-LOCALIZED_C = $(sort $(FOUND_C_SOURCES) $(FOUND_H_SOURCES) $(GENERATED_H))\n\t+LOCALIZED_C = $(sort $(FOUND_C_SOURCES) $(FOUND_H_SOURCES) \\\n\t+\t $(GENERATED_H))\n\nNit: we can avoid the line-wrap here. FWIW I moved it back to the\nversion pre-9f555783c0b, but obviously functionally the same...\n\n\t LOCALIZED_SH = $(sort $(SCRIPT_SH) git-sh-setup.sh)\n\t LOCALIZED_PERL = $(sort $(SCRIPT_PERL))\n\t \n\t@@ -3071,18 +3059,15 @@ GIT-PYTHON-VARS: FORCE\n\t fi\n\t endif\n\t \n\t-test_bindir_programs := $(patsubst %,bin-wrappers/%,$(TEST_BINDIR_PROGRAMS_NEED_X) $(BINDIR_PROGRAMS_NO_X) $(TEST_PROGRAMS_NEED_X))\n\t+test_bindir_programs := $(patsubst %,bin-wrappers/%,$(BINDIR_PROGRAMS_NEED_X) $(BINDIR_PROGRAMS_NO_X) $(TEST_PROGRAMS_NEED_X))\n\t \n\t all:: $(TEST_PROGRAMS) $(test_bindir_programs)\n\t \n\t-# Substitution that's only done on programs named git-* in\n\t-# bin-wrappers/*\n\t-GIT_TEST_BINDIR_PROGRAMS_NEED_X = $(filter-out scalar,$(TEST_BINDIR_PROGRAMS_NEED_X))\n\t bin-wrappers/%: wrap-for-bin.sh\n\t \t$(call mkdir_p_parent_template)\n\t \t$(QUIET_GEN)sed -e '1s|#!.*/sh|#!$(SHELL_PATH_SQ)|' \\\n\t \t -e 's|@@BUILD_DIR@@|$(shell pwd)|' \\\n\t-\t -e 's|@@PROG@@|$(patsubst test-%,t/helper/test-%$(X),$(@F))$(patsubst git%,$(X),$(filter $(@F),$(GIT_TEST_BINDIR_PROGRAMS_NEED_X)))|' < $< > $@ && \\\n\t+\t -e 's|@@PROG@@|$(patsubst test-%,t/helper/test-%,$(@F))$(if $(filter-out $(BINDIR_PROGRAMS_NO_X),$(@F)),$(X),)|' < $< > $@ && \\\n\t \tchmod +x $@\n\t \n\t # GNU make supports exporting all variables by \"export\" without parameters.\n\t@@ -3296,14 +3281,14 @@ ifndef NO_TCLTK\n\t \t$(MAKE) -C git-gui gitexecdir='$(gitexec_instdir_SQ)' install\n\t endif\n\t ifneq (,$X)\n\t-\t$(foreach p,$(patsubst %$X,%,$(filter %$X,$(ALL_COMMANDS_TO_INSTALL) git$X)), test '$(DESTDIR_SQ)$(gitexec_instdir_SQ)/$p' -ef '$(DESTDIR_SQ)$(gitexec_instdir_SQ)/$p$X' || $(RM) '$(DESTDIR_SQ)$(gitexec_instdir_SQ)/$p';)\n\t+\t$(foreach p,$(patsubst %$X,%,$(filter %$X,$(ALL_COMMANDS_TO_INSTALL) $(OTHER_PROGRAMS))), test '$(DESTDIR_SQ)$(gitexec_instdir_SQ)/$p' -ef '$(DESTDIR_SQ)$(gitexec_instdir_SQ)/$p$X' || $(RM) '$(DESTDIR_SQ)$(gitexec_instdir_SQ)/$p';)\n\t endif\n\nLooks correct, just different.\n\t \n\t \tbindir=$$(cd '$(DESTDIR_SQ)$(bindir_SQ)' && pwd) && \\\n\t \texecdir=$$(cd '$(DESTDIR_SQ)$(gitexec_instdir_SQ)' && pwd) && \\\n\t \tdestdir_from_execdir_SQ=$$(echo '$(gitexecdir_relative_SQ)' | sed -e 's|[^/][^/]*|..|g') && \\\n\t \t{ test \"$$bindir/\" = \"$$execdir/\" || \\\n\t-\t for p in git$X $(filter $(install_bindir_programs),$(ALL_PROGRAMS)); do \\\n\t+\t for p in $(OTHER_PROGRAMS) $(filter $(install_bindir_programs),$(ALL_PROGRAMS)); do \\\n\t \t\t$(RM) \"$$execdir/$$p\" && \\\n\t \t\ttest -n \"$(INSTALL_SYMLINKS)\" && \\\n\t \t\tln -s \"$$destdir_from_execdir_SQ/$(bindir_relative_SQ)/$$p\" \"$$execdir/$$p\" || \\\n\nThis part I missed, thanks!\n\n\t@@ -3347,18 +3332,11 @@ endif\n\t .PHONY: install-doc install-man install-man-perl install-html install-info install-pdf\n\t .PHONY: quick-install-doc quick-install-man quick-install-html\n\t \n\t-# We must not \"export\" this as e.g. \"check-docs\" needs to know about\n\t-# scalar.txt. We only exclude scalar.txt for the \"install-*\" targets.\n\t-NO_INSTALL_SCALAR_DOC =\n\t-ifndef INSTALL_SCALAR\n\t-NO_INSTALL_SCALAR_DOC = NoScalarPlease\n\t-endif\n\t-\n\t install-doc: install-man-perl\n\t-\t$(MAKE) -C Documentation install NO_INSTALL_SCALAR_DOC='$(NO_INSTALL_SCALAR_DOC)'\n\t+\t$(MAKE) -C Documentation install\n\t \n\t install-man: install-man-perl\n\t-\t$(MAKE) -C Documentation install-man NO_INSTALL_SCALAR_DOC='$(NO_INSTALL_SCALAR_DOC)'\n\t+\t$(MAKE) -C Documentation install-man\n\t \n\t install-man-perl: man-perl\n\t \t$(INSTALL) -d -m 755 '$(DESTDIR_SQ)$(mandir_SQ)/man3'\n\t@@ -3366,22 +3344,22 @@ install-man-perl: man-perl\n\t \t(cd '$(DESTDIR_SQ)$(mandir_SQ)/man3' && umask 022 && $(TAR) xof -)\n\t \n\t install-html:\n\t-\t$(MAKE) -C Documentation install-html NO_INSTALL_SCALAR_DOC='$(NO_INSTALL_SCALAR_DOC)'\n\t+\t$(MAKE) -C Documentation install-html\n\t \n\t install-info:\n\t-\t$(MAKE) -C Documentation install-info NO_INSTALL_SCALAR_DOC='$(NO_INSTALL_SCALAR_DOC)'\n\t+\t$(MAKE) -C Documentation install-info\n\t \n\t install-pdf:\n\t-\t$(MAKE) -C Documentation install-pdf NO_INSTALL_SCALAR_DOC='$(NO_INSTALL_SCALAR_DOC)'\n\t+\t$(MAKE) -C Documentation install-pdf\n\t \n\t quick-install-doc:\n\t-\t$(MAKE) -C Documentation quick-install NO_INSTALL_SCALAR_DOC='$(NO_INSTALL_SCALAR_DOC)'\n\t+\t$(MAKE) -C Documentation quick-install\n\t \n\t quick-install-man:\n\t-\t$(MAKE) -C Documentation quick-install-man NO_INSTALL_SCALAR_DOC='$(NO_INSTALL_SCALAR_DOC)'\n\t+\t$(MAKE) -C Documentation quick-install-man\n\t \n\t quick-install-html:\n\t-\t$(MAKE) -C Documentation quick-install-html NO_INSTALL_SCALAR_DOC='$(NO_INSTALL_SCALAR_DOC)'\n\t+\t$(MAKE) -C Documentation quick-install-html\n\nThis is to do with making the \"scalar docs\" optional, but if we're not doing that...\n\t \n\t \n\t \n\t@@ -3485,7 +3463,7 @@ clean: profile-clean coverage-clean cocciclean\n\t \t$(RM) git.res\n\t \t$(RM) $(OBJECTS)\n\t \t$(RM) $(LIB_FILE) $(XDIFF_LIB) $(REFTABLE_LIB) $(REFTABLE_TEST_LIB)\n\t-\t$(RM) $(ALL_PROGRAMS) $(SCRIPT_LIB) $(BUILT_INS) git$X\n\t+\t$(RM) $(ALL_PROGRAMS) $(SCRIPT_LIB) $(BUILT_INS) $(OTHER_PROGRAMS)\n\t \t$(RM) $(TEST_PROGRAMS)\n\t \t$(RM) $(FUZZ_PROGRAMS)\n\t \t$(RM) $(SP_OBJ)\n\nMakes sense.\n\n\t@@ -3536,6 +3514,7 @@ ALL_COMMANDS += git-citool\n\t ALL_COMMANDS += git-gui\n\t ALL_COMMANDS += gitk\n\t ALL_COMMANDS += gitweb\n\t+ALL_COMMANDS += scalar\n\t \n\t .PHONY: check-docs\n\t check-docs::\n\t@@ -3571,7 +3550,7 @@ check-docs::\n\t \t\t -e 's/\\.txt//'; \\\n\t \t) | while read how cmd; \\\n\t \tdo \\\n\t-\t\tcase \" $(patsubst %$X,%,$(ALL_COMMANDS) $(BUILT_INS) $(EXCLUDED_PROGRAMS) scalar) \" in \\\n\t+\t\tcase \" $(patsubst %$X,%,$(ALL_COMMANDS) $(BUILT_INS) $(EXCLUDED_PROGRAMS)) \" in \\\n\t \t\t*\" $$cmd \"*)\t;; \\\n\t \t\t*) echo \"removed but $$how: $$cmd\" ;; \\\n\t \t\tesac; \\\n\tdiff --git a/builtin/help.c b/builtin/help.c\n\tindex 09ac4289f13..6f2796f211e 100644\n\t--- a/builtin/help.c\n\t+++ b/builtin/help.c\n\t@@ -440,6 +440,8 @@ static const char *cmd_to_page(const char *git_cmd)\n\t \t\treturn git_cmd;\n\t \telse if (is_git_command(git_cmd))\n\t \t\treturn xstrfmt(\"git-%s\", git_cmd);\n\t+\telse if (!strcmp(\"scalar\", git_cmd))\n\t+\t\treturn xstrdup(git_cmd);\n\t \telse\n\t \t\treturn xstrfmt(\"git%s\", git_cmd);\n\t }\n\nWorks, but I wonder if 3/8 here in your series is what we really want,\ni.e. isn't the point of \"git\" to be a holistic thing for git, and for\n\"scalar\" to be set apart from that?\n\nBut OTOH much of the docs would need to cross-link anyway...\n\n\tdiff --git a/command-list.txt b/command-list.txt\n\tindex 27bd54af49c..f96bdabd7d9 100644\n\t--- a/command-list.txt\n\t+++ b/command-list.txt\n\t@@ -16,7 +16,6 @@\n\t # synchingrepositories\n\t # synchelpers\n\t # purehelpers\n\t-# optionalcontrib\n\t #\n\t # The type names are self explanatory. But if you want to see what\n\t # command belongs to what group to get a better picture, have a look\n\t@@ -236,4 +235,3 @@ gittutorial guide\n\t gittutorial-2 guide\n\t gitweb ancillaryinterrogators\n\t gitworkflows guide\n\t-scalar optionalcontrib\n\nYou don't have it in command-list at all, shouldn't you?\n\n\tdiff --git a/contrib/buildsystems/CMakeLists.txt b/contrib/buildsystems/CMakeLists.txt\n\tindex a5a1dedab4f..bae203c1fb9 100644\n\t--- a/contrib/buildsystems/CMakeLists.txt\n\t+++ b/contrib/buildsystems/CMakeLists.txt\n\t@@ -610,8 +610,7 @@ unset(CMAKE_REQUIRED_INCLUDES)\n\t #programs\n\t set(PROGRAMS_BUILT\n\t \tgit git-daemon git-http-backend git-sh-i18n--envsubst\n\t-\tgit-shell\n\t-\tscalar)\n\t+\tgit-shell scalar)\n\nJust whitespace changes.\n\t \n\t if(NOT CURL_FOUND)\n\t \tlist(APPEND excluded_progs git-http-fetch git-http-push)\n\t@@ -746,9 +745,6 @@ list(TRANSFORM git_SOURCES PREPEND \"${CMAKE_SOURCE_DIR}/\")\n\t add_executable(git ${CMAKE_SOURCE_DIR}/git.c ${git_SOURCES})\n\t target_link_libraries(git common-main)\n\t \n\t-add_executable(scalar ${CMAKE_SOURCE_DIR}/scalar.c)\n\t-target_link_libraries(scalar common-main)\n\t-\n\t add_executable(git-daemon ${CMAKE_SOURCE_DIR}/daemon.c)\n\t target_link_libraries(git-daemon common-main)\n\t \n\t@@ -761,6 +757,9 @@ target_link_libraries(git-sh-i18n--envsubst common-main)\n\t add_executable(git-shell ${CMAKE_SOURCE_DIR}/shell.c)\n\t target_link_libraries(git-shell common-main)\n\t \n\t+add_executable(scalar ${CMAKE_SOURCE_DIR}/scalar.c)\n\t+target_link_libraries(scalar common-main)\n\t+\n\t if(CURL_FOUND)\n\t \tadd_library(http_obj OBJECT ${CMAKE_SOURCE_DIR}/http.c)\n\t \n\t@@ -906,16 +905,10 @@ list(TRANSFORM git_shell_scripts PREPEND \"${CMAKE_BINARY_DIR}/\")\n\t list(TRANSFORM git_perl_scripts PREPEND \"${CMAKE_BINARY_DIR}/\")\n\t \n\t #install\n\t-option(INSTALL_SCALAR \"Install the optional 'scalar' contrib command\")\n\t foreach(program ${PROGRAMS_BUILT})\n\t-if(program STREQUAL \"git\" OR program STREQUAL \"git-shell\")\n\t-install(TARGETS ${program}\n\t-\tRUNTIME DESTINATION bin)\n\t-elseif(program STREQUAL \"scalar\")\n\t-if(INSTALL_SCALAR)\n\t+if(program MATCHES \"^(git|git-shell|scalar)$\")\n\t install(TARGETS ${program}\n\t \tRUNTIME DESTINATION bin)\n\t-endif()\n\t else()\n\t install(TARGETS ${program}\n\t \tRUNTIME DESTINATION libexec/git-core)\n\t@@ -987,8 +980,7 @@ endif()\n\t \n\t #wrapper scripts\n\t set(wrapper_scripts\n\t-\tgit git-upload-pack git-receive-pack git-upload-archive git-shell git-remote-ext\n\t-\tscalar)\n\t+\tgit git-upload-pack git-receive-pack git-upload-archive git-shell git-remote-ext scalar)\n\t \n\t set(wrapper_test_scripts\n\t \ttest-fake-ssh test-tool)\n\tdiff --git a/scalar.c b/scalar.c\n\tindex 642d16124eb..675d7a6b0a9 100644\n\t--- a/scalar.c\n\t+++ b/scalar.c\n\t@@ -819,6 +819,25 @@ static int cmd_delete(int argc, const char **argv)\n\t \treturn res;\n\t }\n\t \n\t+static int cmd_help(int argc, const char **argv)\n\t+{\n\t+\tstruct option options[] = {\n\t+\t\tOPT_END(),\n\t+\t};\n\t+\tconst char * const usage[] = {\n\t+\t\tN_(\"scalar help\"),\n\t+\t\tNULL\n\t+\t};\n\t+\n\t+\targc = parse_options(argc, argv, NULL, options,\n\t+\t\t\t usage, 0);\n\t+\n\t+\tif (argc != 0)\n\t+\t\tusage_with_options(usage, options);\n\t+\n\t+\treturn run_git(\"help\", \"scalar\", NULL);\n\t+}\n\t+\n\t static int cmd_version(int argc, const char **argv)\n\t {\n\t \tint verbose = 0, build_options = 0;\n\t@@ -858,6 +877,7 @@ static struct {\n\t \t{ \"run\", cmd_run },\n\t \t{ \"reconfigure\", cmd_reconfigure },\n\t \t{ \"delete\", cmd_delete },\n\t+\t{ \"help\", cmd_help },\n\t \t{ \"version\", cmd_version },\n\t \t{ \"diagnose\", cmd_diagnose },\n\t \t{ NULL, NULL},\n\tdiff --git a/t/t7990-scalar.sh b/t/t9210-scalar.sh\n\tsimilarity index 98%\n\trename from t/t7990-scalar.sh\n\trename to t/t9210-scalar.sh\n\tindex 62b92d361e2..14ca575a214 100755\n\t--- a/t/t7990-scalar.sh\n\t+++ b/t/t9210-scalar.sh\n\t@@ -4,7 +4,7 @@ test_description='test the `scalar` command'\n\t \n\t . ./test-lib.sh\n\t \n\t-GIT_TEST_MAINT_SCHEDULER=\"crontab:test-tool crontab ../cron.txt,launchctl:true,schtasks:true\"\n\t+GIT_TEST_MAINT_SCHEDULER=\"crontab:test-tool crontab cron.txt,launchctl:true,schtasks:true\"\n\t export GIT_TEST_MAINT_SCHEDULER\n\t \n\t test_expect_success 'scalar shows a usage' '\n\t\nMakes sense.\n\nThanks.\n"},{"id":"462301","messageId":"85ad26ba-8461-fdda-fb26-c99080dfa10d@github.com","threadId":"58383","inReplyTo":"220831.86y1v48h2x.gmgdl@evledraar.gmail.com","subject":"Re: [PATCH 0/8] scalar: integrate into core Git","fromName":"Victoria Dye","fromEmail":"vdye@github.com","sentAt":"2022-08-31T18:42:20Z","receivedAt":"2022-08-31T18:43:22Z","isPatch":true,"sender":{"key":"vdye@github.com","avatar":"https://avatars.githubusercontent.com/u/3619353?v=4"},"body":"Ævar Arnfjörð Bjarmason wrote:\n> \n> On Wed, Aug 31 2022, Victoria Dye via GitGitGadget wrote:\n> \n>> This series completes the initial implementation of the Scalar command as a\n>> core component of Git. For readers new to the topic of Scalar, the\n>> roadmap/design doc [1] provides some background information including how\n>> the project started & evolved, as well as its current intent & high-level\n>> design.\n>>\n>> [...]\n>>\n>> Prior series\n>> ============\n>>\n>> * Add 'scalar' command in 'contrib/':\n>> https://lore.kernel.org/git/pull.1005.v10.git.1638538470.gitgitgadget@gmail.com/\n>> * Introduce 'scalar diagnose':\n>> https://lore.kernel.org/git/pull.1128.v6.git.1653145696.gitgitgadget@gmail.com/\n>> * Add '-c/-C' compatibility:\n>> https://lore.kernel.org/git/pull.1130.v2.git.1643380317358.gitgitgadget@gmail.com/\n>> * [DROPPED] Integrate Scalar into CI builds:\n>> https://lore.kernel.org/git/pull.1129.git.1654160735.gitgitgadget@gmail.com/\n>> * Document Scalar's role in Git & plan remaining work:\n>> https://lore.kernel.org/git/pull.1275.v2.git.1657584367.gitgitgadget@gmail.com/\n>> * Generalize 'scalar diagnose' into 'git diagnose' builtin & 'git bugreport\n>> --diagnose':\n>> https://lore.kernel.org/git/pull.1310.v4.git.1660335019.gitgitgadget@gmail.com/\n>> * Add FSMonitor support to Scalar, refactor enlistment search:\n>> https://lore.kernel.org/git/pull.1324.v3.git.1660858853.gitgitgadget@gmail.com/\n>>\n>> Thanks!\n>>\n>> * Victoria\n> \n> I'm happy to see this finally coming. I can say I've thoroughly reviewed\n> it & tested it for the better part of a year now. Since most of it is\n> the same or functionally the same as previous patches I sent at [1] and\n> [2]. It's odd not to see any mention of that here:\n> \n> \t1. https://lore.kernel.org/git/cover-v2-0.1-00000000000-20220623T100554Z-avarab@gmail.com/\n> \t2. https://lore.kernel.org/git/patch-1.1-86fb8d56307-20211028T185016Z-avarab@gmail.com/\n\nFor what it's worth, the lack of mention wasn't meant as a snub. I\nintentionally wrote this series (with the exception of patches 3 & 4) in a\nvacuum and avoided using any prior approaches as a reference. In this case,\nthat includes the series you linked, the 'microsoft/git' implementation with\n'INCLUDE_SCALAR' [1], and Dscho's dropped CI integration [2]. \n\nLike updating the docs & roadmap, my goal was to start \"fresh\" and keep\nmyself free of any prior discussions/assumptions about the role/purpose of\nScalar that may no longer apply to my approach. Although there's something\nto be said for learning from prior work, I preferred combing through the\ncode to fully understand the *existing* architecture, then using that\nunderstanding to figure out where Scalar best fits in. Then, of course, I\nwould iterate on and revise that design based on review comments. :) \n\n[1] https://github.com/microsoft/git/commit/4f553e0027190484899afba955c7cd3f1de77532\n[2] https://lore.kernel.org/git/pull.1129.git.1654160735.gitgitgadget@gmail.com/\n\n> \n> In any case. I applied this & a rebased version I've kept of [1]\n> locally, and I'll be commenting below on the diff between the two, which\n> I produced with:\n> \n> \tgit diff --stat -p avar/scalar-move-build-from-contrib-3 HEAD -- ':!t/t9211-scalar-clone.sh' ':!Documentation/technical/scalar.txt' ':!t/perf/'\n> \n> I.e. you can get my version at\n> http://github.com/avar/git/tree/avar/scalar-move-build-from-contrib-3 if\n> you're interested, and I omitted the changes to paths unique to yours:\n\nFor the sake of readability/saving space, I cut out parts of the diff you\nnoted as \"ok\" or otherwise didn't seem to need comment (like nits). If I\nneglected to comment on something you'd like me to respond to, though,\nplease let me know!\n\n> \t \n> \tdiff --git a/Documentation/scalar.txt b/Documentation/scalar.txt\n> \tindex f33436c7f65..505a1cea0fd 100644\n> \t--- a/Documentation/scalar.txt\n> \t+++ b/Documentation/scalar.txt\n> \t@@ -163,4 +163,4 @@ linkgit:git-clone[1], linkgit:git-maintenance[1].\n> \t \n> \t GIT\n> \t ---\n> \t-Part of the linkgit:git[1] suite\n> \t+Associated with the linkgit:git[1] suite\n> \n> You just kept this, but it really should be the former. The target\n> audience of this bit of the documentation is some sysadmin that's\n> wondering what this \"scalar\" thing is, and does \"man scalar\".\n> \n> Let's not be cute, we're shipping it as part of git, it's not\n> \"associated\" anymore, it's part of the git suite.\n\nI originally chose to leave the \"associated\" to distinguish it from the\nbuiltins invoked with 'git ' (since 'scalar' is invoked directly).\nHowever, given that every single other \"GIT\" section in the docs (including\nother directly-invoked commands like 'gitk') says \"Part of the\nlinkgit:git[1] suite\", it should probably be changed here as well.\n\n> \t \n> \t+SCALAR_OBJS += scalar.o\n> \t+.PHONY: scalar-objs\n> \t+scalar-objs: $(SCALAR_OBJS)\n> \n> This part looks missing from yours. I.e. we do this with the rest of our\n> \"objects\", just gravy of course...\n\nThis hunk is from my series, I believe it's missing from yours (unless I\nmissed it elsewhere in the diff). It's also mentioned explicitly in the\ncommit message of patch 2 [3].\n\n[3] https://lore.kernel.org/git/4d69e5eaccb8873eece666a3d2bb2b22abdde7ea.1661961746.git.gitgitgadget@gmail.com/\n\n> \n> \t@@ -3536,6 +3514,7 @@ ALL_COMMANDS += git-citool\n> \t ALL_COMMANDS += git-gui\n> \t ALL_COMMANDS += gitk\n> \t ALL_COMMANDS += gitweb\n> \t+ALL_COMMANDS += scalar\n> \t \n> \t .PHONY: check-docs\n> \t check-docs::\n> \t@@ -3571,7 +3550,7 @@ check-docs::\n> \t \t\t -e 's/\\.txt//'; \\\n> \t \t) | while read how cmd; \\\n> \t \tdo \\\n> \t-\t\tcase \" $(patsubst %$X,%,$(ALL_COMMANDS) $(BUILT_INS) $(EXCLUDED_PROGRAMS) scalar) \" in \\\n> \t+\t\tcase \" $(patsubst %$X,%,$(ALL_COMMANDS) $(BUILT_INS) $(EXCLUDED_PROGRAMS)) \" in \\\n> \t \t\t*\" $$cmd \"*)\t;; \\\n> \t \t\t*) echo \"removed but $$how: $$cmd\" ;; \\\n> \t \t\tesac; \\\n> \tdiff --git a/builtin/help.c b/builtin/help.c\n> \tindex 09ac4289f13..6f2796f211e 100644\n> \t--- a/builtin/help.c\n> \t+++ b/builtin/help.c\n> \t@@ -440,6 +440,8 @@ static const char *cmd_to_page(const char *git_cmd)\n> \t \t\treturn git_cmd;\n> \t \telse if (is_git_command(git_cmd))\n> \t \t\treturn xstrfmt(\"git-%s\", git_cmd);\n> \t+\telse if (!strcmp(\"scalar\", git_cmd))\n> \t+\t\treturn xstrdup(git_cmd);\n> \t \telse\n> \t \t\treturn xstrfmt(\"git%s\", git_cmd);\n> \t }\n> \n> Works, but I wonder if 3/8 here in your series is what we really want,\n> i.e. isn't the point of \"git\" to be a holistic thing for git, and for\n> \"scalar\" to be set apart from that?\n> \n> But OTOH much of the docs would need to cross-link anyway...\n\nLike you noted earlier, Scalar is *part of* the Git suite, despite being\ninvoked with 'scalar' (rather than the 'git' executable). Having 'git help\nscalar' work the same way as, e.g., 'git help gitk' is, to me, a sensible\napproach both from a philosophical \"what is Scalar?\" perspective and a user\nexperience/ease-of-access perspective.\n\n> \n> \tdiff --git a/command-list.txt b/command-list.txt\n> \tindex 27bd54af49c..f96bdabd7d9 100644\n> \t--- a/command-list.txt\n> \t+++ b/command-list.txt\n> \t@@ -16,7 +16,6 @@\n> \t # synchingrepositories\n> \t # synchelpers\n> \t # purehelpers\n> \t-# optionalcontrib\n> \t #\n> \t # The type names are self explanatory. But if you want to see what\n> \t # command belongs to what group to get a better picture, have a look\n> \t@@ -236,4 +235,3 @@ gittutorial guide\n> \t gittutorial-2 guide\n> \t gitweb ancillaryinterrogators\n> \t gitworkflows guide\n> \t-scalar optionalcontrib\n> \n> You don't have it in command-list at all, shouldn't you?\n\nI missed that part of the docs, but I agree that it should be in the\ncommand-list. Thanks for pointing this out!\n\n> \n> Thanks.\n\n"},{"id":"462425","messageId":"5q412o8n-60so-52qn-883q-s3spo168o7r9@tzk.qr","threadId":"58383","inReplyTo":"46d0fddfe8fbc2c568cb5a3d14594276db2bc1a9.1661961746.git.gitgitgadget@gmail.com","subject":"Re: [PATCH 4/8] scalar: implement the `help` subcommand","fromName":"Johannes Schindelin","fromEmail":"johannes.schindelin@gmx.de","sentAt":"2022-09-01T08:51:15Z","receivedAt":"2022-09-01T08:51:50Z","isPatch":true,"sender":{"key":"johannes.schindelin@gmx.de","avatar":"https://avatars.githubusercontent.com/u/127790?v=4"},"body":"Hi Victoria,\n\nOn Wed, 31 Aug 2022, Johannes Schindelin via GitGitGadget wrote:\n\n> From: Johannes Schindelin \n\nI probably left that in by mistake. Could I bother you to change this (and\nthe corresponding Signed-off-by: footer) to use my usual email address?\n\nThank you,\nDscho\n\n>\n> It is merely handing off to `git help scalar`.\n>\n> Signed-off-by: Johannes Schindelin \n> Signed-off-by: Victoria Dye \n> ---\n> scalar.c | 20 ++++++++++++++++++++\n> 1 file changed, 20 insertions(+)\n>\n> diff --git a/scalar.c b/scalar.c\n> index 642d16124eb..675d7a6b0a9 100644\n> --- a/scalar.c\n> +++ b/scalar.c\n> @@ -819,6 +819,25 @@ static int cmd_delete(int argc, const char **argv)\n> \treturn res;\n> }\n>\n> +static int cmd_help(int argc, const char **argv)\n> +{\n> +\tstruct option options[] = {\n> +\t\tOPT_END(),\n> +\t};\n> +\tconst char * const usage[] = {\n> +\t\tN_(\"scalar help\"),\n> +\t\tNULL\n> +\t};\n> +\n> +\targc = parse_options(argc, argv, NULL, options,\n> +\t\t\t usage, 0);\n> +\n> +\tif (argc != 0)\n> +\t\tusage_with_options(usage, options);\n> +\n> +\treturn run_git(\"help\", \"scalar\", NULL);\n> +}\n> +\n> static int cmd_version(int argc, const char **argv)\n> {\n> \tint verbose = 0, build_options = 0;\n> @@ -858,6 +877,7 @@ static struct {\n> \t{ \"run\", cmd_run },\n> \t{ \"reconfigure\", cmd_reconfigure },\n> \t{ \"delete\", cmd_delete },\n> +\t{ \"help\", cmd_help },\n> \t{ \"version\", cmd_version },\n> \t{ \"diagnose\", cmd_diagnose },\n> \t{ NULL, NULL},\n> --\n> gitgitgadget\n>\n>\n"},{"id":"462426","messageId":"sso99so6-n28s-rq86-8q20-4456r3pn869r@tzk.qr","threadId":"58383","inReplyTo":"4d69e5eaccb8873eece666a3d2bb2b22abdde7ea.1661961746.git.gitgitgadget@gmail.com","subject":"Re: [PATCH 2/8] scalar: include in standard Git build & installation","fromName":"Johannes Schindelin","fromEmail":"johannes.schindelin@gmx.de","sentAt":"2022-09-01T09:11:18Z","receivedAt":"2022-09-01T09:11:33Z","isPatch":true,"sender":{"key":"johannes.schindelin@gmx.de","avatar":"https://avatars.githubusercontent.com/u/127790?v=4"},"body":"Hi Victoria,\n\nOn Wed, 31 Aug 2022, Victoria Dye via GitGitGadget wrote:\n\n> From: Victoria Dye \n>\n> Move 'scalar' out of 'contrib/' and into the root of the Git tree. The goal\n> of this change is to build 'scalar' as part of the standard Git build &\n> install processes.\n>\n> This patch includes both the physical move of Scalar's files out of\n> 'contrib/' ('scalar.c', 'scalar.txt', and 't9xxx-scalar.sh'), and the\n> changes to the build definitions in 'Makefile' and 'CMakelists.txt' to\n> accommodate the new program.\n>\n> At a high level, Scalar is built so that:\n> - there is a 'scalar-objs' target (similar to those created in 029bac01a8\n> (Makefile: add {program,xdiff,test,git,fuzz}-objs & objects targets,\n> 2021-02-23)) for debugging purposes.\n> - it appears in the root of the install directory (rather than the\n> gitexecdir).\n> - it is included in the 'bin-wrappers/' directory for use in tests.\n> - it receives a platform-specific executable suffix (e.g., '.exe'), if\n> applicable.\n> - 'scalar.txt' is installed as 'man1' documentation.\n> - the 'clean' target removes the 'scalar' executable.\n>\n> Additionally, update the root level '.gitignore' file to ignore the Scalar\n> executable.\n\nA great commit message, and even though the diff is large, it is eminently\nreviewable, with one exception:\n\n> @@ -3062,7 +3067,7 @@ bin-wrappers/%: wrap-for-bin.sh\n> \t$(call mkdir_p_parent_template)\n> \t$(QUIET_GEN)sed -e '1s|#!.*/sh|#!$(SHELL_PATH_SQ)|' \\\n> \t -e 's|@@BUILD_DIR@@|$(shell pwd)|' \\\n> -\t -e 's|@@PROG@@|$(patsubst test-%,t/helper/test-%$(X),$(@F))$(patsubst git%,$(X),$(filter $(@F),$(BINDIR_PROGRAMS_NEED_X)))|' < $< > $@ && \\\n> +\t -e 's|@@PROG@@|$(patsubst test-%,t/helper/test-%,$(@F))$(if $(filter-out $(BINDIR_PROGRAMS_NO_X),$(@F)),$(X),)|' < $< > $@ && \\\n\nIt took me a good while to wrap my head around this (and let me be clear:\nI consider none of this your fault, it's the fault of the design of the\nMakefile syntax).\n\nLet me untangle this, for posterity's benefit. We substitute the\nplaceholder `@@PROG@@` with a concatenation of two strings that are both\nderived from `@F`, i.e. the basename of the to-be-wrapped command.\n\nBefore, the first string would be the dashed command, verbatim, unless its\nname starts with a `test-` prefix, in which case we would add another\nprefix (`t/helper/`) _and_ append the `.exe` suffix on Windows.\n\nAnd the second string would be a `.exe` for all remaining cases that need\nit: the commands starting with `git` and listed as needing the suffix.\nOtherwise, the second string would be empty.\n\nConvoluted, sure, but it worked.\n\nIn the new version, the first string would be the basename of the\nto-be-wrapped command, with `t/helper/` prefixed for the `test-` helpers.\n\nThe second string would be a `.exe` for _all_ commands, except for those\nspecifically listed as not wanting that suffix as per\n`BINDIR_PROGRAMS_NO_X`.\n\nThe new logic is so much simpler to understand!\n\nFeel free to add a `Reviewed-by:` footer for me if you send another\niteration of this patch series.\n\nThank you,\nDscho\n"},{"id":"462427","messageId":"12037768-r742-rs85-3406-rp790p5p3r8n@tzk.qr","threadId":"58383","inReplyTo":"46d0fddfe8fbc2c568cb5a3d14594276db2bc1a9.1661961746.git.gitgitgadget@gmail.com","subject":"Re: [PATCH 4/8] scalar: implement the `help` subcommand","fromName":"Johannes Schindelin","fromEmail":"johannes.schindelin@gmx.de","sentAt":"2022-09-01T09:17:33Z","receivedAt":"2022-09-01T09:17:44Z","isPatch":true,"sender":{"key":"johannes.schindelin@gmx.de","avatar":"https://avatars.githubusercontent.com/u/127790?v=4"},"body":"Hi Victoria,\n\nOn Wed, 31 Aug 2022, Johannes Schindelin via GitGitGadget wrote:\n\n> From: Johannes Schindelin \n>\n> It is merely handing off to `git help scalar`.\n>\n> Signed-off-by: Johannes Schindelin \n> Signed-off-by: Victoria Dye \n> ---\n> scalar.c | 20 ++++++++++++++++++++\n> 1 file changed, 20 insertions(+)\n>\n> diff --git a/scalar.c b/scalar.c\n> index 642d16124eb..675d7a6b0a9 100644\n> --- a/scalar.c\n> +++ b/scalar.c\n> @@ -819,6 +819,25 @@ static int cmd_delete(int argc, const char **argv)\n> \treturn res;\n> }\n>\n> +static int cmd_help(int argc, const char **argv)\n> +{\n> +\tstruct option options[] = {\n> +\t\tOPT_END(),\n> +\t};\n> +\tconst char * const usage[] = {\n> +\t\tN_(\"scalar help\"),\n> +\t\tNULL\n> +\t};\n> +\n> +\targc = parse_options(argc, argv, NULL, options,\n> +\t\t\t usage, 0);\n> +\n> +\tif (argc != 0)\n> +\t\tusage_with_options(usage, options);\n> +\n> +\treturn run_git(\"help\", \"scalar\", NULL);\n> +}\n> +\n> static int cmd_version(int argc, const char **argv)\n> {\n> \tint verbose = 0, build_options = 0;\n> @@ -858,6 +877,7 @@ static struct {\n> \t{ \"run\", cmd_run },\n> \t{ \"reconfigure\", cmd_reconfigure },\n> \t{ \"delete\", cmd_delete },\n> +\t{ \"help\", cmd_help },\n\nMarking this as a tangent (\"optional\", as some peeps suggested in the Git\nstandup on IRC [*1*]) with the suggestion to follow up _after_ this here\npatch series is done cooking, i.e. once it hits the main branch:\n\nWe probably want to migrate `scalar.c` to use the `OPT_SUBCOMMAND` API.\n\nBut as I said, please not in this here patch series, so as to separate\nconcerns properly.\n\nThanks,\nDscho\n\n> \t{ \"version\", cmd_version },\n> \t{ \"diagnose\", cmd_diagnose },\n> \t{ NULL, NULL},\n> --\n> gitgitgadget\n\nFootnote *1*:\nhttps://colabti.org/irclogger/irclogger_log/git-devel?date=2022-08-29#l50\n"},{"id":"462428","messageId":"o429p430-sp18-rr20-p2ro-4r60qs2n83o5@tzk.qr","threadId":"58383","inReplyTo":"a3b7cb0a3bd1f56172db8420b9e80a26be1fda5d.1661961746.git.gitgitgadget@gmail.com","subject":"Re: [PATCH 5/8] scalar-clone: add test coverage","fromName":"Johannes Schindelin","fromEmail":"johannes.schindelin@gmx.de","sentAt":"2022-09-01T09:32:39Z","receivedAt":"2022-09-01T09:32:49Z","isPatch":true,"sender":{"key":"johannes.schindelin@gmx.de","avatar":"https://avatars.githubusercontent.com/u/127790?v=4"},"body":"Hi Victoria,\n\nOn Wed, 31 Aug 2022, Victoria Dye via GitGitGadget wrote:\n\n> From: Victoria Dye \n>\n> Create a new test file ('t9211-scalar-clone.sh') to exercise the options and\n> behavior of the 'scalar clone' command.\n\nGreat catch!\n\nI have one suggestion, given my experience with debugging test failures:\n\n> +test_expect_success 'creates content in enlistment root' '\n> +\ttest_when_finished cleanup_clone cloned &&\n> +\n> +\tscalar clone \"file://$(pwd)/to-clone\" cloned &&\n\nThis pattern of cloning into `cloned` and removing the directory when the\ntest case is done is repeated throughout this test script.\n\nIn instances where all test cases succeed, that poses no problem.\n\nWhen running the test script with `-i`, also no problem.\n\nBut when we run into test failures in CI, those directories will be\nremoved before the workflow run can tar them up and upload them for later\ninspection.\n\nMay I suggest an alternative strategy?\n\nIf we drop all those `test_when_finished cleanup_clone cloned` calls and\ninstead `scalar clone` into different directories (whose names reflect the\ntest cases' intended purpose), I could imagine having a much easier time\nnot only diagnosing but also reproducing and fixing test failures in the\nfuture.\n\nWhen discussing code review practices [*1*], we did not come up with any\nstandard terminology to describe what I am offering here, and I am unsure\nhow to label this in a catchy way. I want to present this suggestion for\nyou to consider, and I would be delighted if you take it, at the same time\nI will happily let it go should you decide against it.\n\nCiao,\nDscho\n\nFootnote *1*:\nhttps://colabti.org/irclogger/irclogger_log/git-devel?date=2022-08-29#l48\n"},{"id":"462429","messageId":"59s7o8n8-5797-q2qo-4q0o-93qppnr92q52@tzk.qr","threadId":"58383","inReplyTo":"42ab39f21212d3da1af3546d3425aa790637056f.1661961746.git.gitgitgadget@gmail.com","subject":"Re: [PATCH 6/8] t/perf: add Scalar performance tests","fromName":"Johannes Schindelin","fromEmail":"johannes.schindelin@gmx.de","sentAt":"2022-09-01T09:39:43Z","receivedAt":"2022-09-01T09:39:54Z","isPatch":true,"sender":{"key":"johannes.schindelin@gmx.de","avatar":"https://avatars.githubusercontent.com/u/127790?v=4"},"body":"Hi Victoria,\n\nOn Wed, 31 Aug 2022, Victoria Dye via GitGitGadget wrote:\n\n> From: Victoria Dye \n>\n> Create 'p9210-scalar.sh' for testing Scalar performance and comparing\n> performance of Git operations in Scalar registrations and standard\n> repositories. Example results:\n>\n> Test this tree\n> ------------------------------------------------------------------------\n> 9210.2: scalar clone 14.82(18.00+3.63)\n> 9210.3: git clone 26.15(36.67+6.90)\n> 9210.4: git status (scalar) 0.04(0.01+0.01)\n> 9210.5: git status (non-scalar) 0.10(0.02+0.11)\n> 9210.6: test_commit --append --no-tag A (scalar) 0.08(0.02+0.03)\n> 9210.7: test_commit --append --no-tag A (non-scalar) 0.13(0.03+0.11)\n\nExcellent!\n\n> [...]\n> +\n> +test_compare_perf () {\n> +\tcommand=\"$@\"\n> +\ttest_perf \"$command (scalar)\" \"\n> +\t\t(\n> +\t\t\tcd scalar-clone/src &&\n> +\t\t\t$command\n> +\t\t)\n> +\t\"\n> +\n> +\ttest_perf \"$command (non-scalar)\" \"\n> +\t\t(\n> +\t\t\tcd git-clone &&\n> +\t\t\t$command\n> +\t\t)\n> +\t\"\n> +}\n> +\n> +test_compare_perf git status\n> +test_compare_perf test_commit --append --no-tag A\n\nGiven the small numbers presented in the commit message, I suspect that\neven so much as running the command in a subshell might skew the timings\nat least on Windows, where subshells are very, very expensive.\n\nGiven that both `git` and `test_commit` understand the `-C `\nsyntax, this variant would resolve my concern:\n\n\ttest_compare_perf () {\n\t command=$1\n shift\n\t args=\"$*\"\n\n\t test_perf \"$command (scalar)\" \"\n\t $command -C scalar-clone/src $args\n\t \"\n\n\t test_perf \"$command (non-scalar)\" \"\n\t $command -C git-clone $args\n\t \"\n\t}\n\nWhat do you think?\n\nCiao,\nDscho\n"},{"id":"462434","messageId":"60p9nos4-n576-qqq2-p8n9-969sro247141@tzk.qr","threadId":"58383","inReplyTo":"96e08a95265ea66839b439ce8abc50b34395aaa3.1661961746.git.gitgitgadget@gmail.com","subject":"Re: [PATCH 7/8] t/perf: add 'GIT_PERF_USE_SCALAR' run option","fromName":"Johannes Schindelin","fromEmail":"johannes.schindelin@gmx.de","sentAt":"2022-09-01T09:43:30Z","receivedAt":"2022-09-01T09:43:58Z","isPatch":true,"sender":{"key":"johannes.schindelin@gmx.de","avatar":"https://avatars.githubusercontent.com/u/127790?v=4"},"body":"Hi Victoria,\n\nOn Wed, 31 Aug 2022, Victoria Dye via GitGitGadget wrote:\n\n> From: Victoria Dye \n>\n> Add a 'GIT_PERF_USE_SCALAR' environment variable (and corresponding perf\n> config 'useScalar') to register a repository created with any of:\n>\n> * test_perf_fresh_repo\n> * test_perf_default_repo\n> * test_perf_large_repo\n>\n> as a Scalar enlistment. This is intended to allow a developer to test the\n> impact of Scalar on already-defined performance scenarios.\n\nGreat idea!\n\n> [...]\n> @@ -130,7 +137,11 @@ test_perf_fresh_repo () {\n> \t\"$MODERN_GIT\" init -q \"$repo\" &&\n> \t(\n> \t\tcd \"$repo\" &&\n> -\t\ttest_perf_do_repo_symlink_config_\n> +\t\ttest_perf_do_repo_symlink_config_ &&\n> +\t\tif test_bool_env \"$GIT_PERF_USE_SCALAR\" false\n> +\t\tthen\n> +\t\t\t\"$MODERN_SCALAR\" register\n\nDo we need to unregister anything here? My guess is that no, the \"global\"\nconfig we're using in tests is \"$TRASH_DIRECTORY/.gitconfig\", and the side\neffect of scheduling the maintenance task won't matter in practice. But I\nmight have missed something and we may want to have an explicit\n`unregister` step.\n\nWhat's your take on this?\n\nCiao,\nDscho\n\n> +\t\tfi\n> \t)\n> }\n"},{"id":"462435","messageId":"21q1o4p0-opq8-31sq-r122-q2rq72732srr@tzk.qr","threadId":"58383","inReplyTo":"pull.1341.git.1661961746.gitgitgadget@gmail.com","subject":"Re: [PATCH 0/8] scalar: integrate into core Git","fromName":"Johannes Schindelin","fromEmail":"johannes.schindelin@gmx.de","sentAt":"2022-09-01T09:56:16Z","receivedAt":"2022-09-01T09:56:27Z","isPatch":true,"sender":{"key":"johannes.schindelin@gmx.de","avatar":"https://avatars.githubusercontent.com/u/127790?v=4"},"body":"Hi Victoria,\n\nOn Wed, 31 Aug 2022, Victoria Dye via GitGitGadget wrote:\n\n> This series completes the initial implementation of the Scalar command as a\n> core component of Git. For readers new to the topic of Scalar, the\n> roadmap/design doc [1] provides some background information including how\n> the project started & evolved, as well as its current intent & high-level\n> design.\n\nThank you for giving this a fresh start.\n\nMy previous attempts all got mired and stuck in \"mine vs yours\" contests,\nand a fresh start was the best that could happen to the Scalar effort to\nget unstuck.\n\nI reviewed all the patches, provided some commentary, cheering on, and\nalso a few suggestions. Feel free to add my `Reviewed-by`.\n\nThank you,\nDscho\n"},{"id":"462441","messageId":"cover-0.5-00000000000-20220901T130817Z-avarab@gmail.com","threadId":"58383","inReplyTo":"sso99so6-n28s-rq86-8q20-4456r3pn869r@tzk.qr","subject":"[PATCH 0/5] Makefile: split up $(test_bindir_programs)","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2022-09-01T13:17:23Z","receivedAt":"2022-09-01T13:20:08Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"On Thu, Sep 01 2022, Johannes Schindelin wrote:\n\n> [...]\n> On Wed, 31 Aug 2022, Victoria Dye via GitGitGadget wrote:\n> [...]\n>> @@ -3062,7 +3067,7 @@ bin-wrappers/%: wrap-for-bin.sh\n>> \t$(call mkdir_p_parent_template)\n>> \t$(QUIET_GEN)sed -e '1s|#!.*/sh|#!$(SHELL_PATH_SQ)|' \\\n>> \t -e 's|@@BUILD_DIR@@|$(shell pwd)|' \\\n>> -\t -e 's|@@PROG@@|$(patsubst test-%,t/helper/test-%$(X),$(@F))$(patsubst git%,$(X),$(filter $(@F),$(BINDIR_PROGRAMS_NEED_X)))|' < $< > $@ && \\\n>> +\t -e 's|@@PROG@@|$(patsubst test-%,t/helper/test-%,$(@F))$(if $(filter-out $(BINDIR_PROGRAMS_NO_X),$(@F)),$(X),)|' < $< > $@ && \\\n>\n> It took me a good while to wrap my head around this (and let me be clear:\n> I consider none of this your fault, it's the fault of the design of the\n> Makefile syntax).\n>\n> Let me untangle this, for posterity's benefit. We substitute the\n> placeholder `@@PROG@@` with a concatenation of two strings that are both\n> derived from `@F`, i.e. the basename of the to-be-wrapped command.\n\nWe could do this later, but the 3/5 here is my reply to the \"fault of\nthe design of the Makfile syntax\".\n\nI really don't think that's the case, the problem here is something\nyou'd get any any language.\n\nWe have three lists which we'd like to treat differently, but for no\nparticularly good reason other than incrementally building on past\nchanges to end up with this we end up having to on-the-fly guess which\nlist a given item came from.\n\nWith this series the end result is instead to do:\n\t\n\tdefine bin_wrappers_template\n\tBW_$(1) = $$($(1):%=bin-wrappers/%)\n\tBIN_WRAPPERS += $$(BW_$(1))\n\tall:: $$(BW_$(1))\n\t$$(BW_$(1)): bin-wrappers/% : $(3)%$(4)\n\t$$(BW_$(1)): wrap-for-bin.sh\n\t\t$$(call mkdir_p_parent_template)\n\t\t$$(QUIET_GEN)$$(call cmd_munge_bin_wrappers_script,$(2),$(3),$(4))\n\tendef\n\t\n\t$(eval $(call bin_wrappers_template,BINDIR_PROGRAMS_NEED_X,'$$(@F)',,$$X))\n\t$(eval $(call bin_wrappers_template,BINDIR_PROGRAMS_NO_X,'$$(@F)'))\n\t$(eval $(call bin_wrappers_template,TEST_PROGRAMS_NEED_X,'$$(@F)',t/helper/,$$X))\n\t\n\tall:: $(BIN_WRAPPERS)\n\nThis obviously conflicts with Victoria's changes here, but if picked\nup the conflict can be entirely solved in favor of this series, and\nthis \"scalar\" series will benefit.\n\nI.e. the only reason this series needs to patch this one liner is\nbecause the Makefile is losing track of where the item(s) came from.\n\nOnce we're not doing that we're perfectly capable of creating a\nbin-wrappers/scalar, because we're no longer running into the logic\nthat uses git% as a heuristic to determine whether something is \"not\nfrom the $(TEST_PROGRAMS_NEED_X) variable\".\n\nA CI run, showing that this also works on Windows etc.:\nhttps://github.com/avar/git/runs/8135048620\n\nÆvar Arnfjörð Bjarmason (5):\n Makefile: factor sed-powered '#!/bin/sh' munging into a variable\n Makefile: define \"TEST_{PROGRAM,OBJS}\" variables earlier\n Makefile: simplify $(test_bindir_programs) rule by splitting it up\n Makefile: define bin-wrappers/% rules with a template\n Makefile: fix \"make clean && make bin-wrappers/$NAME\" dependencies\n\n Makefile | 58 +++++++++++++++++++++++++++++++++++++-------------------\n 1 file changed, 39 insertions(+), 19 deletions(-)\n\n-- \n2.37.3.1426.g360dd7cf8ca\n\n"},{"id":"462440","messageId":"patch-1.5-df150656e72-20220901T130817Z-avarab@gmail.com","threadId":"58383","inReplyTo":"cover-0.5-00000000000-20220901T130817Z-avarab@gmail.com","subject":"[PATCH 1/5] Makefile: factor sed-powered '#!/bin/sh' munging into a variable","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2022-09-01T13:17:24Z","receivedAt":"2022-09-01T13:20:11Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"Reduce the amount of magical copy/pasting in the Makefile by factoring\nthe munging of \"#!/bin/sh\" on the first line of a shellscript into a\nvariable we can re-use in the various rules that need to do so.\n\nSigned-off-by: Ævar Arnfjörð Bjarmason \n---\n Makefile | 12 ++++++++----\n 1 file changed, 8 insertions(+), 4 deletions(-)\n\ndiff --git a/Makefile b/Makefile\nindex eac30126e29..b2304aa93a3 100644\n--- a/Makefile\n+++ b/Makefile\n@@ -2338,8 +2338,12 @@ GIT-SCRIPT-DEFINES: FORCE\n \t\techo \"$$FLAGS\" >$@; \\\n fi\n \n+define cmd_munge_script_sed_shell_path_arg\n+'1s|#!.*/sh|#!$(SHELL_PATH_SQ)|'\n+endef\n+\n define cmd_munge_script\n-sed -e '1s|#!.*/sh|#!$(SHELL_PATH_SQ)|' \\\n+sed -e $(call cmd_munge_script_sed_shell_path_arg) \\\n -e 's|@SHELL_PATH@|$(SHELL_PATH_SQ)|' \\\n -e 's|@@DIFF@@|$(DIFF_SQ)|' \\\n -e 's|@@LOCALEDIR@@|$(localedir_SQ)|g' \\\n@@ -2441,7 +2445,7 @@ git-instaweb: git-instaweb.sh GIT-SCRIPT-DEFINES\n else # NO_PERL\n $(SCRIPT_PERL_GEN) git-instaweb: % : unimplemented.sh\n \t$(QUIET_GEN) \\\n-\tsed -e '1s|#!.*/sh|#!$(SHELL_PATH_SQ)|' \\\n+\tsed -e $(call cmd_munge_script_sed_shell_path_arg) \\\n \t -e 's|@@REASON@@|NO_PERL=$(NO_PERL)|g' \\\n \t unimplemented.sh >$@+ && \\\n \tchmod +x $@+ && \\\n@@ -2462,7 +2466,7 @@ $(SCRIPT_PYTHON_GEN): % : %.py\n else # NO_PYTHON\n $(SCRIPT_PYTHON_GEN): % : unimplemented.sh\n \t$(QUIET_GEN) \\\n-\tsed -e '1s|#!.*/sh|#!$(SHELL_PATH_SQ)|' \\\n+\tsed -e $(call cmd_munge_script_sed_shell_path_arg) \\\n \t -e 's|@@REASON@@|NO_PYTHON=$(NO_PYTHON)|g' \\\n \t unimplemented.sh >$@+ && \\\n \tchmod +x $@+ && \\\n@@ -3060,7 +3064,7 @@ all:: $(TEST_PROGRAMS) $(test_bindir_programs)\n \n bin-wrappers/%: wrap-for-bin.sh\n \t$(call mkdir_p_parent_template)\n-\t$(QUIET_GEN)sed -e '1s|#!.*/sh|#!$(SHELL_PATH_SQ)|' \\\n+\t$(QUIET_GEN)sed -e $(call cmd_munge_script_sed_shell_path_arg) \\\n \t -e 's|@@BUILD_DIR@@|$(shell pwd)|' \\\n \t -e 's|@@PROG@@|$(patsubst test-%,t/helper/test-%$(X),$(@F))$(patsubst git%,$(X),$(filter $(@F),$(BINDIR_PROGRAMS_NEED_X)))|' < $< > $@ && \\\n \tchmod +x $@\n-- \n2.37.3.1426.g360dd7cf8ca\n\n"},{"id":"462442","messageId":"patch-2.5-eeae7ac4ddc-20220901T130817Z-avarab@gmail.com","threadId":"58383","inReplyTo":"cover-0.5-00000000000-20220901T130817Z-avarab@gmail.com","subject":"[PATCH 2/5] Makefile: define \"TEST_{PROGRAM,OBJS}\" variables earlier","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2022-09-01T13:17:25Z","receivedAt":"2022-09-01T13:20:15Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"Define the variables that make up TEST_OBJS earlier, and don't go back\n& forth in their definition. Before we'd first append $X to\n$(TEST_PROGRAMS), and then substitute $X back out of it to define\n$(TEST_OBJS). Let's instead add a new $(TEST_PROGRAM_OBJS) variable,\nwhich avoids this needless back & forth substitution.\n\nSee daa99a91729 (Makefile: make sure test helpers are rebuilt when\nheaders change, 2010-01-26) for how we ended up with the original\n$(TEST_OBJS).\n\nSigned-off-by: Ævar Arnfjörð Bjarmason \n---\n Makefile | 17 +++++++++--------\n 1 file changed, 9 insertions(+), 8 deletions(-)\n\ndiff --git a/Makefile b/Makefile\nindex b2304aa93a3..0d6b2caa7d9 100644\n--- a/Makefile\n+++ b/Makefile\n@@ -617,7 +617,8 @@ SCRIPT_PYTHON =\n SCRIPT_SH =\n SCRIPT_LIB =\n TEST_BUILTINS_OBJS =\n-TEST_OBJS =\n+TEST_PROGRAMS =\n+TEST_PROGRAM_OBJS =\n TEST_PROGRAMS_NEED_X =\n THIRD_PARTY_SOURCES =\n \n@@ -794,6 +795,7 @@ TEST_BUILTINS_OBJS += test-wildmatch.o\n TEST_BUILTINS_OBJS += test-windows-named-pipe.o\n TEST_BUILTINS_OBJS += test-write-cache.o\n TEST_BUILTINS_OBJS += test-xml-encode.o\n+TEST_PROGRAM_OBJS += $(patsubst %,t/helper/%,$(TEST_BUILTINS_OBJS))\n \n # Do not add more tests here unless they have extra dependencies. Add\n # them in TEST_BUILTINS_OBJS above.\n@@ -801,6 +803,9 @@ TEST_PROGRAMS_NEED_X += test-fake-ssh\n TEST_PROGRAMS_NEED_X += test-tool\n \n TEST_PROGRAMS = $(patsubst %,t/helper/%$X,$(TEST_PROGRAMS_NEED_X))\n+all:: $(TEST_PROGRAMS)\n+TEST_PROGRAM_OBJS += $(patsubst %,t/helper/%.o,$(TEST_PROGRAMS_NEED_X))\n+.PRECIOUS: $(TEST_PROGRAM_OBJS)\n \n # List built-in command $C whose implementation cmd_$C() is not in\n # builtin/$C.o but is linked in as part of some other command.\n@@ -2537,10 +2542,8 @@ REFTABLE_TEST_OBJS += reftable/stack_test.o\n REFTABLE_TEST_OBJS += reftable/test_framework.o\n REFTABLE_TEST_OBJS += reftable/tree_test.o\n \n-TEST_OBJS := $(patsubst %$X,%.o,$(TEST_PROGRAMS)) $(patsubst %,t/helper/%,$(TEST_BUILTINS_OBJS))\n-\n .PHONY: test-objs\n-test-objs: $(TEST_OBJS)\n+test-objs: $(TEST_PROGRAM_OBJS)\n \n GIT_OBJS += $(LIB_OBJS)\n GIT_OBJS += $(BUILTIN_OBJS)\n@@ -2551,7 +2554,7 @@ git-objs: $(GIT_OBJS)\n \n OBJECTS += $(GIT_OBJS)\n OBJECTS += $(PROGRAM_OBJS)\n-OBJECTS += $(TEST_OBJS)\n+OBJECTS += $(TEST_PROGRAM_OBJS)\n OBJECTS += $(XDIFF_OBJS)\n OBJECTS += $(FUZZ_OBJS)\n OBJECTS += $(REFTABLE_OBJS) $(REFTABLE_TEST_OBJS)\n@@ -3060,7 +3063,7 @@ endif\n \n test_bindir_programs := $(patsubst %,bin-wrappers/%,$(BINDIR_PROGRAMS_NEED_X) $(BINDIR_PROGRAMS_NO_X) $(TEST_PROGRAMS_NEED_X))\n \n-all:: $(TEST_PROGRAMS) $(test_bindir_programs)\n+all:: $(test_bindir_programs)\n \n bin-wrappers/%: wrap-for-bin.sh\n \t$(call mkdir_p_parent_template)\n@@ -3086,8 +3089,6 @@ perf: all\n \n .PHONY: test perf\n \n-.PRECIOUS: $(TEST_OBJS)\n-\n t/helper/test-tool$X: $(patsubst %,t/helper/%,$(TEST_BUILTINS_OBJS))\n \n t/helper/test-%$X: t/helper/test-%.o GIT-LDFLAGS $(GITLIBS) $(REFTABLE_TEST_LIB)\n-- \n2.37.3.1426.g360dd7cf8ca\n\n"},{"id":"462443","messageId":"patch-3.5-0f66992d5f7-20220901T130817Z-avarab@gmail.com","threadId":"58383","inReplyTo":"cover-0.5-00000000000-20220901T130817Z-avarab@gmail.com","subject":"[PATCH 3/5] Makefile: simplify $(test_bindir_programs) rule by splitting it up","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2022-09-01T13:17:26Z","receivedAt":"2022-09-01T13:20:16Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"When the @@PROG@@ substitution was added in [1] it was a simple matter\nof doing a:\n\n\t's|@@PROG@@|$(@F)|'\n\nThen when t/helpers were added in [2] followed by the \".exe\" needing\nto be appended in [3] this previously simple rule ended up as a dense\none-liner.\n\nIt has been pointed out that this is hard to read[4], but the problem\nisn't the \"design of the Makefile syntax\". It's we now have to\nafter-the-fact determine if we were dealing with a bin-wrapper/ that\nneeded to have $(X) appended, a t/helper/, or a non-binary (currently\nonly git-cvsserver).\n\nThat would be a problem in any language. We're starting out with three\nlists, and then end up having to heuristically determine given a\nmember of any of those lists which list that member came from. Let's\njust stop doing that and keep track of what member belongs to which\nlist.\n\nWe can do this by splitting up the single \"bin-wrappers/%\" rule into a\nrule for each of the three lists. With the\n\"cmd_munge_script_sed_shell_path_arg\" define added in a preceding\ncommit this is easy, we just need to add a sister template to the\nexisting \"cmd_munge_script\" added in [5].\n\nThere is then an an in-flight change[6] that generalizes this to\nsupport \"scalar\". That change needed to further special-case the\npre-image here, as we couldn't rely on the \"git%\" heuristic to mean\n\"not from the $(TEST_PROGRAMS_NEED_X) variable\".\n\nWith this conflicting change that special-casing becomes\nunnecessary. The conflict between the two can be resolved entirely in\nfavor of this change, as the other change adds \"scalar\" to\n\"$(BINDIR_PROGRAMS_NEED_X)\", and we are no longer losing track of\nwhich members belong to that list.\n\nThis change can be tested with e.g.:\n\n\tgit checkout master &&\n\tmake SHELL_PATH=/bin/bash X=.exe &&\n\tmv bin-wrappers bin-wrappers.master &&\n\t &&\n\tmake SHELL_PATH=/bin/bash X=.exe &&\n\tdiff -ru bin-wrappers{.master,}\n\nWhich will show an empty diff, i.e. we've correctly dealt with the\ncombination of $(SHELL_PATH), $(X) and these three variables here.\n\n1. ea925196f1b (build dashless \"bin-wrappers\" directory similar to\n installed bindir, 2009-12-02)\n2. e6e7530d10b (test helpers: move test-* to t/helper/ subdirectory,\n 2016-04-13)\n3. 3a94cb31d52 (bin-wrappers: append `.exe` to target paths if\n necessary, 2019-07-29)\n4. https://lore.kernel.org/git/sso99so6-n28s-rq86-8q20-4456r3pn869r@tzk.qr/\n5. 46bac904581 (Do not install shell libraries executable, 2010-01-31)\n6. https://lore.kernel.org/git/4d69e5eaccb8873eece666a3d2bb2b22abdde7ea.1661961746.git.gitgitgadget@gmail.com/\n\nSigned-off-by: Ævar Arnfjörð Bjarmason \n---\n Makefile | 35 +++++++++++++++++++++++++++--------\n 1 file changed, 27 insertions(+), 8 deletions(-)\n\ndiff --git a/Makefile b/Makefile\nindex 0d6b2caa7d9..96de9e55864 100644\n--- a/Makefile\n+++ b/Makefile\n@@ -597,6 +597,7 @@ export TCL_PATH TCLTK_PATH\n PTHREAD_LIBS = -lpthread\n \n # Guard against environment variables\n+BIN_WRAPPERS =\n BUILTIN_OBJS =\n BUILT_INS =\n COMPAT_CFLAGS =\n@@ -3061,16 +3062,34 @@ GIT-PYTHON-VARS: FORCE\n fi\n endif\n \n-test_bindir_programs := $(patsubst %,bin-wrappers/%,$(BINDIR_PROGRAMS_NEED_X) $(BINDIR_PROGRAMS_NO_X) $(TEST_PROGRAMS_NEED_X))\n+define cmd_munge_bin_wrappers_script\n+sed \\\n+\t-e $(call cmd_munge_script_sed_shell_path_arg) \\\n+\t-e 's|@@BUILD_DIR@@|$(shell pwd)|' \\\n+\t-e 's|@@PROG@@|$(2)$(1)$(3)|' \\\n+\t<$< >$@ && \\\n+\tchmod +x $@\n+endef\n \n-all:: $(test_bindir_programs)\n+BW_BINDIR_PROGRAMS_NEED_X = $(BINDIR_PROGRAMS_NEED_X:%=bin-wrappers/%)\n+BIN_WRAPPERS += $(BW_BINDIR_PROGRAMS_NEED_X)\n+$(BW_BINDIR_PROGRAMS_NEED_X): wrap-for-bin.sh\n+\t$(call mkdir_p_parent_template)\n+\t$(QUIET_GEN)$(call cmd_munge_bin_wrappers_script,$(@F),,$X)\n \n-bin-wrappers/%: wrap-for-bin.sh\n+BW_BINDIR_PROGRAMS_NO_X = $(BINDIR_PROGRAMS_NO_X:%=bin-wrappers/%)\n+BIN_WRAPPERS += $(BW_BINDIR_PROGRAMS_NO_X)\n+$(BW_BINDIR_PROGRAMS_NO_X): wrap-for-bin.sh\n \t$(call mkdir_p_parent_template)\n-\t$(QUIET_GEN)sed -e $(call cmd_munge_script_sed_shell_path_arg) \\\n-\t -e 's|@@BUILD_DIR@@|$(shell pwd)|' \\\n-\t -e 's|@@PROG@@|$(patsubst test-%,t/helper/test-%$(X),$(@F))$(patsubst git%,$(X),$(filter $(@F),$(BINDIR_PROGRAMS_NEED_X)))|' < $< > $@ && \\\n-\tchmod +x $@\n+\t$(QUIET_GEN)$(call cmd_munge_bin_wrappers_script,$(@F))\n+\n+BW_TEST_PROGRAMS_NEED_X = $(TEST_PROGRAMS_NEED_X:%=bin-wrappers/%)\n+BIN_WRAPPERS += $(BW_TEST_PROGRAMS_NEED_X)\n+$(BW_TEST_PROGRAMS_NEED_X): wrap-for-bin.sh\n+\t$(call mkdir_p_parent_template)\n+\t$(QUIET_GEN)$(call cmd_munge_bin_wrappers_script,$(@F),t/helper/,$X)\n+\n+all:: $(BIN_WRAPPERS)\n \n # GNU make supports exporting all variables by \"export\" without parameters.\n # However, the environment gets quite big, and some programs have problems\n@@ -3403,7 +3422,7 @@ OTHER_PROGRAMS += $(shell echo *.dll t/helper/*.dll)\n endif\n \n artifacts-tar:: $(ALL_COMMANDS_TO_INSTALL) $(SCRIPT_LIB) $(OTHER_PROGRAMS) \\\n-\t\tGIT-BUILD-OPTIONS $(TEST_PROGRAMS) $(test_bindir_programs) \\\n+\t\tGIT-BUILD-OPTIONS $(TEST_PROGRAMS) $(BIN_WRAPPERS) \\\n \t\t$(MOFILES)\n \t$(QUIET_SUBDIR0)templates $(QUIET_SUBDIR1) \\\n \t\tSHELL_PATH='$(SHELL_PATH_SQ)' PERL_PATH='$(PERL_PATH_SQ)'\n-- \n2.37.3.1426.g360dd7cf8ca\n\n"},{"id":"462444","messageId":"patch-5.5-d89e2607077-20220901T130817Z-avarab@gmail.com","threadId":"58383","inReplyTo":"cover-0.5-00000000000-20220901T130817Z-avarab@gmail.com","subject":"[PATCH 5/5] Makefile: fix \"make clean && make bin-wrappers/$NAME\" dependencies","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2022-09-01T13:17:28Z","receivedAt":"2022-09-01T13:20:18Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"The \"bin-wrappers/\" scripts have never had properly declared\ndependency information, i.e. this has never worked:\n\n\tmake clean &&\n\tmake bin-wrappers/git &&\n\t./bin-wrappers/git\n\nThere was no reason not to have that work, just as most things\ngenerated by the Makefile have proper dependency information.\n\nBefore the preceding commits doing this would have been painful, but\nnow it's quite easy to simply make use of the third parameter to our\n\"bin_wrappers_template\"\n\nSigned-off-by: Ævar Arnfjörð Bjarmason \n---\n Makefile | 3 ++-\n 1 file changed, 2 insertions(+), 1 deletion(-)\n\ndiff --git a/Makefile b/Makefile\nindex 533858ca481..07a9901c99d 100644\n--- a/Makefile\n+++ b/Makefile\n@@ -3075,9 +3075,10 @@ define bin_wrappers_template\n BW_$(1) = $$($(1):%=bin-wrappers/%)\n BIN_WRAPPERS += $$(BW_$(1))\n all:: $$(BW_$(1))\n+$$(BW_$(1)): bin-wrappers/% : $(3)%$(4)\n $$(BW_$(1)): wrap-for-bin.sh\n \t$$(call mkdir_p_parent_template)\n-\t$$(QUIET_GEN)$$(call cmd_munge_bin_wrappers_script,$(2),$(3))\n+\t$$(QUIET_GEN)$$(call cmd_munge_bin_wrappers_script,$(2),$(3),$(4))\n endef\n \n $(eval $(call bin_wrappers_template,BINDIR_PROGRAMS_NEED_X,'$$(@F)',,$$X))\n-- \n2.37.3.1426.g360dd7cf8ca\n\n"},{"id":"462445","messageId":"patch-4.5-a6d626a5ee1-20220901T130817Z-avarab@gmail.com","threadId":"58383","inReplyTo":"cover-0.5-00000000000-20220901T130817Z-avarab@gmail.com","subject":"[PATCH 4/5] Makefile: define bin-wrappers/% rules with a template","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2022-09-01T13:17:27Z","receivedAt":"2022-09-01T13:20:20Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"Eliminate the copy/pasting among the three rules generating the\nbin-wrappers/% files by refactoring them into a template.\n\nSigned-off-by: Ævar Arnfjörð Bjarmason \n---\n Makefile | 27 +++++++++++----------------\n 1 file changed, 11 insertions(+), 16 deletions(-)\n\ndiff --git a/Makefile b/Makefile\nindex 96de9e55864..533858ca481 100644\n--- a/Makefile\n+++ b/Makefile\n@@ -3071,23 +3071,18 @@ sed \\\n \tchmod +x $@\n endef\n \n-BW_BINDIR_PROGRAMS_NEED_X = $(BINDIR_PROGRAMS_NEED_X:%=bin-wrappers/%)\n-BIN_WRAPPERS += $(BW_BINDIR_PROGRAMS_NEED_X)\n-$(BW_BINDIR_PROGRAMS_NEED_X): wrap-for-bin.sh\n-\t$(call mkdir_p_parent_template)\n-\t$(QUIET_GEN)$(call cmd_munge_bin_wrappers_script,$(@F),,$X)\n-\n-BW_BINDIR_PROGRAMS_NO_X = $(BINDIR_PROGRAMS_NO_X:%=bin-wrappers/%)\n-BIN_WRAPPERS += $(BW_BINDIR_PROGRAMS_NO_X)\n-$(BW_BINDIR_PROGRAMS_NO_X): wrap-for-bin.sh\n-\t$(call mkdir_p_parent_template)\n-\t$(QUIET_GEN)$(call cmd_munge_bin_wrappers_script,$(@F))\n+define bin_wrappers_template\n+BW_$(1) = $$($(1):%=bin-wrappers/%)\n+BIN_WRAPPERS += $$(BW_$(1))\n+all:: $$(BW_$(1))\n+$$(BW_$(1)): wrap-for-bin.sh\n+\t$$(call mkdir_p_parent_template)\n+\t$$(QUIET_GEN)$$(call cmd_munge_bin_wrappers_script,$(2),$(3))\n+endef\n \n-BW_TEST_PROGRAMS_NEED_X = $(TEST_PROGRAMS_NEED_X:%=bin-wrappers/%)\n-BIN_WRAPPERS += $(BW_TEST_PROGRAMS_NEED_X)\n-$(BW_TEST_PROGRAMS_NEED_X): wrap-for-bin.sh\n-\t$(call mkdir_p_parent_template)\n-\t$(QUIET_GEN)$(call cmd_munge_bin_wrappers_script,$(@F),t/helper/,$X)\n+$(eval $(call bin_wrappers_template,BINDIR_PROGRAMS_NEED_X,'$$(@F)',,$$X))\n+$(eval $(call bin_wrappers_template,BINDIR_PROGRAMS_NO_X,'$$(@F)'))\n+$(eval $(call bin_wrappers_template,TEST_PROGRAMS_NEED_X,'$$(@F)',t/helper/,$$X))\n \n all:: $(BIN_WRAPPERS)\n \n-- \n2.37.3.1426.g360dd7cf8ca\n\n"},{"id":"462450","messageId":"d6c46ce8-a4a5-f120-30d2-2388b6b63d7e@github.com","threadId":"58383","inReplyTo":"cover-0.5-00000000000-20220901T130817Z-avarab@gmail.com","subject":"Re: [PATCH 0/5] Makefile: split up $(test_bindir_programs)","fromName":"Derrick Stolee","fromEmail":"derrickstolee@github.com","sentAt":"2022-09-01T15:02:42Z","receivedAt":"2022-09-01T15:02:48Z","isPatch":true,"sender":{"key":"stolee@gmail.com","avatar":"https://avatars.githubusercontent.com/u/570044?v=4"},"body":"On 9/1/2022 9:17 AM, Ævar Arnfjörð Bjarmason wrote:\n\n> Ævar Arnfjörð Bjarmason (5):\n> Makefile: factor sed-powered '#!/bin/sh' munging into a variable\n> Makefile: define \"TEST_{PROGRAM,OBJS}\" variables earlier\n> Makefile: simplify $(test_bindir_programs) rule by splitting it up\n> Makefile: define bin-wrappers/% rules with a template\n> Makefile: fix \"make clean && make bin-wrappers/$NAME\" dependencies\n> \n> Makefile | 58 +++++++++++++++++++++++++++++++++++++-------------------\n> 1 file changed, 39 insertions(+), 19 deletions(-)\n \nI find it very distracting when you send a full patch series in the\nmiddle of someone else's patch series. Viewing the thread now involves\ntwo distinct conversations, assuming that your series gets review and\nrequires multiple versions.\n\nI would rather that you started a new thread with your proposed series\nand then sent a reply in this thread pointing to that one.\n\nThanks,\n-Stolee\n"},{"id":"462462","messageId":"552dc600-9a6e-988c-a4ff-18a6eb45e121@github.com","threadId":"58383","inReplyTo":"220831.8635dc9wzx.gmgdl@evledraar.gmail.com","subject":"Re: [PATCH 4/8] scalar: implement the `help` subcommand","fromName":"Victoria Dye","fromEmail":"vdye@github.com","sentAt":"2022-09-01T16:08:29Z","receivedAt":"2022-09-01T16:08:55Z","isPatch":true,"sender":{"key":"vdye@github.com","avatar":"https://avatars.githubusercontent.com/u/3619353?v=4"},"body":"Ævar Arnfjörð Bjarmason wrote:\n> \n> On Wed, Aug 31 2022, Johannes Schindelin via GitGitGadget wrote:\n> \n>> From: Johannes Schindelin \n>>\n>> It is merely handing off to `git help scalar`.\n>>\n>> Signed-off-by: Johannes Schindelin \n>> Signed-off-by: Victoria Dye \n>> ---\n>> scalar.c | 20 ++++++++++++++++++++\n>> 1 file changed, 20 insertions(+)\n>>\n>> diff --git a/scalar.c b/scalar.c\n>> index 642d16124eb..675d7a6b0a9 100644\n>> --- a/scalar.c\n>> +++ b/scalar.c\n>> @@ -819,6 +819,25 @@ static int cmd_delete(int argc, const char **argv)\n>> \treturn res;\n>> }\n>> \n>> +static int cmd_help(int argc, const char **argv)\n>> +{\n>> +\tstruct option options[] = {\n>> +\t\tOPT_END(),\n>> +\t};\n>> +\tconst char * const usage[] = {\n>> +\t\tN_(\"scalar help\"),\n> \n> \n> This should not have N_(), as it's a literal command.\n\nThanks, will fix. \n\n> \n>> +\t\tNULL\n>> +\t};\n>> +\n>> +\targc = parse_options(argc, argv, NULL, options,\n>> +\t\t\t usage, 0);\n>> +\n>> +\tif (argc != 0)\n> \n> If we're re-rolling anyway we usually just do \"if (argc)\". We don't need\n> to worry about argc < 0 (despite the signed type, which is a historical\n> C wart).\n\nNormally I'd agree, but in this case there's a readability benefit to\nexplicitly comparing 'argc' to 0. 'scalar help' expects exactly zero\npositional arguments, and showing the '!= 0' makes that expectation clearer\n(likewise, 'scalar delete' checks that 'argc != 1' because it expects\nexactly one positional arg). \n\n> \n>> +\t\tusage_with_options(usage, options);\n>> +\n>> +\treturn run_git(\"help\", \"scalar\", NULL);\n> \n> Performance isn't sensitive here, but have you tried just calling\n> cmd_help() instead with the appropriate arguments? It would avoid\n> spawning another command..\n\nAs a matter of design preference, I'd rather avoid invoking builtins via\ntheir 'cmd_*()' entrypoint. Doing so in 'scalar.c' would also introduce some\nfunction name conflicts. While that's an overcomeable problem, precedent\nacross Git doesn't appear to indicate one approach is better than the other,\nso I'm happy with things as they are. \n"},{"id":"462463","messageId":"586fc6ca-b699-14a9-c5a1-10323a1cde0c@github.com","threadId":"58383","inReplyTo":"59s7o8n8-5797-q2qo-4q0o-93qppnr92q52@tzk.qr","subject":"Re: [PATCH 6/8] t/perf: add Scalar performance tests","fromName":"Victoria Dye","fromEmail":"vdye@github.com","sentAt":"2022-09-01T16:15:03Z","receivedAt":"2022-09-01T16:15:39Z","isPatch":true,"sender":{"key":"vdye@github.com","avatar":"https://avatars.githubusercontent.com/u/3619353?v=4"},"body":"Johannes Schindelin wrote:\n> Hi Victoria,\n> \n> On Wed, 31 Aug 2022, Victoria Dye via GitGitGadget wrote:\n> \n>> [...]\n>> +\n>> +test_compare_perf () {\n>> +\tcommand=\"$@\"\n>> +\ttest_perf \"$command (scalar)\" \"\n>> +\t\t(\n>> +\t\t\tcd scalar-clone/src &&\n>> +\t\t\t$command\n>> +\t\t)\n>> +\t\"\n>> +\n>> +\ttest_perf \"$command (non-scalar)\" \"\n>> +\t\t(\n>> +\t\t\tcd git-clone &&\n>> +\t\t\t$command\n>> +\t\t)\n>> +\t\"\n>> +}\n>> +\n>> +test_compare_perf git status\n>> +test_compare_perf test_commit --append --no-tag A\n> \n> Given the small numbers presented in the commit message, I suspect that\n> even so much as running the command in a subshell might skew the timings\n> at least on Windows, where subshells are very, very expensive.\n> \n> Given that both `git` and `test_commit` understand the `-C `\n> syntax, this variant would resolve my concern:\n> \n> \ttest_compare_perf () {\n> \t command=$1\n> shift\n> \t args=\"$*\"\n> \n> \t test_perf \"$command (scalar)\" \"\n> \t $command -C scalar-clone/src $args\n> \t \"\n> \n> \t test_perf \"$command (non-scalar)\" \"\n> \t $command -C git-clone $args\n> \t \"\n> \t}\n> \n> What do you think?\n\nMakes sense to me! Although, out of curiosity, is there a reason you prefer\n\"$1 -> shift -> $*\" over '$1' and '$@'? \n\n> \n> Ciao,\n> Dscho\n\n"},{"id":"462467","messageId":"86b7b9a3-8950-da9c-eae1-19ddb22456ec@github.com","threadId":"58383","inReplyTo":"586fc6ca-b699-14a9-c5a1-10323a1cde0c@github.com","subject":"Re: [PATCH 6/8] t/perf: add Scalar performance tests","fromName":"Victoria Dye","fromEmail":"vdye@github.com","sentAt":"2022-09-01T16:21:00Z","receivedAt":"2022-09-01T16:21:28Z","isPatch":true,"sender":{"key":"vdye@github.com","avatar":"https://avatars.githubusercontent.com/u/3619353?v=4"},"body":"Victoria Dye wrote:\n> Johannes Schindelin wrote:\n>> Hi Victoria,\n>>\n>> On Wed, 31 Aug 2022, Victoria Dye via GitGitGadget wrote:\n>>\n>>> [...]\n>>> +\n>>> +test_compare_perf () {\n>>> +\tcommand=\"$@\"\n>>> +\ttest_perf \"$command (scalar)\" \"\n>>> +\t\t(\n>>> +\t\t\tcd scalar-clone/src &&\n>>> +\t\t\t$command\n>>> +\t\t)\n>>> +\t\"\n>>> +\n>>> +\ttest_perf \"$command (non-scalar)\" \"\n>>> +\t\t(\n>>> +\t\t\tcd git-clone &&\n>>> +\t\t\t$command\n>>> +\t\t)\n>>> +\t\"\n>>> +}\n>>> +\n>>> +test_compare_perf git status\n>>> +test_compare_perf test_commit --append --no-tag A\n>>\n>> Given the small numbers presented in the commit message, I suspect that\n>> even so much as running the command in a subshell might skew the timings\n>> at least on Windows, where subshells are very, very expensive.\n>>\n>> Given that both `git` and `test_commit` understand the `-C `\n>> syntax, this variant would resolve my concern:\n>>\n>> \ttest_compare_perf () {\n>> \t command=$1\n>> shift\n>> \t args=\"$*\"\n>>\n>> \t test_perf \"$command (scalar)\" \"\n>> \t $command -C scalar-clone/src $args\n>> \t \"\n>>\n>> \t test_perf \"$command (non-scalar)\" \"\n>> \t $command -C git-clone $args\n>> \t \"\n>> \t}\n>>\n>> What do you think?\n> \n> Makes sense to me! Although, out of curiosity, is there a reason you prefer\n> \"$1 -> shift -> $*\" over '$1' and '$@'? \n\nWhoops, I completely misread your snippet; the 'shift' is necessary to\nseparate the '$command' out so that we can inject '-C'.\n\nThanks!\n\n> \n>>\n>> Ciao,\n>> Dscho\n> \n\n"},{"id":"462470","messageId":"xmqq7d2navvj.fsf@gitster.g","threadId":"58383","inReplyTo":"42ab39f21212d3da1af3546d3425aa790637056f.1661961746.git.gitgitgadget@gmail.com","subject":"Re: [PATCH 6/8] t/perf: add Scalar performance tests","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2022-09-01T16:43:12Z","receivedAt":"2022-09-01T16:43:20Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"\"Victoria Dye via GitGitGadget\" writes:\n\n> +test_compare_perf () {\n> +\tcommand=\"$@\"\n> +\ttest_perf \"$command (scalar)\" \"\n> +\t\t(\n> +\t\t\tcd scalar-clone/src &&\n> +\t\t\t$command\n> +\t\t)\n> +\t\"\n\nOur preference is to avoid using \"$@\" when you are not taking\nadvantage of the fact that it protects individual parameters from\ngetting split at $IFS whitespaces. Use \"$*\" is preferred here [*].\n\nFor example, this is good:\n\n mytest () {\n for arg in one two \"$@\" three\n do\n do_something_to \"$arg\"\n done\n }\n\n mytest 'a b c' 'd'\n\nThanks to the use of \"$@\", 'a b c' stays together and are kept\ndistinct from 'd'.\n\nThe above is not. command=\"$@\" is used as a misleading synonym for\ncommand=\"$*\" that flattens the arguments to test_compare_perf\nfunction. If you called it with 'a b c' and 'd' like we called\nmytest with, you cannot tell that 'a b c' were together and 'd' was\ndistinct from the other three. The only thing test_perf sees would\nbe that $command without double quotes have four separate tokens.\n\n\n[Footnote]\n\n* Of course, it is tolerated only in tests and perfs where we are in\n total control of the arguments so that we can declare that no args\n to the shell function have whitespace in them. In scripts used by\n end-users, we may not be able to get away with \"$*\".\n"},{"id":"462500","messageId":"23e8901f-b543-dbb5-c7e1-fa653fdf3d57@github.com","threadId":"58383","inReplyTo":"o429p430-sp18-rr20-p2ro-4r60qs2n83o5@tzk.qr","subject":"Re: [PATCH 5/8] scalar-clone: add test coverage","fromName":"Victoria Dye","fromEmail":"vdye@github.com","sentAt":"2022-09-01T23:49:52Z","receivedAt":"2022-09-01T23:50:00Z","isPatch":true,"sender":{"key":"vdye@github.com","avatar":"https://avatars.githubusercontent.com/u/3619353?v=4"},"body":"Johannes Schindelin wrote:\n> Hi Victoria,\n> \n> On Wed, 31 Aug 2022, Victoria Dye via GitGitGadget wrote:\n> \n>> From: Victoria Dye \n>>\n>> Create a new test file ('t9211-scalar-clone.sh') to exercise the options and\n>> behavior of the 'scalar clone' command.\n> \n> Great catch!\n> \n> I have one suggestion, given my experience with debugging test failures:\n> \n>> +test_expect_success 'creates content in enlistment root' '\n>> +\ttest_when_finished cleanup_clone cloned &&\n>> +\n>> +\tscalar clone \"file://$(pwd)/to-clone\" cloned &&\n> \n> This pattern of cloning into `cloned` and removing the directory when the\n> test case is done is repeated throughout this test script.\n> \n> In instances where all test cases succeed, that poses no problem.\n> \n> When running the test script with `-i`, also no problem.\n> \n> But when we run into test failures in CI, those directories will be\n> removed before the workflow run can tar them up and upload them for later\n> inspection.\n> \n> May I suggest an alternative strategy?\n> \n> If we drop all those `test_when_finished cleanup_clone cloned` calls and\n> instead `scalar clone` into different directories (whose names reflect the\n> test cases' intended purpose), I could imagine having a much easier time\n> not only diagnosing but also reproducing and fixing test failures in the\n> future.\n\nWhile I like the simplicity of using 'test_when_finished', I hadn't\nconsidered the value of having the failed tests' artifacts in the CI\nresults. If you think that would be helpful to developers, I'll update\naccordingly (although I'd still clean up the clones whose tests pass to\navoid archiving more data than we need).\n\nThat being said, even if I update 't9211', my experience with Git's test\nsuite is that very few tests preserve test repos this way. Do you expect\nthese artifacts to be especially helpful for 'scalar clone' in particular,\nor is this more of a \"gently nudge contributors to make this standard\npractice\" sort of recommendation? \n\n> \n> When discussing code review practices [*1*], we did not come up with any\n> standard terminology to describe what I am offering here, and I am unsure\n> how to label this in a catchy way. I want to present this suggestion for\n> you to consider, and I would be delighted if you take it, at the same time\n> I will happily let it go should you decide against it.\n> \n> Ciao,\n> Dscho\n> \n> Footnote *1*:\n> https://colabti.org/irclogger/irclogger_log/git-devel?date=2022-08-29#l48\n\n"},{"id":"462507","messageId":"72f6661d-be02-dbb8-7110-b8de6bac77f6@github.com","threadId":"58383","inReplyTo":"60p9nos4-n576-qqq2-p8n9-969sro247141@tzk.qr","subject":"Re: [PATCH 7/8] t/perf: add 'GIT_PERF_USE_SCALAR' run option","fromName":"Victoria Dye","fromEmail":"vdye@github.com","sentAt":"2022-09-02T04:00:33Z","receivedAt":"2022-09-02T04:01:09Z","isPatch":true,"sender":{"key":"vdye@github.com","avatar":"https://avatars.githubusercontent.com/u/3619353?v=4"},"body":"Johannes Schindelin wrote:\n> Hi Victoria,\n> \n> On Wed, 31 Aug 2022, Victoria Dye via GitGitGadget wrote:\n> \n>> From: Victoria Dye \n>>\n>> Add a 'GIT_PERF_USE_SCALAR' environment variable (and corresponding perf\n>> config 'useScalar') to register a repository created with any of:\n>>\n>> * test_perf_fresh_repo\n>> * test_perf_default_repo\n>> * test_perf_large_repo\n>>\n>> as a Scalar enlistment. This is intended to allow a developer to test the\n>> impact of Scalar on already-defined performance scenarios.\n> \n> Great idea!\n> \n>> [...]\n>> @@ -130,7 +137,11 @@ test_perf_fresh_repo () {\n>> \t\"$MODERN_GIT\" init -q \"$repo\" &&\n>> \t(\n>> \t\tcd \"$repo\" &&\n>> -\t\ttest_perf_do_repo_symlink_config_\n>> +\t\ttest_perf_do_repo_symlink_config_ &&\n>> +\t\tif test_bool_env \"$GIT_PERF_USE_SCALAR\" false\n>> +\t\tthen\n>> +\t\t\t\"$MODERN_SCALAR\" register\n> \n> Do we need to unregister anything here? My guess is that no, the \"global\"\n> config we're using in tests is \"$TRASH_DIRECTORY/.gitconfig\", and the side\n> effect of scheduling the maintenance task won't matter in practice. But I\n> might have missed something and we may want to have an explicit\n> `unregister` step.\n> \n> What's your take on this?\n\nAs you guessed, a '.gitconfig' is created in the trash directory of each\ntest containing the Scalar registration and I haven't seen any issues\narising from the scheduled maintenance, so I don't think an 'unregister' is\nnecessary. However, while verifying that, I noticed that the registration\nwasn't happening *at all* because 'test_bool_env' is currently being used\nincorrectly. The fix is straightforward - I'll make sure to correct it in\nthe next version.\n\nThanks!\n\n> \n> Ciao,\n> Dscho\n> \n>> +\t\tfi\n>> \t)\n>> }\n\n"},{"id":"462518","messageId":"9n46p076-q126-o813-8s39-936s6667o7o1@tzk.qr","threadId":"58383","inReplyTo":"23e8901f-b543-dbb5-c7e1-fa653fdf3d57@github.com","subject":"Re: [PATCH 5/8] scalar-clone: add test coverage","fromName":"Johannes Schindelin","fromEmail":"johannes.schindelin@gmx.de","sentAt":"2022-09-02T09:07:13Z","receivedAt":"2022-09-02T09:07:27Z","isPatch":true,"sender":{"key":"johannes.schindelin@gmx.de","avatar":"https://avatars.githubusercontent.com/u/127790?v=4"},"body":"Hi Victoria,\n\nOn Thu, 1 Sep 2022, Victoria Dye wrote:\n\n> Johannes Schindelin wrote:\n>\n> > On Wed, 31 Aug 2022, Victoria Dye via GitGitGadget wrote:\n> >\n> >> From: Victoria Dye \n> >>\n> >> Create a new test file ('t9211-scalar-clone.sh') to exercise the options and\n> >> behavior of the 'scalar clone' command.\n> >\n> > Great catch!\n> >\n> > I have one suggestion, given my experience with debugging test failures:\n> >\n> >> +test_expect_success 'creates content in enlistment root' '\n> >> +\ttest_when_finished cleanup_clone cloned &&\n> >> +\n> >> +\tscalar clone \"file://$(pwd)/to-clone\" cloned &&\n> >\n> > This pattern of cloning into `cloned` and removing the directory when the\n> > test case is done is repeated throughout this test script.\n> >\n> > In instances where all test cases succeed, that poses no problem.\n> >\n> > When running the test script with `-i`, also no problem.\n> >\n> > But when we run into test failures in CI, those directories will be\n> > removed before the workflow run can tar them up and upload them for later\n> > inspection.\n> >\n> > May I suggest an alternative strategy?\n> >\n> > If we drop all those `test_when_finished cleanup_clone cloned` calls and\n> > instead `scalar clone` into different directories (whose names reflect the\n> > test cases' intended purpose), I could imagine having a much easier time\n> > not only diagnosing but also reproducing and fixing test failures in the\n> > future.\n>\n> While I like the simplicity of using 'test_when_finished', I hadn't\n> considered the value of having the failed tests' artifacts in the CI\n> results. If you think that would be helpful to developers, I'll update\n> accordingly (although I'd still clean up the clones whose tests pass to\n> avoid archiving more data than we need).\n\nThank you!\n\n> That being said, even if I update 't9211', my experience with Git's test\n> suite is that very few tests preserve test repos this way. Do you expect\n> these artifacts to be especially helpful for 'scalar clone' in particular,\n> or is this more of a \"gently nudge contributors to make this standard\n> practice\" sort of recommendation?\n\nThank you for this question, which helps me clarify even to myself what my\nintention is.\n\nAfter considering this, yes, I would like this to be a gentle nudge to\ntake a tiny step toward improving Git's test suite by recommending a new\nstandard practice.\n\nCiao,\nDscho\n\n>\n> >\n> > When discussing code review practices [*1*], we did not come up with any\n> > standard terminology to describe what I am offering here, and I am unsure\n> > how to label this in a catchy way. I want to present this suggestion for\n> > you to consider, and I would be delighted if you take it, at the same time\n> > I will happily let it go should you decide against it.\n> >\n> > Ciao,\n> > Dscho\n> >\n> > Footnote *1*:\n> > https://colabti.org/irclogger/irclogger_log/git-devel?date=2022-08-29#l48\n>\n>\n"},{"id":"462519","messageId":"p9221sp1-r3on-ro35-3535-q423r641opn9@tzk.qr","threadId":"58383","inReplyTo":"86b7b9a3-8950-da9c-eae1-19ddb22456ec@github.com","subject":"Re: [PATCH 6/8] t/perf: add Scalar performance tests","fromName":"Johannes Schindelin","fromEmail":"johannes.schindelin@gmx.de","sentAt":"2022-09-02T09:16:13Z","receivedAt":"2022-09-02T09:16:58Z","isPatch":true,"sender":{"key":"johannes.schindelin@gmx.de","avatar":"https://avatars.githubusercontent.com/u/127790?v=4"},"body":"Hi Victoria,\n\nOn Thu, 1 Sep 2022, Victoria Dye wrote:\n\n> Victoria Dye wrote:\n> > Johannes Schindelin wrote:\n> >>\n> >> On Wed, 31 Aug 2022, Victoria Dye via GitGitGadget wrote:\n> >>\n> >>> [...]\n> >>> +\n> >>> +test_compare_perf () {\n> >>> +\tcommand=\"$@\"\n> >>> +\ttest_perf \"$command (scalar)\" \"\n> >>> +\t\t(\n> >>> +\t\t\tcd scalar-clone/src &&\n> >>> +\t\t\t$command\n> >>> +\t\t)\n> >>> +\t\"\n> >>> +\n> >>> +\ttest_perf \"$command (non-scalar)\" \"\n> >>> +\t\t(\n> >>> +\t\t\tcd git-clone &&\n> >>> +\t\t\t$command\n> >>> +\t\t)\n> >>> +\t\"\n> >>> +}\n> >>> +\n> >>> +test_compare_perf git status\n> >>> +test_compare_perf test_commit --append --no-tag A\n> >>\n> >> Given the small numbers presented in the commit message, I suspect that\n> >> even so much as running the command in a subshell might skew the timings\n> >> at least on Windows, where subshells are very, very expensive.\n> >>\n> >> Given that both `git` and `test_commit` understand the `-C `\n> >> syntax, this variant would resolve my concern:\n> >>\n> >> \ttest_compare_perf () {\n> >> \t command=$1\n> >> shift\n> >> \t args=\"$*\"\n> >>\n> >> \t test_perf \"$command (scalar)\" \"\n> >> \t $command -C scalar-clone/src $args\n> >> \t \"\n> >>\n> >> \t test_perf \"$command (non-scalar)\" \"\n> >> \t $command -C git-clone $args\n> >> \t \"\n> >> \t}\n> >>\n> >> What do you think?\n> >\n> > Makes sense to me! Although, out of curiosity, is there a reason you prefer\n> > \"$1 -> shift -> $*\" over '$1' and '$@'?\n>\n> Whoops, I completely misread your snippet; the 'shift' is necessary to\n> separate the '$command' out so that we can inject '-C'.\n\nYes, and I also changed the \"$@\" (which would usually expand to a\nparameter list, except when it is used inside a string, in which case it\nbehaves like $* for convenience) because the \"$*\" conveys more correctly\nwhat we do here.\n\nWhenever I read \"$@\" anywhere, my mind puts a mental check mark behind the\n\"is this code safe with regards to spaces in arguments?\" question.\n\nHowever, this function would mishandle arguments that contain spaces, and\nreading \"$*\" makes me aware of that, so that I can avoid passing such\narguments.\n\nSo for me, using $* here is the right thing to do: It makes it less likely\nthat someone like me adds code in the future that assumes that even\narguments with spaces in them would be handled.\n\nThanks,\nDscho\n"},{"id":"462520","messageId":"p78rqo9o-o1p2-qq6q-6oo4-94828so928nq@tzk.qr","threadId":"58383","inReplyTo":"72f6661d-be02-dbb8-7110-b8de6bac77f6@github.com","subject":"Re: [PATCH 7/8] t/perf: add 'GIT_PERF_USE_SCALAR' run option","fromName":"Johannes Schindelin","fromEmail":"johannes.schindelin@gmx.de","sentAt":"2022-09-02T09:17:17Z","receivedAt":"2022-09-02T09:18:49Z","isPatch":true,"sender":{"key":"johannes.schindelin@gmx.de","avatar":"https://avatars.githubusercontent.com/u/127790?v=4"},"body":"Hi Victoria,\n\nOn Thu, 1 Sep 2022, Victoria Dye wrote:\n\n> Johannes Schindelin wrote:\n> >\n> > On Wed, 31 Aug 2022, Victoria Dye via GitGitGadget wrote:\n> >\n> >> From: Victoria Dye \n> >>\n> >> Add a 'GIT_PERF_USE_SCALAR' environment variable (and corresponding perf\n> >> config 'useScalar') to register a repository created with any of:\n> >>\n> >> * test_perf_fresh_repo\n> >> * test_perf_default_repo\n> >> * test_perf_large_repo\n> >>\n> >> as a Scalar enlistment. This is intended to allow a developer to test the\n> >> impact of Scalar on already-defined performance scenarios.\n> >\n> > Great idea!\n> >\n> >> [...]\n> >> @@ -130,7 +137,11 @@ test_perf_fresh_repo () {\n> >> \t\"$MODERN_GIT\" init -q \"$repo\" &&\n> >> \t(\n> >> \t\tcd \"$repo\" &&\n> >> -\t\ttest_perf_do_repo_symlink_config_\n> >> +\t\ttest_perf_do_repo_symlink_config_ &&\n> >> +\t\tif test_bool_env \"$GIT_PERF_USE_SCALAR\" false\n> >> +\t\tthen\n> >> +\t\t\t\"$MODERN_SCALAR\" register\n> >\n> > Do we need to unregister anything here? My guess is that no, the \"global\"\n> > config we're using in tests is \"$TRASH_DIRECTORY/.gitconfig\", and the side\n> > effect of scheduling the maintenance task won't matter in practice. But I\n> > might have missed something and we may want to have an explicit\n> > `unregister` step.\n> >\n> > What's your take on this?\n>\n> As you guessed, a '.gitconfig' is created in the trash directory of each\n> test containing the Scalar registration and I haven't seen any issues\n> arising from the scheduled maintenance, so I don't think an 'unregister' is\n> necessary.\n\nThank you for checking!\n\n> However, while verifying that, I noticed that the registration wasn't\n> happening *at all* because 'test_bool_env' is currently being used\n> incorrectly. The fix is straightforward - I'll make sure to correct it\n> in the next version.\n\nOh, great, then my feedback was at least _somewhat_ helpful... ;-)\n\nCiao,\nDscho\n"},{"id":"462528","messageId":"62q164ro-pp89-6248-3s1q-66nq2s4s1491@tzk.qr","threadId":"58383","inReplyTo":"cover-0.5-00000000000-20220901T130817Z-avarab@gmail.com","subject":"Re: [PATCH 0/5] Makefile: split up $(test_bindir_programs)","fromName":"Johannes Schindelin","fromEmail":"johannes.schindelin@gmx.de","sentAt":"2022-09-02T12:38:12Z","receivedAt":"2022-09-02T14:34:22Z","isPatch":true,"sender":{"key":"johannes.schindelin@gmx.de","avatar":"https://avatars.githubusercontent.com/u/127790?v=4"},"body":"Hi Ævar,\n\nOn Thu, 1 Sep 2022, Ævar Arnfjörð Bjarmason wrote:\n\n> On Thu, Sep 01 2022, Johannes Schindelin wrote:\n>\n> > [...]\n> > On Wed, 31 Aug 2022, Victoria Dye via GitGitGadget wrote:\n> > [...]\n> >> @@ -3062,7 +3067,7 @@ bin-wrappers/%: wrap-for-bin.sh\n> >> \t$(call mkdir_p_parent_template)\n> >> \t$(QUIET_GEN)sed -e '1s|#!.*/sh|#!$(SHELL_PATH_SQ)|' \\\n> >> \t -e 's|@@BUILD_DIR@@|$(shell pwd)|' \\\n> >> -\t -e 's|@@PROG@@|$(patsubst test-%,t/helper/test-%$(X),$(@F))$(patsubst git%,$(X),$(filter $(@F),$(BINDIR_PROGRAMS_NEED_X)))|' < $< > $@ && \\\n> >> +\t -e 's|@@PROG@@|$(patsubst test-%,t/helper/test-%,$(@F))$(if $(filter-out $(BINDIR_PROGRAMS_NO_X),$(@F)),$(X),)|' < $< > $@ && \\\n> >\n> > It took me a good while to wrap my head around this (and let me be clear:\n> > I consider none of this your fault, it's the fault of the design of the\n> > Makefile syntax).\n> >\n> > Let me untangle this, for posterity's benefit. We substitute the\n> > placeholder `@@PROG@@` with a concatenation of two strings that are both\n> > derived from `@F`, i.e. the basename of the to-be-wrapped command.\n>\n> We could do this later, [... something about touching the same code ...]\n\nYes, and we should.\n\nIt would be better if we kept the focus on Scalar in this mail thread, to\nget it done first.\n\nIt is the stated preference in this project anyway to avoid working on the\ncode that is already being worked on in other in-flight patch series, to\nreduce unnecessary friction.\n\nCiao,\nJohannes\n"},{"id":"462529","messageId":"55aba052c342c7c5e1f8d3b7908f3662a1dcd0dc.1662134210.git.gitgitgadget@gmail.com","threadId":"58383","inReplyTo":"pull.1341.v2.git.1662134210.gitgitgadget@gmail.com","subject":"[PATCH v2 4/9] scalar: implement the `help` subcommand","fromName":"Johannes Schindelin via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2022-09-02T15:56:45Z","receivedAt":"2022-09-02T16:05:05Z","isPatch":true,"sender":{"key":"johannes.schindelin@gmx.de","avatar":"https://avatars.githubusercontent.com/u/127790?v=4"},"body":"From: Johannes Schindelin \n\nIt is merely handing off to `git help scalar`.\n\nSigned-off-by: Johannes Schindelin \nSigned-off-by: Victoria Dye \n---\n scalar.c | 20 ++++++++++++++++++++\n 1 file changed, 20 insertions(+)\n\ndiff --git a/scalar.c b/scalar.c\nindex 642d16124eb..c5c1ce68919 100644\n--- a/scalar.c\n+++ b/scalar.c\n@@ -819,6 +819,25 @@ static int cmd_delete(int argc, const char **argv)\n \treturn res;\n }\n \n+static int cmd_help(int argc, const char **argv)\n+{\n+\tstruct option options[] = {\n+\t\tOPT_END(),\n+\t};\n+\tconst char * const usage[] = {\n+\t\t\"scalar help\",\n+\t\tNULL\n+\t};\n+\n+\targc = parse_options(argc, argv, NULL, options,\n+\t\t\t usage, 0);\n+\n+\tif (argc != 0)\n+\t\tusage_with_options(usage, options);\n+\n+\treturn run_git(\"help\", \"scalar\", NULL);\n+}\n+\n static int cmd_version(int argc, const char **argv)\n {\n \tint verbose = 0, build_options = 0;\n@@ -858,6 +877,7 @@ static struct {\n \t{ \"run\", cmd_run },\n \t{ \"reconfigure\", cmd_reconfigure },\n \t{ \"delete\", cmd_delete },\n+\t{ \"help\", cmd_help },\n \t{ \"version\", cmd_version },\n \t{ \"diagnose\", cmd_diagnose },\n \t{ NULL, NULL},\n-- \ngitgitgadget\n\n"},{"id":"462530","messageId":"eb8663e0300784cc211e02b2590d38d4c4ac610b.1662134210.git.gitgitgadget@gmail.com","threadId":"58383","inReplyTo":"pull.1341.v2.git.1662134210.gitgitgadget@gmail.com","subject":"[PATCH v2 3/9] git help: special-case `scalar`","fromName":"Johannes Schindelin via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2022-09-02T15:56:44Z","receivedAt":"2022-09-02T16:05:09Z","isPatch":true,"sender":{"key":"johannes.schindelin@gmx.de","avatar":"https://avatars.githubusercontent.com/u/127790?v=4"},"body":"From: Johannes Schindelin \n\nWith this commit, `git help scalar` will open the appropriate manual\nor HTML page (instead of looking for `gitscalar`).\n\nSigned-off-by: Johannes Schindelin \nSigned-off-by: Victoria Dye \n---\n builtin/help.c | 2 ++\n 1 file changed, 2 insertions(+)\n\ndiff --git a/builtin/help.c b/builtin/help.c\nindex 09ac4289f13..6f2796f211e 100644\n--- a/builtin/help.c\n+++ b/builtin/help.c\n@@ -440,6 +440,8 @@ static const char *cmd_to_page(const char *git_cmd)\n \t\treturn git_cmd;\n \telse if (is_git_command(git_cmd))\n \t\treturn xstrfmt(\"git-%s\", git_cmd);\n+\telse if (!strcmp(\"scalar\", git_cmd))\n+\t\treturn xstrdup(git_cmd);\n \telse\n \t\treturn xstrfmt(\"git%s\", git_cmd);\n }\n-- \ngitgitgadget\n\n"},{"id":"462531","messageId":"614cbdc629ddb099f4a30ad6d217d1f1c716da83.1662134210.git.gitgitgadget@gmail.com","threadId":"58383","inReplyTo":"pull.1341.v2.git.1662134210.gitgitgadget@gmail.com","subject":"[PATCH v2 1/9] scalar: fix command documentation section header","fromName":"Victoria Dye via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2022-09-02T15:56:42Z","receivedAt":"2022-09-02T16:05:11Z","isPatch":true,"sender":{"key":"vdye@github.com","avatar":"https://avatars.githubusercontent.com/u/3619353?v=4"},"body":"From: Victoria Dye \n\nRename the last section header in 'contrib/scalar/scalar.txt' from \"Scalar\"\nto \"GIT\". The linting rules of the 'documentation' CI build enforce the\nexistence of a \"GIT\" section in command documentation. Although 'scalar.txt'\nis not yet checked, it will be in a future patch.\n\nHere, changing the header name is more appropriate than making a\nScalar-specific exception to the linting rule. The existing \"Scalar\" section\ncontains only a link back to the main Git documentation, essentially the\nsame as the \"GIT\" section in builtin documentation. Changing the section\nname further clarifies the Scalar-Git association and maintains consistency\nwith the rest of Git.\n\nSigned-off-by: Victoria Dye \n---\n contrib/scalar/scalar.txt | 4 ++--\n 1 file changed, 2 insertions(+), 2 deletions(-)\n\ndiff --git a/contrib/scalar/scalar.txt b/contrib/scalar/scalar.txt\nindex 1a12dc45077..f33436c7f65 100644\n--- a/contrib/scalar/scalar.txt\n+++ b/contrib/scalar/scalar.txt\n@@ -161,6 +161,6 @@ SEE ALSO\n --------\n linkgit:git-clone[1], linkgit:git-maintenance[1].\n \n-Scalar\n+GIT\n ---\n-Associated with the linkgit:git[1] suite\n+Part of the linkgit:git[1] suite\n-- \ngitgitgadget\n\n"},{"id":"462532","messageId":"pull.1341.v2.git.1662134210.gitgitgadget@gmail.com","threadId":"58383","inReplyTo":"pull.1341.git.1661961746.gitgitgadget@gmail.com","subject":"[PATCH v2 0/9] scalar: integrate into core Git","fromName":"Victoria Dye via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2022-09-02T15:56:41Z","receivedAt":"2022-09-02T16:05:35Z","isPatch":true,"sender":{"key":"vdye@github.com","avatar":"https://avatars.githubusercontent.com/u/3619353?v=4"},"body":"This series completes the initial implementation of the Scalar command as a\ncore component of Git. For readers new to the topic of Scalar, the\nroadmap/design doc [1] provides some background information including how\nthe project started & evolved, as well as its current intent & high-level\ndesign.\n\n\nChanges since V1\n================\n\n * Updated 'scalar.txt' description from \"Associated with [git]\" -> \"Part of\n [git]\".\n * Fixed Dscho's \"Signed-off-by\" and author emails on patches 3 & 4.\n * Added 'scalar' to 'git help -a' command list.\n * Removed translation tag for literal 'scalar help' usage string.\n * Updated 'scalar clone' tests to clone to unique targets and clean up only\n when test passes.\n * Changed 'p9210' perf tests to invoke 'git' and 'scalar' with '-C', rather\n than creating a subshell to run commands.\n * Fixed use of 'test_env_bool' with 'GIT_PERF_USE_SCALAR', locally tested\n to make sure it was working as-intended.\n\n\nGoals\n=====\n\nThe goal of this series is, as noted in the roadmap linked earlier, to\nintegrate Scalar as a core user-facing component of Git. There's no\nindividual existing precedent that the integration is 100% based on; it's\nmore of an amalgam of the approaches used with builtins installed directly\nto the bindir (e.g. 'git-upload-pack'), contributed tools (e.g. 'gitk'), and\nthe 'git' executable itself. The resulting behavior is:\n\n * Scalar is built & installed unconditionally (like a builtin).\n * Scalar is included in 'bin-wrappers/' and is installed into the 'bin/'\n directory (like 'git-upload-pack').\n * Scalar's documentation is installed as 'man1' docs (like 'gitk').\n * Scalar is not invoked via the 'git' executable (it's called directly,\n like 'git' itself).\n\nThe move out of 'contrib/' is also intended to mean that the relationship\nbetween Git and Scalar should no longer be a strict one-way dependency. In\nthis series, this architectural change is what permits making a change to\n'help.c' specifically for the sake of Scalar. Future contributions can and\nshould (where appropriate!) share code between Scalar and non-Scalar parts\nof Git.\n\n\nOrganization\n============\n\nThe series is broken up as follows:\n\n 1. Fix a CI-breaking documentation error in 'scalar.txt' (patch 1).\n 2. Move Scalar code out of 'contrib/' and into core Git, including changes\n in build definition(s) to build & install Scalar and its resources\n (patch 2).\n 3. Make 'git help scalar' open the 'scalar' command documentation,\n introduce 'scalar help' as an alias. This is done after moving out of\n 'contrib/' due to it involving Scalar-related changes to the non-Scalar\n file 'help.c' (patches 3-4).\n 4. Expand test coverage, including perf testing (patches 5-6)\n 5. Add a perf test configuration allowing users to register standard perf\n test repos with Scalar (patch 7).\n 6. Remove the internally-focused Scalar roadmap from\n 'Documentation/technical/scalar.txt', include it as one of the HTML\n technical docs (patch 8).\n\n\nPrior series\n============\n\n * Add 'scalar' command in 'contrib/':\n https://lore.kernel.org/git/pull.1005.v10.git.1638538470.gitgitgadget@gmail.com/\n * Introduce 'scalar diagnose':\n https://lore.kernel.org/git/pull.1128.v6.git.1653145696.gitgitgadget@gmail.com/\n * Add '-c/-C' compatibility:\n https://lore.kernel.org/git/pull.1130.v2.git.1643380317358.gitgitgadget@gmail.com/\n * [DROPPED] Integrate Scalar into CI builds:\n https://lore.kernel.org/git/pull.1129.git.1654160735.gitgitgadget@gmail.com/\n * Document Scalar's role in Git & plan remaining work:\n https://lore.kernel.org/git/pull.1275.v2.git.1657584367.gitgitgadget@gmail.com/\n * Generalize 'scalar diagnose' into 'git diagnose' builtin & 'git bugreport\n --diagnose':\n https://lore.kernel.org/git/pull.1310.v4.git.1660335019.gitgitgadget@gmail.com/\n * Add FSMonitor support to Scalar, refactor enlistment search:\n https://lore.kernel.org/git/pull.1324.v3.git.1660858853.gitgitgadget@gmail.com/\n\nThanks!\n\n * Victoria\n\n[1]\nhttps://lore.kernel.org/git/65eda9755d2a29e222bf7e6cc719921b24377cc6.1657584367.git.gitgitgadget@gmail.com/\n\nJohannes Schindelin (2):\n git help: special-case `scalar`\n scalar: implement the `help` subcommand\n\nVictoria Dye (7):\n scalar: fix command documentation section header\n scalar: include in standard Git build & installation\n scalar: add to 'git help -a' command list\n scalar-clone: add test coverage\n t/perf: add Scalar performance tests\n t/perf: add 'GIT_PERF_USE_SCALAR' run option\n Documentation/technical: include Scalar technical doc\n\n .gitignore | 1 +\n Documentation/Makefile | 2 +\n Documentation/cmd-list.perl | 2 +-\n {contrib/scalar => Documentation}/scalar.txt | 4 +-\n Documentation/technical/scalar.txt | 61 -------\n Makefile | 31 ++--\n builtin/help.c | 2 +\n command-list.txt | 1 +\n contrib/buildsystems/CMakeLists.txt | 9 +-\n contrib/scalar/.gitignore | 2 -\n contrib/scalar/Makefile | 35 ----\n contrib/scalar/t/Makefile | 81 ----------\n contrib/scalar/scalar.c => scalar.c | 20 +++\n t/perf/README | 4 +\n t/perf/p9210-scalar.sh | 39 +++++\n t/perf/perf-lib.sh | 13 +-\n t/perf/run | 3 +\n .../t/t9099-scalar.sh => t/t9210-scalar.sh | 10 +-\n t/t9211-scalar-clone.sh | 151 ++++++++++++++++++\n 19 files changed, 264 insertions(+), 207 deletions(-)\n rename {contrib/scalar => Documentation}/scalar.txt (99%)\n delete mode 100644 contrib/scalar/.gitignore\n delete mode 100644 contrib/scalar/Makefile\n delete mode 100644 contrib/scalar/t/Makefile\n rename contrib/scalar/scalar.c => scalar.c (98%)\n create mode 100755 t/perf/p9210-scalar.sh\n rename contrib/scalar/t/t9099-scalar.sh => t/t9210-scalar.sh (96%)\n create mode 100755 t/t9211-scalar-clone.sh\n\n\nbase-commit: d42b38dfb5edf1a7fddd9542d722f91038407819\nPublished-As: https://github.com/gitgitgadget/git/releases/tag/pr-1341%2Fvdye%2Ffeature%2Fscalar-toplevel-v2\nFetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-1341/vdye/feature/scalar-toplevel-v2\nPull-Request: https://github.com/gitgitgadget/git/pull/1341\n\nRange-diff vs v1:\n\n 1: fcf73e77574 ! 1: 614cbdc629d scalar: fix command documentation section header\n @@ contrib/scalar/scalar.txt: SEE ALSO\n -Scalar\n +GIT\n ---\n - Associated with the linkgit:git[1] suite\n +-Associated with the linkgit:git[1] suite\n ++Part of the linkgit:git[1] suite\n 2: 4d69e5eaccb ! 2: bc2092a7a7a scalar: include in standard Git build & installation\n @@ Makefile: XGETTEXT_FLAGS_SH = $(XGETTEXT_FLAGS) --language=Shell \\\n \t--keyword=__ --keyword=N__ --keyword=\"__n:1,2\"\n MSGMERGE_FLAGS = --add-location --backup=off --update\n -LOCALIZED_C = $(sort $(FOUND_C_SOURCES) $(FOUND_H_SOURCES) $(SCALAR_SOURCES) \\\n -+LOCALIZED_C = $(sort $(FOUND_C_SOURCES) $(FOUND_H_SOURCES) \\\n - \t $(GENERATED_H))\n +-\t $(GENERATED_H))\n ++LOCALIZED_C = $(sort $(FOUND_C_SOURCES) $(FOUND_H_SOURCES) $(GENERATED_H))\n LOCALIZED_SH = $(sort $(SCRIPT_SH) git-sh-setup.sh)\n LOCALIZED_PERL = $(sort $(SCRIPT_PERL))\n + \n @@ Makefile: bin-wrappers/%: wrap-for-bin.sh\n \t$(call mkdir_p_parent_template)\n \t$(QUIET_GEN)sed -e '1s|#!.*/sh|#!$(SHELL_PATH_SQ)|' \\\n 3: 969160650aa = 3: eb8663e0300 git help: special-case `scalar`\n 4: 46d0fddfe8f ! 4: 55aba052c34 scalar: implement the `help` subcommand\n @@\n ## Metadata ##\n -Author: Johannes Schindelin \n +Author: Johannes Schindelin \n \n ## Commit message ##\n scalar: implement the `help` subcommand\n \n It is merely handing off to `git help scalar`.\n \n - Signed-off-by: Johannes Schindelin \n + Signed-off-by: Johannes Schindelin \n Signed-off-by: Victoria Dye \n \n ## scalar.c ##\n @@ scalar.c: static int cmd_delete(int argc, const char **argv)\n +\t\tOPT_END(),\n +\t};\n +\tconst char * const usage[] = {\n -+\t\tN_(\"scalar help\"),\n ++\t\t\"scalar help\",\n +\t\tNULL\n +\t};\n +\n -: ----------- > 5: 070f195f027 scalar: add to 'git help -a' command list\n 5: a3b7cb0a3bd ! 6: 15acf20274b scalar-clone: add test coverage\n @@ Commit message\n scalar-clone: add test coverage\n \n Create a new test file ('t9211-scalar-clone.sh') to exercise the options and\n - behavior of the 'scalar clone' command.\n + behavior of the 'scalar clone' command. Each test clones to a unique target\n + location and cleans up the cloned repo only when the test passes. This\n + ensures that failed tests' artifacts are captured in CI artifacts for\n + further debugging.\n \n + Helped-by: Johannes Schindelin \n Signed-off-by: Victoria Dye \n \n ## t/t9211-scalar-clone.sh (new) ##\n @@ t/t9211-scalar-clone.sh (new)\n +}\n +\n +test_expect_success 'creates content in enlistment root' '\n -+\ttest_when_finished cleanup_clone cloned &&\n ++\tenlistment=cloned &&\n +\n -+\tscalar clone \"file://$(pwd)/to-clone\" cloned &&\n -+\tls -A cloned >enlistment-root &&\n ++\tscalar clone \"file://$(pwd)/to-clone\" $enlistment &&\n ++\tls -A $enlistment >enlistment-root &&\n +\ttest_line_count = 1 enlistment-root &&\n -+\ttest_path_is_dir cloned/src &&\n -+\ttest_path_is_dir cloned/src/.git\n ++\ttest_path_is_dir $enlistment/src &&\n ++\ttest_path_is_dir $enlistment/src/.git &&\n ++\n ++\tcleanup_clone $enlistment\n +'\n +\n +test_expect_success 'with spaces' '\n -+\ttest_when_finished cleanup_clone \"cloned with space\" &&\n ++\tenlistment=\"cloned with space\" &&\n ++\n ++\tscalar clone \"file://$(pwd)/to-clone\" \"$enlistment\" &&\n ++\ttest_path_is_dir \"$enlistment\" &&\n ++\ttest_path_is_dir \"$enlistment/src\" &&\n ++\ttest_path_is_dir \"$enlistment/src/.git\" &&\n +\n -+\tscalar clone \"file://$(pwd)/to-clone\" \"cloned with space\" &&\n -+\ttest_path_is_dir \"cloned with space\" &&\n -+\ttest_path_is_dir \"cloned with space/src\" &&\n -+\ttest_path_is_dir \"cloned with space/src/.git\"\n ++\tcleanup_clone \"$enlistment\"\n +'\n +\n +test_expect_success 'partial clone if supported by server' '\n -+\ttest_when_finished cleanup_clone cloned &&\n ++\tenlistment=partial-clone &&\n +\n -+\tscalar clone \"file://$(pwd)/to-clone\" cloned &&\n ++\tscalar clone \"file://$(pwd)/to-clone\" $enlistment &&\n +\n +\t(\n -+\t\tcd cloned/src &&\n ++\t\tcd $enlistment/src &&\n +\n +\t\t# Two promisor packs: one for refs, the other for blobs\n +\t\tls .git/objects/pack/pack-*.promisor >promisorlist &&\n +\t\ttest_line_count = 2 promisorlist\n -+\t)\n ++\t) &&\n ++\n ++\tcleanup_clone $enlistment\n +'\n +\n +test_expect_success 'fall back on full clone if partial unsupported' '\n -+\ttest_when_finished cleanup_clone cloned &&\n ++\tenlistment=no-partial-support &&\n +\n +\ttest_config -C to-clone uploadpack.allowfilter false &&\n +\ttest_config -C to-clone uploadpack.allowanysha1inwant false &&\n +\n -+\tscalar clone \"file://$(pwd)/to-clone\" cloned 2>err &&\n ++\tscalar clone \"file://$(pwd)/to-clone\" $enlistment 2>err &&\n +\tgrep \"filtering not recognized by server, ignoring\" err &&\n +\n +\t(\n -+\t\tcd cloned/src &&\n ++\t\tcd $enlistment/src &&\n +\n +\t\t# Still get a refs promisor file, but none for blobs\n +\t\tls .git/objects/pack/pack-*.promisor >promisorlist &&\n +\t\ttest_line_count = 1 promisorlist\n -+\t)\n ++\t) &&\n ++\n ++\tcleanup_clone $enlistment\n +'\n +\n +test_expect_success 'initializes sparse-checkout by default' '\n -+\ttest_when_finished cleanup_clone cloned &&\n ++\tenlistment=sparse &&\n +\n -+\tscalar clone \"file://$(pwd)/to-clone\" cloned &&\n ++\tscalar clone \"file://$(pwd)/to-clone\" $enlistment &&\n +\t(\n -+\t\tcd cloned/src &&\n ++\t\tcd $enlistment/src &&\n +\t\ttest_cmp_config true core.sparseCheckout &&\n +\t\ttest_cmp_config true core.sparseCheckoutCone\n -+\t)\n ++\t) &&\n ++\n ++\tcleanup_clone $enlistment\n +'\n +\n +test_expect_success '--full-clone does not create sparse-checkout' '\n -+\ttest_when_finished cleanup_clone cloned &&\n ++\tenlistment=full-clone &&\n +\n -+\tscalar clone --full-clone \"file://$(pwd)/to-clone\" cloned &&\n ++\tscalar clone --full-clone \"file://$(pwd)/to-clone\" $enlistment &&\n +\t(\n -+\t\tcd cloned/src &&\n ++\t\tcd $enlistment/src &&\n +\t\ttest_cmp_config \"\" --default \"\" core.sparseCheckout &&\n +\t\ttest_cmp_config \"\" --default \"\" core.sparseCheckoutCone\n -+\t)\n ++\t) &&\n ++\n ++\tcleanup_clone $enlistment\n +'\n +\n +test_expect_success '--single-branch clones HEAD only' '\n -+\ttest_when_finished cleanup_clone cloned &&\n ++\tenlistment=single-branch &&\n +\n -+\tscalar clone --single-branch \"file://$(pwd)/to-clone\" cloned &&\n ++\tscalar clone --single-branch \"file://$(pwd)/to-clone\" $enlistment &&\n +\t(\n -+\t\tcd cloned/src &&\n ++\t\tcd $enlistment/src &&\n +\t\tgit for-each-ref refs/remotes/origin >out &&\n +\t\ttest_line_count = 1 out &&\n +\t\tgrep \"refs/remotes/origin/base\" out\n -+\t)\n ++\t) &&\n ++\n ++\tcleanup_clone $enlistment\n +'\n +\n +test_expect_success '--no-single-branch clones all branches' '\n -+\ttest_when_finished cleanup_clone cloned &&\n ++\tenlistment=no-single-branch &&\n +\n -+\tscalar clone --no-single-branch \"file://$(pwd)/to-clone\" cloned &&\n ++\tscalar clone --no-single-branch \"file://$(pwd)/to-clone\" $enlistment &&\n +\t(\n -+\t\tcd cloned/src &&\n ++\t\tcd $enlistment/src &&\n +\t\tgit for-each-ref refs/remotes/origin >out &&\n +\t\ttest_line_count = 2 out &&\n +\t\tgrep \"refs/remotes/origin/base\" out &&\n +\t\tgrep \"refs/remotes/origin/parallel\" out\n -+\t)\n ++\t) &&\n ++\n ++\tcleanup_clone $enlistment\n +'\n +\n +test_done\n 6: 42ab39f2121 ! 7: d5b68c65163 t/perf: add Scalar performance tests\n @@ Commit message\n 9210.6: test_commit --append --no-tag A (scalar) 0.08(0.02+0.03)\n 9210.7: test_commit --append --no-tag A (non-scalar) 0.13(0.03+0.11)\n \n + Helped-by: Johannes Schindelin \n Signed-off-by: Victoria Dye \n \n ## t/perf/p9210-scalar.sh (new) ##\n @@ t/perf/p9210-scalar.sh (new)\n +'\n +\n +test_compare_perf () {\n -+\tcommand=\"$@\"\n -+\ttest_perf \"$command (scalar)\" \"\n -+\t\t(\n -+\t\t\tcd scalar-clone/src &&\n -+\t\t\t$command\n -+\t\t)\n ++\tcommand=$1\n ++\tshift\n ++\targs=$*\n ++\ttest_perf \"$command $args (scalar)\" \"\n ++\t\t$command -C scalar-clone/src $args\n +\t\"\n +\n -+\ttest_perf \"$command (non-scalar)\" \"\n -+\t\t(\n -+\t\t\tcd git-clone &&\n -+\t\t\t$command\n -+\t\t)\n ++\ttest_perf \"$command $args (non-scalar)\" \"\n ++\t\t$command -C git-clone $args\n +\t\"\n +}\n +\n 7: 96e08a95265 ! 8: 2e6dd03beba t/perf: add 'GIT_PERF_USE_SCALAR' run option\n @@ t/perf/perf-lib.sh: test_perf_create_repo_from () {\n \t\t\t# a copy it's fine to remove the lock.\n \t\t\trm .git/index.lock\n +\t\tfi &&\n -+\t\tif test_bool_env \"$GIT_PERF_USE_SCALAR\" false\n ++\t\tif test_bool_env GIT_PERF_USE_SCALAR false\n +\t\tthen\n +\t\t\t\"$MODERN_SCALAR\" register\n \t\tfi\n @@ t/perf/perf-lib.sh: test_perf_fresh_repo () {\n \t\tcd \"$repo\" &&\n -\t\ttest_perf_do_repo_symlink_config_\n +\t\ttest_perf_do_repo_symlink_config_ &&\n -+\t\tif test_bool_env \"$GIT_PERF_USE_SCALAR\" false\n ++\t\tif test_bool_env GIT_PERF_USE_SCALAR false\n +\t\tthen\n +\t\t\t\"$MODERN_SCALAR\" register\n +\t\tfi\n 8: 8bbe913dccf = 9: 6bedd80c398 Documentation/technical: include Scalar technical doc\n\n-- \ngitgitgadget\n"},{"id":"462533","messageId":"d5b68c65163a384d86f08d371d4b1abc774b66db.1662134210.git.gitgitgadget@gmail.com","threadId":"58383","inReplyTo":"pull.1341.v2.git.1662134210.gitgitgadget@gmail.com","subject":"[PATCH v2 7/9] t/perf: add Scalar performance tests","fromName":"Victoria Dye via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2022-09-02T15:56:48Z","receivedAt":"2022-09-02T16:05:41Z","isPatch":true,"sender":{"key":"vdye@github.com","avatar":"https://avatars.githubusercontent.com/u/3619353?v=4"},"body":"From: Victoria Dye \n\nCreate 'p9210-scalar.sh' for testing Scalar performance and comparing\nperformance of Git operations in Scalar registrations and standard\nrepositories. Example results:\n\nTest this tree\n------------------------------------------------------------------------\n9210.2: scalar clone 14.82(18.00+3.63)\n9210.3: git clone 26.15(36.67+6.90)\n9210.4: git status (scalar) 0.04(0.01+0.01)\n9210.5: git status (non-scalar) 0.10(0.02+0.11)\n9210.6: test_commit --append --no-tag A (scalar) 0.08(0.02+0.03)\n9210.7: test_commit --append --no-tag A (non-scalar) 0.13(0.03+0.11)\n\nHelped-by: Johannes Schindelin \nSigned-off-by: Victoria Dye \n---\n t/perf/p9210-scalar.sh | 39 +++++++++++++++++++++++++++++++++++++++\n 1 file changed, 39 insertions(+)\n create mode 100755 t/perf/p9210-scalar.sh\n\ndiff --git a/t/perf/p9210-scalar.sh b/t/perf/p9210-scalar.sh\nnew file mode 100755\nindex 00000000000..265f7cd1fe2\n--- /dev/null\n+++ b/t/perf/p9210-scalar.sh\n@@ -0,0 +1,39 @@\n+#!/bin/sh\n+\n+test_description='test scalar performance'\n+. ./perf-lib.sh\n+\n+test_perf_large_repo \"$TRASH_DIRECTORY/to-clone\"\n+\n+test_expect_success 'enable server-side partial clone' '\n+\tgit -C to-clone config uploadpack.allowFilter true &&\n+\tgit -C to-clone config uploadpack.allowAnySHA1InWant true\n+'\n+\n+test_perf 'scalar clone' '\n+\trm -rf scalar-clone &&\n+\tscalar clone \"file://$(pwd)/to-clone\" scalar-clone\n+'\n+\n+test_perf 'git clone' '\n+\trm -rf git-clone &&\n+\tgit clone \"file://$(pwd)/to-clone\" git-clone\n+'\n+\n+test_compare_perf () {\n+\tcommand=$1\n+\tshift\n+\targs=$*\n+\ttest_perf \"$command $args (scalar)\" \"\n+\t\t$command -C scalar-clone/src $args\n+\t\"\n+\n+\ttest_perf \"$command $args (non-scalar)\" \"\n+\t\t$command -C git-clone $args\n+\t\"\n+}\n+\n+test_compare_perf git status\n+test_compare_perf test_commit --append --no-tag A\n+\n+test_done\n-- \ngitgitgadget\n\n"},{"id":"462534","messageId":"15acf20274bc78231cbbcb456b303d24fc2ba425.1662134210.git.gitgitgadget@gmail.com","threadId":"58383","inReplyTo":"pull.1341.v2.git.1662134210.gitgitgadget@gmail.com","subject":"[PATCH v2 6/9] scalar-clone: add test coverage","fromName":"Victoria Dye via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2022-09-02T15:56:47Z","receivedAt":"2022-09-02T16:06:18Z","isPatch":true,"sender":{"key":"vdye@github.com","avatar":"https://avatars.githubusercontent.com/u/3619353?v=4"},"body":"From: Victoria Dye \n\nCreate a new test file ('t9211-scalar-clone.sh') to exercise the options and\nbehavior of the 'scalar clone' command. Each test clones to a unique target\nlocation and cleans up the cloned repo only when the test passes. This\nensures that failed tests' artifacts are captured in CI artifacts for\nfurther debugging.\n\nHelped-by: Johannes Schindelin \nSigned-off-by: Victoria Dye \n---\n t/t9211-scalar-clone.sh | 151 ++++++++++++++++++++++++++++++++++++++++\n 1 file changed, 151 insertions(+)\n create mode 100755 t/t9211-scalar-clone.sh\n\ndiff --git a/t/t9211-scalar-clone.sh b/t/t9211-scalar-clone.sh\nnew file mode 100755\nindex 00000000000..dd33d87e9be\n--- /dev/null\n+++ b/t/t9211-scalar-clone.sh\n@@ -0,0 +1,151 @@\n+#!/bin/sh\n+\n+test_description='test the `scalar clone` subcommand'\n+\n+. ./test-lib.sh\n+\n+GIT_TEST_MAINT_SCHEDULER=\"crontab:test-tool crontab cron.txt,launchctl:true,schtasks:true\"\n+export GIT_TEST_MAINT_SCHEDULER\n+\n+test_expect_success 'set up repository to clone' '\n+\trm -rf .git &&\n+\tgit init to-clone &&\n+\t(\n+\t\tcd to-clone &&\n+\t\tgit branch -m base &&\n+\n+\t\ttest_commit first &&\n+\t\ttest_commit second &&\n+\t\ttest_commit third &&\n+\n+\t\tgit switch -c parallel first &&\n+\t\tmkdir -p 1/2 &&\n+\t\ttest_commit 1/2/3 &&\n+\n+\t\tgit switch base &&\n+\n+\t\t# By default, permit\n+\t\tgit config uploadpack.allowfilter true &&\n+\t\tgit config uploadpack.allowanysha1inwant true\n+\t)\n+'\n+\n+cleanup_clone () {\n+\trm -rf \"$1\"\n+}\n+\n+test_expect_success 'creates content in enlistment root' '\n+\tenlistment=cloned &&\n+\n+\tscalar clone \"file://$(pwd)/to-clone\" $enlistment &&\n+\tls -A $enlistment >enlistment-root &&\n+\ttest_line_count = 1 enlistment-root &&\n+\ttest_path_is_dir $enlistment/src &&\n+\ttest_path_is_dir $enlistment/src/.git &&\n+\n+\tcleanup_clone $enlistment\n+'\n+\n+test_expect_success 'with spaces' '\n+\tenlistment=\"cloned with space\" &&\n+\n+\tscalar clone \"file://$(pwd)/to-clone\" \"$enlistment\" &&\n+\ttest_path_is_dir \"$enlistment\" &&\n+\ttest_path_is_dir \"$enlistment/src\" &&\n+\ttest_path_is_dir \"$enlistment/src/.git\" &&\n+\n+\tcleanup_clone \"$enlistment\"\n+'\n+\n+test_expect_success 'partial clone if supported by server' '\n+\tenlistment=partial-clone &&\n+\n+\tscalar clone \"file://$(pwd)/to-clone\" $enlistment &&\n+\n+\t(\n+\t\tcd $enlistment/src &&\n+\n+\t\t# Two promisor packs: one for refs, the other for blobs\n+\t\tls .git/objects/pack/pack-*.promisor >promisorlist &&\n+\t\ttest_line_count = 2 promisorlist\n+\t) &&\n+\n+\tcleanup_clone $enlistment\n+'\n+\n+test_expect_success 'fall back on full clone if partial unsupported' '\n+\tenlistment=no-partial-support &&\n+\n+\ttest_config -C to-clone uploadpack.allowfilter false &&\n+\ttest_config -C to-clone uploadpack.allowanysha1inwant false &&\n+\n+\tscalar clone \"file://$(pwd)/to-clone\" $enlistment 2>err &&\n+\tgrep \"filtering not recognized by server, ignoring\" err &&\n+\n+\t(\n+\t\tcd $enlistment/src &&\n+\n+\t\t# Still get a refs promisor file, but none for blobs\n+\t\tls .git/objects/pack/pack-*.promisor >promisorlist &&\n+\t\ttest_line_count = 1 promisorlist\n+\t) &&\n+\n+\tcleanup_clone $enlistment\n+'\n+\n+test_expect_success 'initializes sparse-checkout by default' '\n+\tenlistment=sparse &&\n+\n+\tscalar clone \"file://$(pwd)/to-clone\" $enlistment &&\n+\t(\n+\t\tcd $enlistment/src &&\n+\t\ttest_cmp_config true core.sparseCheckout &&\n+\t\ttest_cmp_config true core.sparseCheckoutCone\n+\t) &&\n+\n+\tcleanup_clone $enlistment\n+'\n+\n+test_expect_success '--full-clone does not create sparse-checkout' '\n+\tenlistment=full-clone &&\n+\n+\tscalar clone --full-clone \"file://$(pwd)/to-clone\" $enlistment &&\n+\t(\n+\t\tcd $enlistment/src &&\n+\t\ttest_cmp_config \"\" --default \"\" core.sparseCheckout &&\n+\t\ttest_cmp_config \"\" --default \"\" core.sparseCheckoutCone\n+\t) &&\n+\n+\tcleanup_clone $enlistment\n+'\n+\n+test_expect_success '--single-branch clones HEAD only' '\n+\tenlistment=single-branch &&\n+\n+\tscalar clone --single-branch \"file://$(pwd)/to-clone\" $enlistment &&\n+\t(\n+\t\tcd $enlistment/src &&\n+\t\tgit for-each-ref refs/remotes/origin >out &&\n+\t\ttest_line_count = 1 out &&\n+\t\tgrep \"refs/remotes/origin/base\" out\n+\t) &&\n+\n+\tcleanup_clone $enlistment\n+'\n+\n+test_expect_success '--no-single-branch clones all branches' '\n+\tenlistment=no-single-branch &&\n+\n+\tscalar clone --no-single-branch \"file://$(pwd)/to-clone\" $enlistment &&\n+\t(\n+\t\tcd $enlistment/src &&\n+\t\tgit for-each-ref refs/remotes/origin >out &&\n+\t\ttest_line_count = 2 out &&\n+\t\tgrep \"refs/remotes/origin/base\" out &&\n+\t\tgrep \"refs/remotes/origin/parallel\" out\n+\t) &&\n+\n+\tcleanup_clone $enlistment\n+'\n+\n+test_done\n-- \ngitgitgadget\n\n"},{"id":"462535","messageId":"bc2092a7a7aae640b547bde8db0c0a26ce31278a.1662134210.git.gitgitgadget@gmail.com","threadId":"58383","inReplyTo":"pull.1341.v2.git.1662134210.gitgitgadget@gmail.com","subject":"[PATCH v2 2/9] scalar: include in standard Git build & installation","fromName":"Victoria Dye via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2022-09-02T15:56:43Z","receivedAt":"2022-09-02T16:09:59Z","isPatch":true,"sender":{"key":"vdye@github.com","avatar":"https://avatars.githubusercontent.com/u/3619353?v=4"},"body":"From: Victoria Dye \n\nMove 'scalar' out of 'contrib/' and into the root of the Git tree. The goal\nof this change is to build 'scalar' as part of the standard Git build &\ninstall processes.\n\nThis patch includes both the physical move of Scalar's files out of\n'contrib/' ('scalar.c', 'scalar.txt', and 't9xxx-scalar.sh'), and the\nchanges to the build definitions in 'Makefile' and 'CMakelists.txt' to\naccommodate the new program.\n\nAt a high level, Scalar is built so that:\n- there is a 'scalar-objs' target (similar to those created in 029bac01a8\n (Makefile: add {program,xdiff,test,git,fuzz}-objs & objects targets,\n 2021-02-23)) for debugging purposes.\n- it appears in the root of the install directory (rather than the\n gitexecdir).\n- it is included in the 'bin-wrappers/' directory for use in tests.\n- it receives a platform-specific executable suffix (e.g., '.exe'), if\n applicable.\n- 'scalar.txt' is installed as 'man1' documentation.\n- the 'clean' target removes the 'scalar' executable.\n\nAdditionally, update the root level '.gitignore' file to ignore the Scalar\nexecutable.\n\nSigned-off-by: Victoria Dye \n---\n .gitignore | 1 +\n Documentation/Makefile | 1 +\n {contrib/scalar => Documentation}/scalar.txt | 0\n Makefile | 31 ++++---\n contrib/buildsystems/CMakeLists.txt | 9 ++-\n contrib/scalar/.gitignore | 2 -\n contrib/scalar/Makefile | 35 --------\n contrib/scalar/t/Makefile | 81 -------------------\n contrib/scalar/scalar.c => scalar.c | 0\n .../t/t9099-scalar.sh => t/t9210-scalar.sh | 10 +--\n 10 files changed, 28 insertions(+), 142 deletions(-)\n rename {contrib/scalar => Documentation}/scalar.txt (100%)\n delete mode 100644 contrib/scalar/.gitignore\n delete mode 100644 contrib/scalar/Makefile\n delete mode 100644 contrib/scalar/t/Makefile\n rename contrib/scalar/scalar.c => scalar.c (100%)\n rename contrib/scalar/t/t9099-scalar.sh => t/t9210-scalar.sh (96%)\n\ndiff --git a/.gitignore b/.gitignore\nindex 80b530bbed2..3d1b880101e 100644\n--- a/.gitignore\n+++ b/.gitignore\n@@ -185,6 +185,7 @@\n /git-whatchanged\n /git-worktree\n /git-write-tree\n+/scalar\n /git-core-*/?*\n /git.res\n /gitweb/GITWEB-BUILD-OPTIONS\ndiff --git a/Documentation/Makefile b/Documentation/Makefile\nindex bd6b6fcb930..16c9e062390 100644\n--- a/Documentation/Makefile\n+++ b/Documentation/Makefile\n@@ -21,6 +21,7 @@ MAN1_TXT += $(filter-out \\\n MAN1_TXT += git.txt\n MAN1_TXT += gitk.txt\n MAN1_TXT += gitweb.txt\n+MAN1_TXT += scalar.txt\n \n # man5 / man7 guides (note: new guides should also be added to command-list.txt)\n MAN5_TXT += gitattributes.txt\ndiff --git a/contrib/scalar/scalar.txt b/Documentation/scalar.txt\nsimilarity index 100%\nrename from contrib/scalar/scalar.txt\nrename to Documentation/scalar.txt\ndiff --git a/Makefile b/Makefile\nindex eac30126e29..e03f32ec1e7 100644\n--- a/Makefile\n+++ b/Makefile\n@@ -608,7 +608,9 @@ FUZZ_OBJS =\n FUZZ_PROGRAMS =\n GIT_OBJS =\n LIB_OBJS =\n+SCALAR_OBJS =\n OBJECTS =\n+OTHER_PROGRAMS =\n PROGRAM_OBJS =\n PROGRAMS =\n EXCLUDED_PROGRAMS =\n@@ -821,10 +823,12 @@ BUILT_INS += git-switch$X\n BUILT_INS += git-whatchanged$X\n \n # what 'all' will build but not install in gitexecdir\n-OTHER_PROGRAMS = git$X\n+OTHER_PROGRAMS += git$X\n+OTHER_PROGRAMS += scalar$X\n \n # what test wrappers are needed and 'install' will install, in bindir\n BINDIR_PROGRAMS_NEED_X += git\n+BINDIR_PROGRAMS_NEED_X += scalar\n BINDIR_PROGRAMS_NEED_X += git-receive-pack\n BINDIR_PROGRAMS_NEED_X += git-shell\n BINDIR_PROGRAMS_NEED_X += git-upload-archive\n@@ -2222,7 +2226,7 @@ profile-fast: profile-clean\n \n all:: $(ALL_COMMANDS_TO_INSTALL) $(SCRIPT_LIB) $(OTHER_PROGRAMS) GIT-BUILD-OPTIONS\n ifneq (,$X)\n-\t$(QUIET_BUILT_IN)$(foreach p,$(patsubst %$X,%,$(filter %$X,$(ALL_COMMANDS_TO_INSTALL) git$X)), test -d '$p' -o '$p' -ef '$p$X' || $(RM) '$p';)\n+\t$(QUIET_BUILT_IN)$(foreach p,$(patsubst %$X,%,$(filter %$X,$(ALL_COMMANDS_TO_INSTALL) $(OTHER_PROGRAMS))), test -d '$p' -o '$p' -ef '$p$X' || $(RM) '$p';)\n endif\n \n all::\n@@ -2545,7 +2549,12 @@ GIT_OBJS += git.o\n .PHONY: git-objs\n git-objs: $(GIT_OBJS)\n \n+SCALAR_OBJS += scalar.o\n+.PHONY: scalar-objs\n+scalar-objs: $(SCALAR_OBJS)\n+\n OBJECTS += $(GIT_OBJS)\n+OBJECTS += $(SCALAR_OBJS)\n OBJECTS += $(PROGRAM_OBJS)\n OBJECTS += $(TEST_OBJS)\n OBJECTS += $(XDIFF_OBJS)\n@@ -2556,10 +2565,6 @@ ifndef NO_CURL\n \tOBJECTS += http.o http-walker.o remote-curl.o\n endif\n \n-SCALAR_SOURCES := contrib/scalar/scalar.c\n-SCALAR_OBJECTS := $(SCALAR_SOURCES:c=o)\n-OBJECTS += $(SCALAR_OBJECTS)\n-\n .PHONY: objects\n objects: $(OBJECTS)\n \n@@ -2691,7 +2696,7 @@ $(REMOTE_CURL_PRIMARY): remote-curl.o http.o http-walker.o GIT-LDFLAGS $(GITLIBS\n \t$(QUIET_LINK)$(CC) $(ALL_CFLAGS) -o $@ $(ALL_LDFLAGS) $(filter %.o,$^) \\\n \t\t$(CURL_LIBCURL) $(EXPAT_LIBEXPAT) $(LIBS)\n \n-contrib/scalar/scalar$X: $(SCALAR_OBJECTS) GIT-LDFLAGS $(GITLIBS)\n+scalar$X: scalar.o GIT-LDFLAGS $(GITLIBS)\n \t$(QUIET_LINK)$(CC) $(ALL_CFLAGS) -o $@ $(ALL_LDFLAGS) \\\n \t\t$(filter %.o,$^) $(LIBS)\n \n@@ -2747,8 +2752,7 @@ XGETTEXT_FLAGS_SH = $(XGETTEXT_FLAGS) --language=Shell \\\n XGETTEXT_FLAGS_PERL = $(XGETTEXT_FLAGS) --language=Perl \\\n \t--keyword=__ --keyword=N__ --keyword=\"__n:1,2\"\n MSGMERGE_FLAGS = --add-location --backup=off --update\n-LOCALIZED_C = $(sort $(FOUND_C_SOURCES) $(FOUND_H_SOURCES) $(SCALAR_SOURCES) \\\n-\t $(GENERATED_H))\n+LOCALIZED_C = $(sort $(FOUND_C_SOURCES) $(FOUND_H_SOURCES) $(GENERATED_H))\n LOCALIZED_SH = $(sort $(SCRIPT_SH) git-sh-setup.sh)\n LOCALIZED_PERL = $(sort $(SCRIPT_PERL))\n \n@@ -3062,7 +3066,7 @@ bin-wrappers/%: wrap-for-bin.sh\n \t$(call mkdir_p_parent_template)\n \t$(QUIET_GEN)sed -e '1s|#!.*/sh|#!$(SHELL_PATH_SQ)|' \\\n \t -e 's|@@BUILD_DIR@@|$(shell pwd)|' \\\n-\t -e 's|@@PROG@@|$(patsubst test-%,t/helper/test-%$(X),$(@F))$(patsubst git%,$(X),$(filter $(@F),$(BINDIR_PROGRAMS_NEED_X)))|' < $< > $@ && \\\n+\t -e 's|@@PROG@@|$(patsubst test-%,t/helper/test-%,$(@F))$(if $(filter-out $(BINDIR_PROGRAMS_NO_X),$(@F)),$(X),)|' < $< > $@ && \\\n \tchmod +x $@\n \n # GNU make supports exporting all variables by \"export\" without parameters.\n@@ -3276,14 +3280,14 @@ ifndef NO_TCLTK\n \t$(MAKE) -C git-gui gitexecdir='$(gitexec_instdir_SQ)' install\n endif\n ifneq (,$X)\n-\t$(foreach p,$(patsubst %$X,%,$(filter %$X,$(ALL_COMMANDS_TO_INSTALL) git$X)), test '$(DESTDIR_SQ)$(gitexec_instdir_SQ)/$p' -ef '$(DESTDIR_SQ)$(gitexec_instdir_SQ)/$p$X' || $(RM) '$(DESTDIR_SQ)$(gitexec_instdir_SQ)/$p';)\n+\t$(foreach p,$(patsubst %$X,%,$(filter %$X,$(ALL_COMMANDS_TO_INSTALL) $(OTHER_PROGRAMS))), test '$(DESTDIR_SQ)$(gitexec_instdir_SQ)/$p' -ef '$(DESTDIR_SQ)$(gitexec_instdir_SQ)/$p$X' || $(RM) '$(DESTDIR_SQ)$(gitexec_instdir_SQ)/$p';)\n endif\n \n \tbindir=$$(cd '$(DESTDIR_SQ)$(bindir_SQ)' && pwd) && \\\n \texecdir=$$(cd '$(DESTDIR_SQ)$(gitexec_instdir_SQ)' && pwd) && \\\n \tdestdir_from_execdir_SQ=$$(echo '$(gitexecdir_relative_SQ)' | sed -e 's|[^/][^/]*|..|g') && \\\n \t{ test \"$$bindir/\" = \"$$execdir/\" || \\\n-\t for p in git$X $(filter $(install_bindir_programs),$(ALL_PROGRAMS)); do \\\n+\t for p in $(OTHER_PROGRAMS) $(filter $(install_bindir_programs),$(ALL_PROGRAMS)); do \\\n \t\t$(RM) \"$$execdir/$$p\" && \\\n \t\ttest -n \"$(INSTALL_SYMLINKS)\" && \\\n \t\tln -s \"$$destdir_from_execdir_SQ/$(bindir_relative_SQ)/$$p\" \"$$execdir/$$p\" || \\\n@@ -3458,7 +3462,7 @@ clean: profile-clean coverage-clean cocciclean\n \t$(RM) git.res\n \t$(RM) $(OBJECTS)\n \t$(RM) $(LIB_FILE) $(XDIFF_LIB) $(REFTABLE_LIB) $(REFTABLE_TEST_LIB)\n-\t$(RM) $(ALL_PROGRAMS) $(SCRIPT_LIB) $(BUILT_INS) git$X\n+\t$(RM) $(ALL_PROGRAMS) $(SCRIPT_LIB) $(BUILT_INS) $(OTHER_PROGRAMS)\n \t$(RM) $(TEST_PROGRAMS)\n \t$(RM) $(FUZZ_PROGRAMS)\n \t$(RM) $(SP_OBJ)\n@@ -3509,6 +3513,7 @@ ALL_COMMANDS += git-citool\n ALL_COMMANDS += git-gui\n ALL_COMMANDS += gitk\n ALL_COMMANDS += gitweb\n+ALL_COMMANDS += scalar\n \n .PHONY: check-docs\n check-docs::\ndiff --git a/contrib/buildsystems/CMakeLists.txt b/contrib/buildsystems/CMakeLists.txt\nindex 2237109b57f..bae203c1fb9 100644\n--- a/contrib/buildsystems/CMakeLists.txt\n+++ b/contrib/buildsystems/CMakeLists.txt\n@@ -610,7 +610,7 @@ unset(CMAKE_REQUIRED_INCLUDES)\n #programs\n set(PROGRAMS_BUILT\n \tgit git-daemon git-http-backend git-sh-i18n--envsubst\n-\tgit-shell)\n+\tgit-shell scalar)\n \n if(NOT CURL_FOUND)\n \tlist(APPEND excluded_progs git-http-fetch git-http-push)\n@@ -757,6 +757,9 @@ target_link_libraries(git-sh-i18n--envsubst common-main)\n add_executable(git-shell ${CMAKE_SOURCE_DIR}/shell.c)\n target_link_libraries(git-shell common-main)\n \n+add_executable(scalar ${CMAKE_SOURCE_DIR}/scalar.c)\n+target_link_libraries(scalar common-main)\n+\n if(CURL_FOUND)\n \tadd_library(http_obj OBJECT ${CMAKE_SOURCE_DIR}/http.c)\n \n@@ -903,7 +906,7 @@ list(TRANSFORM git_perl_scripts PREPEND \"${CMAKE_BINARY_DIR}/\")\n \n #install\n foreach(program ${PROGRAMS_BUILT})\n-if(program STREQUAL \"git\" OR program STREQUAL \"git-shell\")\n+if(program MATCHES \"^(git|git-shell|scalar)$\")\n install(TARGETS ${program}\n \tRUNTIME DESTINATION bin)\n else()\n@@ -977,7 +980,7 @@ endif()\n \n #wrapper scripts\n set(wrapper_scripts\n-\tgit git-upload-pack git-receive-pack git-upload-archive git-shell git-remote-ext)\n+\tgit git-upload-pack git-receive-pack git-upload-archive git-shell git-remote-ext scalar)\n \n set(wrapper_test_scripts\n \ttest-fake-ssh test-tool)\ndiff --git a/contrib/scalar/.gitignore b/contrib/scalar/.gitignore\ndeleted file mode 100644\nindex ff3d47e84d0..00000000000\n--- a/contrib/scalar/.gitignore\n+++ /dev/null\n@@ -1,2 +0,0 @@\n-/*.exe\n-/scalar\ndiff --git a/contrib/scalar/Makefile b/contrib/scalar/Makefile\ndeleted file mode 100644\nindex 37f283f35d7..00000000000\n--- a/contrib/scalar/Makefile\n+++ /dev/null\n@@ -1,35 +0,0 @@\n-# The default target of this Makefile is...\n-all::\n-\n-# Import tree-wide shared Makefile behavior and libraries\n-include ../../shared.mak\n-\n-include ../../config.mak.uname\n--include ../../config.mak.autogen\n--include ../../config.mak\n-\n-TARGETS = scalar$(X) scalar.o\n-GITLIBS = ../../common-main.o ../../libgit.a ../../xdiff/lib.a\n-\n-all:: scalar$(X) ../../bin-wrappers/scalar\n-\n-$(GITLIBS):\n-\t$(QUIET_SUBDIR0)../.. $(QUIET_SUBDIR1) $(subst ../../,,$@)\n-\n-$(TARGETS): $(GITLIBS) scalar.c\n-\t$(QUIET_SUBDIR0)../.. $(QUIET_SUBDIR1) $(patsubst %,contrib/scalar/%,$@)\n-\n-clean:\n-\t$(RM) $(TARGETS) ../../bin-wrappers/scalar\n-\n-../../bin-wrappers/scalar: ../../wrap-for-bin.sh Makefile\n-\t@mkdir -p ../../bin-wrappers\n-\t$(QUIET_GEN)sed -e '1s|#!.*/sh|#!$(SHELL_PATH_SQ)|' \\\n-\t -e 's|@@BUILD_DIR@@|$(shell cd ../.. && pwd)|' \\\n-\t -e 's|@@PROG@@|contrib/scalar/scalar$(X)|' < $< > $@ && \\\n-\tchmod +x $@\n-\n-test: all\n-\t$(MAKE) -C t\n-\n-.PHONY: $(GITLIBS) all clean test FORCE\ndiff --git a/contrib/scalar/t/Makefile b/contrib/scalar/t/Makefile\ndeleted file mode 100644\nindex 1ed174a8cf3..00000000000\n--- a/contrib/scalar/t/Makefile\n+++ /dev/null\n@@ -1,81 +0,0 @@\n-# Import tree-wide shared Makefile behavior and libraries\n-include ../../../shared.mak\n-\n-# Run scalar tests\n-#\n-# Copyright (c) 2005,2021 Junio C Hamano, Johannes Schindelin\n-#\n-\n--include ../../../config.mak.autogen\n--include ../../../config.mak\n-\n-SHELL_PATH ?= $(SHELL)\n-PERL_PATH ?= /usr/bin/perl\n-RM ?= rm -f\n-PROVE ?= prove\n-DEFAULT_TEST_TARGET ?= test\n-TEST_LINT ?= test-lint\n-\n-ifdef TEST_OUTPUT_DIRECTORY\n-TEST_RESULTS_DIRECTORY = $(TEST_OUTPUT_DIRECTORY)/test-results\n-else\n-TEST_RESULTS_DIRECTORY = ../../../t/test-results\n-endif\n-\n-# Shell quote;\n-SHELL_PATH_SQ = $(subst ','\\'',$(SHELL_PATH))\n-PERL_PATH_SQ = $(subst ','\\'',$(PERL_PATH))\n-TEST_RESULTS_DIRECTORY_SQ = $(subst ','\\'',$(TEST_RESULTS_DIRECTORY))\n-\n-T = $(sort $(wildcard t[0-9][0-9][0-9][0-9]-*.sh))\n-\n-all: $(DEFAULT_TEST_TARGET)\n-\n-test: $(TEST_LINT)\n-\t$(MAKE) aggregate-results-and-cleanup\n-\n-prove: $(TEST_LINT)\n-\t@echo \"*** prove ***\"; GIT_CONFIG=.git/config $(PROVE) --exec '$(SHELL_PATH_SQ)' $(GIT_PROVE_OPTS) $(T) :: $(GIT_TEST_OPTS)\n-\t$(MAKE) clean-except-prove-cache\n-\n-$(T):\n-\t@echo \"*** $@ ***\"; GIT_CONFIG=.git/config '$(SHELL_PATH_SQ)' $@ $(GIT_TEST_OPTS)\n-\n-clean-except-prove-cache:\n-\t$(RM) -r 'trash directory'.*\n-\t$(RM) -r valgrind/bin\n-\n-clean: clean-except-prove-cache\n-\t$(RM) .prove\n-\n-test-lint: test-lint-duplicates test-lint-executable test-lint-shell-syntax\n-\n-test-lint-duplicates:\n-\t@dups=`echo $(T) | tr ' ' '\\n' | sed 's/-.*//' | sort | uniq -d` && \\\n-\t\ttest -z \"$$dups\" || { \\\n-\t\techo >&2 \"duplicate test numbers:\" $$dups; exit 1; }\n-\n-test-lint-executable:\n-\t@bad=`for i in $(T); do test -x \"$$i\" || echo $$i; done` && \\\n-\t\ttest -z \"$$bad\" || { \\\n-\t\techo >&2 \"non-executable tests:\" $$bad; exit 1; }\n-\n-test-lint-shell-syntax:\n-\t@'$(PERL_PATH_SQ)' ../../../t/check-non-portable-shell.pl $(T)\n-\n-aggregate-results-and-cleanup: $(T)\n-\t$(MAKE) aggregate-results\n-\t$(MAKE) clean\n-\n-aggregate-results:\n-\tfor f in '$(TEST_RESULTS_DIRECTORY_SQ)'/t*-*.counts; do \\\n-\t\techo \"$$f\"; \\\n-\tdone | '$(SHELL_PATH_SQ)' ../../../t/aggregate-results.sh\n-\n-valgrind:\n-\t$(MAKE) GIT_TEST_OPTS=\"$(GIT_TEST_OPTS) --valgrind\"\n-\n-test-results:\n-\tmkdir -p test-results\n-\n-.PHONY: $(T) aggregate-results clean valgrind\ndiff --git a/contrib/scalar/scalar.c b/scalar.c\nsimilarity index 100%\nrename from contrib/scalar/scalar.c\nrename to scalar.c\ndiff --git a/contrib/scalar/t/t9099-scalar.sh b/t/t9210-scalar.sh\nsimilarity index 96%\nrename from contrib/scalar/t/t9099-scalar.sh\nrename to t/t9210-scalar.sh\nindex dfb949f52ee..14ca575a214 100755\n--- a/contrib/scalar/t/t9099-scalar.sh\n+++ b/t/t9210-scalar.sh\n@@ -2,15 +2,9 @@\n \n test_description='test the `scalar` command'\n \n-TEST_DIRECTORY=$PWD/../../../t\n-export TEST_DIRECTORY\n+. ./test-lib.sh\n \n-# Make it work with --no-bin-wrappers\n-PATH=$PWD/..:$PATH\n-\n-. ../../../t/test-lib.sh\n-\n-GIT_TEST_MAINT_SCHEDULER=\"crontab:test-tool crontab ../cron.txt,launchctl:true,schtasks:true\"\n+GIT_TEST_MAINT_SCHEDULER=\"crontab:test-tool crontab cron.txt,launchctl:true,schtasks:true\"\n export GIT_TEST_MAINT_SCHEDULER\n \n test_expect_success 'scalar shows a usage' '\n-- \ngitgitgadget\n\n"},{"id":"462536","messageId":"2e6dd03beba221e68f1af71aabf17e63541401db.1662134210.git.gitgitgadget@gmail.com","threadId":"58383","inReplyTo":"pull.1341.v2.git.1662134210.gitgitgadget@gmail.com","subject":"[PATCH v2 8/9] t/perf: add 'GIT_PERF_USE_SCALAR' run option","fromName":"Victoria Dye via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2022-09-02T15:56:49Z","receivedAt":"2022-09-02T16:11:24Z","isPatch":true,"sender":{"key":"vdye@github.com","avatar":"https://avatars.githubusercontent.com/u/3619353?v=4"},"body":"From: Victoria Dye \n\nAdd a 'GIT_PERF_USE_SCALAR' environment variable (and corresponding perf\nconfig 'useScalar') to register a repository created with any of:\n\n* test_perf_fresh_repo\n* test_perf_default_repo\n* test_perf_large_repo\n\nas a Scalar enlistment. This is intended to allow a developer to test the\nimpact of Scalar on already-defined performance scenarios.\n\nSuggested-by: Derrick Stolee \nSigned-off-by: Victoria Dye \n---\n t/perf/README | 4 ++++\n t/perf/perf-lib.sh | 13 ++++++++++++-\n t/perf/run | 3 +++\n 3 files changed, 19 insertions(+), 1 deletion(-)\n\ndiff --git a/t/perf/README b/t/perf/README\nindex fb9127a66f7..8f217d7be7d 100644\n--- a/t/perf/README\n+++ b/t/perf/README\n@@ -95,6 +95,10 @@ You can set the following variables (also in your config.mak):\n \tGit (e.g., performance of index-pack as the number of threads\n \tchanges). These can be enabled with GIT_PERF_EXTRA.\n \n+ GIT_PERF_USE_SCALAR\n+\tBoolean indicating whether to register test repo(s) with Scalar\n+\tbefore executing tests.\n+\n You can also pass the options taken by ordinary git tests; the most\n useful one is:\n \ndiff --git a/t/perf/perf-lib.sh b/t/perf/perf-lib.sh\nindex 27c28017921..e7786775a90 100644\n--- a/t/perf/perf-lib.sh\n+++ b/t/perf/perf-lib.sh\n@@ -49,6 +49,9 @@ export TEST_DIRECTORY TRASH_DIRECTORY GIT_BUILD_DIR GIT_TEST_CMP\n MODERN_GIT=$GIT_BUILD_DIR/bin-wrappers/git\n export MODERN_GIT\n \n+MODERN_SCALAR=$GIT_BUILD_DIR/bin-wrappers/scalar\n+export MODERN_SCALAR\n+\n perf_results_dir=$TEST_RESULTS_DIR\n test -n \"$GIT_PERF_SUBSECTION\" && perf_results_dir=\"$perf_results_dir/$GIT_PERF_SUBSECTION\"\n mkdir -p \"$perf_results_dir\"\n@@ -120,6 +123,10 @@ test_perf_create_repo_from () {\n \t\t\t# status\" due to a locked index. Since we have\n \t\t\t# a copy it's fine to remove the lock.\n \t\t\trm .git/index.lock\n+\t\tfi &&\n+\t\tif test_bool_env GIT_PERF_USE_SCALAR false\n+\t\tthen\n+\t\t\t\"$MODERN_SCALAR\" register\n \t\tfi\n \t) || error \"failed to copy repository '$source' to '$repo'\"\n }\n@@ -130,7 +137,11 @@ test_perf_fresh_repo () {\n \t\"$MODERN_GIT\" init -q \"$repo\" &&\n \t(\n \t\tcd \"$repo\" &&\n-\t\ttest_perf_do_repo_symlink_config_\n+\t\ttest_perf_do_repo_symlink_config_ &&\n+\t\tif test_bool_env GIT_PERF_USE_SCALAR false\n+\t\tthen\n+\t\t\t\"$MODERN_SCALAR\" register\n+\t\tfi\n \t)\n }\n \ndiff --git a/t/perf/run b/t/perf/run\nindex 55219aa4056..33da4d2aba2 100755\n--- a/t/perf/run\n+++ b/t/perf/run\n@@ -171,6 +171,9 @@ run_subsection () {\n \tget_var_from_env_or_config \"GIT_PERF_MAKE_COMMAND\" \"perf\" \"makeCommand\"\n \tget_var_from_env_or_config \"GIT_PERF_MAKE_OPTS\" \"perf\" \"makeOpts\"\n \n+\tget_var_from_env_or_config \"GIT_PERF_USE_SCALAR\" \"perf\" \"useScalar\" \"--bool\"\n+\texport GIT_PERF_USE_SCALAR\n+\n \tget_var_from_env_or_config \"GIT_PERF_REPO_NAME\" \"perf\" \"repoName\"\n \texport GIT_PERF_REPO_NAME\n \n-- \ngitgitgadget\n\n"},{"id":"462537","messageId":"6bedd80c398dd862d17b03528cc14f17e9484391.1662134210.git.gitgitgadget@gmail.com","threadId":"58383","inReplyTo":"pull.1341.v2.git.1662134210.gitgitgadget@gmail.com","subject":"[PATCH v2 9/9] Documentation/technical: include Scalar technical doc","fromName":"Victoria Dye via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2022-09-02T15:56:50Z","receivedAt":"2022-09-02T16:11:31Z","isPatch":true,"sender":{"key":"vdye@github.com","avatar":"https://avatars.githubusercontent.com/u/3619353?v=4"},"body":"From: Victoria Dye \n\nInclude 'Documentation/technical/scalar.txt' alongside the other HTML\ntechnical docs when installing them.\n\nNow that the document is intended as a widely-accessible reference, remove\nthe internal work-in-progress roadmap from the document. Those details\nshould no longer be needed to guide Scalar's development and, if they were\nleft, they could fall out-of-date and be misleading to readers.\n\nSigned-off-by: Victoria Dye \n---\n Documentation/Makefile | 1 +\n Documentation/technical/scalar.txt | 61 ------------------------------\n 2 files changed, 1 insertion(+), 61 deletions(-)\n\ndiff --git a/Documentation/Makefile b/Documentation/Makefile\nindex 16c9e062390..9ec53afdf18 100644\n--- a/Documentation/Makefile\n+++ b/Documentation/Makefile\n@@ -116,6 +116,7 @@ TECH_DOCS += technical/parallel-checkout\n TECH_DOCS += technical/partial-clone\n TECH_DOCS += technical/racy-git\n TECH_DOCS += technical/reftable\n+TECH_DOCS += technical/scalar\n TECH_DOCS += technical/send-pack-pipeline\n TECH_DOCS += technical/shallow\n TECH_DOCS += technical/trivial-merge\ndiff --git a/Documentation/technical/scalar.txt b/Documentation/technical/scalar.txt\nindex 0600150b3ad..921cb104c3c 100644\n--- a/Documentation/technical/scalar.txt\n+++ b/Documentation/technical/scalar.txt\n@@ -64,64 +64,3 @@ some \"global\" `git` options (e.g., `-c` and `-C`).\n Because `scalar` is not invoked as a Git subcommand (like `git scalar`), it is\n built and installed as its own executable in the `bin/` directory, alongside\n `git`, `git-gui`, etc.\n-\n-Roadmap\n--------\n-\n-NOTE: this section will be removed once the remaining tasks outlined in this\n-roadmap are complete.\n-\n-Scalar is a large enough project that it is being upstreamed incrementally,\n-living in `contrib/` until it is feature-complete. So far, the following patch\n-series have been accepted:\n-\n-- `scalar-the-beginning`: The initial patch series which sets up\n- `contrib/scalar/` and populates it with a minimal `scalar` command that\n- demonstrates the fundamental ideas.\n-\n-- `scalar-c-and-C`: The `scalar` command learns about two options that can be\n- specified before the command, `-c =` and `-C `.\n-\n-- `scalar-diagnose`: The `scalar` command is taught the `diagnose` subcommand.\n-\n-- `scalar-generalize-diagnose`: Move the functionality of `scalar diagnose`\n- into `git diagnose` and `git bugreport --diagnose`.\n-\n-- 'scalar-add-fsmonitor: Enable the built-in FSMonitor in Scalar\n- enlistments. At the end of this series, Scalar should be feature-complete\n- from the perspective of a user.\n-\n-Roughly speaking (and subject to change), the following series are needed to\n-\"finish\" this initial version of Scalar:\n-\n-- Move Scalar to toplevel: Move Scalar out of `contrib/` and into the root of\n- `git`. This includes a variety of related updates, including:\n- - building & installing Scalar in the Git root-level 'make [install]'.\n- - builing & testing Scalar as part of CI.\n- - moving and expanding test coverage of Scalar (including perf tests).\n- - implementing 'scalar help'/'git help scalar' to display scalar\n- documentation.\n-\n-Finally, there are two additional patch series that exist in Microsoft's fork of\n-Git, but there is no current plan to upstream them. There are some interesting\n-ideas there, but the implementation is too specific to Azure Repos and/or VFS\n-for Git to be of much help in general.\n-\n-These still exist mainly because the GVFS protocol is what Azure Repos has\n-instead of partial clone, while Git is focused on improving partial clone:\n-\n-- `scalar-with-gvfs`: The primary purpose of this patch series is to support\n- existing Scalar users whose repositories are hosted in Azure Repos (which does\n- not support Git's partial clones, but supports its predecessor, the GVFS\n- protocol, which is used by Scalar to emulate the partial clone).\n-\n- Since the GVFS protocol will never be supported by core Git, this patch series\n- will remain in Microsoft's fork of Git.\n-\n-- `run-scalar-functional-tests`: The Scalar project developed a quite\n- comprehensive set of integration tests (or, \"Functional Tests\"). They are the\n- sole remaining part of the original C#-based Scalar project, and this patch\n- adds a GitHub workflow that runs them all.\n-\n- Since the tests partially depend on features that are only provided in the\n- `scalar-with-gvfs` patch series, this patch cannot be upstreamed.\n-- \ngitgitgadget\n"},{"id":"462538","messageId":"070f195f027e5601b88ca6a0d4c9991b472ad9ab.1662134210.git.gitgitgadget@gmail.com","threadId":"58383","inReplyTo":"pull.1341.v2.git.1662134210.gitgitgadget@gmail.com","subject":"[PATCH v2 5/9] scalar: add to 'git help -a' command list","fromName":"Victoria Dye via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2022-09-02T15:56:46Z","receivedAt":"2022-09-02T16:13:42Z","isPatch":true,"sender":{"key":"vdye@github.com","avatar":"https://avatars.githubusercontent.com/u/3619353?v=4"},"body":"From: Victoria Dye \n\nAdd 'scalar' as a 'mainporcelain' command in the Git command list. Update\nthe regex in 'cmd-list.perl' used to match the first line of command\ndocumentation to find 'scalar(1)'.\n\nHelped-by: Ævar Arnfjörð Bjarmason \nSigned-off-by: Victoria Dye \n---\n Documentation/cmd-list.perl | 2 +-\n command-list.txt | 1 +\n 2 files changed, 2 insertions(+), 1 deletion(-)\n\ndiff --git a/Documentation/cmd-list.perl b/Documentation/cmd-list.perl\nindex af5da45d287..9515a499a31 100755\n--- a/Documentation/cmd-list.perl\n+++ b/Documentation/cmd-list.perl\n@@ -10,7 +10,7 @@ sub format_one {\n \t$state = 0;\n \topen I, '<', \"$name.txt\" or die \"No such file $name.txt\";\n \twhile () {\n-\t\tif (/^git[a-z0-9-]*\\(([0-9])\\)$/) {\n+\t\tif (/^(git|scalar)[a-z0-9-]*\\(([0-9])\\)$/) {\n \t\t\t$mansection = $1;\n \t\t\tnext;\n \t\t}\ndiff --git a/command-list.txt b/command-list.txt\nindex f96bdabd7d9..93f94e42ab7 100644\n--- a/command-list.txt\n+++ b/command-list.txt\n@@ -235,3 +235,4 @@ gittutorial guide\n gittutorial-2 guide\n gitweb ancillaryinterrogators\n gitworkflows guide\n+scalar mainporcelain\n-- \ngitgitgadget\n\n"},{"id":"462541","messageId":"xmqqwnal90s6.fsf@gitster.g","threadId":"58383","inReplyTo":"9n46p076-q126-o813-8s39-936s6667o7o1@tzk.qr","subject":"Re: [PATCH 5/8] scalar-clone: add test coverage","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2022-09-02T16:52:25Z","receivedAt":"2022-09-02T16:52:31Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Johannes Schindelin writes:\n\n>> That being said, even if I update 't9211', my experience with Git's test\n>> suite is that very few tests preserve test repos this way. Do you expect\n>> these artifacts to be especially helpful for 'scalar clone' in particular,\n>> or is this more of a \"gently nudge contributors to make this standard\n>> practice\" sort of recommendation?\n>\n> Thank you for this question, which helps me clarify even to myself what my\n> intention is.\n>\n> After considering this, yes, I would like this to be a gentle nudge to\n> take a tiny step toward improving Git's test suite by recommending a new\n> standard practice.\n\nI agree that it would in general be a good idea to leave more cruft\nfor those who want to postmortem. It will nudge a sequence of tests\nto use distinct test directories and output files, in order to avoid\nletting the later ones overwrite the earlier ones, which would be a\ndeviation from what we have done historically.\n\nFor those who diagnose breakage manually after seeing \"tXXXX.sh -i\"\nfail, cruft left by earlier successful tests are quite distracting\nnuisance and that certainly was why we tried to remove them after\neach test piece succeeds. These days, we seem to be relying much\nmore on unattended bulk tests at CI than before, which changes the\nequation.\n\n\n"},{"id":"462625","messageId":"73sr2o20-q1np-sn3n-n3p0-8s904n966344@tzk.qr","threadId":"58383","inReplyTo":"pull.1341.v2.git.1662134210.gitgitgadget@gmail.com","subject":"Re: [PATCH v2 0/9] scalar: integrate into core Git","fromName":"Johannes Schindelin","fromEmail":"johannes.schindelin@gmx.de","sentAt":"2022-09-05T10:36:09Z","receivedAt":"2022-09-05T10:36:22Z","isPatch":true,"sender":{"key":"johannes.schindelin@gmx.de","avatar":"https://avatars.githubusercontent.com/u/127790?v=4"},"body":"Hi Victoria,\n\nOn Fri, 2 Sep 2022, Victoria Dye via GitGitGadget wrote:\n\n> Changes since V1\n> ================\n>\n> * Updated 'scalar.txt' description from \"Associated with [git]\" -> \"Part of\n> [git]\".\n> * Fixed Dscho's \"Signed-off-by\" and author emails on patches 3 & 4.\n> * Added 'scalar' to 'git help -a' command list.\n> * Removed translation tag for literal 'scalar help' usage string.\n> * Updated 'scalar clone' tests to clone to unique targets and clean up only\n> when test passes.\n> * Changed 'p9210' perf tests to invoke 'git' and 'scalar' with '-C', rather\n> than creating a subshell to run commands.\n> * Fixed use of 'test_env_bool' with 'GIT_PERF_USE_SCALAR', locally tested\n> to make sure it was working as-intended.\n\nThis iteration addresses all concerns I had.\n\nThank you so much!\nDscho\n"},{"id":"462831","messageId":"dade8c50-3456-3538-e151-a1deffbb1c1d@github.com","threadId":"58383","inReplyTo":"pull.1341.v2.git.1662134210.gitgitgadget@gmail.com","subject":"Re: [PATCH v2 0/9] scalar: integrate into core Git","fromName":"Derrick Stolee","fromEmail":"derrickstolee@github.com","sentAt":"2022-09-08T20:54:51Z","receivedAt":"2022-09-08T20:55:08Z","isPatch":true,"sender":{"key":"stolee@gmail.com","avatar":"https://avatars.githubusercontent.com/u/570044?v=4"},"body":"On 9/2/2022 11:56 AM, Victoria Dye via GitGitGadget wrote:\n> This series completes the initial implementation of the Scalar command as a\n> core component of Git. For readers new to the topic of Scalar, the\n> roadmap/design doc [1] provides some background information including how\n> the project started & evolved, as well as its current intent & high-level\n> design.\n\nI just took a read through this series, and everything seemed\nwell justified and cleanly implemented. I built and installed\nthis version of Scalar locally as a way to see that all the\nbuild steps worked as expected.\n\nThanks!\n-Stolee\n"},{"id":"465779","messageId":"cover-v2-0.3-00000000000-20221026T143533Z-avarab@gmail.com","threadId":"58383","inReplyTo":"cover-0.5-00000000000-20220901T130817Z-avarab@gmail.com","subject":"[PATCH v2 0/3] Makefile: fix issues with bin-wrappers/% rule","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2022-10-26T14:42:34Z","receivedAt":"2022-10-26T14:42:47Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"This simple topic fixes issues with the bin-wrappers/% rules, as seen\nin the range-diff below this never worked properly:\n\n\tmake bin-wrappers/git\n\nI.e. we'd make the scirpt, but not \"git\".\n\nI originally sent this as [1] in response to a topic that's since\nlanded, i.e. that topic needed to add more conditions to the \"match\nthis, but not that\" part of the current rule.\n\nAs 3/3 notes that's because we were previously squashing 3 Makefile\nvariables into one, and then having to heurisitcally match their\ncontents to figure out which item originally came from which variable.\n\nThe 3/3 here simply avoids squashing all that data together, so we\ndon't need to guess after the fact.\n\nFor the v2 I squashed the previous 3 commits together, they were\nincrementally improving the rule, now we do it all at once.\n\n1. https://lore.kernel.org/git/cover-0.5-00000000000-20220901T130817Z-avarab@gmail.com/\n\nÆvar Arnfjörð Bjarmason (3):\n Makefile: factor sed-powered '#!/bin/sh' munging into a variable\n Makefile: define \"TEST_{PROGRAM,OBJS}\" variables earlier\n Makefile: simplify $(test_bindir_programs) rule by splitting it up\n\n Makefile | 67 ++++++++++++++++++++++++++++++++++++++++----------------\n 1 file changed, 48 insertions(+), 19 deletions(-)\n\nRange-diff against v1:\n1: 40033143cdd = 1: fc6c5a6a8df Makefile: factor sed-powered '#!/bin/sh' munging into a variable\n2: fe54dacaad2 = 2: 6dcb49f25c4 Makefile: define \"TEST_{PROGRAM,OBJS}\" variables earlier\n3: 9d4ac628f0c ! 3: 400f487e30d Makefile: simplify $(test_bindir_programs) rule by splitting it up\n @@ Commit message\n Which will show an empty diff, i.e. we've correctly dealt with the\n combination of $(SHELL_PATH), $(X) and these three variables here.\n \n + This also fixes an issue with the \"bin-wrappers/\" scripts have never had properly declared\n + dependency information, i.e. this has never worked:\n +\n + make clean &&\n + make bin-wrappers/git &&\n + # the script is there, but no \"./git\" is built\n + ./bin-wrappers/git\n +\n + There was no reason not to have that work, just as most things\n + generated by the Makefile have proper dependency information.\n +\n + Before this commit doing this would have been painful, but now it's\n + easy to pass this as a parameter to our \"bin_wrappers_template\"\n +\n 1. ea925196f1b (build dashless \"bin-wrappers\" directory similar to\n installed bindir, 2009-12-02)\n 2. e6e7530d10b (test helpers: move test-* to t/helper/ subdirectory,\n @@ Makefile: GIT-PYTHON-VARS: FORCE\n +endef\n \n -all:: $(test_bindir_programs)\n -+BW_BINDIR_PROGRAMS_NEED_X = $(BINDIR_PROGRAMS_NEED_X:%=bin-wrappers/%)\n -+BIN_WRAPPERS += $(BW_BINDIR_PROGRAMS_NEED_X)\n -+$(BW_BINDIR_PROGRAMS_NEED_X): wrap-for-bin.sh\n -+\t$(call mkdir_p_parent_template)\n -+\t$(QUIET_GEN)$(call cmd_munge_bin_wrappers_script,$(@F),,$X)\n ++define bin_wrappers_template\n ++\n ++## bin_wrappers_template\n ++# 1 = $(1)\n ++# 2 = $(2)\n ++# 3 = $(3)\n ++# 4 = $(4)\n ++BW_$(1) = $$($(1):%=bin-wrappers/%)\n ++BIN_WRAPPERS += $$(BW_$(1))\n ++all:: $$(BW_$(1))\n ++$$(BW_$(1)): bin-wrappers/% : $(3)%$(4)\n ++$$(BW_$(1)): wrap-for-bin.sh\n ++\t$$(call mkdir_p_parent_template)\n ++\t$$(QUIET_GEN)$$(call cmd_munge_bin_wrappers_script,$(2),$(3),$(4))\n ++endef\n \n -bin-wrappers/%: wrap-for-bin.sh\n -+BW_BINDIR_PROGRAMS_NO_X = $(BINDIR_PROGRAMS_NO_X:%=bin-wrappers/%)\n -+BIN_WRAPPERS += $(BW_BINDIR_PROGRAMS_NO_X)\n -+$(BW_BINDIR_PROGRAMS_NO_X): wrap-for-bin.sh\n - \t$(call mkdir_p_parent_template)\n +-\t$(call mkdir_p_parent_template)\n -\t$(QUIET_GEN)sed -e $(call cmd_munge_script_sed_shell_path_arg) \\\n -\t -e 's|@@BUILD_DIR@@|$(shell pwd)|' \\\n -\t -e 's|@@PROG@@|$(patsubst test-%,t/helper/test-%,$(@F))$(if $(filter-out $(BINDIR_PROGRAMS_NO_X),$(@F)),$(X),)|' < $< > $@ && \\\n -\tchmod +x $@\n -+\t$(QUIET_GEN)$(call cmd_munge_bin_wrappers_script,$(@F))\n -+\n -+BW_TEST_PROGRAMS_NEED_X = $(TEST_PROGRAMS_NEED_X:%=bin-wrappers/%)\n -+BIN_WRAPPERS += $(BW_TEST_PROGRAMS_NEED_X)\n -+$(BW_TEST_PROGRAMS_NEED_X): wrap-for-bin.sh\n -+\t$(call mkdir_p_parent_template)\n -+\t$(QUIET_GEN)$(call cmd_munge_bin_wrappers_script,$(@F),t/helper/,$X)\n ++define bin_wrappers_templates\n ++$(call bin_wrappers_template,BINDIR_PROGRAMS_NEED_X,'$$(@F)',,$$X)\n ++$(call bin_wrappers_template,BINDIR_PROGRAMS_NO_X,'$$(@F)')\n ++$(call bin_wrappers_template,TEST_PROGRAMS_NEED_X,'$$(@F)',t/helper/,$$X)\n ++endef\n ++$(eval $(call bin_wrappers_templates))\n +\n +all:: $(BIN_WRAPPERS)\n \n4: cbbf458433f < -: ----------- Makefile: define bin-wrappers/% rules with a template\n5: 560dee80b4a < -: ----------- Makefile: fix \"make clean && make bin-wrappers/$NAME\" dependencies\n-- \n2.38.0.1251.g3eefdfb5e7a\n\n"},{"id":"465780","messageId":"patch-v2-1.3-fc6c5a6a8df-20221026T143534Z-avarab@gmail.com","threadId":"58383","inReplyTo":"cover-v2-0.3-00000000000-20221026T143533Z-avarab@gmail.com","subject":"[PATCH v2 1/3] Makefile: factor sed-powered '#!/bin/sh' munging into a variable","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2022-10-26T14:42:35Z","receivedAt":"2022-10-26T14:43:00Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"Reduce the amount of magical copy/pasting in the Makefile by factoring\nthe munging of \"#!/bin/sh\" on the first line of a shellscript into a\nvariable we can re-use in the various rules that need to do so.\n\nSigned-off-by: Ævar Arnfjörð Bjarmason \n---\n Makefile | 12 ++++++++----\n 1 file changed, 8 insertions(+), 4 deletions(-)\n\ndiff --git a/Makefile b/Makefile\nindex 85f03c6aed1..45b22d33513 100644\n--- a/Makefile\n+++ b/Makefile\n@@ -2344,8 +2344,12 @@ GIT-SCRIPT-DEFINES: FORCE\n \t\techo \"$$FLAGS\" >$@; \\\n fi\n \n+define cmd_munge_script_sed_shell_path_arg\n+'1s|#!.*/sh|#!$(SHELL_PATH_SQ)|'\n+endef\n+\n define cmd_munge_script\n-sed -e '1s|#!.*/sh|#!$(SHELL_PATH_SQ)|' \\\n+sed -e $(call cmd_munge_script_sed_shell_path_arg) \\\n -e 's|@SHELL_PATH@|$(SHELL_PATH_SQ)|' \\\n -e 's|@@DIFF@@|$(DIFF_SQ)|' \\\n -e 's|@@LOCALEDIR@@|$(localedir_SQ)|g' \\\n@@ -2447,7 +2451,7 @@ git-instaweb: git-instaweb.sh GIT-SCRIPT-DEFINES\n else # NO_PERL\n $(SCRIPT_PERL_GEN) git-instaweb: % : unimplemented.sh\n \t$(QUIET_GEN) \\\n-\tsed -e '1s|#!.*/sh|#!$(SHELL_PATH_SQ)|' \\\n+\tsed -e $(call cmd_munge_script_sed_shell_path_arg) \\\n \t -e 's|@@REASON@@|NO_PERL=$(NO_PERL)|g' \\\n \t unimplemented.sh >$@+ && \\\n \tchmod +x $@+ && \\\n@@ -2468,7 +2472,7 @@ $(SCRIPT_PYTHON_GEN): % : %.py\n else # NO_PYTHON\n $(SCRIPT_PYTHON_GEN): % : unimplemented.sh\n \t$(QUIET_GEN) \\\n-\tsed -e '1s|#!.*/sh|#!$(SHELL_PATH_SQ)|' \\\n+\tsed -e $(call cmd_munge_script_sed_shell_path_arg) \\\n \t -e 's|@@REASON@@|NO_PYTHON=$(NO_PYTHON)|g' \\\n \t unimplemented.sh >$@+ && \\\n \tchmod +x $@+ && \\\n@@ -3061,7 +3065,7 @@ all:: $(TEST_PROGRAMS) $(test_bindir_programs)\n \n bin-wrappers/%: wrap-for-bin.sh\n \t$(call mkdir_p_parent_template)\n-\t$(QUIET_GEN)sed -e '1s|#!.*/sh|#!$(SHELL_PATH_SQ)|' \\\n+\t$(QUIET_GEN)sed -e $(call cmd_munge_script_sed_shell_path_arg) \\\n \t -e 's|@@BUILD_DIR@@|$(shell pwd)|' \\\n \t -e 's|@@PROG@@|$(patsubst test-%,t/helper/test-%,$(@F))$(if $(filter-out $(BINDIR_PROGRAMS_NO_X),$(@F)),$(X),)|' < $< > $@ && \\\n \tchmod +x $@\n-- \n2.38.0.1251.g3eefdfb5e7a\n\n"},{"id":"465781","messageId":"patch-v2-3.3-400f487e30d-20221026T143534Z-avarab@gmail.com","threadId":"58383","inReplyTo":"cover-v2-0.3-00000000000-20221026T143533Z-avarab@gmail.com","subject":"[PATCH v2 3/3] Makefile: simplify $(test_bindir_programs) rule by splitting it up","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2022-10-26T14:42:37Z","receivedAt":"2022-10-26T14:43:13Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"When the @@PROG@@ substitution was added in [1] it was a simple matter\nof doing a:\n\n\t's|@@PROG@@|$(@F)|'\n\nThen when t/helpers were added in [2] followed by the \".exe\" needing\nto be appended in [3] this previously simple rule ended up as a dense\none-liner.\n\nIt has been pointed out that this is hard to read[4], but the problem\nisn't the \"design of the Makefile syntax\". It's we now have to\nafter-the-fact determine if we were dealing with a bin-wrapper/ that\nneeded to have $(X) appended, a t/helper/, or a non-binary (currently\nonly git-cvsserver).\n\nThat would be a problem in any language. We're starting out with three\nlists, and then end up having to heuristically determine given a\nmember of any of those lists which list that member came from. Let's\njust stop doing that and keep track of what member belongs to which\nlist.\n\nWe can do this by splitting up the single \"bin-wrappers/%\" rule into a\nrule for each of the three lists. With the\n\"cmd_munge_script_sed_shell_path_arg\" define added in a preceding\ncommit this is easy, we just need to add a sister template to the\nexisting \"cmd_munge_script\" added in [5].\n\nThe \"filter-out\" added in [6] has now become unnecessary per the\nabove, it was an artifact of us losing track of what was in which of\nour lists to begin with.\n\nThis change can be tested with e.g.:\n\n\tgit checkout master &&\n\tmake SHELL_PATH=/bin/bash X=.exe &&\n\tmv bin-wrappers bin-wrappers.master &&\n\t &&\n\tmake SHELL_PATH=/bin/bash X=.exe &&\n\tdiff -ru bin-wrappers{.master,}\n\nWhich will show an empty diff, i.e. we've correctly dealt with the\ncombination of $(SHELL_PATH), $(X) and these three variables here.\n\nThis also fixes an issue with the \"bin-wrappers/\" scripts have never had properly declared\ndependency information, i.e. this has never worked:\n\n\tmake clean &&\n\tmake bin-wrappers/git &&\n\t# the script is there, but no \"./git\" is built\n\t./bin-wrappers/git\n\nThere was no reason not to have that work, just as most things\ngenerated by the Makefile have proper dependency information.\n\nBefore this commit doing this would have been painful, but now it's\neasy to pass this as a parameter to our \"bin_wrappers_template\"\n\n1. ea925196f1b (build dashless \"bin-wrappers\" directory similar to\n installed bindir, 2009-12-02)\n2. e6e7530d10b (test helpers: move test-* to t/helper/ subdirectory,\n 2016-04-13)\n3. 3a94cb31d52 (bin-wrappers: append `.exe` to target paths if\n necessary, 2019-07-29)\n4. https://lore.kernel.org/git/sso99so6-n28s-rq86-8q20-4456r3pn869r@tzk.qr/\n5. 46bac904581 (Do not install shell libraries executable, 2010-01-31)\n6. 7b5c93c6c68\" (scalar: include in standard Git build & installation,\n 2022-09-02)\n\nSigned-off-by: Ævar Arnfjörð Bjarmason \n---\n Makefile | 42 +++++++++++++++++++++++++++++++++---------\n 1 file changed, 33 insertions(+), 9 deletions(-)\n\ndiff --git a/Makefile b/Makefile\nindex a8cfa096dc1..d551a89bbff 100644\n--- a/Makefile\n+++ b/Makefile\n@@ -595,6 +595,7 @@ export TCL_PATH TCLTK_PATH\n PTHREAD_LIBS = -lpthread\n \n # Guard against environment variables\n+BIN_WRAPPERS =\n BUILTIN_OBJS =\n BUILT_INS =\n COMPAT_CFLAGS =\n@@ -3062,16 +3063,39 @@ GIT-PYTHON-VARS: FORCE\n fi\n endif\n \n-test_bindir_programs := $(patsubst %,bin-wrappers/%,$(BINDIR_PROGRAMS_NEED_X) $(BINDIR_PROGRAMS_NO_X) $(TEST_PROGRAMS_NEED_X))\n+define cmd_munge_bin_wrappers_script\n+sed \\\n+\t-e $(call cmd_munge_script_sed_shell_path_arg) \\\n+\t-e 's|@@BUILD_DIR@@|$(shell pwd)|' \\\n+\t-e 's|@@PROG@@|$(2)$(1)$(3)|' \\\n+\t<$< >$@ && \\\n+\tchmod +x $@\n+endef\n \n-all:: $(test_bindir_programs)\n+define bin_wrappers_template\n+\n+## bin_wrappers_template\n+# 1 = $(1)\n+# 2 = $(2)\n+# 3 = $(3)\n+# 4 = $(4)\n+BW_$(1) = $$($(1):%=bin-wrappers/%)\n+BIN_WRAPPERS += $$(BW_$(1))\n+all:: $$(BW_$(1))\n+$$(BW_$(1)): bin-wrappers/% : $(3)%$(4)\n+$$(BW_$(1)): wrap-for-bin.sh\n+\t$$(call mkdir_p_parent_template)\n+\t$$(QUIET_GEN)$$(call cmd_munge_bin_wrappers_script,$(2),$(3),$(4))\n+endef\n \n-bin-wrappers/%: wrap-for-bin.sh\n-\t$(call mkdir_p_parent_template)\n-\t$(QUIET_GEN)sed -e $(call cmd_munge_script_sed_shell_path_arg) \\\n-\t -e 's|@@BUILD_DIR@@|$(shell pwd)|' \\\n-\t -e 's|@@PROG@@|$(patsubst test-%,t/helper/test-%,$(@F))$(if $(filter-out $(BINDIR_PROGRAMS_NO_X),$(@F)),$(X),)|' < $< > $@ && \\\n-\tchmod +x $@\n+define bin_wrappers_templates\n+$(call bin_wrappers_template,BINDIR_PROGRAMS_NEED_X,'$$(@F)',,$$X)\n+$(call bin_wrappers_template,BINDIR_PROGRAMS_NO_X,'$$(@F)')\n+$(call bin_wrappers_template,TEST_PROGRAMS_NEED_X,'$$(@F)',t/helper/,$$X)\n+endef\n+$(eval $(call bin_wrappers_templates))\n+\n+all:: $(BIN_WRAPPERS)\n \n # GNU make supports exporting all variables by \"export\" without parameters.\n # However, the environment gets quite big, and some programs have problems\n@@ -3404,7 +3428,7 @@ OTHER_PROGRAMS += $(shell echo *.dll t/helper/*.dll)\n endif\n \n artifacts-tar:: $(ALL_COMMANDS_TO_INSTALL) $(SCRIPT_LIB) $(OTHER_PROGRAMS) \\\n-\t\tGIT-BUILD-OPTIONS $(TEST_PROGRAMS) $(test_bindir_programs) \\\n+\t\tGIT-BUILD-OPTIONS $(TEST_PROGRAMS) $(BIN_WRAPPERS) \\\n \t\t$(MOFILES)\n \t$(QUIET_SUBDIR0)templates $(QUIET_SUBDIR1) \\\n \t\tSHELL_PATH='$(SHELL_PATH_SQ)' PERL_PATH='$(PERL_PATH_SQ)'\n-- \n2.38.0.1251.g3eefdfb5e7a\n\n"},{"id":"465782","messageId":"patch-v2-2.3-6dcb49f25c4-20221026T143534Z-avarab@gmail.com","threadId":"58383","inReplyTo":"cover-v2-0.3-00000000000-20221026T143533Z-avarab@gmail.com","subject":"[PATCH v2 2/3] Makefile: define \"TEST_{PROGRAM,OBJS}\" variables earlier","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2022-10-26T14:42:36Z","receivedAt":"2022-10-26T14:43:17Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"Define the variables that make up TEST_OBJS earlier, and don't go back\n& forth in their definition. Before we'd first append $X to\n$(TEST_PROGRAMS), and then substitute $X back out of it to define\n$(TEST_OBJS). Let's instead add a new $(TEST_PROGRAM_OBJS) variable,\nwhich avoids this needless back & forth substitution.\n\nSee daa99a91729 (Makefile: make sure test helpers are rebuilt when\nheaders change, 2010-01-26) for how we ended up with the original\n$(TEST_OBJS).\n\nSigned-off-by: Ævar Arnfjörð Bjarmason \n---\n Makefile | 17 +++++++++--------\n 1 file changed, 9 insertions(+), 8 deletions(-)\n\ndiff --git a/Makefile b/Makefile\nindex 45b22d33513..a8cfa096dc1 100644\n--- a/Makefile\n+++ b/Makefile\n@@ -617,7 +617,8 @@ SCRIPT_PYTHON =\n SCRIPT_SH =\n SCRIPT_LIB =\n TEST_BUILTINS_OBJS =\n-TEST_OBJS =\n+TEST_PROGRAMS =\n+TEST_PROGRAM_OBJS =\n TEST_PROGRAMS_NEED_X =\n THIRD_PARTY_SOURCES =\n \n@@ -795,6 +796,7 @@ TEST_BUILTINS_OBJS += test-wildmatch.o\n TEST_BUILTINS_OBJS += test-windows-named-pipe.o\n TEST_BUILTINS_OBJS += test-write-cache.o\n TEST_BUILTINS_OBJS += test-xml-encode.o\n+TEST_PROGRAM_OBJS += $(patsubst %,t/helper/%,$(TEST_BUILTINS_OBJS))\n \n # Do not add more tests here unless they have extra dependencies. Add\n # them in TEST_BUILTINS_OBJS above.\n@@ -802,6 +804,9 @@ TEST_PROGRAMS_NEED_X += test-fake-ssh\n TEST_PROGRAMS_NEED_X += test-tool\n \n TEST_PROGRAMS = $(patsubst %,t/helper/%$X,$(TEST_PROGRAMS_NEED_X))\n+all:: $(TEST_PROGRAMS)\n+TEST_PROGRAM_OBJS += $(patsubst %,t/helper/%.o,$(TEST_PROGRAMS_NEED_X))\n+.PRECIOUS: $(TEST_PROGRAM_OBJS)\n \n # List built-in command $C whose implementation cmd_$C() is not in\n # builtin/$C.o but is linked in as part of some other command.\n@@ -2543,10 +2548,8 @@ REFTABLE_TEST_OBJS += reftable/stack_test.o\n REFTABLE_TEST_OBJS += reftable/test_framework.o\n REFTABLE_TEST_OBJS += reftable/tree_test.o\n \n-TEST_OBJS := $(patsubst %$X,%.o,$(TEST_PROGRAMS)) $(patsubst %,t/helper/%,$(TEST_BUILTINS_OBJS))\n-\n .PHONY: test-objs\n-test-objs: $(TEST_OBJS)\n+test-objs: $(TEST_PROGRAM_OBJS)\n \n GIT_OBJS += $(LIB_OBJS)\n GIT_OBJS += $(BUILTIN_OBJS)\n@@ -2562,7 +2565,7 @@ scalar-objs: $(SCALAR_OBJS)\n OBJECTS += $(GIT_OBJS)\n OBJECTS += $(SCALAR_OBJS)\n OBJECTS += $(PROGRAM_OBJS)\n-OBJECTS += $(TEST_OBJS)\n+OBJECTS += $(TEST_PROGRAM_OBJS)\n OBJECTS += $(XDIFF_OBJS)\n OBJECTS += $(FUZZ_OBJS)\n OBJECTS += $(REFTABLE_OBJS) $(REFTABLE_TEST_OBJS)\n@@ -3061,7 +3064,7 @@ endif\n \n test_bindir_programs := $(patsubst %,bin-wrappers/%,$(BINDIR_PROGRAMS_NEED_X) $(BINDIR_PROGRAMS_NO_X) $(TEST_PROGRAMS_NEED_X))\n \n-all:: $(TEST_PROGRAMS) $(test_bindir_programs)\n+all:: $(test_bindir_programs)\n \n bin-wrappers/%: wrap-for-bin.sh\n \t$(call mkdir_p_parent_template)\n@@ -3087,8 +3090,6 @@ perf: all\n \n .PHONY: test perf\n \n-.PRECIOUS: $(TEST_OBJS)\n-\n t/helper/test-tool$X: $(patsubst %,t/helper/%,$(TEST_BUILTINS_OBJS))\n \n t/helper/test-%$X: t/helper/test-%.o GIT-LDFLAGS $(GITLIBS) $(REFTABLE_TEST_LIB)\n-- \n2.38.0.1251.g3eefdfb5e7a\n\n"},{"id":"465807","messageId":"xmqqsfjailak.fsf@gitster.g","threadId":"58383","inReplyTo":"patch-v2-2.3-6dcb49f25c4-20221026T143534Z-avarab@gmail.com","subject":"Re: [PATCH v2 2/3] Makefile: define \"TEST_{PROGRAM,OBJS}\" variables earlier","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2022-10-26T16:47:47Z","receivedAt":"2022-10-26T16:47:53Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Ævar Arnfjörð Bjarmason writes:\n\n> Define the variables that make up TEST_OBJS earlier, and don't go back\n> & forth in their definition. Before we'd first append $X to\n> $(TEST_PROGRAMS), and then substitute $X back out of it to define\n> $(TEST_OBJS). Let's instead add a new $(TEST_PROGRAM_OBJS) variable,\n> which avoids this needless back & forth substitution.\n\nMakes sense, I guess. So TEST_OBJS is no longer used?\n\n> TEST_PROGRAMS = $(patsubst %,t/helper/%$X,$(TEST_PROGRAMS_NEED_X))\n> +all:: $(TEST_PROGRAMS)\n\nThis change is not necessary to achieve the stated goal of this\nstep, though. It is one of those \"while at it\" distraction that\nconsumes our already constrained reviewer bandwidth, no?\n\nHaving said that, \"all::\" being able to be built up with independent\npieces shine here in this split from the original. It probably is\neasier to reason about while seeing this isolated area of Makefile\nwhat is being done to TEST_PROGRAMS.\n\nThe rest of the patch is quite straight-forward renaming of\nTEST_OBJS to TEST_PROGRAM_OBJS and an improvement of how the\nelements on the list are computed from the source-of-truth list that\nis TEST_PROGRAMS_NEED_X that looks correct.\n\n> +TEST_PROGRAM_OBJS += $(patsubst %,t/helper/%.o,$(TEST_PROGRAMS_NEED_X))\n> +.PRECIOUS: $(TEST_PROGRAM_OBJS)\n\n> # List built-in command $C whose implementation cmd_$C() is not in\n> # builtin/$C.o but is linked in as part of some other command.\n> @@ -2543,10 +2548,8 @@ REFTABLE_TEST_OBJS += reftable/stack_test.o\n> REFTABLE_TEST_OBJS += reftable/test_framework.o\n> REFTABLE_TEST_OBJS += reftable/tree_test.o\n> \n> -TEST_OBJS := $(patsubst %$X,%.o,$(TEST_PROGRAMS)) $(patsubst %,t/helper/%,$(TEST_BUILTINS_OBJS))\n> -\n> .PHONY: test-objs\n> -test-objs: $(TEST_OBJS)\n> +test-objs: $(TEST_PROGRAM_OBJS)\n> \n> GIT_OBJS += $(LIB_OBJS)\n> GIT_OBJS += $(BUILTIN_OBJS)\n> @@ -2562,7 +2565,7 @@ scalar-objs: $(SCALAR_OBJS)\n> OBJECTS += $(GIT_OBJS)\n> OBJECTS += $(SCALAR_OBJS)\n> OBJECTS += $(PROGRAM_OBJS)\n> -OBJECTS += $(TEST_OBJS)\n> +OBJECTS += $(TEST_PROGRAM_OBJS)\n> OBJECTS += $(XDIFF_OBJS)\n> OBJECTS += $(FUZZ_OBJS)\n> OBJECTS += $(REFTABLE_OBJS) $(REFTABLE_TEST_OBJS)\n> @@ -3061,7 +3064,7 @@ endif\n> \n> test_bindir_programs := $(patsubst %,bin-wrappers/%,$(BINDIR_PROGRAMS_NEED_X) $(BINDIR_PROGRAMS_NO_X) $(TEST_PROGRAMS_NEED_X))\n> \n> -all:: $(TEST_PROGRAMS) $(test_bindir_programs)\n> +all:: $(test_bindir_programs)\n> \n> bin-wrappers/%: wrap-for-bin.sh\n> \t$(call mkdir_p_parent_template)\n> @@ -3087,8 +3090,6 @@ perf: all\n> \n> .PHONY: test perf\n> \n> -.PRECIOUS: $(TEST_OBJS)\n> -\n> t/helper/test-tool$X: $(patsubst %,t/helper/%,$(TEST_BUILTINS_OBJS))\n> \n> t/helper/test-%$X: t/helper/test-%.o GIT-LDFLAGS $(GITLIBS) $(REFTABLE_TEST_LIB)\n"},{"id":"465809","messageId":"xmqq1qquiibv.fsf@gitster.g","threadId":"58383","inReplyTo":"patch-v2-1.3-fc6c5a6a8df-20221026T143534Z-avarab@gmail.com","subject":"Re: [PATCH v2 1/3] Makefile: factor sed-powered '#!/bin/sh' munging into a variable","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2022-10-26T17:51:48Z","receivedAt":"2022-10-26T17:51:54Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Ævar Arnfjörð Bjarmason writes:\n\n> Reduce the amount of magical copy/pasting in the Makefile by factoring\n> the munging of \"#!/bin/sh\" on the first line of a shellscript into a\n> variable we can re-use in the various rules that need to do so.\n\nAt least when taken standalone, this looks more like replacing one\nmagical copy pasting with another magical one, the difference being\nthat the latter is not immediately obvious without referring back to\nthe definition of the variable.\n\nIf we need to change the replacement wholesale in a later step, then\nit might give us a good trade off, but otherwise I am not sure why\nthis is a good idea that is worth the churn. \n\nWhen adding or updating these actions, in the original, you can typo\nSHELL_PATH_SQ, but you can typo cmd_munge_script_sed_shell_path_arg\nin the updated to break it the same way. So...?\n\n> Signed-off-by: Ævar Arnfjörð Bjarmason \n> ---\n> Makefile | 12 ++++++++----\n> 1 file changed, 8 insertions(+), 4 deletions(-)\n>\n> diff --git a/Makefile b/Makefile\n> index 85f03c6aed1..45b22d33513 100644\n> --- a/Makefile\n> +++ b/Makefile\n> @@ -2344,8 +2344,12 @@ GIT-SCRIPT-DEFINES: FORCE\n> \t\techo \"$$FLAGS\" >$@; \\\n> fi\n> \n> +define cmd_munge_script_sed_shell_path_arg\n> +'1s|#!.*/sh|#!$(SHELL_PATH_SQ)|'\n> +endef\n> +\n> define cmd_munge_script\n> -sed -e '1s|#!.*/sh|#!$(SHELL_PATH_SQ)|' \\\n> +sed -e $(call cmd_munge_script_sed_shell_path_arg) \\\n> -e 's|@SHELL_PATH@|$(SHELL_PATH_SQ)|' \\\n> -e 's|@@DIFF@@|$(DIFF_SQ)|' \\\n> -e 's|@@LOCALEDIR@@|$(localedir_SQ)|g' \\\n> @@ -2447,7 +2451,7 @@ git-instaweb: git-instaweb.sh GIT-SCRIPT-DEFINES\n> else # NO_PERL\n> $(SCRIPT_PERL_GEN) git-instaweb: % : unimplemented.sh\n> \t$(QUIET_GEN) \\\n> -\tsed -e '1s|#!.*/sh|#!$(SHELL_PATH_SQ)|' \\\n> +\tsed -e $(call cmd_munge_script_sed_shell_path_arg) \\\n> \t -e 's|@@REASON@@|NO_PERL=$(NO_PERL)|g' \\\n> \t unimplemented.sh >$@+ && \\\n> \tchmod +x $@+ && \\\n> @@ -2468,7 +2472,7 @@ $(SCRIPT_PYTHON_GEN): % : %.py\n> else # NO_PYTHON\n> $(SCRIPT_PYTHON_GEN): % : unimplemented.sh\n> \t$(QUIET_GEN) \\\n> -\tsed -e '1s|#!.*/sh|#!$(SHELL_PATH_SQ)|' \\\n> +\tsed -e $(call cmd_munge_script_sed_shell_path_arg) \\\n> \t -e 's|@@REASON@@|NO_PYTHON=$(NO_PYTHON)|g' \\\n> \t unimplemented.sh >$@+ && \\\n> \tchmod +x $@+ && \\\n> @@ -3061,7 +3065,7 @@ all:: $(TEST_PROGRAMS) $(test_bindir_programs)\n> \n> bin-wrappers/%: wrap-for-bin.sh\n> \t$(call mkdir_p_parent_template)\n> -\t$(QUIET_GEN)sed -e '1s|#!.*/sh|#!$(SHELL_PATH_SQ)|' \\\n> +\t$(QUIET_GEN)sed -e $(call cmd_munge_script_sed_shell_path_arg) \\\n> \t -e 's|@@BUILD_DIR@@|$(shell pwd)|' \\\n> \t -e 's|@@PROG@@|$(patsubst test-%,t/helper/test-%,$(@F))$(if $(filter-out $(BINDIR_PROGRAMS_NO_X),$(@F)),$(X),)|' < $< > $@ && \\\n> \tchmod +x $@\n"},{"id":"465812","messageId":"xmqqr0yuh144.fsf@gitster.g","threadId":"58383","inReplyTo":"patch-v2-3.3-400f487e30d-20221026T143534Z-avarab@gmail.com","subject":"Re: [PATCH v2 3/3] Makefile: simplify $(test_bindir_programs) rule by splitting it up","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2022-10-26T18:48:59Z","receivedAt":"2022-10-26T18:49:06Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Ævar Arnfjörð Bjarmason writes:\n\n> -test_bindir_programs := $(patsubst %,bin-wrappers/%,$(BINDIR_PROGRAMS_NEED_X) $(BINDIR_PROGRAMS_NO_X) $(TEST_PROGRAMS_NEED_X))\n> +define cmd_munge_bin_wrappers_script\n> +sed \\\n> +\t-e $(call cmd_munge_script_sed_shell_path_arg) \\\n> +\t-e 's|@@BUILD_DIR@@|$(shell pwd)|' \\\n> +\t-e 's|@@PROG@@|$(2)$(1)$(3)|' \\\n> +\t<$< >$@ && \\\n> +\tchmod +x $@\n> +endef\n\nWe've talked about relying on another feature of GNU make (which we\nalready depend on) to automatically remove the target build artifact\nwhen a rule to build it fails, but I see quite a many old world best\npractice pattern \"generate >$@+ && chmod +x $@+ && mv $@+ $@\" still\nin today's Makefile. What happens when a rule that uses the above\nfails in the middle? Do we correctly remove the half-built target?\n\n> -all:: $(test_bindir_programs)\n> +define bin_wrappers_template\n> +\n> +## bin_wrappers_template\n> +# 1 = $(1)\n> +# 2 = $(2)\n> +# 3 = $(3)\n> +# 4 = $(4)\n\nWhatever the above comment wants to convey to readers, it seems to\nfail to do so at least to me, while ...\n\n> +BW_$(1) = $$($(1):%=bin-wrappers/%)\n> +BIN_WRAPPERS += $$(BW_$(1))\n> +all:: $$(BW_$(1))\n> +$$(BW_$(1)): bin-wrappers/% : $(3)%$(4)\n> +$$(BW_$(1)): wrap-for-bin.sh\n> +\t$$(call mkdir_p_parent_template)\n> +\t$$(QUIET_GEN)$$(call cmd_munge_bin_wrappers_script,$(2),$(3),$(4))\n> +endef\n> \n> +define bin_wrappers_templates\n> +$(call bin_wrappers_template,BINDIR_PROGRAMS_NEED_X,'$$(@F)',,$$X)\n> +$(call bin_wrappers_template,BINDIR_PROGRAMS_NO_X,'$$(@F)')\n> +$(call bin_wrappers_template,TEST_PROGRAMS_NEED_X,'$$(@F)',t/helper/,$$X)\n> +endef\n> +$(eval $(call bin_wrappers_templates))\n> +\n> +all:: $(BIN_WRAPPERS)\n\n... what it expands to looked quite straight-forward.\n"},{"id":"465814","messageId":"221026.86y1t2beoj.gmgdl@evledraar.gmail.com","threadId":"58383","inReplyTo":"xmqqsfjailak.fsf@gitster.g","subject":"Re: [PATCH v2 2/3] Makefile: define \"TEST_{PROGRAM,OBJS}\" variables earlier","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2022-10-26T18:47:43Z","receivedAt":"2022-10-26T18:52:37Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"\nOn Wed, Oct 26 2022, Junio C Hamano wrote:\n\n> Ævar Arnfjörð Bjarmason writes:\n>\n>> Define the variables that make up TEST_OBJS earlier, and don't go back\n>> & forth in their definition. Before we'd first append $X to\n>> $(TEST_PROGRAMS), and then substitute $X back out of it to define\n>> $(TEST_OBJS). Let's instead add a new $(TEST_PROGRAM_OBJS) variable,\n>> which avoids this needless back & forth substitution.\n>\n> Makes sense, I guess. So TEST_OBJS is no longer used?\n\nYes, sorry I'll clarify that in a re-roll.\n\n>> TEST_PROGRAMS = $(patsubst %,t/helper/%$X,$(TEST_PROGRAMS_NEED_X))\n>> +all:: $(TEST_PROGRAMS)\n>\n> This change is not necessary to achieve the stated goal of this\n> step, though. It is one of those \"while at it\" distraction that\n> consumes our already constrained reviewer bandwidth, no?\n\nI figured this would be better use of that bandwith, since the reviewer\ndoesn't need to wonder why these are still spread befo/after the main\nbody of the change.\n\nNot everyone is keenly aware of the at first odd way a Makefile is read\n(per \"3.7 How 'make' Reads a Makefile\" in the GNU make manual).\n\nBut I'm happy to eject this part if that helps, but...\n\n> Having said that, \"all::\" being able to be built up with independent\n> pieces shine here in this split from the original. It probably is\n> easier to reason about while seeing this isolated area of Makefile\n> what is being done to TEST_PROGRAMS.\n\n...here I'm not quite sure if you want to keep it after all or nat...\n\n> The rest of the patch is quite straight-forward renaming of\n> TEST_OBJS to TEST_PROGRAM_OBJS and an improvement of how the\n> elements on the list are computed from the source-of-truth list that\n> is TEST_PROGRAMS_NEED_X that looks correct.\n\nThanks for the quick review!\n"},{"id":"465817","messageId":"xmqqeduugzzd.fsf@gitster.g","threadId":"58383","inReplyTo":"221026.86y1t2beoj.gmgdl@evledraar.gmail.com","subject":"Re: [PATCH v2 2/3] Makefile: define \"TEST_{PROGRAM,OBJS}\" variables earlier","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2022-10-26T19:13:26Z","receivedAt":"2022-10-26T19:15:17Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Ævar Arnfjörð Bjarmason writes:\n\n> On Wed, Oct 26 2022, Junio C Hamano wrote:\n>\n>> Ævar Arnfjörð Bjarmason writes:\n>>\n>>> Define the variables that make up TEST_OBJS earlier, and don't go back\n>>> & forth in their definition. Before we'd first append $X to\n>>> $(TEST_PROGRAMS), and then substitute $X back out of it to define\n>>> $(TEST_OBJS). Let's instead add a new $(TEST_PROGRAM_OBJS) variable,\n>>> which avoids this needless back & forth substitution.\n>>\n>> Makes sense, I guess. So TEST_OBJS is no longer used?\n>\n> Yes, sorry I'll clarify that in a re-roll.\n\nIt invites the question why it needed to be renamed in the first\nplace, though. If we needed to name something that is different\nfrom the original TEST_OBJS, it makes perfect sense to add a\nvariable with a new name, but if it is not the case here, then...?\n\n>> Having said that, \"all::\" being able to be built up with independent\n>> pieces shine here in this split from the original. It probably is\n>> easier to reason about while seeing this isolated area of Makefile\n>> what is being done to TEST_PROGRAMS.\n>\n> ...here I'm not quite sure if you want to keep it after all or nat...\n\nI'll let it pass this time ;-) but my patience is not infinite.\n\n"},{"id":"465818","messageId":"221026.86tu3qbd03.gmgdl@evledraar.gmail.com","threadId":"58383","inReplyTo":"xmqqr0yuh144.fsf@gitster.g","subject":"Re: [PATCH v2 3/3] Makefile: simplify $(test_bindir_programs) rule by splitting it up","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2022-10-26T19:14:29Z","receivedAt":"2022-10-26T19:28:51Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"\nOn Wed, Oct 26 2022, Junio C Hamano wrote:\n\n> Ævar Arnfjörð Bjarmason writes:\n>\n>> -test_bindir_programs := $(patsubst %,bin-wrappers/%,$(BINDIR_PROGRAMS_NEED_X) $(BINDIR_PROGRAMS_NO_X) $(TEST_PROGRAMS_NEED_X))\n>> +define cmd_munge_bin_wrappers_script\n>> +sed \\\n>> +\t-e $(call cmd_munge_script_sed_shell_path_arg) \\\n>> +\t-e 's|@@BUILD_DIR@@|$(shell pwd)|' \\\n>> +\t-e 's|@@PROG@@|$(2)$(1)$(3)|' \\\n>> +\t<$< >$@ && \\\n>> +\tchmod +x $@\n>> +endef\n\n[Re-arranged your question a bit for the reply...]\n\n> We've talked about relying on another feature of GNU make (which we\n> already depend on) to automatically remove the target build artifact\n> when a rule to build it fails, [...] What happens when a rule that\n> uses the above fails in the middle? Do we correctly remove the\n> half-built target?\n\nAre you asking if \"define\"'s execute in some context outside the purview\nof rules, so that if you $(call) one from within a rule and it errors,\nthat we won't clean up the file?\n\nNo, that won't happen, they'll be cleaned up properly. First, these are\nbasically macros, but secondly it wouldn't matter if they\nweren't. E.g. you can execute an arbitrary program that happens to write\nto a file that matches your $@.\n\nThe only thing that matters is that we carry forward the exit code,\nwhich we do here. I.e. the \"&&\" after the $(call).\n\n(Even then we'd still error properly, unless you ran \"make -i\" to ignore\nerrors in the rule, the && is more in service of the $(QUIET_*) in our\nMakefile).\n\n> but I see quite a many old world best practice pattern \"generate >$@+\n> && chmod +x $@+ && mv $@+ $@\" still in today's Makefile.\n\nThose are mostly there because nobody's cleaned them up, we don't need\nthem anymore.\n\n\"Mostly\" because one thing they still do is not replace the file until\nit's ready, we had a previous discussion on a related topic in reply to\nhttps://lore.kernel.org/git/patch-1.6-3330cdbccc0-20210329T161723Z-avarab@gmail.com/\n(it didn't go in).\n\n>> -all:: $(test_bindir_programs)\n>> +define bin_wrappers_template\n>> +\n>> +## bin_wrappers_template\n>> +# 1 = $(1)\n>> +# 2 = $(2)\n>> +# 3 = $(3)\n>> +# 4 = $(4)\n>\n> Whatever the above comment wants to convey to readers, it seems to\n> fail to do so at least to me, while ...\n\nI can drop them, FWIW I've found it quite handy to add these to ad-hoc\ndebug templates. E.g. you can run:\n\t\n\t$ make -f /dev/null -E '$(eval $(file writes:\n\n> Are you asking if \"define\"'s execute in some context outside the purview\n> of rules, so that if you $(call) one from within a rule and it errors,\n> that we won't clean up the file?\n\nNot at all.\n\nI was wondering why the defined sequance does not end in\n\n\t...\n\t<$> >$@+ && \\\n\tchmod +x $@+ && \\\n\tmv $@+ $@\n\nlike many other command sequences in the Makefile. As I said, I did\nremember we had discussed delete-on-error; I just didn't recall if\nwe do depend on it already.\n\n> I can drop them, FWIW I've found it quite handy to add these to ad-hoc\n> debug templates. E.g. you can run:\n> \t\n> \t$ make -f /dev/null -E '$(eval $(file \tmake: *** \n> \t## bin_wrappers_template\n> \t# 1 = a\n> \t# 2 = b\n> \t# 3 = c\n> \t# 4 = d\n\nOK. That use pattern was what I was missing.\n> ...\n> So you see what the parameters expand to. Maybe just changing the\n> heading to:\n>\n> \t## bin_wrappers_template: $(1..N) below for manual \"$(error $(call ...\" deubgging\n\nYeah, it would be totally useless without such an instruction.\n"},{"id":"465830","messageId":"221026.86h6zqb9cu.gmgdl@evledraar.gmail.com","threadId":"58383","inReplyTo":"xmqq5yg6gwhs.fsf@gitster.g","subject":"Re: [PATCH v2 3/3] Makefile: simplify $(test_bindir_programs) rule by splitting it up","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2022-10-26T20:43:50Z","receivedAt":"2022-10-26T20:47:36Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"\nOn Wed, Oct 26 2022, Junio C Hamano wrote:\n\n> Ævar Arnfjörð Bjarmason writes:\n>\n>> Are you asking if \"define\"'s execute in some context outside the purview\n>> of rules, so that if you $(call) one from within a rule and it errors,\n>> that we won't clean up the file?\n>\n> Not at all.\n>\n> I was wondering why the defined sequance does not end in\n>\n> \t...\n> \t<$> >$@+ && \\\n> \tchmod +x $@+ && \\\n> \tmv $@+ $@\n>\n> like many other command sequences in the Makefile. As I said, I did\n> remember we had discussed delete-on-error; I just didn't recall if\n> we do depend on it already.\n\nAh! For the purposes of this patch the actual answer to that is that\nit's just moving existing code around.\n\nSo this happens to be the pattern we should prefer with\n.DELETE_ON_ERROR before & after 3/3.\n\nBut if it was doing that \"$@+ $@\" mv dance I'd have just blindly\nretained that too, to avoid \"while at it\" and all that...\n\nThe code I touched in 1/3 uses the \"mv $@+ $@\" for no good\npost-.DELETE_ON_ERROR reason, but I just left that in place as-is...\n\n>> I can drop them, FWIW I've found it quite handy to add these to ad-hoc\n>> debug templates. E.g. you can run:\n>> \t\n>> \t$ make -f /dev/null -E '$(eval $(file > \tmake: *** \n>> \t## bin_wrappers_template\n>> \t# 1 = a\n>> \t# 2 = b\n>> \t# 3 = c\n>> \t# 4 = d\n>\n> OK. That use pattern was what I was missing.\n>> ...\n>> So you see what the parameters expand to. Maybe just changing the\n>> heading to:\n>>\n>> \t## bin_wrappers_template: $(1..N) below for manual \"$(error $(call ...\" deubgging\n>\n> Yeah, it would be totally useless without such an instruction.\n\nWill update it with that.\n"},{"id":"465923","messageId":"Y1so+kU1sZqYAS4y@coredump.intra.peff.net","threadId":"58383","inReplyTo":"cover-v2-0.3-00000000000-20221026T143533Z-avarab@gmail.com","subject":"Re: [PATCH v2 0/3] Makefile: fix issues with bin-wrappers/% rule","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2022-10-28T00:57:30Z","receivedAt":"2022-10-28T00:57:34Z","isPatch":true,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Wed, Oct 26, 2022 at 04:42:34PM +0200, Ævar Arnfjörð Bjarmason wrote:\n\n> This simple topic fixes issues with the bin-wrappers/% rules, as seen\n> in the range-diff below this never worked properly:\n> \n> \tmake bin-wrappers/git\n> \n> I.e. we'd make the scirpt, but not \"git\".\n\nMaybe I'm missing something, but the current behavior is what I'd expect\nto happen. Sure, bin-wrappers/git depends on \"git\" at run-time if you\nwant it to do something useful. But AFAIK it doesn't at build-time. Why\ndo we want a build-time dependency? Are we expecting rules to depend on\nbin-wrappers/foo but not also on foo? Or people to ask manually for\nthose targets?\n\nThis kind of feels like making git-repack depend on git-pack-objects.\nOne will call the other at run-time, but there's no reason the builds\nhave to be together. And if you try to build and run one without\nbuilding the other, well...if it hurts, don't do it.\n\n-Peff\n"},{"id":"465927","messageId":"221028.86o7tw8wyd.gmgdl@evledraar.gmail.com","threadId":"58383","inReplyTo":"Y1so+kU1sZqYAS4y@coredump.intra.peff.net","subject":"Re: [PATCH v2 0/3] Makefile: fix issues with bin-wrappers/% rule","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2022-10-28T03:03:33Z","receivedAt":"2022-10-28T03:10:45Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"\nOn Thu, Oct 27 2022, Jeff King wrote:\n\n> On Wed, Oct 26, 2022 at 04:42:34PM +0200, Ævar Arnfjörð Bjarmason wrote:\n>\n>> This simple topic fixes issues with the bin-wrappers/% rules, as seen\n>> in the range-diff below this never worked properly:\n>> \n>> \tmake bin-wrappers/git\n>> \n>> I.e. we'd make the scirpt, but not \"git\".\n>\n> Maybe I'm missing something, but the current behavior is what I'd expect\n> to happen. Sure, bin-wrappers/git depends on \"git\" at run-time if you\n> want it to do something useful. But AFAIK it doesn't at build-time. Why\n> do we want a build-time dependency? Are we expecting rules to depend on\n> bin-wrappers/foo but not also on foo? Or people to ask manually for\n> those targets?\n\nI didn't have an immediate practical reason for it beyond just not\nleaving holes in the dependency DAG.\n\nBut yes, nothing is only building bin-wrappers/% now, so nothing needs\nthis. It just helps to not have to worry about Makefile targets not\nneeding other targets to work.\n\n> This kind of feels like making git-repack depend on git-pack-objects.\n> One will call the other at run-time, but there's no reason the builds\n> have to be together. And if you try to build and run one without\n> building the other, well...if it hurts, don't do it.\n\nOr wget depending on your router being plugged in? Beacuse that's what\nyou use it for?:)\n\nI get the general point, but I don't think these are comparable to those\nsorts of loose or semi-loose dependencies. For repack it'll service up\ne.g. \"-h\" without pack-objects, at least that's something...\n\nWhereas these scripts all end with invoking the binary corresponding to\ntheir name, so they really are useless without that target. So I think\nit makes sense to make it a dependency of the target in the Makefile.\n"},{"id":"466123","messageId":"cover-v3-0.4-00000000000-20221031T222249Z-avarab@gmail.com","threadId":"58383","inReplyTo":"cover-v2-0.3-00000000000-20221026T143533Z-avarab@gmail.com","subject":"[PATCH v3 0/4] Makefile: untangle bin-wrappers/% rule complexity","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2022-10-31T22:28:05Z","receivedAt":"2022-10-31T22:28:47Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"This series untangles the gnarly rule we use to generate\nbin-wrappers/%.\n\nIt's now complex because we generate it from 3x separate variables,\nand then end up having to do inline pattern matching to decide which\nvalue comes from where.\n\nInstead, we can avoid squashing those values together, so we don't\nhave to guess.\n\nSee[1] for the v2. This hopefully addresses all outstanding\nissues/points that were raised. An added 3/4 here makes the eventual\n4/4 smaler.\n\n1. https://lore.kernel.org/git/cover-v2-0.3-00000000000-20221026T143533Z-avarab@gmail.com/\n\nÆvar Arnfjörð Bjarmason (4):\n Makefile: factor sed-powered '#!/bin/sh' munging into a variable\n Makefile: define \"TEST_{PROGRAM,OBJS}\" variables earlier\n Makefile: rename \"test_bindir_programs\" variable, pre-declare\n Makefile: simplify $(test_bindir_programs) rule by splitting it up\n\n Makefile | 70 +++++++++++++++++++++++++++++++++++++++++---------------\n 1 file changed, 51 insertions(+), 19 deletions(-)\n\nRange-diff against v2:\n1: fc6c5a6a8df = 1: c9ce5b78a3a Makefile: factor sed-powered '#!/bin/sh' munging into a variable\n2: 6dcb49f25c4 = 2: b95c296b6de Makefile: define \"TEST_{PROGRAM,OBJS}\" variables earlier\n-: ----------- > 3: fea93c45898 Makefile: rename \"test_bindir_programs\" variable, pre-declare\n3: 400f487e30d ! 4: 0ff09495476 Makefile: simplify $(test_bindir_programs) rule by splitting it up\n @@ Commit message\n Which will show an empty diff, i.e. we've correctly dealt with the\n combination of $(SHELL_PATH), $(X) and these three variables here.\n \n - This also fixes an issue with the \"bin-wrappers/\" scripts have never had properly declared\n - dependency information, i.e. this has never worked:\n + This also fixes an issue with the \"bin-wrappers/\" scripts have never\n + had properly declared dependency information, i.e. this has never\n + worked:\n \n make clean &&\n make bin-wrappers/git &&\n @@ Commit message\n Signed-off-by: Ævar Arnfjörð Bjarmason \n \n ## Makefile ##\n -@@ Makefile: export TCL_PATH TCLTK_PATH\n - PTHREAD_LIBS = -lpthread\n - \n - # Guard against environment variables\n -+BIN_WRAPPERS =\n - BUILTIN_OBJS =\n - BUILT_INS =\n - COMPAT_CFLAGS =\n @@ Makefile: GIT-PYTHON-VARS: FORCE\n fi\n endif\n \n --test_bindir_programs := $(patsubst %,bin-wrappers/%,$(BINDIR_PROGRAMS_NEED_X) $(BINDIR_PROGRAMS_NO_X) $(TEST_PROGRAMS_NEED_X))\n +-BIN_WRAPPERS = $(patsubst %,bin-wrappers/%,$(BINDIR_PROGRAMS_NEED_X) $(BINDIR_PROGRAMS_NO_X) $(TEST_PROGRAMS_NEED_X))\n +define cmd_munge_bin_wrappers_script\n +sed \\\n +\t-e $(call cmd_munge_script_sed_shell_path_arg) \\\n @@ Makefile: GIT-PYTHON-VARS: FORCE\n +\tchmod +x $@\n +endef\n \n --all:: $(test_bindir_programs)\n +-all:: $(BIN_WRAPPERS)\n +define bin_wrappers_template\n +\n -+## bin_wrappers_template\n -+# 1 = $(1)\n -+# 2 = $(2)\n -+# 3 = $(3)\n -+# 4 = $(4)\n ++### bin_wrappers_template; Parameters:\n ++## E.g. \"BINDIR_PROGRAMS_NEED_X\": Variable reference\n ++# 1='$(1)'\n ++## E.g. \"$(@F)\": Passed as $$(1)) to \"cmd_munge_bin_wrappers_script\"\n ++# 2='$(2)'\n ++## E.g. \"\" or \"t/helper\": Directory prefix for the wrapped binary\n ++# 3='$(3)'\n ++## E.g. \"\" or \"$$X\": If $$X: wrapped binary needs X=.exe (for Windows)\n ++# 4='$(4)'\n +BW_$(1) = $$($(1):%=bin-wrappers/%)\n +BIN_WRAPPERS += $$(BW_$(1))\n -+all:: $$(BW_$(1))\n +$$(BW_$(1)): bin-wrappers/% : $(3)%$(4)\n +$$(BW_$(1)): wrap-for-bin.sh\n +\t$$(call mkdir_p_parent_template)\n @@ Makefile: GIT-PYTHON-VARS: FORCE\n \n # GNU make supports exporting all variables by \"export\" without parameters.\n # However, the environment gets quite big, and some programs have problems\n -@@ Makefile: OTHER_PROGRAMS += $(shell echo *.dll t/helper/*.dll)\n - endif\n - \n - artifacts-tar:: $(ALL_COMMANDS_TO_INSTALL) $(SCRIPT_LIB) $(OTHER_PROGRAMS) \\\n --\t\tGIT-BUILD-OPTIONS $(TEST_PROGRAMS) $(test_bindir_programs) \\\n -+\t\tGIT-BUILD-OPTIONS $(TEST_PROGRAMS) $(BIN_WRAPPERS) \\\n - \t\t$(MOFILES)\n - \t$(QUIET_SUBDIR0)templates $(QUIET_SUBDIR1) \\\n - \t\tSHELL_PATH='$(SHELL_PATH_SQ)' PERL_PATH='$(PERL_PATH_SQ)'\n-- \n2.38.0.1280.g8136eb6fab2\n\n"},{"id":"466124","messageId":"patch-v3-1.4-c9ce5b78a3a-20221031T222249Z-avarab@gmail.com","threadId":"58383","inReplyTo":"cover-v3-0.4-00000000000-20221031T222249Z-avarab@gmail.com","subject":"[PATCH v3 1/4] Makefile: factor sed-powered '#!/bin/sh' munging into a variable","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2022-10-31T22:28:06Z","receivedAt":"2022-10-31T22:28:49Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"Reduce the amount of magical copy/pasting in the Makefile by factoring\nthe munging of \"#!/bin/sh\" on the first line of a shellscript into a\nvariable we can re-use in the various rules that need to do so.\n\nSigned-off-by: Ævar Arnfjörð Bjarmason \n---\n Makefile | 12 ++++++++----\n 1 file changed, 8 insertions(+), 4 deletions(-)\n\ndiff --git a/Makefile b/Makefile\nindex 4927379184c..679c24377f4 100644\n--- a/Makefile\n+++ b/Makefile\n@@ -2347,8 +2347,12 @@ GIT-SCRIPT-DEFINES: FORCE\n \t\techo \"$$FLAGS\" >$@; \\\n fi\n \n+define cmd_munge_script_sed_shell_path_arg\n+'1s|#!.*/sh|#!$(SHELL_PATH_SQ)|'\n+endef\n+\n define cmd_munge_script\n-sed -e '1s|#!.*/sh|#!$(SHELL_PATH_SQ)|' \\\n+sed -e $(call cmd_munge_script_sed_shell_path_arg) \\\n -e 's|@SHELL_PATH@|$(SHELL_PATH_SQ)|' \\\n -e 's|@@DIFF@@|$(DIFF_SQ)|' \\\n -e 's|@@LOCALEDIR@@|$(localedir_SQ)|g' \\\n@@ -2450,7 +2454,7 @@ git-instaweb: git-instaweb.sh GIT-SCRIPT-DEFINES\n else # NO_PERL\n $(SCRIPT_PERL_GEN) git-instaweb: % : unimplemented.sh\n \t$(QUIET_GEN) \\\n-\tsed -e '1s|#!.*/sh|#!$(SHELL_PATH_SQ)|' \\\n+\tsed -e $(call cmd_munge_script_sed_shell_path_arg) \\\n \t -e 's|@@REASON@@|NO_PERL=$(NO_PERL)|g' \\\n \t unimplemented.sh >$@+ && \\\n \tchmod +x $@+ && \\\n@@ -2471,7 +2475,7 @@ $(SCRIPT_PYTHON_GEN): % : %.py\n else # NO_PYTHON\n $(SCRIPT_PYTHON_GEN): % : unimplemented.sh\n \t$(QUIET_GEN) \\\n-\tsed -e '1s|#!.*/sh|#!$(SHELL_PATH_SQ)|' \\\n+\tsed -e $(call cmd_munge_script_sed_shell_path_arg) \\\n \t -e 's|@@REASON@@|NO_PYTHON=$(NO_PYTHON)|g' \\\n \t unimplemented.sh >$@+ && \\\n \tchmod +x $@+ && \\\n@@ -3065,7 +3069,7 @@ all:: $(TEST_PROGRAMS) $(test_bindir_programs)\n \n bin-wrappers/%: wrap-for-bin.sh\n \t$(call mkdir_p_parent_template)\n-\t$(QUIET_GEN)sed -e '1s|#!.*/sh|#!$(SHELL_PATH_SQ)|' \\\n+\t$(QUIET_GEN)sed -e $(call cmd_munge_script_sed_shell_path_arg) \\\n \t -e 's|@@BUILD_DIR@@|$(shell pwd)|' \\\n \t -e 's|@@PROG@@|$(patsubst test-%,t/helper/test-%,$(@F))$(if $(filter-out $(BINDIR_PROGRAMS_NO_X),$(@F)),$(X),)|' < $< > $@ && \\\n \tchmod +x $@\n-- \n2.38.0.1280.g8136eb6fab2\n\n"},{"id":"466125","messageId":"patch-v3-3.4-fea93c45898-20221031T222249Z-avarab@gmail.com","threadId":"58383","inReplyTo":"cover-v3-0.4-00000000000-20221031T222249Z-avarab@gmail.com","subject":"[PATCH v3 3/4] Makefile: rename \"test_bindir_programs\" variable, pre-declare","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2022-10-31T22:28:08Z","receivedAt":"2022-10-31T22:28:51Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"Pre-declare the \"BIN-WRAPPERS\" variable, renamed from\n\"test_bindir_programs\", which is more in line with our usual naming\nconventions in the Makefile.\n\nThere's also no reason to use a \"simply expanded variable\" here, so\nlet's use a normal \"=\" assignment. It doesn't make a difference in the\nend as we're about to use to to define a rule, so it'll have\n\"IMMEDIATE\" expansion here.\n\nSee ea925196f1b (build dashless \"bin-wrappers\" directory similar to\ninstalled bindir, 2009-12-02) for the introduction of\n\"test_bindir_programs\".\n\nSigned-off-by: Ævar Arnfjörð Bjarmason \n---\n Makefile | 7 ++++---\n 1 file changed, 4 insertions(+), 3 deletions(-)\n\ndiff --git a/Makefile b/Makefile\nindex 911cce6d487..032654640c5 100644\n--- a/Makefile\n+++ b/Makefile\n@@ -595,6 +595,7 @@ export TCL_PATH TCLTK_PATH\n PTHREAD_LIBS = -lpthread\n \n # Guard against environment variables\n+BIN_WRAPPERS =\n BUILTIN_OBJS =\n BUILT_INS =\n COMPAT_CFLAGS =\n@@ -3066,9 +3067,9 @@ GIT-PYTHON-VARS: FORCE\n fi\n endif\n \n-test_bindir_programs := $(patsubst %,bin-wrappers/%,$(BINDIR_PROGRAMS_NEED_X) $(BINDIR_PROGRAMS_NO_X) $(TEST_PROGRAMS_NEED_X))\n+BIN_WRAPPERS = $(patsubst %,bin-wrappers/%,$(BINDIR_PROGRAMS_NEED_X) $(BINDIR_PROGRAMS_NO_X) $(TEST_PROGRAMS_NEED_X))\n \n-all:: $(test_bindir_programs)\n+all:: $(BIN_WRAPPERS)\n \n bin-wrappers/%: wrap-for-bin.sh\n \t$(call mkdir_p_parent_template)\n@@ -3408,7 +3409,7 @@ OTHER_PROGRAMS += $(shell echo *.dll t/helper/*.dll)\n endif\n \n artifacts-tar:: $(ALL_COMMANDS_TO_INSTALL) $(SCRIPT_LIB) $(OTHER_PROGRAMS) \\\n-\t\tGIT-BUILD-OPTIONS $(TEST_PROGRAMS) $(test_bindir_programs) \\\n+\t\tGIT-BUILD-OPTIONS $(TEST_PROGRAMS) $(BIN_WRAPPERS) \\\n \t\t$(MOFILES)\n \t$(QUIET_SUBDIR0)templates $(QUIET_SUBDIR1) \\\n \t\tSHELL_PATH='$(SHELL_PATH_SQ)' PERL_PATH='$(PERL_PATH_SQ)'\n-- \n2.38.0.1280.g8136eb6fab2\n\n"},{"id":"466126","messageId":"patch-v3-2.4-b95c296b6de-20221031T222249Z-avarab@gmail.com","threadId":"58383","inReplyTo":"cover-v3-0.4-00000000000-20221031T222249Z-avarab@gmail.com","subject":"[PATCH v3 2/4] Makefile: define \"TEST_{PROGRAM,OBJS}\" variables earlier","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2022-10-31T22:28:07Z","receivedAt":"2022-10-31T22:28:52Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"Define the variables that make up TEST_OBJS earlier, and don't go back\n& forth in their definition. Before we'd first append $X to\n$(TEST_PROGRAMS), and then substitute $X back out of it to define\n$(TEST_OBJS). Let's instead add a new $(TEST_PROGRAM_OBJS) variable,\nwhich avoids this needless back & forth substitution.\n\nSee daa99a91729 (Makefile: make sure test helpers are rebuilt when\nheaders change, 2010-01-26) for how we ended up with the original\n$(TEST_OBJS).\n\nSigned-off-by: Ævar Arnfjörð Bjarmason \n---\n Makefile | 17 +++++++++--------\n 1 file changed, 9 insertions(+), 8 deletions(-)\n\ndiff --git a/Makefile b/Makefile\nindex 679c24377f4..911cce6d487 100644\n--- a/Makefile\n+++ b/Makefile\n@@ -617,7 +617,8 @@ SCRIPT_PYTHON =\n SCRIPT_SH =\n SCRIPT_LIB =\n TEST_BUILTINS_OBJS =\n-TEST_OBJS =\n+TEST_PROGRAMS =\n+TEST_PROGRAM_OBJS =\n TEST_PROGRAMS_NEED_X =\n THIRD_PARTY_SOURCES =\n \n@@ -796,6 +797,7 @@ TEST_BUILTINS_OBJS += test-wildmatch.o\n TEST_BUILTINS_OBJS += test-windows-named-pipe.o\n TEST_BUILTINS_OBJS += test-write-cache.o\n TEST_BUILTINS_OBJS += test-xml-encode.o\n+TEST_PROGRAM_OBJS += $(patsubst %,t/helper/%,$(TEST_BUILTINS_OBJS))\n \n # Do not add more tests here unless they have extra dependencies. Add\n # them in TEST_BUILTINS_OBJS above.\n@@ -803,6 +805,9 @@ TEST_PROGRAMS_NEED_X += test-fake-ssh\n TEST_PROGRAMS_NEED_X += test-tool\n \n TEST_PROGRAMS = $(patsubst %,t/helper/%$X,$(TEST_PROGRAMS_NEED_X))\n+all:: $(TEST_PROGRAMS)\n+TEST_PROGRAM_OBJS += $(patsubst %,t/helper/%.o,$(TEST_PROGRAMS_NEED_X))\n+.PRECIOUS: $(TEST_PROGRAM_OBJS)\n \n # List built-in command $C whose implementation cmd_$C() is not in\n # builtin/$C.o but is linked in as part of some other command.\n@@ -2546,10 +2551,8 @@ REFTABLE_TEST_OBJS += reftable/stack_test.o\n REFTABLE_TEST_OBJS += reftable/test_framework.o\n REFTABLE_TEST_OBJS += reftable/tree_test.o\n \n-TEST_OBJS := $(patsubst %$X,%.o,$(TEST_PROGRAMS)) $(patsubst %,t/helper/%,$(TEST_BUILTINS_OBJS))\n-\n .PHONY: test-objs\n-test-objs: $(TEST_OBJS)\n+test-objs: $(TEST_PROGRAM_OBJS)\n \n GIT_OBJS += $(LIB_OBJS)\n GIT_OBJS += $(BUILTIN_OBJS)\n@@ -2565,7 +2568,7 @@ scalar-objs: $(SCALAR_OBJS)\n OBJECTS += $(GIT_OBJS)\n OBJECTS += $(SCALAR_OBJS)\n OBJECTS += $(PROGRAM_OBJS)\n-OBJECTS += $(TEST_OBJS)\n+OBJECTS += $(TEST_PROGRAM_OBJS)\n OBJECTS += $(XDIFF_OBJS)\n OBJECTS += $(FUZZ_OBJS)\n OBJECTS += $(REFTABLE_OBJS) $(REFTABLE_TEST_OBJS)\n@@ -3065,7 +3068,7 @@ endif\n \n test_bindir_programs := $(patsubst %,bin-wrappers/%,$(BINDIR_PROGRAMS_NEED_X) $(BINDIR_PROGRAMS_NO_X) $(TEST_PROGRAMS_NEED_X))\n \n-all:: $(TEST_PROGRAMS) $(test_bindir_programs)\n+all:: $(test_bindir_programs)\n \n bin-wrappers/%: wrap-for-bin.sh\n \t$(call mkdir_p_parent_template)\n@@ -3091,8 +3094,6 @@ perf: all\n \n .PHONY: test perf\n \n-.PRECIOUS: $(TEST_OBJS)\n-\n t/helper/test-tool$X: $(patsubst %,t/helper/%,$(TEST_BUILTINS_OBJS))\n \n t/helper/test-%$X: t/helper/test-%.o GIT-LDFLAGS $(GITLIBS) $(REFTABLE_TEST_LIB)\n-- \n2.38.0.1280.g8136eb6fab2\n\n"},{"id":"466127","messageId":"patch-v3-4.4-0ff09495476-20221031T222249Z-avarab@gmail.com","threadId":"58383","inReplyTo":"cover-v3-0.4-00000000000-20221031T222249Z-avarab@gmail.com","subject":"[PATCH v3 4/4] Makefile: simplify $(test_bindir_programs) rule by splitting it up","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2022-10-31T22:28:09Z","receivedAt":"2022-10-31T22:28:55Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"When the @@PROG@@ substitution was added in [1] it was a simple matter\nof doing a:\n\n\t's|@@PROG@@|$(@F)|'\n\nThen when t/helpers were added in [2] followed by the \".exe\" needing\nto be appended in [3] this previously simple rule ended up as a dense\none-liner.\n\nIt has been pointed out that this is hard to read[4], but the problem\nisn't the \"design of the Makefile syntax\". It's we now have to\nafter-the-fact determine if we were dealing with a bin-wrapper/ that\nneeded to have $(X) appended, a t/helper/, or a non-binary (currently\nonly git-cvsserver).\n\nThat would be a problem in any language. We're starting out with three\nlists, and then end up having to heuristically determine given a\nmember of any of those lists which list that member came from. Let's\njust stop doing that and keep track of what member belongs to which\nlist.\n\nWe can do this by splitting up the single \"bin-wrappers/%\" rule into a\nrule for each of the three lists. With the\n\"cmd_munge_script_sed_shell_path_arg\" define added in a preceding\ncommit this is easy, we just need to add a sister template to the\nexisting \"cmd_munge_script\" added in [5].\n\nThe \"filter-out\" added in [6] has now become unnecessary per the\nabove, it was an artifact of us losing track of what was in which of\nour lists to begin with.\n\nThis change can be tested with e.g.:\n\n\tgit checkout master &&\n\tmake SHELL_PATH=/bin/bash X=.exe &&\n\tmv bin-wrappers bin-wrappers.master &&\n\t &&\n\tmake SHELL_PATH=/bin/bash X=.exe &&\n\tdiff -ru bin-wrappers{.master,}\n\nWhich will show an empty diff, i.e. we've correctly dealt with the\ncombination of $(SHELL_PATH), $(X) and these three variables here.\n\nThis also fixes an issue with the \"bin-wrappers/\" scripts have never\nhad properly declared dependency information, i.e. this has never\nworked:\n\n\tmake clean &&\n\tmake bin-wrappers/git &&\n\t# the script is there, but no \"./git\" is built\n\t./bin-wrappers/git\n\nThere was no reason not to have that work, just as most things\ngenerated by the Makefile have proper dependency information.\n\nBefore this commit doing this would have been painful, but now it's\neasy to pass this as a parameter to our \"bin_wrappers_template\"\n\n1. ea925196f1b (build dashless \"bin-wrappers\" directory similar to\n installed bindir, 2009-12-02)\n2. e6e7530d10b (test helpers: move test-* to t/helper/ subdirectory,\n 2016-04-13)\n3. 3a94cb31d52 (bin-wrappers: append `.exe` to target paths if\n necessary, 2019-07-29)\n4. https://lore.kernel.org/git/sso99so6-n28s-rq86-8q20-4456r3pn869r@tzk.qr/\n5. 46bac904581 (Do not install shell libraries executable, 2010-01-31)\n6. 7b5c93c6c68\" (scalar: include in standard Git build & installation,\n 2022-09-02)\n\nSigned-off-by: Ævar Arnfjörð Bjarmason \n---\n Makefile | 42 ++++++++++++++++++++++++++++++++++--------\n 1 file changed, 34 insertions(+), 8 deletions(-)\n\ndiff --git a/Makefile b/Makefile\nindex 032654640c5..d7ab68e3db8 100644\n--- a/Makefile\n+++ b/Makefile\n@@ -3067,16 +3067,42 @@ GIT-PYTHON-VARS: FORCE\n fi\n endif\n \n-BIN_WRAPPERS = $(patsubst %,bin-wrappers/%,$(BINDIR_PROGRAMS_NEED_X) $(BINDIR_PROGRAMS_NO_X) $(TEST_PROGRAMS_NEED_X))\n+define cmd_munge_bin_wrappers_script\n+sed \\\n+\t-e $(call cmd_munge_script_sed_shell_path_arg) \\\n+\t-e 's|@@BUILD_DIR@@|$(shell pwd)|' \\\n+\t-e 's|@@PROG@@|$(2)$(1)$(3)|' \\\n+\t<$< >$@ && \\\n+\tchmod +x $@\n+endef\n \n-all:: $(BIN_WRAPPERS)\n+define bin_wrappers_template\n+\n+### bin_wrappers_template; Parameters:\n+## E.g. \"BINDIR_PROGRAMS_NEED_X\": Variable reference\n+# 1='$(1)'\n+## E.g. \"$(@F)\": Passed as $$(1)) to \"cmd_munge_bin_wrappers_script\"\n+# 2='$(2)'\n+## E.g. \"\" or \"t/helper\": Directory prefix for the wrapped binary\n+# 3='$(3)'\n+## E.g. \"\" or \"$$X\": If $$X: wrapped binary needs X=.exe (for Windows)\n+# 4='$(4)'\n+BW_$(1) = $$($(1):%=bin-wrappers/%)\n+BIN_WRAPPERS += $$(BW_$(1))\n+$$(BW_$(1)): bin-wrappers/% : $(3)%$(4)\n+$$(BW_$(1)): wrap-for-bin.sh\n+\t$$(call mkdir_p_parent_template)\n+\t$$(QUIET_GEN)$$(call cmd_munge_bin_wrappers_script,$(2),$(3),$(4))\n+endef\n \n-bin-wrappers/%: wrap-for-bin.sh\n-\t$(call mkdir_p_parent_template)\n-\t$(QUIET_GEN)sed -e $(call cmd_munge_script_sed_shell_path_arg) \\\n-\t -e 's|@@BUILD_DIR@@|$(shell pwd)|' \\\n-\t -e 's|@@PROG@@|$(patsubst test-%,t/helper/test-%,$(@F))$(if $(filter-out $(BINDIR_PROGRAMS_NO_X),$(@F)),$(X),)|' < $< > $@ && \\\n-\tchmod +x $@\n+define bin_wrappers_templates\n+$(call bin_wrappers_template,BINDIR_PROGRAMS_NEED_X,'$$(@F)',,$$X)\n+$(call bin_wrappers_template,BINDIR_PROGRAMS_NO_X,'$$(@F)')\n+$(call bin_wrappers_template,TEST_PROGRAMS_NEED_X,'$$(@F)',t/helper/,$$X)\n+endef\n+$(eval $(call bin_wrappers_templates))\n+\n+all:: $(BIN_WRAPPERS)\n \n # GNU make supports exporting all variables by \"export\" without parameters.\n # However, the environment gets quite big, and some programs have problems\n-- \n2.38.0.1280.g8136eb6fab2\n\n"},{"id":"466140","messageId":"Y2BgRU+HnJjWPzkn@nand.local","threadId":"58383","inReplyTo":"cover-v3-0.4-00000000000-20221031T222249Z-avarab@gmail.com","subject":"Re: [PATCH v3 0/4] Makefile: untangle bin-wrappers/% rule complexity","fromName":"Taylor Blau","fromEmail":"me@ttaylorr.com","sentAt":"2022-10-31T23:54:45Z","receivedAt":"2022-10-31T23:54:50Z","isPatch":true,"sender":{"key":"me@ttaylorr.com","avatar":"https://avatars.githubusercontent.com/u/301000140?v=4"},"body":"On Mon, Oct 31, 2022 at 11:28:05PM +0100, Ævar Arnfjörð Bjarmason wrote:\n> Ævar Arnfjörð Bjarmason (4):\n> Makefile: factor sed-powered '#!/bin/sh' munging into a variable\n> Makefile: define \"TEST_{PROGRAM,OBJS}\" variables earlier\n> Makefile: rename \"test_bindir_programs\" variable, pre-declare\n> Makefile: simplify $(test_bindir_programs) rule by splitting it up\n>\n> Makefile | 70 +++++++++++++++++++++++++++++++++++++++++---------------\n> 1 file changed, 51 insertions(+), 19 deletions(-)\n\nThanks. I replaced the earlier round with this one and pushed the result\nout to ttaylorr/git.\n\nBut having read the topic over again, I have to say that I also find the\npre-existing behavior to be as expected. \"make bin-wrappers/git\"\nproduces its target as expected, but the target is useless because the\nrelationship it has with \"git\" is a runtime dependency, not a Make-time\ndependency.\n\nSo I'm not inclined to pick this one up, honestly. Perhaps other\nreviewers feel differently.\n\n\nThanks,\nTaylor\n"},{"id":"466164","messageId":"221101.86edun5tgn.gmgdl@evledraar.gmail.com","threadId":"58383","inReplyTo":"Y2BgRU+HnJjWPzkn@nand.local","subject":"Re: [PATCH v3 0/4] Makefile: untangle bin-wrappers/% rule complexity","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2022-11-01T01:29:40Z","receivedAt":"2022-11-01T01:55:44Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"\nOn Mon, Oct 31 2022, Taylor Blau wrote:\n\n> On Mon, Oct 31, 2022 at 11:28:05PM +0100, Ævar Arnfjörð Bjarmason wrote:\n>> Ævar Arnfjörð Bjarmason (4):\n>> Makefile: factor sed-powered '#!/bin/sh' munging into a variable\n>> Makefile: define \"TEST_{PROGRAM,OBJS}\" variables earlier\n>> Makefile: rename \"test_bindir_programs\" variable, pre-declare\n>> Makefile: simplify $(test_bindir_programs) rule by splitting it up\n>>\n>> Makefile | 70 +++++++++++++++++++++++++++++++++++++++++---------------\n>> 1 file changed, 51 insertions(+), 19 deletions(-)\n>\n> Thanks. I replaced the earlier round with this one and pushed the result\n> out to ttaylorr/git.\n>\n> But having read the topic over again, I have to say that I also find the\n> pre-existing behavior to be as expected. \"make bin-wrappers/git\"\n> produces its target as expected, but the target is useless because the\n> relationship it has with \"git\" is a runtime dependency, not a Make-time\n> dependency.\n>\n> So I'm not inclined to pick this one up, honestly. Perhaps other\n> reviewers feel differently.\n\nI'm fine with changing that. The part where it makes:\n\n\tmake clean &&\n\tmake bin-wrappers/git &&\n\t./git ...\n\nwork is a 1-line change to remove, i.e.:\n\n\tdiff --git a/Makefile b/Makefile\n\tindex d7ab68e3db8..d3c0a66b4b4 100644\n\t--- a/Makefile\n\t+++ b/Makefile\n\t@@ -3089,7 +3089,6 @@ define bin_wrappers_template\n\t # 4='$(4)'\n\t BW_$(1) = $$($(1):%=bin-wrappers/%)\n\t BIN_WRAPPERS += $$(BW_$(1))\n\t-$$(BW_$(1)): bin-wrappers/% : $(3)%$(4)\n\t $$(BW_$(1)): wrap-for-bin.sh\n\t \t$$(call mkdir_p_parent_template)\n\t \t$$(QUIET_GEN)$$(call cmd_munge_bin_wrappers_script,$(2),$(3),$(4))\n\nThis topic is mainly about untangling the dense $(filter) mess in the\nexisting rule, which last came up as we need do to special-case\n\"scalar\", and we'd need to special-case any future binaries.\n\nBut I see it's my own fault for making that clear, the \"as seen in the\nrange-diff below\" in v2's CL [1] was meant to refer to \"here's what's\nnew in v2\", not the topic's reason for existing.\n\nSo, while I'm happy to re-roll it and remove that one line (and the\nassociated part of the commit message) I'm surprised that there's such a\nhang-up about this aspect of it, to the point where I think I'm missing\nsomething.\n\nNothing depends on \"bin-wrappers/%\" now, so having this harms nothing,\nbut just seems to me to nicely tie off a loose end.\n\nI'm aware that e.g. git-submodule which depends on git-submodule.sh\ndoesn't depend on e.g. \"submodule--helper\" being built, so in that sense\nthis is inconsistent with the rest of our shellscripts.\n\nBut in the case of wrap-for-bin.sh its generation is driven by the\nMakefile, and it's *only* there to to exec() the target binary (at least\nsome of those other *.sh's can serve up -h on their own).\n\nAnd if we don't include that line, then (like on \"master\"):\n\n\t$ make clean >/dev/null; make bin-wrappers/git; ./git; echo $?\n\tGIT_VERSION = 2.38.GIT-dev\n\t MKDIR -p bin-wrappers\n\t GEN bin-wrappers/git\n\tbash: ./git: No such file or directory\n\t127\n\nBut *shrug*, I just don't see how it's useful, when it's so easy to make\nit do something useful.\n\n1. https://lore.kernel.org/git/cover-v2-0.3-00000000000-20221026T143533Z-avarab@gmail.com/\n"}]}