{"thread":{"id":"54562","subject":"[PATCH 1/3] maintenance: extract platform-specific scheduling","startedAt":"2020-11-03T14:04:52Z","lastAt":"2021-01-10T06:35:14Z","messageCount":83,"participants":["Derrick Stolee via GitGitGadget","Eric Sunshine","Junio C Hamano","Derrick Stolee"],"isPatch":true,"patchVersion":1,"patchTotal":3},"messages":[{"id":"408995","messageId":"d35f1aa162097d4dfe468b3ac1717bca92a41cfa.1604412196.git.gitgitgadget@gmail.com","threadId":"54562","inReplyTo":"pull.776.git.1604412196.gitgitgadget@gmail.com","subject":"[PATCH 1/3] maintenance: extract platform-specific scheduling","fromName":"Derrick Stolee via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2020-11-03T14:03:14Z","receivedAt":"2020-11-03T14:04:52Z","isPatch":true,"sender":{"key":"stolee@gmail.com","avatar":"https://avatars.githubusercontent.com/u/570044?v=4"},"body":"From: Derrick Stolee \n\nThe existing schedule mechanism using 'cron' is supported by POSIX\nplatforms, but not Windows. It also works slightly differently on\nmacOS to significant detriment of the user experience. To allow for\nnew implementations on these platforms, extract a method that\nperforms the platform-specific scheduling mechanism. This will be\nswapped at compile time with new implementations on specialized\nplatforms.\n\nSigned-off-by: Derrick Stolee \n---\n builtin/gc.c | 38 +++++++++++++++++++++-----------------\n 1 file changed, 21 insertions(+), 17 deletions(-)\n\ndiff --git a/builtin/gc.c b/builtin/gc.c\nindex e3098ef6a1..c1f7d9bdc2 100644\n--- a/builtin/gc.c\n+++ b/builtin/gc.c\n@@ -1494,7 +1494,7 @@ static int maintenance_unregister(void)\n #define BEGIN_LINE \"# BEGIN GIT MAINTENANCE SCHEDULE\"\n #define END_LINE \"# END GIT MAINTENANCE SCHEDULE\"\n \n-static int update_background_schedule(int run_maintenance)\n+static int platform_update_schedule(int run_maintenance, int fd)\n {\n \tint result = 0;\n \tint in_old_region = 0;\n@@ -1503,11 +1503,6 @@ static int update_background_schedule(int run_maintenance)\n \tFILE *cron_list, *cron_in;\n \tconst char *crontab_name;\n \tstruct strbuf line = STRBUF_INIT;\n-\tstruct lock_file lk;\n-\tchar *lock_path = xstrfmt(\"%s/schedule\", the_repository->objects->odb->path);\n-\n-\tif (hold_lock_file_for_update(&lk, lock_path, LOCK_NO_DEREF) < 0)\n-\t\treturn error(_(\"another process is scheduling background maintenance\"));\n \n \tcrontab_name = getenv(\"GIT_TEST_CRONTAB\");\n \tif (!crontab_name)\n@@ -1516,12 +1511,11 @@ static int update_background_schedule(int run_maintenance)\n \tstrvec_split(&crontab_list.args, crontab_name);\n \tstrvec_push(&crontab_list.args, \"-l\");\n \tcrontab_list.in = -1;\n-\tcrontab_list.out = dup(lk.tempfile->fd);\n+\tcrontab_list.out = dup(fd);\n \tcrontab_list.git_cmd = 0;\n \n \tif (start_command(&crontab_list)) {\n-\t\tresult = error(_(\"failed to run 'crontab -l'; your system might not support 'cron'\"));\n-\t\tgoto cleanup;\n+\t\treturn error(_(\"failed to run 'crontab -l'; your system might not support 'cron'\"));\n \t}\n \n \t/* Ignore exit code, as an empty crontab will return error. */\n@@ -1531,7 +1525,7 @@ static int update_background_schedule(int run_maintenance)\n \t * Read from the .lock file, filtering out the old\n \t * schedule while appending the new schedule.\n \t */\n-\tcron_list = fdopen(lk.tempfile->fd, \"r\");\n+\tcron_list = fdopen(fd, \"r\");\n \trewind(cron_list);\n \n \tstrvec_split(&crontab_edit.args, crontab_name);\n@@ -1539,8 +1533,7 @@ static int update_background_schedule(int run_maintenance)\n \tcrontab_edit.git_cmd = 0;\n \n \tif (start_command(&crontab_edit)) {\n-\t\tresult = error(_(\"failed to run 'crontab'; your system might not support 'cron'\"));\n-\t\tgoto cleanup;\n+\t\treturn error(_(\"failed to run 'crontab'; your system might not support 'cron'\"));\n \t}\n \n \tcron_in = fdopen(crontab_edit.in, \"w\");\n@@ -1586,13 +1579,24 @@ static int update_background_schedule(int run_maintenance)\n \tclose(crontab_edit.in);\n \n done_editing:\n-\tif (finish_command(&crontab_edit)) {\n+\tif (finish_command(&crontab_edit))\n \t\tresult = error(_(\"'crontab' died\"));\n-\t\tgoto cleanup;\n-\t}\n-\tfclose(cron_list);\n+\telse\n+\t\tfclose(cron_list);\n+\treturn result;\n+}\n+\n+static int update_background_schedule(int run_maintenance)\n+{\n+\tint result;\n+\tstruct lock_file lk;\n+\tchar *lock_path = xstrfmt(\"%s/schedule\", the_repository->objects->odb->path);\n+\n+\tif (hold_lock_file_for_update(&lk, lock_path, LOCK_NO_DEREF) < 0)\n+\t\treturn error(_(\"another process is scheduling background maintenance\"));\n+\n+\tresult = platform_update_schedule(run_maintenance, lk.tempfile->fd);\n \n-cleanup:\n \trollback_lock_file(&lk);\n \treturn result;\n }\n-- \ngitgitgadget\n\n"},{"id":"408996","messageId":"a9221cc4aa12192e9a691f8e1b77a3cc2d7e4952.1604412197.git.gitgitgadget@gmail.com","threadId":"54562","inReplyTo":"pull.776.git.1604412196.gitgitgadget@gmail.com","subject":"[PATCH 3/3] maintenance: use Windows scheduled tasks","fromName":"Derrick Stolee via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2020-11-03T14:03:16Z","receivedAt":"2020-11-03T14:04:54Z","isPatch":true,"sender":{"key":"stolee@gmail.com","avatar":"https://avatars.githubusercontent.com/u/570044?v=4"},"body":"From: Derrick Stolee \n\nGit's background maintenance uses cron by default, but this is not\navailable on Windows. Instead, integrate with Task Scheduler.\n\nTasks can be scheduled using the 'schtasks' command. There are several\ncommand-line options that can allow for some advanced scheduling, but\nunfortunately these seem to all require authenticating using a password.\n\nInstead, use the \"/xml\" option to pass an XML file that contains the\nconfiguration for the necessary schedule. These XML files are based on\nsome that I exported after constructing a schedule in the Task Scheduler\nGUI. These options only run background maintenance when the user is\nlogged in, and more fields are populated with the current username and\nSID at run-time by 'schtasks'.\n\nThere is a deficiency in the current design. Windows has two kinds of\napplications: GUI applications that start by \"winmain()\" and console\napplications that start by \"main()\". Console applications are attached\nto a new Console window if they are not already associated with a GUI\napplication. This means that every hour the scheudled task launches a\ncommand window for the scheduled tasks. Not only is this visually\nobtrusive, but it also takes focus from whatever else the user is\ndoing!\n\nA simple fix would be to insert a GUI application that acts as a shim\nbetween the scheduled task and Git. This is currently possible in Git\nfor Windows by setting the tag equal to\n\n C:\\Program Files\\Git\\git-bash.exe\n\nwith options \"--hide --no-needs-console --command=cmd\\git.exe\"\nfollowed by the arguments currently used. Since git-bash.exe is not\nincluded in Windows builds of core Git, I chose to leave out this\nfeature. My plan is to submit a small patch to Git for Windows that\nconverts the use of git.exe with this use of git-bash.exe in the\nshort term. In the long term, we can consider creating this GUI\nshim application within core Git, perhaps in contrib/.\n\nSigned-off-by: Derrick Stolee \n---\n builtin/gc.c | 181 +++++++++++++++++++++++++++++++++++++++++\n t/t7900-maintenance.sh | 40 ++++++++-\n 2 files changed, 218 insertions(+), 3 deletions(-)\n\ndiff --git a/builtin/gc.c b/builtin/gc.c\nindex fa0ae63a80..24511fec2e 100644\n--- a/builtin/gc.c\n+++ b/builtin/gc.c\n@@ -1698,6 +1698,187 @@ static int platform_update_schedule(int run_maintenance, int fd)\n \telse\n \t\treturn remove_plists();\n }\n+\n+#elif defined(GIT_WINDOWS_NATIVE)\n+\n+static const char *get_frequency(enum schedule_priority schedule)\n+{\n+\tswitch (schedule) {\n+\tcase SCHEDULE_HOURLY:\n+\t\treturn \"hourly\";\n+\tcase SCHEDULE_DAILY:\n+\t\treturn \"daily\";\n+\tcase SCHEDULE_WEEKLY:\n+\t\treturn \"weekly\";\n+\tdefault:\n+\t\tBUG(\"invalid schedule %d\", schedule);\n+\t}\n+}\n+\n+static char *get_task_name(const char *frequency)\n+{\n+\tstruct strbuf label = STRBUF_INIT;\n+\tstrbuf_addf(&label, \"Git Maintenance (%s)\", frequency);\n+\treturn strbuf_detach(&label, NULL);\n+}\n+\n+static int remove_task(enum schedule_priority schedule)\n+{\n+\tint result;\n+\tstruct strvec args = STRVEC_INIT;\n+\tconst char *frequency = get_frequency(schedule);\n+\tchar *name = get_task_name(frequency);\n+\tconst char *schtasks = getenv(\"GIT_TEST_CRONTAB\");\n+\tif (!schtasks)\n+\t\tschtasks = \"schtasks\";\n+\n+\tstrvec_split(&args, schtasks);\n+\tstrvec_pushl(&args, \"/delete\", \"/tn\", name, \"/f\", NULL);\n+\n+\tresult = run_command_v_opt(args.v, 0);\n+\n+\tstrvec_clear(&args);\n+\tfree(name);\n+\treturn result;\n+}\n+\n+static int remove_scheduled_tasks(void)\n+{\n+\treturn remove_task(SCHEDULE_HOURLY) ||\n+\t\tremove_task(SCHEDULE_DAILY) ||\n+\t\tremove_task(SCHEDULE_WEEKLY);\n+}\n+\n+static int schedule_task(const char *exec_path, enum schedule_priority schedule)\n+{\n+\tint result;\n+\tstruct strvec args = STRVEC_INIT;\n+\tconst char *xml, *schtasks;\n+\tchar *xmlpath;\n+\tFILE *xmlfp;\n+\tconst char *frequency = get_frequency(schedule);\n+\tchar *name = get_task_name(frequency);\n+\n+\txmlpath = xstrfmt(\"%s/schedule-%s.xml\",\n+\t\t\t the_repository->objects->odb->path,\n+\t\t\t frequency);\n+\txmlfp = fopen(xmlpath, \"w\");\n+\tif (!xmlfp)\n+\t\tdie(_(\"failed to open '%s'\"), xmlpath);\n+\n+\txml = \"\\n\"\n+\t \"\\n\"\n+\t \"\\n\"\n+\t \"\\n\";\n+\tfprintf(xmlfp, xml);\n+\n+\tswitch (schedule) {\n+\tcase SCHEDULE_HOURLY:\n+\t\tfprintf(xmlfp,\n+\t\t\t\"2020-01-01T01:00:00\\n\"\n+\t\t\t\"true\\n\"\n+\t\t\t\"\\n\"\n+\t\t\t\"1\\n\"\n+\t\t\t\"\\n\"\n+\t\t\t\"\\n\"\n+\t\t\t\"PT1H\\n\"\n+\t\t\t\"PT23H\\n\"\n+\t\t\t\"false\\n\"\n+\t\t\t\"\\n\");\n+\t\tbreak;\n+\n+\tcase SCHEDULE_DAILY:\n+\t\tfprintf(xmlfp,\n+\t\t\t\"2020-01-01T00:00:00\\n\"\n+\t\t\t\"true\\n\"\n+\t\t\t\"\\n\"\n+\t\t\t\"\\n\"\n+\t\t\t\"\\n\"\n+\t\t\t\"\\n\"\n+\t\t\t\"\\n\"\n+\t\t\t\"\\n\"\n+\t\t\t\"\\n\"\n+\t\t\t\"\\n\"\n+\t\t\t\"\\n\"\n+\t\t\t\"1\\n\"\n+\t\t\t\"\\n\");\n+\t\tbreak;\n+\n+\tcase SCHEDULE_WEEKLY:\n+\t\tfprintf(xmlfp,\n+\t\t\t\"2020-01-01T00:00:00\\n\"\n+\t\t\t\"true\\n\"\n+\t\t\t\"\\n\"\n+\t\t\t\"\\n\"\n+\t\t\t\"\\n\"\n+\t\t\t\"\\n\"\n+\t\t\t\"1\\n\"\n+\t\t\t\"\\n\");\n+\t\tbreak;\n+\n+\tdefault:\n+\t\tbreak;\n+\t}\n+\n+\txml= \"\\n\"\n+\t \"\\n\"\n+\t \"\\n\"\n+\t \"\\n\"\n+\t \"InteractiveToken\\n\"\n+\t \"LeastPrivilege\\n\"\n+\t \"\\n\"\n+\t \"\\n\"\n+\t \"\\n\"\n+\t \"IgnoreNew\\n\"\n+\t \"true\\n\"\n+\t \"true\\n\"\n+\t \"true\\n\"\n+\t \"false\\n\"\n+\t \"PT72H\\n\"\n+\t \"7\\n\"\n+\t \"\\n\"\n+\t \"\\n\"\n+\t \"\\n\"\n+\t \"\\\"%s\\\\git.exe\\\"\\n\"\n+\t \"--exec-path=\\\"%s\\\" for-each-repo --config=maintenance.repo maintenance run --schedule=%s\\n\"\n+\t \"\\n\"\n+\t \"\\n\"\n+\t \"\\n\";\n+\tfprintf(xmlfp, xml, exec_path, exec_path, frequency);\n+\tfclose(xmlfp);\n+\n+\tschtasks = getenv(\"GIT_TEST_CRONTAB\");\n+\tif (!schtasks)\n+\t\tschtasks = \"schtasks\";\n+\tstrvec_split(&args, schtasks);\n+\tstrvec_pushl(&args, \"/create\", \"/tn\", name, \"/f\", \"/xml\", xmlpath, NULL);\n+\n+\tresult = run_command_v_opt(args.v, 0);\n+\n+\tstrvec_clear(&args);\n+\tunlink(xmlpath);\n+\tfree(xmlpath);\n+\tfree(name);\n+\treturn result;\n+}\n+\n+static int add_scheduled_tasks(void)\n+{\n+\tconst char *exec_path = git_exec_path();\n+\n+\treturn schedule_task(exec_path, SCHEDULE_HOURLY) ||\n+\t\tschedule_task(exec_path, SCHEDULE_DAILY) ||\n+\t\tschedule_task(exec_path, SCHEDULE_WEEKLY);\n+}\n+\n+static int platform_update_schedule(int run_maintenance, int fd)\n+{\n+\tif (run_maintenance)\n+\t\treturn add_scheduled_tasks();\n+\telse\n+\t\treturn remove_scheduled_tasks();\n+}\n+\n #else\n #define BEGIN_LINE \"# BEGIN GIT MAINTENANCE SCHEDULE\"\n #define END_LINE \"# END GIT MAINTENANCE SCHEDULE\"\ndiff --git a/t/t7900-maintenance.sh b/t/t7900-maintenance.sh\nindex f0210aa206..73dc0078da 100755\n--- a/t/t7900-maintenance.sh\n+++ b/t/t7900-maintenance.sh\n@@ -367,7 +367,7 @@ test_expect_success 'register and unregister' '\n \ttest_cmp before actual\n '\n \n-test_expect_success !MACOS_MAINTENANCE 'start from empty cron table' '\n+test_expect_success !MACOS_MAINTENANCE,!MINGW 'start from empty cron table' '\n \tGIT_TEST_CRONTAB=\"test-tool crontab cron.txt\" git maintenance start &&\n \n \t# start registers the repo\n@@ -378,7 +378,7 @@ test_expect_success !MACOS_MAINTENANCE 'start from empty cron table' '\n \tgrep \"for-each-repo --config=maintenance.repo maintenance run --schedule=weekly\" cron.txt\n '\n \n-test_expect_success !MACOS_MAINTENANCE 'stop from existing schedule' '\n+test_expect_success !MACOS_MAINTENANCE,!MINGW 'stop from existing schedule' '\n \tGIT_TEST_CRONTAB=\"test-tool crontab cron.txt\" git maintenance stop &&\n \n \t# stop does not unregister the repo\n@@ -389,7 +389,7 @@ test_expect_success !MACOS_MAINTENANCE 'stop from existing schedule' '\n \ttest_must_be_empty cron.txt\n '\n \n-test_expect_success !MACOS_MAINTENANCE 'start preserves existing schedule' '\n+test_expect_success !MACOS_MAINTENANCE,!MINGW 'start preserves existing schedule' '\n \techo \"Important information!\" >cron.txt &&\n \tGIT_TEST_CRONTAB=\"test-tool crontab cron.txt\" git maintenance start &&\n \tgrep \"Important information!\" cron.txt\n@@ -441,6 +441,40 @@ test_expect_success MACOS_MAINTENANCE 'start and stop macOS maintenance' '\n \ttest_cmp expect args\n '\n \n+test_expect_success MINGW 'start and stop Windows maintenance' '\n+\techo \"echo \\$@ >>args\" >print-args &&\n+\tchmod a+x print-args &&\n+\n+\trm -f args &&\n+\tGIT_TEST_CRONTAB=\"/bin/sh print-args\" git maintenance start &&\n+\tcat args &&\n+\n+\t# start registers the repo\n+\tgit config --get --global maintenance.repo \"$(pwd)\" &&\n+\n+\trm expect &&\n+\tfor frequency in hourly daily weekly\n+\tdo\n+\t\techo \"/create /tn Git Maintenance ($frequency) /f /xml .git/objects/schedule-$frequency.xml\" >>expect \\\n+\t\t\t|| return 1\n+\tdone &&\n+\ttest_cmp expect args &&\n+\n+\trm -f args &&\n+\tGIT_TEST_CRONTAB=\"/bin/sh print-args\" git maintenance stop &&\n+\n+\t# stop does not unregister the repo\n+\tgit config --get --global maintenance.repo \"$(pwd)\" &&\n+\n+\trm expect &&\n+\tfor frequency in hourly daily weekly\n+\tdo\n+\t\techo \"/delete /tn Git Maintenance ($frequency) /f\" >>expect \\\n+\t\t\t|| return 1\n+\tdone &&\n+\ttest_cmp expect args\n+'\n+\n test_expect_success 'register preserves existing strategy' '\n \tgit config maintenance.strategy none &&\n \tgit maintenance register &&\n-- \ngitgitgadget\n"},{"id":"408997","messageId":"pull.776.git.1604412196.gitgitgadget@gmail.com","threadId":"54562","inReplyTo":null,"subject":"[PATCH 0/3] Maintenance IV: Platform-specific background maintenance","fromName":"Derrick Stolee via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2020-11-03T14:03:13Z","receivedAt":"2020-11-03T14:05:04Z","isPatch":true,"sender":{"key":"stolee@gmail.com","avatar":"https://avatars.githubusercontent.com/u/570044?v=4"},"body":"This is based on ds/maintenance-part-3.\n\nAfter sitting with the background maintenance as it has been cooking, I\nwanted to come back around and implement the background maintenance for\nWindows. However, I noticed that there were some things bothering me with\nbackground maintenance on my macOS machine. These are detailed in PATCH 2,\nbut the tl;dr is that 'cron' is not recommended by Apple and instead\n'launchd' satisfies our needs.\n\nThis series implements the background scheduling so git maintenance\n(start|stop) works on those platforms. I've been operating with these\nschedules for a while now without the problems described in the patches.\n\nThere is a particularly annoying case about console windows popping up on\nWindows, but PATCH 3 describes a plan to get around that.\n\nThanks, -Stolee\n\nDerrick Stolee (3):\n maintenance: extract platform-specific scheduling\n maintenance: use launchctl on macOS\n maintenance: use Windows scheduled tasks\n\n builtin/gc.c | 428 +++++++++++++++++++++++++++++++++++++++--\n t/t7900-maintenance.sh | 86 ++++++++-\n t/test-lib.sh | 4 +\n 3 files changed, 498 insertions(+), 20 deletions(-)\n\n\nbase-commit: 0016b618182f642771dc589cf0090289f9fe1b4f\nPublished-As: https://github.com/gitgitgadget/git/releases/tag/pr-776%2Fderrickstolee%2Fmaintenance%2FmacOS-v1\nFetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-776/derrickstolee/maintenance/macOS-v1\nPull-Request: https://github.com/gitgitgadget/git/pull/776\n-- \ngitgitgadget\n"},{"id":"408998","messageId":"832fdf16872cbfee4a5e15b559b2b40dabd545f4.1604412197.git.gitgitgadget@gmail.com","threadId":"54562","inReplyTo":"pull.776.git.1604412196.gitgitgadget@gmail.com","subject":"[PATCH 2/3] maintenance: use launchctl on macOS","fromName":"Derrick Stolee via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2020-11-03T14:03:15Z","receivedAt":"2020-11-03T14:05:04Z","isPatch":true,"sender":{"key":"stolee@gmail.com","avatar":"https://avatars.githubusercontent.com/u/570044?v=4"},"body":"From: Derrick Stolee \n\nThe existing mechanism for scheduling background maintenance is done\nthrough cron. The 'crontab -e' command allows updating the schedule\nwhile cron itself runs those commands. While this is technically\nsupported by macOS, it has some significant deficiencies:\n\n1. Every run of 'crontab -e' must request elevated privileges through\n the user interface. When running 'git maintenance start' from the\n Terminal app, it presents a dialog box saying \"Terminal.app would\n like to administer your computer. Administration can include\n modifying passwords, networking, and system settings.\" This is more\n alarming than what we are hoping to achieve. If this alert had some\n information about how \"git\" is trying to run \"crontab\" then we would\n have some reason to believe that this dialog might be fine. However,\n it also doesn't help that some scenarios just leave Git waiting for\n a response without presenting anything to the user. I experienced\n this when executing the command from a Bash terminal view inside\n Visual Studio Code.\n\n2. While cron initializes a user environment enough for \"git config\n --global --show-origin\" to show the correct config file information,\n it does not set up the environment enough for Git Credential Manager\n Core to load credentials during a 'prefetch' task. My prefetches\n against private repositories required re-authenticating through UI\n pop-ups in a way that should not be required.\n\nThe solution is to switch from cron to the Apple-recommended [1]\n'launchd' tool.\n\n[1] https://developer.apple.com/library/archive/documentation/MacOSX/Conceptual/BPSystemStartup/Chapters/ScheduledJobs.html\n\nThe basics of this tool is that we need to create XML-formatted\n\"plist\" files inside \"~/Library/LaunchAgents/\" and then use the\n'launchctl' tool to make launchd aware of them. The plist files\ninclude all of the scheduling information, along with the command-line\narguments split across an array of tags.\n\nFor example, here is my plist file for the weekly scheduled tasks:\n\n\n\n\nLabelorg.git-scm.git.weekly\nProgramArguments\n\n/usr/local/libexec/git-core/git\n--exec-path=/usr/local/libexec/git-core\nfor-each-repo\n--config=maintenance.repo\nmaintenance\nrun\n--schedule=weekly\n\nStartCalendarInterval\n\n\nDay0\nHour0\nMinute0\n\n\n\n\n\nThe schedules for the daily and hourly tasks are more complicated\nsince we need to use an array for the StartCalendarInterval with\nan entry for each of the six days other than the 0th day (to avoid\ncolliding with the weekly task), and each of the 23 hours other\nthan the 0th hour (to avoid colliding with the daily task).\n\nThe \"Label\" value is currently filled with \"org.git-scm.git.X\"\nwhere X is the frequency. We need a different plist file for each\nfrequency.\n\nThe launchctl command needs to be aligned with a user id in order\nto initialize the command environment. This must be done using\nthe 'launchctl bootstrap' subcommand. This subcommand is new as\nof macOS 10.11, which was released in September 2015. Before that\nrelease the 'launchctl load' subcommand was recommended. The best\nsource of information on this transition I have seen is available\nat [2].\n\n[2] https://babodee.wordpress.com/2016/04/09/launchctl-2-0-syntax/\n\nTo remove a schedule, we must run 'launchctl bootout' with a valid\nplist file. We also need to 'bootout' a task before the 'bootstrap'\nsubcommand will succeed, if such a task already exists.\n\nWe can verify the commands that were run by 'git maintenance start'\nand 'git maintenance stop' by injecting a script that writes the\ncommand-line arguments into GIT_TEST_CRONTAB.\n\nSigned-off-by: Derrick Stolee \n---\n builtin/gc.c | 209 +++++++++++++++++++++++++++++++++++++++++\n t/t7900-maintenance.sh | 52 +++++++++-\n t/test-lib.sh | 4 +\n 3 files changed, 262 insertions(+), 3 deletions(-)\n\ndiff --git a/builtin/gc.c b/builtin/gc.c\nindex c1f7d9bdc2..fa0ae63a80 100644\n--- a/builtin/gc.c\n+++ b/builtin/gc.c\n@@ -1491,6 +1491,214 @@ static int maintenance_unregister(void)\n \treturn run_command(&config_unset);\n }\n \n+#if defined(__APPLE__)\n+\n+static char *get_service_name(const char *frequency)\n+{\n+\tstruct strbuf label = STRBUF_INIT;\n+\tstrbuf_addf(&label, \"org.git-scm.git.%s\", frequency);\n+\treturn strbuf_detach(&label, NULL);\n+}\n+\n+static char *get_service_filename(const char *name)\n+{\n+\tchar *expanded;\n+\tstruct strbuf filename = STRBUF_INIT;\n+\tstrbuf_addf(&filename, \"~/Library/LaunchAgents/%s.plist\", name);\n+\n+\texpanded = expand_user_path(filename.buf, 1);\n+\tif (!expanded)\n+\t\tdie(_(\"failed to expand path '%s'\"), filename.buf);\n+\n+\tstrbuf_release(&filename);\n+\treturn expanded;\n+}\n+\n+static const char *get_frequency(enum schedule_priority schedule)\n+{\n+\tswitch (schedule) {\n+\tcase SCHEDULE_HOURLY:\n+\t\treturn \"hourly\";\n+\tcase SCHEDULE_DAILY:\n+\t\treturn \"daily\";\n+\tcase SCHEDULE_WEEKLY:\n+\t\treturn \"weekly\";\n+\tdefault:\n+\t\tBUG(\"invalid schedule %d\", schedule);\n+\t}\n+}\n+\n+static char *get_uid(void)\n+{\n+\tstruct strbuf output = STRBUF_INIT;\n+\tstruct child_process id = CHILD_PROCESS_INIT;\n+\n+\tstrvec_pushl(&id.args, \"/usr/bin/id\", \"-u\", NULL);\n+\tif (capture_command(&id, &output, 0))\n+\t\tdie(_(\"failed to discover user id\"));\n+\n+\tstrbuf_trim_trailing_newline(&output);\n+\treturn strbuf_detach(&output, NULL);\n+}\n+\n+static int bootout(const char *filename)\n+{\n+\tint result;\n+\tstruct strvec args = STRVEC_INIT;\n+\tchar *uid = get_uid();\n+\tconst char *launchctl = getenv(\"GIT_TEST_CRONTAB\");\n+\tif (!launchctl)\n+\t\tlaunchctl = \"/bin/launchctl\";\n+\n+\tstrvec_split(&args, launchctl);\n+\tstrvec_push(&args, \"bootout\");\n+\tstrvec_pushf(&args, \"gui/%s\", uid);\n+\tstrvec_push(&args, filename);\n+\n+\tresult = run_command_v_opt(args.v, 0);\n+\n+\tstrvec_clear(&args);\n+\tfree(uid);\n+\treturn result;\n+}\n+\n+static int bootstrap(const char *filename)\n+{\n+\tint result;\n+\tstruct strvec args = STRVEC_INIT;\n+\tchar *uid = get_uid();\n+\tconst char *launchctl = getenv(\"GIT_TEST_CRONTAB\");\n+\tif (!launchctl)\n+\t\tlaunchctl = \"/bin/launchctl\";\n+\n+\tstrvec_split(&args, launchctl);\n+\tstrvec_push(&args, \"bootstrap\");\n+\tstrvec_pushf(&args, \"gui/%s\", uid);\n+\tstrvec_push(&args, filename);\n+\n+\tresult = run_command_v_opt(args.v, 0);\n+\n+\tstrvec_clear(&args);\n+\tfree(uid);\n+\treturn result;\n+}\n+\n+static int remove_plist(enum schedule_priority schedule)\n+{\n+\tconst char *frequency = get_frequency(schedule);\n+\tchar *name = get_service_name(frequency);\n+\tchar *filename = get_service_filename(name);\n+\tint result = bootout(filename);\n+\tfree(filename);\n+\tfree(name);\n+\treturn result;\n+}\n+\n+static int remove_plists(void)\n+{\n+\treturn remove_plist(SCHEDULE_HOURLY) ||\n+\t\tremove_plist(SCHEDULE_DAILY) ||\n+\t\tremove_plist(SCHEDULE_WEEKLY);\n+}\n+\n+static int schedule_plist(const char *exec_path, enum schedule_priority schedule)\n+{\n+\tFILE *plist;\n+\tint i;\n+\tconst char *preamble, *repeat;\n+\tconst char *frequency = get_frequency(schedule);\n+\tchar *name = get_service_name(frequency);\n+\tchar *filename = get_service_filename(name);\n+\n+\tif (safe_create_leading_directories(filename))\n+\t\tdie(_(\"failed to create directories for '%s'\"), filename);\n+\tplist = fopen(filename, \"w\");\n+\n+\tif (!plist)\n+\t\tdie(_(\"failed to open '%s'\"), filename);\n+\n+\tpreamble = \"\\n\"\n+\t\t \"\\n\"\n+\t\t \"\"\n+\t\t \"\\n\"\n+\t\t \"Label%s\\n\"\n+\t\t \"ProgramArguments\\n\"\n+\t\t \"\\n\"\n+\t\t \"%s/git\\n\"\n+\t\t \"--exec-path=%s\\n\"\n+\t\t \"for-each-repo\\n\"\n+\t\t \"--config=maintenance.repo\\n\"\n+\t\t \"maintenance\\n\"\n+\t\t \"run\\n\"\n+\t\t \"--schedule=%s\\n\"\n+\t\t \"\\n\"\n+\t\t \"StartCalendarInterval\\n\"\n+\t\t \"\\n\";\n+\tfprintf(plist, preamble, name, exec_path, exec_path, frequency);\n+\n+\tswitch (schedule) {\n+\tcase SCHEDULE_HOURLY:\n+\t\trepeat = \"\\n\"\n+\t\t\t \"Hour%d\\n\"\n+\t\t\t \"Minute0\\n\"\n+\t\t\t \"\\n\";\n+\t\tfor (i = 1; i <= 23; i++)\n+\t\t\tfprintf(plist, repeat, i);\n+\t\tbreak;\n+\n+\tcase SCHEDULE_DAILY:\n+\t\trepeat = \"\\n\"\n+\t\t\t \"Day%d\\n\"\n+\t\t\t \"Hour0\\n\"\n+\t\t\t \"Minute0\\n\"\n+\t\t\t \"\\n\";\n+\t\tfor (i = 1; i <= 6; i++)\n+\t\t\tfprintf(plist, repeat, i);\n+\t\tbreak;\n+\n+\tcase SCHEDULE_WEEKLY:\n+\t\tfprintf(plist,\n+\t\t\t\"\\n\"\n+\t\t\t\"Day0\\n\"\n+\t\t\t\"Hour0\\n\"\n+\t\t\t\"Minute0\\n\"\n+\t\t\t\"\\n\");\n+\t\tbreak;\n+\n+\tdefault:\n+\t\t/* unreachable */\n+\t\tbreak;\n+\t}\n+\tfprintf(plist, \"\\n\\n\\n\");\n+\n+\t/* bootout might fail if not already running, so ignore */\n+\tbootout(filename);\n+\tif (bootstrap(filename))\n+\t\tdie(_(\"failed to bootstrap service %s\"), filename);\n+\n+\tfclose(plist);\n+\tfree(filename);\n+\tfree(name);\n+\treturn 0;\n+}\n+\n+static int add_plists(void)\n+{\n+\tconst char *exec_path = git_exec_path();\n+\n+\treturn schedule_plist(exec_path, SCHEDULE_HOURLY) ||\n+\t\tschedule_plist(exec_path, SCHEDULE_DAILY) ||\n+\t\tschedule_plist(exec_path, SCHEDULE_WEEKLY);\n+}\n+\n+static int platform_update_schedule(int run_maintenance, int fd)\n+{\n+\tif (run_maintenance)\n+\t\treturn add_plists();\n+\telse\n+\t\treturn remove_plists();\n+}\n+#else\n #define BEGIN_LINE \"# BEGIN GIT MAINTENANCE SCHEDULE\"\n #define END_LINE \"# END GIT MAINTENANCE SCHEDULE\"\n \n@@ -1585,6 +1793,7 @@ static int platform_update_schedule(int run_maintenance, int fd)\n \t\tfclose(cron_list);\n \treturn result;\n }\n+#endif\n \n static int update_background_schedule(int run_maintenance)\n {\ndiff --git a/t/t7900-maintenance.sh b/t/t7900-maintenance.sh\nindex 20184e96e1..f0210aa206 100755\n--- a/t/t7900-maintenance.sh\n+++ b/t/t7900-maintenance.sh\n@@ -367,7 +367,7 @@ test_expect_success 'register and unregister' '\n \ttest_cmp before actual\n '\n \n-test_expect_success 'start from empty cron table' '\n+test_expect_success !MACOS_MAINTENANCE 'start from empty cron table' '\n \tGIT_TEST_CRONTAB=\"test-tool crontab cron.txt\" git maintenance start &&\n \n \t# start registers the repo\n@@ -378,7 +378,7 @@ test_expect_success 'start from empty cron table' '\n \tgrep \"for-each-repo --config=maintenance.repo maintenance run --schedule=weekly\" cron.txt\n '\n \n-test_expect_success 'stop from existing schedule' '\n+test_expect_success !MACOS_MAINTENANCE 'stop from existing schedule' '\n \tGIT_TEST_CRONTAB=\"test-tool crontab cron.txt\" git maintenance stop &&\n \n \t# stop does not unregister the repo\n@@ -389,12 +389,58 @@ test_expect_success 'stop from existing schedule' '\n \ttest_must_be_empty cron.txt\n '\n \n-test_expect_success 'start preserves existing schedule' '\n+test_expect_success !MACOS_MAINTENANCE 'start preserves existing schedule' '\n \techo \"Important information!\" >cron.txt &&\n \tGIT_TEST_CRONTAB=\"test-tool crontab cron.txt\" git maintenance start &&\n \tgrep \"Important information!\" cron.txt\n '\n \n+test_expect_success MACOS_MAINTENANCE 'start and stop macOS maintenance' '\n+\techo \"#!/bin/sh\\necho \\$@ >>args\" >print-args &&\n+\tchmod a+x print-args &&\n+\n+\trm -f args &&\n+\tGIT_TEST_CRONTAB=\"./print-args\" git maintenance start &&\n+\n+\t# start registers the repo\n+\tgit config --get --global maintenance.repo \"$(pwd)\" &&\n+\n+\t# ~/Library/LaunchAgents\n+\tls \"$HOME/Library/LaunchAgents\" >actual &&\n+\tcat >expect <<-\\EOF &&\n+\torg.git-scm.git.daily.plist\n+\torg.git-scm.git.hourly.plist\n+\torg.git-scm.git.weekly.plist\n+\tEOF\n+\ttest_cmp expect actual &&\n+\n+\trm expect &&\n+\tfor frequency in hourly daily weekly\n+\tdo\n+\t\tPLIST=\"$HOME/Library/LaunchAgents/org.git-scm.git.$frequency.plist\" &&\n+\t\tgrep schedule=$frequency \"$PLIST\" &&\n+\t\techo \"bootout gui/$UID $PLIST\" >>expect &&\n+\t\techo \"bootstrap gui/$UID $PLIST\" >>expect || return 1\n+\tdone &&\n+\ttest_cmp expect args &&\n+\n+\trm -f args &&\n+\tGIT_TEST_CRONTAB=\"./print-args\" git maintenance stop &&\n+\n+\t# stop does not unregister the repo\n+\tgit config --get --global maintenance.repo \"$(pwd)\" &&\n+\n+\t# stop does not remove plist files, but boots them out\n+\trm expect &&\n+\tfor frequency in hourly daily weekly\n+\tdo\n+\t\tPLIST=\"$HOME/Library/LaunchAgents/org.git-scm.git.$frequency.plist\" &&\n+\t\tgrep schedule=$frequency \"$PLIST\" &&\n+\t\techo \"bootout gui/$UID $PLIST\" >>expect || return 1\n+\tdone &&\n+\ttest_cmp expect args\n+'\n+\n test_expect_success 'register preserves existing strategy' '\n \tgit config maintenance.strategy none &&\n \tgit maintenance register &&\ndiff --git a/t/test-lib.sh b/t/test-lib.sh\nindex 4a60d1ed76..620ffbf3af 100644\n--- a/t/test-lib.sh\n+++ b/t/test-lib.sh\n@@ -1703,6 +1703,10 @@ test_lazy_prereq REBASE_P '\n \ttest -z \"$GIT_TEST_SKIP_REBASE_P\"\n '\n \n+test_lazy_prereq MACOS_MAINTENANCE '\n+\tlaunchctl list\n+'\n+\n # Ensure that no test accidentally triggers a Git command\n # that runs 'crontab', affecting a user's cron schedule.\n # Tests that verify the cron integration must set this locally\n-- \ngitgitgadget\n\n"},{"id":"409030","messageId":"CAPig+cR1zSqY_y02JBdtjh_+9Pp2dOtGrHHmf-ah5sU6Lhzp6Q@mail.gmail.com","threadId":"54562","inReplyTo":"832fdf16872cbfee4a5e15b559b2b40dabd545f4.1604412197.git.gitgitgadget@gmail.com","subject":"Re: [PATCH 2/3] maintenance: use launchctl on macOS","fromName":"Eric Sunshine","fromEmail":"sunshine@sunshineco.com","sentAt":"2020-11-03T18:45:36Z","receivedAt":"2020-11-03T18:45:52Z","isPatch":true,"sender":{"key":"sunshine@sunshineco.com","avatar":"https://avatars.githubusercontent.com/u/163641?v=4"},"body":"On Tue, Nov 3, 2020 at 9:05 AM Derrick Stolee via GitGitGadget\n wrote:\n> maintenance: use launchctl on macOS\n\nA few comments below (not necessarily worth a re-roll)...\n\n> The launchctl command needs to be aligned with a user id in order\n> to initialize the command environment. This must be done using\n> the 'launchctl bootstrap' subcommand. This subcommand is new as\n> of macOS 10.11, which was released in September 2015. Before that\n> release the 'launchctl load' subcommand was recommended. The best\n> source of information on this transition I have seen is available\n> at [2].\n\nIt's not clear whether or not this is saying that git-maintenance will\ndynamically adapt to work with modern and older 'launchctl'. A glance\nat the actual code reveals that it knows only about modern\n'bootstrap'. Perhaps this could be a bit clearer by saying that it\nonly supports modern versions, and that support for older versions can\nbe added later if needed. (For those of us who are stuck with 10-20\nyear old hardware and OS versions, 2015 isn't that long ago.)\n\n> Signed-off-by: Derrick Stolee \n> ---\n> diff --git a/builtin/gc.c b/builtin/gc.c\n> @@ -1491,6 +1491,214 @@ static int maintenance_unregister(void)\n> +static int bootout(const char *filename)\n> +{\n> + int result;\n> + struct strvec args = STRVEC_INIT;\n> + char *uid = get_uid();\n> + const char *launchctl = getenv(\"GIT_TEST_CRONTAB\");\n> + if (!launchctl)\n> + launchctl = \"/bin/launchctl\";\n> +\n> + strvec_split(&args, launchctl);\n> + strvec_push(&args, \"bootout\");\n> + strvec_pushf(&args, \"gui/%s\", uid);\n> + strvec_push(&args, filename);\n> +\n> + result = run_command_v_opt(args.v, 0);\n> +\n> + strvec_clear(&args);\n> + free(uid);\n> + return result;\n> +}\n> +\n> +static int bootstrap(const char *filename)\n> +{\n> + int result;\n> + struct strvec args = STRVEC_INIT;\n> + char *uid = get_uid();\n> + const char *launchctl = getenv(\"GIT_TEST_CRONTAB\");\n> + if (!launchctl)\n> + launchctl = \"/bin/launchctl\";\n> +\n> + strvec_split(&args, launchctl);\n> + strvec_push(&args, \"bootstrap\");\n> + strvec_pushf(&args, \"gui/%s\", uid);\n> + strvec_push(&args, filename);\n> +\n> + result = run_command_v_opt(args.v, 0);\n> +\n> + strvec_clear(&args);\n> + free(uid);\n> + return result;\n> +}\n\nThe bootout() and bootstrap() functions seem to be identical except\nfor one string literal. Code could be reduced by refactoring and\npassing that string literal in as an argument.\n\n> +static int remove_plist(enum schedule_priority schedule)\n> +{\n> + const char *frequency = get_frequency(schedule);\n> + char *name = get_service_name(frequency);\n> + char *filename = get_service_filename(name);\n> + int result = bootout(filename);\n> + free(filename);\n> + free(name);\n> + return result;\n> +}\n\nThe result of get_service_name() is only ever passed to\nget_service_filename(). If get_service_filename() made the call to\nget_service_name() itself, it would free up callers from having to\nremember to free(name), thus reducing the likelihood of a possible\nleak.\n\n> +static int schedule_plist(const char *exec_path, enum schedule_priority schedule)\n> +{\n> + plist = fopen(filename, \"w\");\n> +\n> + if (!plist)\n> + die(_(\"failed to open '%s'\"), filename);\n\nYou can replace the fopen() and die() with a single call to xfopen().\n\n> + /* bootout might fail if not already running, so ignore */\n> + bootout(filename);\n> + if (bootstrap(filename))\n> + die(_(\"failed to bootstrap service %s\"), filename);\n\nI'm guessing that 'launchctl bootout' won't print a confusing and\nunexpected error message if the plist is not presently registered?\n\n> diff --git a/t/t7900-maintenance.sh b/t/t7900-maintenance.sh\n> @@ -389,12 +389,58 @@ test_expect_success 'stop from existing schedule' '\n> +test_expect_success MACOS_MAINTENANCE 'start and stop macOS maintenance' '\n> + echo \"#!/bin/sh\\necho \\$@ >>args\" >print-args &&\n> + chmod a+x print-args &&\n\nNote that $@ loses its special magic if not surrounded by quotes, thus\nacts just like $*. So, either use \"$@\" or $* depending upon your\nrequirements, but in the case of 'echo', it's just not going to matter\nat all, so $* is fine.\n\nTo construct the script, you can do this instead, which is easier to\nread and handles the 'chmod' for you:\n\n write_script print-args <<-\\EOF\n echo $* >>args\n EOF\n\n> + for frequency in hourly daily weekly\n> + do\n> + PLIST=\"$HOME/Library/LaunchAgents/org.git-scm.git.$frequency.plist\" &&\n> + grep schedule=$frequency \"$PLIST\" &&\n> + echo \"bootout gui/$UID $PLIST\" >>expect &&\n> + echo \"bootstrap gui/$UID $PLIST\" >>expect || return 1\n> + done &&\n\nMy gut feeling is that this would be more robust if you manually\ndetermine UID in the test script the same way as the git-maintenance\ncommand itself does using '/usr/bin/id -u' rather than relying upon\ninheriting UID from the user's environment.\n\n> + # stop does not remove plist files, but boots them out\n\nIs this desirable? Should git-maintenance do a better job of cleaning\nup after itself?\n"},{"id":"409034","messageId":"CAPig+cRE3RNkUCrmmTvVC4mm1crT+1OEU5zSwu80pjXmDdAkfg@mail.gmail.com","threadId":"54562","inReplyTo":"a9221cc4aa12192e9a691f8e1b77a3cc2d7e4952.1604412197.git.gitgitgadget@gmail.com","subject":"Re: [PATCH 3/3] maintenance: use Windows scheduled tasks","fromName":"Eric Sunshine","fromEmail":"sunshine@sunshineco.com","sentAt":"2020-11-03T19:06:38Z","receivedAt":"2020-11-03T19:06:54Z","isPatch":true,"sender":{"key":"sunshine@sunshineco.com","avatar":"https://avatars.githubusercontent.com/u/163641?v=4"},"body":"On Tue, Nov 3, 2020 at 9:05 AM Derrick Stolee via GitGitGadget\n wrote:\n> There is a deficiency in the current design. Windows has two kinds of\n> applications: GUI applications that start by \"winmain()\" and console\n> applications that start by \"main()\". Console applications are attached\n> to a new Console window if they are not already associated with a GUI\n> application. This means that every hour the scheudled task launches a\n> command window for the scheduled tasks. Not only is this visually\n> obtrusive, but it also takes focus from whatever else the user is\n> doing!\n\nI wonder if you could use the technique explained in [1] to prevent\nthe console window from popping up.\n\n[1]: https://pureinfotech.com/prevent-command-window-appearing-scheduled-tasks-windows-10/\n\n> Signed-off-by: Derrick Stolee \n> ---\n> diff --git a/t/t7900-maintenance.sh b/t/t7900-maintenance.sh\n> @@ -441,6 +441,40 @@ test_expect_success MACOS_MAINTENANCE 'start and stop macOS maintenance' '\n> +test_expect_success MINGW 'start and stop Windows maintenance' '\n> + echo \"echo \\$@ >>args\" >print-args &&\n> + chmod a+x print-args &&\n\nSame comments as my review of [2/3] regarding $@ and write_script().\n\n> + rm -f args &&\n> + GIT_TEST_CRONTAB=\"/bin/sh print-args\" git maintenance start &&\n> + cat args &&\n\nIs this 'cat' leftover debugging gunk?\n\n> + # start registers the repo\n> + git config --get --global maintenance.repo \"$(pwd)\" &&\n> +\n> + rm expect &&\n> + for frequency in hourly daily weekly\n> + do\n> + echo \"/create /tn Git Maintenance ($frequency) /f /xml .git/objects/schedule-$frequency.xml\" >>expect \\\n> + || return 1\n> + done &&\n\nRather than using >> within the loop, it's often simpler to capture\nthe output of the for-loop in its entirety:\n\n for frequency in hourly daily weekly\n do\n echo \"/create ...\" || return 1\n done >expect &&\n\nHowever, in this case 'printf' may be even simpler:\n\n printf \"/create /tn ... .git/objects/schedule-%s.xml\\n\" \\\n hourly daily weekly >expect &&\n\n> + GIT_TEST_CRONTAB=\"/bin/sh print-args\" git maintenance stop &&\n\nToo many spaces before the 'git' command.\n"},{"id":"409039","messageId":"xmqqtuu6qjnp.fsf@gitster.c.googlers.com","threadId":"54562","inReplyTo":"pull.776.git.1604412196.gitgitgadget@gmail.com","subject":"Re: [PATCH 0/3] Maintenance IV: Platform-specific background maintenance","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2020-11-03T20:18:02Z","receivedAt":"2020-11-03T20:18:07Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"\"Derrick Stolee via GitGitGadget\" writes:\n\n> This is based on ds/maintenance-part-3.\n>\n> After sitting with the background maintenance as it has been cooking, I\n> wanted to come back around and implement the background maintenance for\n> Windows. However, I noticed that there were some things bothering me with\n> background maintenance on my macOS machine. These are detailed in PATCH 2,\n> but the tl;dr is that 'cron' is not recommended by Apple and instead\n> 'launchd' satisfies our needs.\n\nNicely done.\n\n> This series implements the background scheduling so git maintenance\n> (start|stop) works on those platforms. I've been operating with these\n> schedules for a while now without the problems described in the patches.\n>\n> There is a particularly annoying case about console windows popping up on\n> Windows, but PATCH 3 describes a plan to get around that.\n>\n> Thanks, -Stolee\n>\n> Derrick Stolee (3):\n> maintenance: extract platform-specific scheduling\n> maintenance: use launchctl on macOS\n> maintenance: use Windows scheduled tasks\n>\n> builtin/gc.c | 428 +++++++++++++++++++++++++++++++++++++++--\n> t/t7900-maintenance.sh | 86 ++++++++-\n> t/test-lib.sh | 4 +\n> 3 files changed, 498 insertions(+), 20 deletions(-)\n>\n>\n> base-commit: 0016b618182f642771dc589cf0090289f9fe1b4f\n> Published-As: https://github.com/gitgitgadget/git/releases/tag/pr-776%2Fderrickstolee%2Fmaintenance%2FmacOS-v1\n> Fetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-776/derrickstolee/maintenance/macOS-v1\n> Pull-Request: https://github.com/gitgitgadget/git/pull/776\n"},{"id":"409040","messageId":"xmqqmtzyqjil.fsf@gitster.c.googlers.com","threadId":"54562","inReplyTo":"pull.776.git.1604412196.gitgitgadget@gmail.com","subject":"Re: [PATCH 0/3] Maintenance IV: Platform-specific background maintenance","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2020-11-03T20:21:06Z","receivedAt":"2020-11-03T20:21:13Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"\"Derrick Stolee via GitGitGadget\" writes:\n\n> This is based on ds/maintenance-part-3.\n\nAh, I forgot to ask those on CC list how carefully they read the\npart 3 of the series, as it's been left on 'seen' for some time, and\nI do not know if it is ready to start cooking in 'next'.\n\nThanks.\n"},{"id":"409043","messageId":"022c5792-e66c-71d8-fbf4-ead9231030ad@gmail.com","threadId":"54562","inReplyTo":"xmqqmtzyqjil.fsf@gitster.c.googlers.com","subject":"Re: [PATCH 0/3] Maintenance IV: Platform-specific background maintenance","fromName":"Derrick Stolee","fromEmail":"stolee@gmail.com","sentAt":"2020-11-03T21:09:46Z","receivedAt":"2020-11-03T21:17:57Z","isPatch":true,"sender":{"key":"stolee@gmail.com","avatar":"https://avatars.githubusercontent.com/u/570044?v=4"},"body":"On 11/3/2020 3:21 PM, Junio C Hamano wrote:\n> \"Derrick Stolee via GitGitGadget\" writes:\n> \n>> This is based on ds/maintenance-part-3.\n> \n> Ah, I forgot to ask those on CC list how carefully they read the\n> part 3 of the series, as it's been left on 'seen' for some time, and\n> I do not know if it is ready to start cooking in 'next'.\n\nIt has been a while since anyone has commented, and I've been\nrunning the patches locally for quite a while. I'd be very\nhappy to see them cook in next. I wasn't quite to the place\nto be pushy about it.\n\nI'm hoping that parts 3 and 4 can make it in time for 2.30,\nso the feature is universally available for all platforms.\nI realize that's not entirely up to just me.\n\nThanks,\n-Stolee\n"},{"id":"409044","messageId":"e493b759-4565-e7d2-4a19-dee23c651119@gmail.com","threadId":"54562","inReplyTo":"CAPig+cR1zSqY_y02JBdtjh_+9Pp2dOtGrHHmf-ah5sU6Lhzp6Q@mail.gmail.com","subject":"Re: [PATCH 2/3] maintenance: use launchctl on macOS","fromName":"Derrick Stolee","fromEmail":"stolee@gmail.com","sentAt":"2020-11-03T21:21:44Z","receivedAt":"2020-11-03T21:21:54Z","isPatch":true,"sender":{"key":"stolee@gmail.com","avatar":"https://avatars.githubusercontent.com/u/570044?v=4"},"body":"On 11/3/2020 1:45 PM, Eric Sunshine wrote:\n> On Tue, Nov 3, 2020 at 9:05 AM Derrick Stolee via GitGitGadget\n> wrote:\n>> maintenance: use launchctl on macOS\n> \n> A few comments below (not necessarily worth a re-roll)...\n> \n>> The launchctl command needs to be aligned with a user id in order\n>> to initialize the command environment. This must be done using\n>> the 'launchctl bootstrap' subcommand. This subcommand is new as\n>> of macOS 10.11, which was released in September 2015. Before that\n>> release the 'launchctl load' subcommand was recommended. The best\n>> source of information on this transition I have seen is available\n>> at [2].\n> \n> It's not clear whether or not this is saying that git-maintenance will\n> dynamically adapt to work with modern and older 'launchctl'. A glance\n> at the actual code reveals that it knows only about modern\n> 'bootstrap'. Perhaps this could be a bit clearer by saying that it\n> only supports modern versions, and that support for older versions can\n> be added later if needed. (For those of us who are stuck with 10-20\n> year old hardware and OS versions, 2015 isn't that long ago.)\n\nYes, this is a strange place to be. How far do we go back to support\nas many users as possible? How many users will be simultaneously\nstuck on an old version of macOS _and_ interested in updating to this\nlatest version of Git? Is that worth the extra functionality to detect\nthe the OS version and change commands?\n\nThe good news is that this patch doesn't lock us in to the\nboot(strap|out) subcommands too much. We could add in load/unload\nsubcommands for systems that are too old. However, I did think it\nwas prudent to take the currently-recommended option for fear that\nApple will completely _delete_ the load/unload options in an\nupcoming release.\n\nThis makes me realize that I should update the documentation to give\npointers for how to view the schedules for each platform:\n\n- Windows: Open \"Task Scheduler\"\n- macOS: 'launchctl list | grep org.git-scm.git'\n- Others: 'crontab -l'\n\n>> +static int bootstrap(const char *filename)\n>> +{\n>> + int result;\n>> + struct strvec args = STRVEC_INIT;\n>> + char *uid = get_uid();\n>> + const char *launchctl = getenv(\"GIT_TEST_CRONTAB\");\n>> + if (!launchctl)\n>> + launchctl = \"/bin/launchctl\";\n>> +\n>> + strvec_split(&args, launchctl);\n>> + strvec_push(&args, \"bootstrap\");\n>> + strvec_pushf(&args, \"gui/%s\", uid);\n>> + strvec_push(&args, filename);\n>> +\n>> + result = run_command_v_opt(args.v, 0);\n>> +\n>> + strvec_clear(&args);\n>> + free(uid);\n>> + return result;\n>> +}\n> \n> The bootout() and bootstrap() functions seem to be identical except\n> for one string literal. Code could be reduced by refactoring and\n> passing that string literal in as an argument.\n\nGood point. Or a simple boolean value for \"add\" or \"remove\".\n \n>> +static int remove_plist(enum schedule_priority schedule)\n>> +{\n>> + const char *frequency = get_frequency(schedule);\n>> + char *name = get_service_name(frequency);\n>> + char *filename = get_service_filename(name);\n>> + int result = bootout(filename);\n>> + free(filename);\n>> + free(name);\n>> + return result;\n>> +}\n> \n> The result of get_service_name() is only ever passed to\n> get_service_filename(). If get_service_filename() made the call to\n> get_service_name() itself, it would free up callers from having to\n> remember to free(name), thus reducing the likelihood of a possible\n> leak.\n\nYou're right. In an earlier version I thought I needed to add the\nname in the XML, but it turns out I did not.\n\n>> +static int schedule_plist(const char *exec_path, enum schedule_priority schedule)\n>> +{\n>> + plist = fopen(filename, \"w\");\n>> +\n>> + if (!plist)\n>> + die(_(\"failed to open '%s'\"), filename);\n> \n> You can replace the fopen() and die() with a single call to xfopen().\n\nThanks! I'll use that in several places and try to remember next time.\n\n>> + /* bootout might fail if not already running, so ignore */\n>> + bootout(filename);\n>> + if (bootstrap(filename))\n>> + die(_(\"failed to bootstrap service %s\"), filename);\n> \n> I'm guessing that 'launchctl bootout' won't print a confusing and\n> unexpected error message if the plist is not presently registered?\n\nYou're right, it does. It also returns with a non-zero exit code.\nAlong with your later suggestion to clear the .plist files, we will\nwant to have several conditions to not error out during a case where\nthe task is not scheduled.\n\n>> diff --git a/t/t7900-maintenance.sh b/t/t7900-maintenance.sh\n>> @@ -389,12 +389,58 @@ test_expect_success 'stop from existing schedule' '\n>> +test_expect_success MACOS_MAINTENANCE 'start and stop macOS maintenance' '\n>> + echo \"#!/bin/sh\\necho \\$@ >>args\" >print-args &&\n>> + chmod a+x print-args &&\n> \n> Note that $@ loses its special magic if not surrounded by quotes, thus\n> acts just like $*. So, either use \"$@\" or $* depending upon your\n> requirements, but in the case of 'echo', it's just not going to matter\n> at all, so $* is fine.\n> \n> To construct the script, you can do this instead, which is easier to\n> read and handles the 'chmod' for you:\n> \n> write_script print-args <<-\\EOF\n> echo $* >>args\n> EOF\n\nTIL. thanks.\n\n>> + for frequency in hourly daily weekly\n>> + do\n>> + PLIST=\"$HOME/Library/LaunchAgents/org.git-scm.git.$frequency.plist\" &&\n>> + grep schedule=$frequency \"$PLIST\" &&\n>> + echo \"bootout gui/$UID $PLIST\" >>expect &&\n>> + echo \"bootstrap gui/$UID $PLIST\" >>expect || return 1\n>> + done &&\n> \n> My gut feeling is that this would be more robust if you manually\n> determine UID in the test script the same way as the git-maintenance\n> command itself does using '/usr/bin/id -u' rather than relying upon\n> inheriting UID from the user's environment.\n\nYeah, you're right. Thanks!\n\n>> + # stop does not remove plist files, but boots them out\n> \n> Is this desirable? Should git-maintenance do a better job of cleaning\n> up after itself?\n\nYes, let's clear up these .plist files.\n\nThanks!\n-Stolee\n"},{"id":"409047","messageId":"3268cfb4-22b4-301d-abbc-bb36fa737a2a@gmail.com","threadId":"54562","inReplyTo":"CAPig+cRE3RNkUCrmmTvVC4mm1crT+1OEU5zSwu80pjXmDdAkfg@mail.gmail.com","subject":"Re: [PATCH 3/3] maintenance: use Windows scheduled tasks","fromName":"Derrick Stolee","fromEmail":"stolee@gmail.com","sentAt":"2020-11-03T21:23:52Z","receivedAt":"2020-11-03T21:23:59Z","isPatch":true,"sender":{"key":"stolee@gmail.com","avatar":"https://avatars.githubusercontent.com/u/570044?v=4"},"body":"On 11/3/2020 2:06 PM, Eric Sunshine wrote:\n> On Tue, Nov 3, 2020 at 9:05 AM Derrick Stolee via GitGitGadget\n> wrote:\n>> There is a deficiency in the current design. Windows has two kinds of\n>> applications: GUI applications that start by \"winmain()\" and console\n>> applications that start by \"main()\". Console applications are attached\n>> to a new Console window if they are not already associated with a GUI\n>> application. This means that every hour the scheudled task launches a\n>> command window for the scheduled tasks. Not only is this visually\n>> obtrusive, but it also takes focus from whatever else the user is\n>> doing!\n> \n> I wonder if you could use the technique explained in [1] to prevent\n> the console window from popping up.\n> \n> [1]: https://pureinfotech.com/prevent-command-window-appearing-scheduled-tasks-windows-10/\n\nThe critical part of that strategy is the \"Run whether the user is\nlogged in or not\". The resulting option that triggers causes the\nschtasks command to require a password prompt (or a password passed\nas a command-line argument). I found that interaction to be too\ndisruptive.\n\n>> Signed-off-by: Derrick Stolee \n>> ---\n>> diff --git a/t/t7900-maintenance.sh b/t/t7900-maintenance.sh\n>> @@ -441,6 +441,40 @@ test_expect_success MACOS_MAINTENANCE 'start and stop macOS maintenance' '\n>> +test_expect_success MINGW 'start and stop Windows maintenance' '\n>> + echo \"echo \\$@ >>args\" >print-args &&\n>> + chmod a+x print-args &&\n> \n> Same comments as my review of [2/3] regarding $@ and write_script().\n\nNoted!\n\n>> + rm -f args &&\n>> + GIT_TEST_CRONTAB=\"/bin/sh print-args\" git maintenance start &&\n>> + cat args &&\n> \n> Is this 'cat' leftover debugging gunk?\n\nYes. Thanks.\n\n>> + # start registers the repo\n>> + git config --get --global maintenance.repo \"$(pwd)\" &&\n>> +\n>> + rm expect &&\n>> + for frequency in hourly daily weekly\n>> + do\n>> + echo \"/create /tn Git Maintenance ($frequency) /f /xml .git/objects/schedule-$frequency.xml\" >>expect \\\n>> + || return 1\n>> + done &&\n> \n> Rather than using >> within the loop, it's often simpler to capture\n> the output of the for-loop in its entirety:\n> \n> for frequency in hourly daily weekly\n> do\n> echo \"/create ...\" || return 1\n> done >expect &&\n> \n> However, in this case 'printf' may be even simpler:\n> \n> printf \"/create /tn ... .git/objects/schedule-%s.xml\\n\" \\\n> hourly daily weekly >expect &&\n\nExcellent.\n\n>> + GIT_TEST_CRONTAB=\"/bin/sh print-args\" git maintenance stop &&\n> \n> Too many spaces before the 'git' command.\n\nThanks!\n-Stolee\n\n"},{"id":"409052","messageId":"CAPig+cSbNGdRA11cMCugJH1HtY65smfuE9ThARX0EvuQZnA5rA@mail.gmail.com","threadId":"54562","inReplyTo":"e493b759-4565-e7d2-4a19-dee23c651119@gmail.com","subject":"Re: [PATCH 2/3] maintenance: use launchctl on macOS","fromName":"Eric Sunshine","fromEmail":"sunshine@sunshineco.com","sentAt":"2020-11-03T22:27:55Z","receivedAt":"2020-11-03T22:28:09Z","isPatch":true,"sender":{"key":"sunshine@sunshineco.com","avatar":"https://avatars.githubusercontent.com/u/163641?v=4"},"body":"On Tue, Nov 3, 2020 at 4:22 PM Derrick Stolee wrote:\n> On 11/3/2020 1:45 PM, Eric Sunshine wrote:\n> > It's not clear whether or not this is saying that git-maintenance will\n> > dynamically adapt to work with modern and older 'launchctl'. A glance\n> > at the actual code reveals that it knows only about modern\n> > 'bootstrap'. Perhaps this could be a bit clearer by saying that it\n> > only supports modern versions, and that support for older versions can\n> > be added later if needed. (For those of us who are stuck with 10-20\n> > year old hardware and OS versions, 2015 isn't that long ago.)\n>\n> Yes, this is a strange place to be. How far do we go back to support\n> as many users as possible? How many users will be simultaneously\n> stuck on an old version of macOS _and_ interested in updating to this\n> latest version of Git? Is that worth the extra functionality to detect\n> the the OS version and change commands?\n\nI don't think this patch series needs to answer these questions\nprovided that it doesn't close the door to someone adding\nolder-version support down the road. My review comment was more about\nthe commit message being clearer about the choice -- supporting only\nrecent 'launchctl' -- being made by this series. (And perhaps the\ndocumentation could mention that it requires a reasonably modern\n'launchctl'.)\n\n> This makes me realize that I should update the documentation to give\n> pointers for how to view the schedules for each platform:\n>\n> - Windows: Open \"Task Scheduler\"\n> - macOS: 'launchctl list | grep org.git-scm.git'\n> - Others: 'crontab -l'\n\nGood idea.\n\nI haven't looked at the documentation, but if it doesn't already, I\nwonder if it should give examples of how to set these up by hand or\nhow to customize the ones created by git-maintenance itself. I was\nalso wondering if git-maintenance could have a mode in which it\ngenerates the template file(s) for you but doesn't actually\nactivate/install it, instead providing instructions for\nactivation/installation. That way, people could modify the scheduling\nfile before actually activating it. However, this may all be outside\nthe scope of the patch series, and could be done later if desired.\n"},{"id":"409053","messageId":"xmqqo8keoyy8.fsf@gitster.c.googlers.com","threadId":"54562","inReplyTo":"022c5792-e66c-71d8-fbf4-ead9231030ad@gmail.com","subject":"Re: [PATCH 0/3] Maintenance IV: Platform-specific background maintenance","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2020-11-03T22:30:39Z","receivedAt":"2020-11-03T22:30:44Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Derrick Stolee writes:\n\n> On 11/3/2020 3:21 PM, Junio C Hamano wrote:\n>> \"Derrick Stolee via GitGitGadget\" writes:\n>> \n>>> This is based on ds/maintenance-part-3.\n>> \n>> Ah, I forgot to ask those on CC list how carefully they read the\n>> part 3 of the series, as it's been left on 'seen' for some time, and\n>> I do not know if it is ready to start cooking in 'next'.\n>\n> It has been a while since anyone has commented, and I've been\n> running the patches locally for quite a while. I'd be very\n> happy to see them cook in next. I wasn't quite to the place\n> to be pushy about it.\n>\n> I'm hoping that parts 3 and 4 can make it in time for 2.30,\n> so the feature is universally available for all platforms.\n> I realize that's not entirely up to just me.\n\nAfter writing this entry in the What's cooking report:\n\n * ds/maintenance-part-4 (2020-11-03) 3 commits\n - maintenance: use Windows scheduled tasks\n - maintenance: use launchctl on macOS\n - maintenance: extract platform-specific scheduling\n (this branch uses ds/maintenance-part-3.)\n\n Follow-up on the \"maintenance part-3\" which introduced scheduled\n maintenance tasks to support platforms whose native scheduling\n methods are not 'cron'.\n\nI was wondering if I should propose making these two parts into one,\nso we may be pretty much on the same page.\n\n"},{"id":"409069","messageId":"af2befd7-7d93-5981-42c8-700e49f3999a@gmail.com","threadId":"54562","inReplyTo":"xmqqo8keoyy8.fsf@gitster.c.googlers.com","subject":"Re: [PATCH 0/3] Maintenance IV: Platform-specific background maintenance","fromName":"Derrick Stolee","fromEmail":"stolee@gmail.com","sentAt":"2020-11-04T13:02:34Z","receivedAt":"2020-11-04T13:02:40Z","isPatch":true,"sender":{"key":"stolee@gmail.com","avatar":"https://avatars.githubusercontent.com/u/570044?v=4"},"body":"On 11/3/2020 5:30 PM, Junio C Hamano wrote:\n> Derrick Stolee writes:\n>> I'm hoping that parts 3 and 4 can make it in time for 2.30,\n>> so the feature is universally available for all platforms.\n>> I realize that's not entirely up to just me.\n> \n> After writing this entry in the What's cooking report:\n> \n> * ds/maintenance-part-4 (2020-11-03) 3 commits\n> - maintenance: use Windows scheduled tasks\n> - maintenance: use launchctl on macOS\n> - maintenance: extract platform-specific scheduling\n> (this branch uses ds/maintenance-part-3.)\n> \n> Follow-up on the \"maintenance part-3\" which introduced scheduled\n> maintenance tasks to support platforms whose native scheduling\n> methods are not 'cron'.\n\nThis is a good summary of this series.\n\n> I was wondering if I should propose making these two parts into one,\n> so we may be pretty much on the same page.\n\nI'm happy to have the two topics be merged into one series, but\nI'd prefer to only re-roll these three patches during review.\nOf course, by keeping them together we have the ability to re-\nroll all of the patches, but part-3 has a decent length and has\nbeen stable for a while.\n\nI defer to what is easiest for you.\n\nThanks,\n-Stolee\n\n"},{"id":"409078","messageId":"42151abf-6a1a-d56b-48b8-cf122103fbc5@gmail.com","threadId":"54562","inReplyTo":"CAPig+cSbNGdRA11cMCugJH1HtY65smfuE9ThARX0EvuQZnA5rA@mail.gmail.com","subject":"Re: [PATCH 2/3] maintenance: use launchctl on macOS","fromName":"Derrick Stolee","fromEmail":"stolee@gmail.com","sentAt":"2020-11-04T13:33:51Z","receivedAt":"2020-11-04T13:33:56Z","isPatch":true,"sender":{"key":"stolee@gmail.com","avatar":"https://avatars.githubusercontent.com/u/570044?v=4"},"body":"On 11/3/2020 5:27 PM, Eric Sunshine wrote:\n> On Tue, Nov 3, 2020 at 4:22 PM Derrick Stolee wrote:\n>> On 11/3/2020 1:45 PM, Eric Sunshine wrote:\n>>> It's not clear whether or not this is saying that git-maintenance will\n>>> dynamically adapt to work with modern and older 'launchctl'. A glance\n>>> at the actual code reveals that it knows only about modern\n>>> 'bootstrap'. Perhaps this could be a bit clearer by saying that it\n>>> only supports modern versions, and that support for older versions can\n>>> be added later if needed. (For those of us who are stuck with 10-20\n>>> year old hardware and OS versions, 2015 isn't that long ago.)\n>>\n>> Yes, this is a strange place to be. How far do we go back to support\n>> as many users as possible? How many users will be simultaneously\n>> stuck on an old version of macOS _and_ interested in updating to this\n>> latest version of Git? Is that worth the extra functionality to detect\n>> the the OS version and change commands?\n> \n> I don't think this patch series needs to answer these questions\n> provided that it doesn't close the door to someone adding\n> older-version support down the road. My review comment was more about\n> the commit message being clearer about the choice -- supporting only\n> recent 'launchctl' -- being made by this series. (And perhaps the\n> documentation could mention that it requires a reasonably modern\n> 'launchctl'.)\n\nThanks. I'll be sure to make the commit message more clear.\n \n>> This makes me realize that I should update the documentation to give\n>> pointers for how to view the schedules for each platform:\n>>\n>> - Windows: Open \"Task Scheduler\"\n>> - macOS: 'launchctl list | grep org.git-scm.git'\n>> - Others: 'crontab -l'\n> \n> Good idea.\n> \n> I haven't looked at the documentation, but if it doesn't already, I\n> wonder if it should give examples of how to set these up by hand or\n> how to customize the ones created by git-maintenance itself. I was\n> also wondering if git-maintenance could have a mode in which it\n> generates the template file(s) for you but doesn't actually\n> activate/install it, instead providing instructions for\n> activation/installation. That way, people could modify the scheduling\n> file before actually activating it. However, this may all be outside\n> the scope of the patch series, and could be done later if desired.\n\nOutside of the technical details, the biggest questions I've\ntried to handle with the background maintenance feature has\nbeen to balance customization with ease-of-use. My philosophy\nis that users fall into a few expertise buckets, and have\ndifferent expectations:\n\n* Beginners will never know about background maintenance and\n so will never run \"git maintenance start\" or set the config\n values.\n\n* Intermediate users might discover \"git maintenance start\"\n and will appreciate that they don't need to learn cron to\n set up a good default schedule.\n\n* Advanced users will read the documentation and use Git\n config settings to customize their maintenance tasks and\n schedule.\n\n* Expert users might decide the task schedule available by\n \"git maintenance start\" is too restrictive, so they create\n their own background schedule with custom tasks. They might\n not even run the maintenance builtin and opt instead for\n 'git gc' or 'git repack' directly!\n\nMy main target has been these \"intermediate\" users who might\nrun the command and forget about it. However, I've also tried\nto keep the advanced users in mind with the advanced config\noptions available.\n\nYour comment about documentation demonstrates a way to serve\nthe advanced and expert users by providing a clear framework\nfor discovering what Git is doing under the hood and how to\nmodify or adapt that to their needs. It is also important to\ndemonstrate how to set up schedules in a similar way without\nhaving them be overwritten by a later \"git maintenance start\"\ncommand.\n\nI will give this a shot in v2. Thanks.\n\n-Stolee\n"},{"id":"409082","messageId":"51d193c0-ef0c-aa5c-56c6-388499eaf2a7@gmail.com","threadId":"54562","inReplyTo":"e493b759-4565-e7d2-4a19-dee23c651119@gmail.com","subject":"Re: [PATCH 2/3] maintenance: use launchctl on macOS","fromName":"Derrick Stolee","fromEmail":"stolee@gmail.com","sentAt":"2020-11-04T14:17:13Z","receivedAt":"2020-11-04T14:17:19Z","isPatch":true,"sender":{"key":"stolee@gmail.com","avatar":"https://avatars.githubusercontent.com/u/570044?v=4"},"body":"On 11/3/2020 4:21 PM, Derrick Stolee wrote:\n> On 11/3/2020 1:45 PM, Eric Sunshine wrote:\n>> On Tue, Nov 3, 2020 at 9:05 AM Derrick Stolee via GitGitGadget\n>>> +static int remove_plist(enum schedule_priority schedule)\n>>> +{\n>>> + const char *frequency = get_frequency(schedule);\n>>> + char *name = get_service_name(frequency);\n>>> + char *filename = get_service_filename(name);\n>>> + int result = bootout(filename);\n>>> + free(filename);\n>>> + free(name);\n>>> + return result;\n>>> +}\n>>\n>> The result of get_service_name() is only ever passed to\n>> get_service_filename(). If get_service_filename() made the call to\n>> get_service_name() itself, it would free up callers from having to\n>> remember to free(name), thus reducing the likelihood of a possible\n>> leak.\n> \n> You're right. In an earlier version I thought I needed to add the\n> name in the XML, but it turns out I did not.\n\nAs I go through the effort to remove get_service_name() I find that\nactually the name is used in one place in the XML file:\n\n+\t\t \"Label%s\\n\"\n\nThis \"Label\" should match the filename, I believe. I can still\nbe more careful about how often this name is actually required.\n\nThanks,\n-Stolee\n"},{"id":"409098","messageId":"xmqqblgdnjjx.fsf@gitster.c.googlers.com","threadId":"54562","inReplyTo":"af2befd7-7d93-5981-42c8-700e49f3999a@gmail.com","subject":"Re: [PATCH 0/3] Maintenance IV: Platform-specific background maintenance","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2020-11-04T17:00:50Z","receivedAt":"2020-11-04T17:00:54Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Derrick Stolee writes:\n\n> On 11/3/2020 5:30 PM, Junio C Hamano wrote:\n>> Derrick Stolee writes:\n>>> I'm hoping that parts 3 and 4 can make it in time for 2.30,\n>>> so the feature is universally available for all platforms.\n>>> I realize that's not entirely up to just me.\n>> ...\n>> I was wondering if I should propose making these two parts into one,\n>> so we may be pretty much on the same page.\n>\n> I'm happy to have the two topics be merged into one series, but\n> I'd prefer to only re-roll these three patches during review.\n> Of course, by keeping them together we have the ability to re-\n> roll all of the patches, but part-3 has a decent length and has\n> been stable for a while.\n>\n> I defer to what is easiest for you.\n\nTwo topics, one on top of the other, is fine, as long as I can\nremember (and you help me by reminding) to keep them together,\nand the bottom one is reasonably solid that I do not have to do the\nrebasing myself ;-)\n\n"},{"id":"409122","messageId":"56d6524f-c8ac-a303-d9e4-aef1b082dc74@gmail.com","threadId":"54562","inReplyTo":"xmqqblgdnjjx.fsf@gitster.c.googlers.com","subject":"Re: [PATCH 0/3] Maintenance IV: Platform-specific background maintenance","fromName":"Derrick Stolee","fromEmail":"stolee@gmail.com","sentAt":"2020-11-04T18:43:27Z","receivedAt":"2020-11-04T18:43:32Z","isPatch":true,"sender":{"key":"stolee@gmail.com","avatar":"https://avatars.githubusercontent.com/u/570044?v=4"},"body":"On 11/4/2020 12:00 PM, Junio C Hamano wrote:\n> Derrick Stolee writes:\n> \n>> On 11/3/2020 5:30 PM, Junio C Hamano wrote:\n>>> Derrick Stolee writes:\n>>>> I'm hoping that parts 3 and 4 can make it in time for 2.30,\n>>>> so the feature is universally available for all platforms.\n>>>> I realize that's not entirely up to just me.\n>>> ...\n>>> I was wondering if I should propose making these two parts into one,\n>>> so we may be pretty much on the same page.\n>>\n>> I'm happy to have the two topics be merged into one series, but\n>> I'd prefer to only re-roll these three patches during review.\n>> Of course, by keeping them together we have the ability to re-\n>> roll all of the patches, but part-3 has a decent length and has\n>> been stable for a while.\n>>\n>> I defer to what is easiest for you.\n> \n> Two topics, one on top of the other, is fine, as long as I can\n> remember (and you help me by reminding) to keep them together,\n> and the bottom one is reasonably solid that I do not have to do the\n> rebasing myself ;-)\n\nSounds good to me. If I need to re-roll part 3, then\nI'll consider replacing both parts with a new series.\n\nThanks,\n-Stolee\n\n"},{"id":"409131","messageId":"pull.776.v2.git.1604520368.gitgitgadget@gmail.com","threadId":"54562","inReplyTo":"pull.776.git.1604412196.gitgitgadget@gmail.com","subject":"[PATCH v2 0/4] Maintenance IV: Platform-specific background maintenance","fromName":"Derrick Stolee via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2020-11-04T20:06:04Z","receivedAt":"2020-11-04T20:06:16Z","isPatch":true,"sender":{"key":"stolee@gmail.com","avatar":"https://avatars.githubusercontent.com/u/570044?v=4"},"body":"This is based on ds/maintenance-part-3.\n\nAfter sitting with the background maintenance as it has been cooking, I\nwanted to come back around and implement the background maintenance for\nWindows. However, I noticed that there were some things bothering me with\nbackground maintenance on my macOS machine. These are detailed in PATCH 3,\nbut the tl;dr is that 'cron' is not recommended by Apple and instead\n'launchd' satisfies our needs.\n\nThis series implements the background scheduling so git maintenance\n(start|stop) works on those platforms. I've been operating with these\nschedules for a while now without the problems described in the patches.\n\nThere is a particularly annoying case about console windows popping up on\nWindows, but PATCH 4 describes a plan to get around that.\n\nUpdates in V2\n=============\n\n * This is a faster turnaround for a v2 than I would normally like, but Eric\n inspired extra documentation about how to customize background schedules.\n \n \n * New extensions to git-maintenance.txt include guidelines for inspecting\n what git maintenance start does and how to customize beyond that. This\n includes a new PATCH 2 that includes documentation for 'cron' on\n non-macOS non-Windows systems.\n \n \n * Several improvements, especially in the tests, are included.\n \n \n * While testing manually, I noticed that somehow I had incorrectly had an\n opening tag instead of a closing tag in the hourly format\n on macOS. I found that the xmllint tool can verify the XML format of a\n file, which catches the bug. This seems like a good approach since the\n test is macOS-only. Does anyone have concerns about adding this\n dependency?\n \n \n\nThanks, -Stolee\n\nDerrick Stolee (4):\n maintenance: extract platform-specific scheduling\n maintenance: include 'cron' details in docs\n maintenance: use launchctl on macOS\n maintenance: use Windows scheduled tasks\n\n Documentation/git-maintenance.txt | 119 +++++++++\n builtin/gc.c | 428 ++++++++++++++++++++++++++++--\n t/t7900-maintenance.sh | 83 +++++-\n t/test-lib.sh | 4 +\n 4 files changed, 614 insertions(+), 20 deletions(-)\n\n\nbase-commit: 0016b618182f642771dc589cf0090289f9fe1b4f\nPublished-As: https://github.com/gitgitgadget/git/releases/tag/pr-776%2Fderrickstolee%2Fmaintenance%2FmacOS-v2\nFetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-776/derrickstolee/maintenance/macOS-v2\nPull-Request: https://github.com/gitgitgadget/git/pull/776\n\nRange-diff vs v1:\n\n 1: d35f1aa162 = 1: d35f1aa162 maintenance: extract platform-specific scheduling\n -: ---------- > 2: 709a173720 maintenance: include 'cron' details in docs\n 2: 832fdf1687 ! 3: 0fafd75d10 maintenance: use launchctl on macOS\n @@ Commit message\n and 'git maintenance stop' by injecting a script that writes the\n command-line arguments into GIT_TEST_CRONTAB.\n \n + An earlier version of this patch accidentally had an opening\n + \"\" tag when it should have had a closing \"\" tag. This\n + was caught during manual testing with actual 'launchctl' commands,\n + but we do not want to update developers' tasks when running tests.\n + It appears that macOS includes the \"xmllint\" tool which can verify\n + the XML format, so call it from the macOS-specific tests to ensure\n + the .plist files are well-formatted.\n +\n + Helped-by: Eric Sunshine \n Signed-off-by: Derrick Stolee \n \n + ## Documentation/git-maintenance.txt ##\n +@@ Documentation/git-maintenance.txt: for advanced scheduling techniques. Please do use the full path and\n + executing the correct binaries in your schedule.\n + \n + \n ++BACKGROUND MAINTENANCE ON MACOS SYSTEMS\n ++---------------------------------------\n ++\n ++While macOS technically supports `cron`, using `crontab -e` requires\n ++elevated privileges and the executed process do not have a full user\n ++context. Without a full user context, Git and its credential helpers\n ++cannot access stored credentials, so some maintenance tasks are not\n ++functional.\n ++\n ++Instead, `git maintenance start` interacts with the `launchctl` tool,\n ++which is the recommended way to\n ++https://developer.apple.com/library/archive/documentation/MacOSX/Conceptual/BPSystemStartup/Chapters/ScheduledJobs.html[schedule timed jobs in macOS].\n ++\n ++Scheduling maintenance through `git maintenance (start|stop)` requires\n ++some `launchctl` features available only in macOS 10.11 or later.\n ++\n ++Your user-specific scheduled tasks are stored as XML-formatted `.plist`\n ++files in `~/Library/LaunchAgents/`. You can see the currently-registered\n ++tasks using the following command:\n ++\n ++-----------------------------------------------------------------------\n ++$ ls ~/Library/LaunchAgents/ | grep org.git-scm.git\n ++org.git-scm.git.daily.plist\n ++org.git-scm.git.hourly.plist\n ++org.git-scm.git.weekly.plist\n ++-----------------------------------------------------------------------\n ++\n ++One task is registered for each `--schedule=` option. To\n ++inspect how the XML format describes each schedule, open one of these\n ++`.plist` files in an editor and inspect the `` element following\n ++the `StartCalendarInterval` element.\n ++\n ++`git maintenance start` will overwrite these files and register the\n ++tasks again with `launchctl`, so any customizations should be done by\n ++creating your own `.plist` files with distinct names. Similarly, the\n ++`git maintenance stop` command will unregister the tasks with `launchctl`\n ++and delete the `.plist` files.\n ++\n ++To create more advanced customizations to your background tasks, see\n ++https://developer.apple.com/library/archive/documentation/MacOSX/Conceptual/BPSystemStartup/Chapters/CreatingLaunchdJobs.html#//apple_ref/doc/uid/TP40001762-104142[the `launchctl` documentation]\n ++for more information.\n ++\n ++\n + GIT\n + ---\n + Part of the linkgit:git[1] suite\n +\n ## builtin/gc.c ##\n @@ builtin/gc.c: static int maintenance_unregister(void)\n \treturn run_command(&config_unset);\n @@ builtin/gc.c: static int maintenance_unregister(void)\n +\t\trepeat = \"\\n\"\n +\t\t\t \"Hour%d\\n\"\n +\t\t\t \"Minute0\\n\"\n -+\t\t\t \"\\n\";\n ++\t\t\t \"\\n\";\n +\t\tfor (i = 1; i <= 23; i++)\n +\t\t\tfprintf(plist, repeat, i);\n +\t\tbreak;\n @@ t/t7900-maintenance.sh: test_expect_success 'stop from existing schedule' '\n +\tfor frequency in hourly daily weekly\n +\tdo\n +\t\tPLIST=\"$HOME/Library/LaunchAgents/org.git-scm.git.$frequency.plist\" &&\n ++\t\txmllint \"$PLIST\" >/dev/null &&\n +\t\tgrep schedule=$frequency \"$PLIST\" &&\n +\t\techo \"bootout gui/$UID $PLIST\" >>expect &&\n +\t\techo \"bootstrap gui/$UID $PLIST\" >>expect || return 1\n 3: a9221cc4aa ! 4: 84eb44de31 maintenance: use Windows scheduled tasks\n @@ Commit message\n short term. In the long term, we can consider creating this GUI\n shim application within core Git, perhaps in contrib/.\n \n + Helped-by: Eric Sunshine \n Signed-off-by: Derrick Stolee \n \n + ## Documentation/git-maintenance.txt ##\n +@@ Documentation/git-maintenance.txt: https://developer.apple.com/library/archive/documentation/MacOSX/Conceptual/BPSy\n + for more information.\n + \n + \n ++BACKGROUND MAINTENANCE ON WINDOWS SYSTEMS\n ++-----------------------------------------\n ++\n ++Windows does not support `cron` and instead has its own system for\n ++scheduling background tasks. The `git maintenance start` command uses\n ++the `schtasks` command to submit tasks to this system. You can inspect\n ++all background tasks using the Task Scheduler application. The tasks\n ++added by Git have names of the form `Git Maintenance ()`.\n ++The Task Scheduler GUI has ways to inspect these tasks, but you can also\n ++export the tasks to XML files and view the details there.\n ++\n ++Note that since Git is a console application, these background tasks\n ++create a console window visible to the current user. This can be changed\n ++manually by selecting the \"Run whether user is logged in or not\" option\n ++in Task Scheduler. This change requires a password input, which is why\n ++`git maintenance start` does not select it by default.\n ++\n ++If you want to customize the background tasks, please rename the tasks\n ++so future calls to `git maintenance (start|stop)` do not overwrite your\n ++custom tasks.\n ++\n ++\n + GIT\n + ---\n + Part of the linkgit:git[1] suite\n +\n ## builtin/gc.c ##\n @@ builtin/gc.c: static int platform_update_schedule(int run_maintenance, int fd)\n \telse\n @@ t/t7900-maintenance.sh: test_expect_success MACOS_MAINTENANCE 'start and stop ma\n '\n \n +test_expect_success MINGW 'start and stop Windows maintenance' '\n -+\techo \"echo \\$@ >>args\" >print-args &&\n -+\tchmod a+x print-args &&\n ++\twrite_script print-args <<-\\EOF &&\n ++\techo $* >>args\n ++\tEOF\n +\n +\trm -f args &&\n +\tGIT_TEST_CRONTAB=\"/bin/sh print-args\" git maintenance start &&\n -+\tcat args &&\n +\n +\t# start registers the repo\n +\tgit config --get --global maintenance.repo \"$(pwd)\" &&\n +\n -+\trm expect &&\n +\tfor frequency in hourly daily weekly\n +\tdo\n -+\t\techo \"/create /tn Git Maintenance ($frequency) /f /xml .git/objects/schedule-$frequency.xml\" >>expect \\\n -+\t\t\t|| return 1\n -+\tdone &&\n ++\t\tprintf \"/create /tn Git Maintenance (%s) /f /xml .git/objects/schedule-%s.xml\\n\" \\\n ++\t\t\t$frequency $frequency\n ++\tdone >expect &&\n +\ttest_cmp expect args &&\n +\n +\trm -f args &&\n -+\tGIT_TEST_CRONTAB=\"/bin/sh print-args\" git maintenance stop &&\n ++\tGIT_TEST_CRONTAB=\"/bin/sh print-args\" git maintenance stop &&\n +\n +\t# stop does not unregister the repo\n +\tgit config --get --global maintenance.repo \"$(pwd)\" &&\n +\n +\trm expect &&\n -+\tfor frequency in hourly daily weekly\n -+\tdo\n -+\t\techo \"/delete /tn Git Maintenance ($frequency) /f\" >>expect \\\n -+\t\t\t|| return 1\n -+\tdone &&\n ++\tprintf \"/delete /tn Git Maintenance (%s) /f\\n\" \\\n ++\t\thourly daily weekly >expect &&\n +\ttest_cmp expect args\n +'\n +\n\n-- \ngitgitgadget\n"},{"id":"409132","messageId":"0fafd75d100f343f7cff6471772ed9b12793f81e.1604520368.git.gitgitgadget@gmail.com","threadId":"54562","inReplyTo":"pull.776.v2.git.1604520368.gitgitgadget@gmail.com","subject":"[PATCH v2 3/4] maintenance: use launchctl on macOS","fromName":"Derrick Stolee via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2020-11-04T20:06:07Z","receivedAt":"2020-11-04T20:06:17Z","isPatch":true,"sender":{"key":"stolee@gmail.com","avatar":"https://avatars.githubusercontent.com/u/570044?v=4"},"body":"From: Derrick Stolee \n\nThe existing mechanism for scheduling background maintenance is done\nthrough cron. The 'crontab -e' command allows updating the schedule\nwhile cron itself runs those commands. While this is technically\nsupported by macOS, it has some significant deficiencies:\n\n1. Every run of 'crontab -e' must request elevated privileges through\n the user interface. When running 'git maintenance start' from the\n Terminal app, it presents a dialog box saying \"Terminal.app would\n like to administer your computer. Administration can include\n modifying passwords, networking, and system settings.\" This is more\n alarming than what we are hoping to achieve. If this alert had some\n information about how \"git\" is trying to run \"crontab\" then we would\n have some reason to believe that this dialog might be fine. However,\n it also doesn't help that some scenarios just leave Git waiting for\n a response without presenting anything to the user. I experienced\n this when executing the command from a Bash terminal view inside\n Visual Studio Code.\n\n2. While cron initializes a user environment enough for \"git config\n --global --show-origin\" to show the correct config file information,\n it does not set up the environment enough for Git Credential Manager\n Core to load credentials during a 'prefetch' task. My prefetches\n against private repositories required re-authenticating through UI\n pop-ups in a way that should not be required.\n\nThe solution is to switch from cron to the Apple-recommended [1]\n'launchd' tool.\n\n[1] https://developer.apple.com/library/archive/documentation/MacOSX/Conceptual/BPSystemStartup/Chapters/ScheduledJobs.html\n\nThe basics of this tool is that we need to create XML-formatted\n\"plist\" files inside \"~/Library/LaunchAgents/\" and then use the\n'launchctl' tool to make launchd aware of them. The plist files\ninclude all of the scheduling information, along with the command-line\narguments split across an array of tags.\n\nFor example, here is my plist file for the weekly scheduled tasks:\n\n\n\n\nLabelorg.git-scm.git.weekly\nProgramArguments\n\n/usr/local/libexec/git-core/git\n--exec-path=/usr/local/libexec/git-core\nfor-each-repo\n--config=maintenance.repo\nmaintenance\nrun\n--schedule=weekly\n\nStartCalendarInterval\n\n\nDay0\nHour0\nMinute0\n\n\n\n\n\nThe schedules for the daily and hourly tasks are more complicated\nsince we need to use an array for the StartCalendarInterval with\nan entry for each of the six days other than the 0th day (to avoid\ncolliding with the weekly task), and each of the 23 hours other\nthan the 0th hour (to avoid colliding with the daily task).\n\nThe \"Label\" value is currently filled with \"org.git-scm.git.X\"\nwhere X is the frequency. We need a different plist file for each\nfrequency.\n\nThe launchctl command needs to be aligned with a user id in order\nto initialize the command environment. This must be done using\nthe 'launchctl bootstrap' subcommand. This subcommand is new as\nof macOS 10.11, which was released in September 2015. Before that\nrelease the 'launchctl load' subcommand was recommended. The best\nsource of information on this transition I have seen is available\nat [2].\n\n[2] https://babodee.wordpress.com/2016/04/09/launchctl-2-0-syntax/\n\nTo remove a schedule, we must run 'launchctl bootout' with a valid\nplist file. We also need to 'bootout' a task before the 'bootstrap'\nsubcommand will succeed, if such a task already exists.\n\nWe can verify the commands that were run by 'git maintenance start'\nand 'git maintenance stop' by injecting a script that writes the\ncommand-line arguments into GIT_TEST_CRONTAB.\n\nAn earlier version of this patch accidentally had an opening\n\"\" tag when it should have had a closing \"\" tag. This\nwas caught during manual testing with actual 'launchctl' commands,\nbut we do not want to update developers' tasks when running tests.\nIt appears that macOS includes the \"xmllint\" tool which can verify\nthe XML format, so call it from the macOS-specific tests to ensure\nthe .plist files are well-formatted.\n\nHelped-by: Eric Sunshine \nSigned-off-by: Derrick Stolee \n---\n Documentation/git-maintenance.txt | 43 ++++++\n builtin/gc.c | 209 ++++++++++++++++++++++++++++++\n t/t7900-maintenance.sh | 53 +++++++-\n t/test-lib.sh | 4 +\n 4 files changed, 306 insertions(+), 3 deletions(-)\n\ndiff --git a/Documentation/git-maintenance.txt b/Documentation/git-maintenance.txt\nindex 4c7aac877d..451ebac131 100644\n--- a/Documentation/git-maintenance.txt\n+++ b/Documentation/git-maintenance.txt\n@@ -273,6 +273,49 @@ for advanced scheduling techniques. Please do use the full path and\n executing the correct binaries in your schedule.\n \n \n+BACKGROUND MAINTENANCE ON MACOS SYSTEMS\n+---------------------------------------\n+\n+While macOS technically supports `cron`, using `crontab -e` requires\n+elevated privileges and the executed process do not have a full user\n+context. Without a full user context, Git and its credential helpers\n+cannot access stored credentials, so some maintenance tasks are not\n+functional.\n+\n+Instead, `git maintenance start` interacts with the `launchctl` tool,\n+which is the recommended way to\n+https://developer.apple.com/library/archive/documentation/MacOSX/Conceptual/BPSystemStartup/Chapters/ScheduledJobs.html[schedule timed jobs in macOS].\n+\n+Scheduling maintenance through `git maintenance (start|stop)` requires\n+some `launchctl` features available only in macOS 10.11 or later.\n+\n+Your user-specific scheduled tasks are stored as XML-formatted `.plist`\n+files in `~/Library/LaunchAgents/`. You can see the currently-registered\n+tasks using the following command:\n+\n+-----------------------------------------------------------------------\n+$ ls ~/Library/LaunchAgents/ | grep org.git-scm.git\n+org.git-scm.git.daily.plist\n+org.git-scm.git.hourly.plist\n+org.git-scm.git.weekly.plist\n+-----------------------------------------------------------------------\n+\n+One task is registered for each `--schedule=` option. To\n+inspect how the XML format describes each schedule, open one of these\n+`.plist` files in an editor and inspect the `` element following\n+the `StartCalendarInterval` element.\n+\n+`git maintenance start` will overwrite these files and register the\n+tasks again with `launchctl`, so any customizations should be done by\n+creating your own `.plist` files with distinct names. Similarly, the\n+`git maintenance stop` command will unregister the tasks with `launchctl`\n+and delete the `.plist` files.\n+\n+To create more advanced customizations to your background tasks, see\n+https://developer.apple.com/library/archive/documentation/MacOSX/Conceptual/BPSystemStartup/Chapters/CreatingLaunchdJobs.html#//apple_ref/doc/uid/TP40001762-104142[the `launchctl` documentation]\n+for more information.\n+\n+\n GIT\n ---\n Part of the linkgit:git[1] suite\ndiff --git a/builtin/gc.c b/builtin/gc.c\nindex c1f7d9bdc2..7604064a8d 100644\n--- a/builtin/gc.c\n+++ b/builtin/gc.c\n@@ -1491,6 +1491,214 @@ static int maintenance_unregister(void)\n \treturn run_command(&config_unset);\n }\n \n+#if defined(__APPLE__)\n+\n+static char *get_service_name(const char *frequency)\n+{\n+\tstruct strbuf label = STRBUF_INIT;\n+\tstrbuf_addf(&label, \"org.git-scm.git.%s\", frequency);\n+\treturn strbuf_detach(&label, NULL);\n+}\n+\n+static char *get_service_filename(const char *name)\n+{\n+\tchar *expanded;\n+\tstruct strbuf filename = STRBUF_INIT;\n+\tstrbuf_addf(&filename, \"~/Library/LaunchAgents/%s.plist\", name);\n+\n+\texpanded = expand_user_path(filename.buf, 1);\n+\tif (!expanded)\n+\t\tdie(_(\"failed to expand path '%s'\"), filename.buf);\n+\n+\tstrbuf_release(&filename);\n+\treturn expanded;\n+}\n+\n+static const char *get_frequency(enum schedule_priority schedule)\n+{\n+\tswitch (schedule) {\n+\tcase SCHEDULE_HOURLY:\n+\t\treturn \"hourly\";\n+\tcase SCHEDULE_DAILY:\n+\t\treturn \"daily\";\n+\tcase SCHEDULE_WEEKLY:\n+\t\treturn \"weekly\";\n+\tdefault:\n+\t\tBUG(\"invalid schedule %d\", schedule);\n+\t}\n+}\n+\n+static char *get_uid(void)\n+{\n+\tstruct strbuf output = STRBUF_INIT;\n+\tstruct child_process id = CHILD_PROCESS_INIT;\n+\n+\tstrvec_pushl(&id.args, \"/usr/bin/id\", \"-u\", NULL);\n+\tif (capture_command(&id, &output, 0))\n+\t\tdie(_(\"failed to discover user id\"));\n+\n+\tstrbuf_trim_trailing_newline(&output);\n+\treturn strbuf_detach(&output, NULL);\n+}\n+\n+static int bootout(const char *filename)\n+{\n+\tint result;\n+\tstruct strvec args = STRVEC_INIT;\n+\tchar *uid = get_uid();\n+\tconst char *launchctl = getenv(\"GIT_TEST_CRONTAB\");\n+\tif (!launchctl)\n+\t\tlaunchctl = \"/bin/launchctl\";\n+\n+\tstrvec_split(&args, launchctl);\n+\tstrvec_push(&args, \"bootout\");\n+\tstrvec_pushf(&args, \"gui/%s\", uid);\n+\tstrvec_push(&args, filename);\n+\n+\tresult = run_command_v_opt(args.v, 0);\n+\n+\tstrvec_clear(&args);\n+\tfree(uid);\n+\treturn result;\n+}\n+\n+static int bootstrap(const char *filename)\n+{\n+\tint result;\n+\tstruct strvec args = STRVEC_INIT;\n+\tchar *uid = get_uid();\n+\tconst char *launchctl = getenv(\"GIT_TEST_CRONTAB\");\n+\tif (!launchctl)\n+\t\tlaunchctl = \"/bin/launchctl\";\n+\n+\tstrvec_split(&args, launchctl);\n+\tstrvec_push(&args, \"bootstrap\");\n+\tstrvec_pushf(&args, \"gui/%s\", uid);\n+\tstrvec_push(&args, filename);\n+\n+\tresult = run_command_v_opt(args.v, 0);\n+\n+\tstrvec_clear(&args);\n+\tfree(uid);\n+\treturn result;\n+}\n+\n+static int remove_plist(enum schedule_priority schedule)\n+{\n+\tconst char *frequency = get_frequency(schedule);\n+\tchar *name = get_service_name(frequency);\n+\tchar *filename = get_service_filename(name);\n+\tint result = bootout(filename);\n+\tfree(filename);\n+\tfree(name);\n+\treturn result;\n+}\n+\n+static int remove_plists(void)\n+{\n+\treturn remove_plist(SCHEDULE_HOURLY) ||\n+\t\tremove_plist(SCHEDULE_DAILY) ||\n+\t\tremove_plist(SCHEDULE_WEEKLY);\n+}\n+\n+static int schedule_plist(const char *exec_path, enum schedule_priority schedule)\n+{\n+\tFILE *plist;\n+\tint i;\n+\tconst char *preamble, *repeat;\n+\tconst char *frequency = get_frequency(schedule);\n+\tchar *name = get_service_name(frequency);\n+\tchar *filename = get_service_filename(name);\n+\n+\tif (safe_create_leading_directories(filename))\n+\t\tdie(_(\"failed to create directories for '%s'\"), filename);\n+\tplist = fopen(filename, \"w\");\n+\n+\tif (!plist)\n+\t\tdie(_(\"failed to open '%s'\"), filename);\n+\n+\tpreamble = \"\\n\"\n+\t\t \"\\n\"\n+\t\t \"\"\n+\t\t \"\\n\"\n+\t\t \"Label%s\\n\"\n+\t\t \"ProgramArguments\\n\"\n+\t\t \"\\n\"\n+\t\t \"%s/git\\n\"\n+\t\t \"--exec-path=%s\\n\"\n+\t\t \"for-each-repo\\n\"\n+\t\t \"--config=maintenance.repo\\n\"\n+\t\t \"maintenance\\n\"\n+\t\t \"run\\n\"\n+\t\t \"--schedule=%s\\n\"\n+\t\t \"\\n\"\n+\t\t \"StartCalendarInterval\\n\"\n+\t\t \"\\n\";\n+\tfprintf(plist, preamble, name, exec_path, exec_path, frequency);\n+\n+\tswitch (schedule) {\n+\tcase SCHEDULE_HOURLY:\n+\t\trepeat = \"\\n\"\n+\t\t\t \"Hour%d\\n\"\n+\t\t\t \"Minute0\\n\"\n+\t\t\t \"\\n\";\n+\t\tfor (i = 1; i <= 23; i++)\n+\t\t\tfprintf(plist, repeat, i);\n+\t\tbreak;\n+\n+\tcase SCHEDULE_DAILY:\n+\t\trepeat = \"\\n\"\n+\t\t\t \"Day%d\\n\"\n+\t\t\t \"Hour0\\n\"\n+\t\t\t \"Minute0\\n\"\n+\t\t\t \"\\n\";\n+\t\tfor (i = 1; i <= 6; i++)\n+\t\t\tfprintf(plist, repeat, i);\n+\t\tbreak;\n+\n+\tcase SCHEDULE_WEEKLY:\n+\t\tfprintf(plist,\n+\t\t\t\"\\n\"\n+\t\t\t\"Day0\\n\"\n+\t\t\t\"Hour0\\n\"\n+\t\t\t\"Minute0\\n\"\n+\t\t\t\"\\n\");\n+\t\tbreak;\n+\n+\tdefault:\n+\t\t/* unreachable */\n+\t\tbreak;\n+\t}\n+\tfprintf(plist, \"\\n\\n\\n\");\n+\n+\t/* bootout might fail if not already running, so ignore */\n+\tbootout(filename);\n+\tif (bootstrap(filename))\n+\t\tdie(_(\"failed to bootstrap service %s\"), filename);\n+\n+\tfclose(plist);\n+\tfree(filename);\n+\tfree(name);\n+\treturn 0;\n+}\n+\n+static int add_plists(void)\n+{\n+\tconst char *exec_path = git_exec_path();\n+\n+\treturn schedule_plist(exec_path, SCHEDULE_HOURLY) ||\n+\t\tschedule_plist(exec_path, SCHEDULE_DAILY) ||\n+\t\tschedule_plist(exec_path, SCHEDULE_WEEKLY);\n+}\n+\n+static int platform_update_schedule(int run_maintenance, int fd)\n+{\n+\tif (run_maintenance)\n+\t\treturn add_plists();\n+\telse\n+\t\treturn remove_plists();\n+}\n+#else\n #define BEGIN_LINE \"# BEGIN GIT MAINTENANCE SCHEDULE\"\n #define END_LINE \"# END GIT MAINTENANCE SCHEDULE\"\n \n@@ -1585,6 +1793,7 @@ static int platform_update_schedule(int run_maintenance, int fd)\n \t\tfclose(cron_list);\n \treturn result;\n }\n+#endif\n \n static int update_background_schedule(int run_maintenance)\n {\ndiff --git a/t/t7900-maintenance.sh b/t/t7900-maintenance.sh\nindex 20184e96e1..1c43b34a93 100755\n--- a/t/t7900-maintenance.sh\n+++ b/t/t7900-maintenance.sh\n@@ -367,7 +367,7 @@ test_expect_success 'register and unregister' '\n \ttest_cmp before actual\n '\n \n-test_expect_success 'start from empty cron table' '\n+test_expect_success !MACOS_MAINTENANCE 'start from empty cron table' '\n \tGIT_TEST_CRONTAB=\"test-tool crontab cron.txt\" git maintenance start &&\n \n \t# start registers the repo\n@@ -378,7 +378,7 @@ test_expect_success 'start from empty cron table' '\n \tgrep \"for-each-repo --config=maintenance.repo maintenance run --schedule=weekly\" cron.txt\n '\n \n-test_expect_success 'stop from existing schedule' '\n+test_expect_success !MACOS_MAINTENANCE 'stop from existing schedule' '\n \tGIT_TEST_CRONTAB=\"test-tool crontab cron.txt\" git maintenance stop &&\n \n \t# stop does not unregister the repo\n@@ -389,12 +389,59 @@ test_expect_success 'stop from existing schedule' '\n \ttest_must_be_empty cron.txt\n '\n \n-test_expect_success 'start preserves existing schedule' '\n+test_expect_success !MACOS_MAINTENANCE 'start preserves existing schedule' '\n \techo \"Important information!\" >cron.txt &&\n \tGIT_TEST_CRONTAB=\"test-tool crontab cron.txt\" git maintenance start &&\n \tgrep \"Important information!\" cron.txt\n '\n \n+test_expect_success MACOS_MAINTENANCE 'start and stop macOS maintenance' '\n+\techo \"#!/bin/sh\\necho \\$@ >>args\" >print-args &&\n+\tchmod a+x print-args &&\n+\n+\trm -f args &&\n+\tGIT_TEST_CRONTAB=\"./print-args\" git maintenance start &&\n+\n+\t# start registers the repo\n+\tgit config --get --global maintenance.repo \"$(pwd)\" &&\n+\n+\t# ~/Library/LaunchAgents\n+\tls \"$HOME/Library/LaunchAgents\" >actual &&\n+\tcat >expect <<-\\EOF &&\n+\torg.git-scm.git.daily.plist\n+\torg.git-scm.git.hourly.plist\n+\torg.git-scm.git.weekly.plist\n+\tEOF\n+\ttest_cmp expect actual &&\n+\n+\trm expect &&\n+\tfor frequency in hourly daily weekly\n+\tdo\n+\t\tPLIST=\"$HOME/Library/LaunchAgents/org.git-scm.git.$frequency.plist\" &&\n+\t\txmllint \"$PLIST\" >/dev/null &&\n+\t\tgrep schedule=$frequency \"$PLIST\" &&\n+\t\techo \"bootout gui/$UID $PLIST\" >>expect &&\n+\t\techo \"bootstrap gui/$UID $PLIST\" >>expect || return 1\n+\tdone &&\n+\ttest_cmp expect args &&\n+\n+\trm -f args &&\n+\tGIT_TEST_CRONTAB=\"./print-args\" git maintenance stop &&\n+\n+\t# stop does not unregister the repo\n+\tgit config --get --global maintenance.repo \"$(pwd)\" &&\n+\n+\t# stop does not remove plist files, but boots them out\n+\trm expect &&\n+\tfor frequency in hourly daily weekly\n+\tdo\n+\t\tPLIST=\"$HOME/Library/LaunchAgents/org.git-scm.git.$frequency.plist\" &&\n+\t\tgrep schedule=$frequency \"$PLIST\" &&\n+\t\techo \"bootout gui/$UID $PLIST\" >>expect || return 1\n+\tdone &&\n+\ttest_cmp expect args\n+'\n+\n test_expect_success 'register preserves existing strategy' '\n \tgit config maintenance.strategy none &&\n \tgit maintenance register &&\ndiff --git a/t/test-lib.sh b/t/test-lib.sh\nindex 4a60d1ed76..620ffbf3af 100644\n--- a/t/test-lib.sh\n+++ b/t/test-lib.sh\n@@ -1703,6 +1703,10 @@ test_lazy_prereq REBASE_P '\n \ttest -z \"$GIT_TEST_SKIP_REBASE_P\"\n '\n \n+test_lazy_prereq MACOS_MAINTENANCE '\n+\tlaunchctl list\n+'\n+\n # Ensure that no test accidentally triggers a Git command\n # that runs 'crontab', affecting a user's cron schedule.\n # Tests that verify the cron integration must set this locally\n-- \ngitgitgadget\n\n"},{"id":"409134","messageId":"709a17372099351af3e3b9f2dddcd9ce220a9fcb.1604520368.git.gitgitgadget@gmail.com","threadId":"54562","inReplyTo":"pull.776.v2.git.1604520368.gitgitgadget@gmail.com","subject":"[PATCH v2 2/4] maintenance: include 'cron' details in docs","fromName":"Derrick Stolee via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2020-11-04T20:06:06Z","receivedAt":"2020-11-04T20:06:18Z","isPatch":true,"sender":{"key":"stolee@gmail.com","avatar":"https://avatars.githubusercontent.com/u/570044?v=4"},"body":"From: Derrick Stolee \n\nAdvanced and expert users may want to know how 'git maintenance start'\nschedules background maintenance in order to customize their own\nschedules beyond what the maintenance.* config values allow. Start a new\nset of sections in git-maintenance.txt that describe how 'cron' is used\nto run these tasks.\n\nThis is particularly valuable for users who want to inspect what Git is\ndoing or for users who want to customize the schedule further. Having a\nbaseline can provide a way forward for users who have never worked with\ncron schedules.\n\nSigned-off-by: Derrick Stolee \n---\n Documentation/git-maintenance.txt | 54 +++++++++++++++++++++++++++++++\n 1 file changed, 54 insertions(+)\n\ndiff --git a/Documentation/git-maintenance.txt b/Documentation/git-maintenance.txt\nindex 6fec1eb8dc..4c7aac877d 100644\n--- a/Documentation/git-maintenance.txt\n+++ b/Documentation/git-maintenance.txt\n@@ -218,6 +218,60 @@ Further, the `git gc` command should not be combined with\n but does not take the lock in the same way as `git maintenance run`. If\n possible, use `git maintenance run --task=gc` instead of `git gc`.\n \n+The following sections describe the mechanisms put in place to run\n+background maintenance by `git maintenance start` and how to customize\n+them.\n+\n+BACKGROUND MAINTENANCE ON POSIX SYSTEMS\n+---------------------------------------\n+\n+The standard mechanism for scheduling background tasks on POSIX systems\n+is `cron`. This tool executes commands based on a given schedule. The\n+current list of user-scheduled tasks can be found by running `crontab -l`.\n+The schedule written by `git maintenance start` is similar to this:\n+\n+-----------------------------------------------------------------------\n+# BEGIN GIT MAINTENANCE SCHEDULE\n+# The following schedule was created by Git\n+# Any edits made in this region might be\n+# replaced in the future by a Git command.\n+\n+0 1-23 * * * \"//git\" --exec-path=\"/\" for-each-repo --config=maintenance.repo maintenance run --schedule=hourly\n+0 0 * * 1-6 \"//git\" --exec-path=\"/\" for-each-repo --config=maintenance.repo maintenance run --schedule=daily\n+0 0 * * 0 \"//git\" --exec-path=\"/\" for-each-repo --config=maintenance.repo maintenance run --schedule=weekly\n+\n+# END GIT MAINTENANCE SCHEDULE\n+-----------------------------------------------------------------------\n+\n+The comments are used as a region to mark the schedule as written by Git.\n+Any modifications within this region will be completely deleted by\n+`git maintenance stop` or overwritten by `git maintenance start`.\n+\n+The `` string is loaded to specifically use the location for the\n+`git` executable used in the `git maintenance start` command. This allows\n+for multiple versions to be compatible. However, if the same user runs\n+`git maintenance start` with multiple Git executables, then only the\n+latest executable will be used.\n+\n+These commands use `git for-each-repo --config=maintenance.repo` to run\n+`git maintenance run --schedule=` on each repository listed in\n+the multi-valued `maintenance.repo` config option. These are typically\n+loaded from the user-specific global config located at `~/.gitconfig`.\n+The `git maintenance` process then determines which maintenance tasks\n+are configured to run on each repository with each `` using\n+the `maintenance..schedule` config options. These values are loaded\n+from the global or repository config values.\n+\n+If the config values are insufficient to achieve your desired background\n+maintenance schedule, then you can create your own schedule. If you run\n+`crontab -e`, then an editor will load with your user-specific `cron`\n+schedule. In that editor, you can add your own schedule lines. You could\n+start by adapting the default schedule listed earlier, or you could read\n+https://man7.org/linux/man-pages/man5/crontab.5.html[the `crontab` documentation]\n+for advanced scheduling techniques. Please do use the full path and\n+`--exec-path` techniques from the default schedule to ensure you are\n+executing the correct binaries in your schedule.\n+\n \n GIT\n ---\n-- \ngitgitgadget\n\n"},{"id":"409133","messageId":"84eb44de31f04b2a94f57ee11d70be81f5bbeee2.1604520368.git.gitgitgadget@gmail.com","threadId":"54562","inReplyTo":"pull.776.v2.git.1604520368.gitgitgadget@gmail.com","subject":"[PATCH v2 4/4] maintenance: use Windows scheduled tasks","fromName":"Derrick Stolee via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2020-11-04T20:06:08Z","receivedAt":"2020-11-04T20:06:19Z","isPatch":true,"sender":{"key":"stolee@gmail.com","avatar":"https://avatars.githubusercontent.com/u/570044?v=4"},"body":"From: Derrick Stolee \n\nGit's background maintenance uses cron by default, but this is not\navailable on Windows. Instead, integrate with Task Scheduler.\n\nTasks can be scheduled using the 'schtasks' command. There are several\ncommand-line options that can allow for some advanced scheduling, but\nunfortunately these seem to all require authenticating using a password.\n\nInstead, use the \"/xml\" option to pass an XML file that contains the\nconfiguration for the necessary schedule. These XML files are based on\nsome that I exported after constructing a schedule in the Task Scheduler\nGUI. These options only run background maintenance when the user is\nlogged in, and more fields are populated with the current username and\nSID at run-time by 'schtasks'.\n\nThere is a deficiency in the current design. Windows has two kinds of\napplications: GUI applications that start by \"winmain()\" and console\napplications that start by \"main()\". Console applications are attached\nto a new Console window if they are not already associated with a GUI\napplication. This means that every hour the scheudled task launches a\ncommand window for the scheduled tasks. Not only is this visually\nobtrusive, but it also takes focus from whatever else the user is\ndoing!\n\nA simple fix would be to insert a GUI application that acts as a shim\nbetween the scheduled task and Git. This is currently possible in Git\nfor Windows by setting the tag equal to\n\n C:\\Program Files\\Git\\git-bash.exe\n\nwith options \"--hide --no-needs-console --command=cmd\\git.exe\"\nfollowed by the arguments currently used. Since git-bash.exe is not\nincluded in Windows builds of core Git, I chose to leave out this\nfeature. My plan is to submit a small patch to Git for Windows that\nconverts the use of git.exe with this use of git-bash.exe in the\nshort term. In the long term, we can consider creating this GUI\nshim application within core Git, perhaps in contrib/.\n\nHelped-by: Eric Sunshine \nSigned-off-by: Derrick Stolee \n---\n Documentation/git-maintenance.txt | 22 ++++\n builtin/gc.c | 181 ++++++++++++++++++++++++++++++\n t/t7900-maintenance.sh | 36 +++++-\n 3 files changed, 236 insertions(+), 3 deletions(-)\n\ndiff --git a/Documentation/git-maintenance.txt b/Documentation/git-maintenance.txt\nindex 451ebac131..f4f6a4091b 100644\n--- a/Documentation/git-maintenance.txt\n+++ b/Documentation/git-maintenance.txt\n@@ -316,6 +316,28 @@ https://developer.apple.com/library/archive/documentation/MacOSX/Conceptual/BPSy\n for more information.\n \n \n+BACKGROUND MAINTENANCE ON WINDOWS SYSTEMS\n+-----------------------------------------\n+\n+Windows does not support `cron` and instead has its own system for\n+scheduling background tasks. The `git maintenance start` command uses\n+the `schtasks` command to submit tasks to this system. You can inspect\n+all background tasks using the Task Scheduler application. The tasks\n+added by Git have names of the form `Git Maintenance ()`.\n+The Task Scheduler GUI has ways to inspect these tasks, but you can also\n+export the tasks to XML files and view the details there.\n+\n+Note that since Git is a console application, these background tasks\n+create a console window visible to the current user. This can be changed\n+manually by selecting the \"Run whether user is logged in or not\" option\n+in Task Scheduler. This change requires a password input, which is why\n+`git maintenance start` does not select it by default.\n+\n+If you want to customize the background tasks, please rename the tasks\n+so future calls to `git maintenance (start|stop)` do not overwrite your\n+custom tasks.\n+\n+\n GIT\n ---\n Part of the linkgit:git[1] suite\ndiff --git a/builtin/gc.c b/builtin/gc.c\nindex 7604064a8d..80f43a59ce 100644\n--- a/builtin/gc.c\n+++ b/builtin/gc.c\n@@ -1698,6 +1698,187 @@ static int platform_update_schedule(int run_maintenance, int fd)\n \telse\n \t\treturn remove_plists();\n }\n+\n+#elif defined(GIT_WINDOWS_NATIVE)\n+\n+static const char *get_frequency(enum schedule_priority schedule)\n+{\n+\tswitch (schedule) {\n+\tcase SCHEDULE_HOURLY:\n+\t\treturn \"hourly\";\n+\tcase SCHEDULE_DAILY:\n+\t\treturn \"daily\";\n+\tcase SCHEDULE_WEEKLY:\n+\t\treturn \"weekly\";\n+\tdefault:\n+\t\tBUG(\"invalid schedule %d\", schedule);\n+\t}\n+}\n+\n+static char *get_task_name(const char *frequency)\n+{\n+\tstruct strbuf label = STRBUF_INIT;\n+\tstrbuf_addf(&label, \"Git Maintenance (%s)\", frequency);\n+\treturn strbuf_detach(&label, NULL);\n+}\n+\n+static int remove_task(enum schedule_priority schedule)\n+{\n+\tint result;\n+\tstruct strvec args = STRVEC_INIT;\n+\tconst char *frequency = get_frequency(schedule);\n+\tchar *name = get_task_name(frequency);\n+\tconst char *schtasks = getenv(\"GIT_TEST_CRONTAB\");\n+\tif (!schtasks)\n+\t\tschtasks = \"schtasks\";\n+\n+\tstrvec_split(&args, schtasks);\n+\tstrvec_pushl(&args, \"/delete\", \"/tn\", name, \"/f\", NULL);\n+\n+\tresult = run_command_v_opt(args.v, 0);\n+\n+\tstrvec_clear(&args);\n+\tfree(name);\n+\treturn result;\n+}\n+\n+static int remove_scheduled_tasks(void)\n+{\n+\treturn remove_task(SCHEDULE_HOURLY) ||\n+\t\tremove_task(SCHEDULE_DAILY) ||\n+\t\tremove_task(SCHEDULE_WEEKLY);\n+}\n+\n+static int schedule_task(const char *exec_path, enum schedule_priority schedule)\n+{\n+\tint result;\n+\tstruct strvec args = STRVEC_INIT;\n+\tconst char *xml, *schtasks;\n+\tchar *xmlpath;\n+\tFILE *xmlfp;\n+\tconst char *frequency = get_frequency(schedule);\n+\tchar *name = get_task_name(frequency);\n+\n+\txmlpath = xstrfmt(\"%s/schedule-%s.xml\",\n+\t\t\t the_repository->objects->odb->path,\n+\t\t\t frequency);\n+\txmlfp = fopen(xmlpath, \"w\");\n+\tif (!xmlfp)\n+\t\tdie(_(\"failed to open '%s'\"), xmlpath);\n+\n+\txml = \"\\n\"\n+\t \"\\n\"\n+\t \"\\n\"\n+\t \"\\n\";\n+\tfprintf(xmlfp, xml);\n+\n+\tswitch (schedule) {\n+\tcase SCHEDULE_HOURLY:\n+\t\tfprintf(xmlfp,\n+\t\t\t\"2020-01-01T01:00:00\\n\"\n+\t\t\t\"true\\n\"\n+\t\t\t\"\\n\"\n+\t\t\t\"1\\n\"\n+\t\t\t\"\\n\"\n+\t\t\t\"\\n\"\n+\t\t\t\"PT1H\\n\"\n+\t\t\t\"PT23H\\n\"\n+\t\t\t\"false\\n\"\n+\t\t\t\"\\n\");\n+\t\tbreak;\n+\n+\tcase SCHEDULE_DAILY:\n+\t\tfprintf(xmlfp,\n+\t\t\t\"2020-01-01T00:00:00\\n\"\n+\t\t\t\"true\\n\"\n+\t\t\t\"\\n\"\n+\t\t\t\"\\n\"\n+\t\t\t\"\\n\"\n+\t\t\t\"\\n\"\n+\t\t\t\"\\n\"\n+\t\t\t\"\\n\"\n+\t\t\t\"\\n\"\n+\t\t\t\"\\n\"\n+\t\t\t\"\\n\"\n+\t\t\t\"1\\n\"\n+\t\t\t\"\\n\");\n+\t\tbreak;\n+\n+\tcase SCHEDULE_WEEKLY:\n+\t\tfprintf(xmlfp,\n+\t\t\t\"2020-01-01T00:00:00\\n\"\n+\t\t\t\"true\\n\"\n+\t\t\t\"\\n\"\n+\t\t\t\"\\n\"\n+\t\t\t\"\\n\"\n+\t\t\t\"\\n\"\n+\t\t\t\"1\\n\"\n+\t\t\t\"\\n\");\n+\t\tbreak;\n+\n+\tdefault:\n+\t\tbreak;\n+\t}\n+\n+\txml= \"\\n\"\n+\t \"\\n\"\n+\t \"\\n\"\n+\t \"\\n\"\n+\t \"InteractiveToken\\n\"\n+\t \"LeastPrivilege\\n\"\n+\t \"\\n\"\n+\t \"\\n\"\n+\t \"\\n\"\n+\t \"IgnoreNew\\n\"\n+\t \"true\\n\"\n+\t \"true\\n\"\n+\t \"true\\n\"\n+\t \"false\\n\"\n+\t \"PT72H\\n\"\n+\t \"7\\n\"\n+\t \"\\n\"\n+\t \"\\n\"\n+\t \"\\n\"\n+\t \"\\\"%s\\\\git.exe\\\"\\n\"\n+\t \"--exec-path=\\\"%s\\\" for-each-repo --config=maintenance.repo maintenance run --schedule=%s\\n\"\n+\t \"\\n\"\n+\t \"\\n\"\n+\t \"\\n\";\n+\tfprintf(xmlfp, xml, exec_path, exec_path, frequency);\n+\tfclose(xmlfp);\n+\n+\tschtasks = getenv(\"GIT_TEST_CRONTAB\");\n+\tif (!schtasks)\n+\t\tschtasks = \"schtasks\";\n+\tstrvec_split(&args, schtasks);\n+\tstrvec_pushl(&args, \"/create\", \"/tn\", name, \"/f\", \"/xml\", xmlpath, NULL);\n+\n+\tresult = run_command_v_opt(args.v, 0);\n+\n+\tstrvec_clear(&args);\n+\tunlink(xmlpath);\n+\tfree(xmlpath);\n+\tfree(name);\n+\treturn result;\n+}\n+\n+static int add_scheduled_tasks(void)\n+{\n+\tconst char *exec_path = git_exec_path();\n+\n+\treturn schedule_task(exec_path, SCHEDULE_HOURLY) ||\n+\t\tschedule_task(exec_path, SCHEDULE_DAILY) ||\n+\t\tschedule_task(exec_path, SCHEDULE_WEEKLY);\n+}\n+\n+static int platform_update_schedule(int run_maintenance, int fd)\n+{\n+\tif (run_maintenance)\n+\t\treturn add_scheduled_tasks();\n+\telse\n+\t\treturn remove_scheduled_tasks();\n+}\n+\n #else\n #define BEGIN_LINE \"# BEGIN GIT MAINTENANCE SCHEDULE\"\n #define END_LINE \"# END GIT MAINTENANCE SCHEDULE\"\ndiff --git a/t/t7900-maintenance.sh b/t/t7900-maintenance.sh\nindex 1c43b34a93..e7ad130cbc 100755\n--- a/t/t7900-maintenance.sh\n+++ b/t/t7900-maintenance.sh\n@@ -367,7 +367,7 @@ test_expect_success 'register and unregister' '\n \ttest_cmp before actual\n '\n \n-test_expect_success !MACOS_MAINTENANCE 'start from empty cron table' '\n+test_expect_success !MACOS_MAINTENANCE,!MINGW 'start from empty cron table' '\n \tGIT_TEST_CRONTAB=\"test-tool crontab cron.txt\" git maintenance start &&\n \n \t# start registers the repo\n@@ -378,7 +378,7 @@ test_expect_success !MACOS_MAINTENANCE 'start from empty cron table' '\n \tgrep \"for-each-repo --config=maintenance.repo maintenance run --schedule=weekly\" cron.txt\n '\n \n-test_expect_success !MACOS_MAINTENANCE 'stop from existing schedule' '\n+test_expect_success !MACOS_MAINTENANCE,!MINGW 'stop from existing schedule' '\n \tGIT_TEST_CRONTAB=\"test-tool crontab cron.txt\" git maintenance stop &&\n \n \t# stop does not unregister the repo\n@@ -389,7 +389,7 @@ test_expect_success !MACOS_MAINTENANCE 'stop from existing schedule' '\n \ttest_must_be_empty cron.txt\n '\n \n-test_expect_success !MACOS_MAINTENANCE 'start preserves existing schedule' '\n+test_expect_success !MACOS_MAINTENANCE,!MINGW 'start preserves existing schedule' '\n \techo \"Important information!\" >cron.txt &&\n \tGIT_TEST_CRONTAB=\"test-tool crontab cron.txt\" git maintenance start &&\n \tgrep \"Important information!\" cron.txt\n@@ -442,6 +442,36 @@ test_expect_success MACOS_MAINTENANCE 'start and stop macOS maintenance' '\n \ttest_cmp expect args\n '\n \n+test_expect_success MINGW 'start and stop Windows maintenance' '\n+\twrite_script print-args <<-\\EOF &&\n+\techo $* >>args\n+\tEOF\n+\n+\trm -f args &&\n+\tGIT_TEST_CRONTAB=\"/bin/sh print-args\" git maintenance start &&\n+\n+\t# start registers the repo\n+\tgit config --get --global maintenance.repo \"$(pwd)\" &&\n+\n+\tfor frequency in hourly daily weekly\n+\tdo\n+\t\tprintf \"/create /tn Git Maintenance (%s) /f /xml .git/objects/schedule-%s.xml\\n\" \\\n+\t\t\t$frequency $frequency\n+\tdone >expect &&\n+\ttest_cmp expect args &&\n+\n+\trm -f args &&\n+\tGIT_TEST_CRONTAB=\"/bin/sh print-args\" git maintenance stop &&\n+\n+\t# stop does not unregister the repo\n+\tgit config --get --global maintenance.repo \"$(pwd)\" &&\n+\n+\trm expect &&\n+\tprintf \"/delete /tn Git Maintenance (%s) /f\\n\" \\\n+\t\thourly daily weekly >expect &&\n+\ttest_cmp expect args\n+'\n+\n test_expect_success 'register preserves existing strategy' '\n \tgit config maintenance.strategy none &&\n \tgit maintenance register &&\n-- \ngitgitgadget\n"},{"id":"409135","messageId":"d35f1aa162097d4dfe468b3ac1717bca92a41cfa.1604520368.git.gitgitgadget@gmail.com","threadId":"54562","inReplyTo":"pull.776.v2.git.1604520368.gitgitgadget@gmail.com","subject":"[PATCH v2 1/4] maintenance: extract platform-specific scheduling","fromName":"Derrick Stolee via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2020-11-04T20:06:05Z","receivedAt":"2020-11-04T20:06:20Z","isPatch":true,"sender":{"key":"stolee@gmail.com","avatar":"https://avatars.githubusercontent.com/u/570044?v=4"},"body":"From: Derrick Stolee \n\nThe existing schedule mechanism using 'cron' is supported by POSIX\nplatforms, but not Windows. It also works slightly differently on\nmacOS to significant detriment of the user experience. To allow for\nnew implementations on these platforms, extract a method that\nperforms the platform-specific scheduling mechanism. This will be\nswapped at compile time with new implementations on specialized\nplatforms.\n\nSigned-off-by: Derrick Stolee \n---\n builtin/gc.c | 38 +++++++++++++++++++++-----------------\n 1 file changed, 21 insertions(+), 17 deletions(-)\n\ndiff --git a/builtin/gc.c b/builtin/gc.c\nindex e3098ef6a1..c1f7d9bdc2 100644\n--- a/builtin/gc.c\n+++ b/builtin/gc.c\n@@ -1494,7 +1494,7 @@ static int maintenance_unregister(void)\n #define BEGIN_LINE \"# BEGIN GIT MAINTENANCE SCHEDULE\"\n #define END_LINE \"# END GIT MAINTENANCE SCHEDULE\"\n \n-static int update_background_schedule(int run_maintenance)\n+static int platform_update_schedule(int run_maintenance, int fd)\n {\n \tint result = 0;\n \tint in_old_region = 0;\n@@ -1503,11 +1503,6 @@ static int update_background_schedule(int run_maintenance)\n \tFILE *cron_list, *cron_in;\n \tconst char *crontab_name;\n \tstruct strbuf line = STRBUF_INIT;\n-\tstruct lock_file lk;\n-\tchar *lock_path = xstrfmt(\"%s/schedule\", the_repository->objects->odb->path);\n-\n-\tif (hold_lock_file_for_update(&lk, lock_path, LOCK_NO_DEREF) < 0)\n-\t\treturn error(_(\"another process is scheduling background maintenance\"));\n \n \tcrontab_name = getenv(\"GIT_TEST_CRONTAB\");\n \tif (!crontab_name)\n@@ -1516,12 +1511,11 @@ static int update_background_schedule(int run_maintenance)\n \tstrvec_split(&crontab_list.args, crontab_name);\n \tstrvec_push(&crontab_list.args, \"-l\");\n \tcrontab_list.in = -1;\n-\tcrontab_list.out = dup(lk.tempfile->fd);\n+\tcrontab_list.out = dup(fd);\n \tcrontab_list.git_cmd = 0;\n \n \tif (start_command(&crontab_list)) {\n-\t\tresult = error(_(\"failed to run 'crontab -l'; your system might not support 'cron'\"));\n-\t\tgoto cleanup;\n+\t\treturn error(_(\"failed to run 'crontab -l'; your system might not support 'cron'\"));\n \t}\n \n \t/* Ignore exit code, as an empty crontab will return error. */\n@@ -1531,7 +1525,7 @@ static int update_background_schedule(int run_maintenance)\n \t * Read from the .lock file, filtering out the old\n \t * schedule while appending the new schedule.\n \t */\n-\tcron_list = fdopen(lk.tempfile->fd, \"r\");\n+\tcron_list = fdopen(fd, \"r\");\n \trewind(cron_list);\n \n \tstrvec_split(&crontab_edit.args, crontab_name);\n@@ -1539,8 +1533,7 @@ static int update_background_schedule(int run_maintenance)\n \tcrontab_edit.git_cmd = 0;\n \n \tif (start_command(&crontab_edit)) {\n-\t\tresult = error(_(\"failed to run 'crontab'; your system might not support 'cron'\"));\n-\t\tgoto cleanup;\n+\t\treturn error(_(\"failed to run 'crontab'; your system might not support 'cron'\"));\n \t}\n \n \tcron_in = fdopen(crontab_edit.in, \"w\");\n@@ -1586,13 +1579,24 @@ static int update_background_schedule(int run_maintenance)\n \tclose(crontab_edit.in);\n \n done_editing:\n-\tif (finish_command(&crontab_edit)) {\n+\tif (finish_command(&crontab_edit))\n \t\tresult = error(_(\"'crontab' died\"));\n-\t\tgoto cleanup;\n-\t}\n-\tfclose(cron_list);\n+\telse\n+\t\tfclose(cron_list);\n+\treturn result;\n+}\n+\n+static int update_background_schedule(int run_maintenance)\n+{\n+\tint result;\n+\tstruct lock_file lk;\n+\tchar *lock_path = xstrfmt(\"%s/schedule\", the_repository->objects->odb->path);\n+\n+\tif (hold_lock_file_for_update(&lk, lock_path, LOCK_NO_DEREF) < 0)\n+\t\treturn error(_(\"another process is scheduling background maintenance\"));\n+\n+\tresult = platform_update_schedule(run_maintenance, lk.tempfile->fd);\n \n-cleanup:\n \trollback_lock_file(&lk);\n \treturn result;\n }\n-- \ngitgitgadget\n\n"},{"id":"409588","messageId":"CAPig+cReatZ-iSLWreQqYLM7qONWvBLbDUNVH1dJV37VnmPBgw@mail.gmail.com","threadId":"54562","inReplyTo":"709a17372099351af3e3b9f2dddcd9ce220a9fcb.1604520368.git.gitgitgadget@gmail.com","subject":"Re: [PATCH v2 2/4] maintenance: include 'cron' details in docs","fromName":"Eric Sunshine","fromEmail":"sunshine@sunshineco.com","sentAt":"2020-11-11T07:10:50Z","receivedAt":"2020-11-11T07:11:08Z","isPatch":true,"sender":{"key":"sunshine@sunshineco.com","avatar":"https://avatars.githubusercontent.com/u/163641?v=4"},"body":"On Wed, Nov 4, 2020 at 3:06 PM Derrick Stolee via GitGitGadget\n wrote:\n> Advanced and expert users may want to know how 'git maintenance start'\n> schedules background maintenance in order to customize their own\n> schedules beyond what the maintenance.* config values allow. Start a new\n> set of sections in git-maintenance.txt that describe how 'cron' is used\n> to run these tasks.\n>\n> This is particularly valuable for users who want to inspect what Git is\n> doing or for users who want to customize the schedule further. Having a\n> baseline can provide a way forward for users who have never worked with\n> cron schedules.\n\nA few comments below, not necessarily worth a re-roll...\n\n> Signed-off-by: Derrick Stolee \n> ---\n> diff --git a/Documentation/git-maintenance.txt b/Documentation/git-maintenance.txt\n> @@ -218,6 +218,60 @@ Further, the `git gc` command should not be combined with\n> +The standard mechanism for scheduling background tasks on POSIX systems\n> +is `cron`. This tool executes commands based on a given schedule. The\n\nIt is common in Git (and other Unix) documentation to refer to a Unix\ntool by its man page reference. So, for instance, instead of `cron`,\nwe would say cron(8) since the `cron` man page is in section 8 of the\nUnix manual.\n\n> +The `` string is loaded to specifically use the location for the\n\nThe word \"loaded\" sounds odd in this context.\n\n> +`git` executable used in the `git maintenance start` command. This allows\n> +for multiple versions to be compatible. However, if the same user runs\n> +`git maintenance start` with multiple Git executables, then only the\n> +latest executable will be used.\n\nI had to read this paragraph four or five times to understand what it\nis trying to say (assuming I do understand it). Perhaps it can be\nrewritten more succinctly something like this:\n\n The `crontab` entry specifies the full path of the `git`\n executable to ensure that the `git` command run is the same one\n with which `git maintenance start` was issued independent of\n `PATH`.\n\n> +These commands use `git for-each-repo --config=maintenance.repo` to run\n> +`git maintenance run --schedule=` on each repository listed in\n> +the multi-valued `maintenance.repo` config option. These are typically\n> +loaded from the user-specific global config located at `~/.gitconfig`.\n\nI wonder if mentioning `~/.gitconfig` explicitly is wise since it\nmight also be at $XDG_CONFIG_HOME/git/config or some other location on\nWindows. Perhaps it would be sufficient to mention only \"global Git\nconfiguration\" or something.\n\n> +If the config values are insufficient to achieve your desired background\n> +maintenance schedule, then you can create your own schedule. If you run\n> +`crontab -e`, then an editor will load with your user-specific `cron`\n> +schedule. In that editor, you can add your own schedule lines. You could\n> +start by adapting the default schedule listed earlier, or you could read\n> +https://man7.org/linux/man-pages/man5/crontab.5.html[the `crontab` documentation]\n\nFor consistency with other parts of Git documentation, it might be\nbetter to reference the `crontab` man page simply as crontab(5) rather\nthan providing an explicit URL.\n\n> +for advanced scheduling techniques. Please do use the full path and\n> +`--exec-path` techniques from the default schedule to ensure you are\n> +executing the correct binaries in your schedule.\n"},{"id":"409590","messageId":"CAPig+cT=DytbMH6KkC6ipD3jbWNa7jgW9G0Q76rwJoEsLGn_ow@mail.gmail.com","threadId":"54562","inReplyTo":"0fafd75d100f343f7cff6471772ed9b12793f81e.1604520368.git.gitgitgadget@gmail.com","subject":"Re: [PATCH v2 3/4] maintenance: use launchctl on macOS","fromName":"Eric Sunshine","fromEmail":"sunshine@sunshineco.com","sentAt":"2020-11-11T08:12:19Z","receivedAt":"2020-11-11T08:12:35Z","isPatch":true,"sender":{"key":"sunshine@sunshineco.com","avatar":"https://avatars.githubusercontent.com/u/163641?v=4"},"body":"On Wed, Nov 4, 2020 at 3:06 PM Derrick Stolee via GitGitGadget\n wrote:\n> [...]\n> The solution is to switch from cron to the Apple-recommended [1]\n> 'launchd' tool.\n> [...]\n> Signed-off-by: Derrick Stolee \n> ---\n> diff --git a/Documentation/git-maintenance.txt b/Documentation/git-maintenance.txt\n> +While macOS technically supports `cron`, using `crontab -e` requires\n> +elevated privileges and the executed process do not have a full user\n\nEither s/process/processes/ or s/do/does/\n\n> +context. Without a full user context, Git and its credential helpers\n> +cannot access stored credentials, so some maintenance tasks are not\n> +functional.\n\nNicely explained.\n\n> +Instead, `git maintenance start` interacts with the `launchctl` tool,\n> +which is the recommended way to\n> +https://developer.apple.com/library/archive/documentation/MacOSX/Conceptual/BPSystemStartup/Chapters/ScheduledJobs.html[schedule timed jobs in macOS].\n\nNit: I worry a bit about links to Apple documentation becoming\noutdated. It might not hurt to omit this link altogether, or perhaps\ndemote it to a footnote (which might allow it to be somewhat usable\neven when Git documentation is rendered into something other than\nHTML).\n\n> +Scheduling maintenance through `git maintenance (start|stop)` requires\n> +some `launchctl` features available only in macOS 10.11 or later.\n\nNit: This leaves the reader wondering what modern features are needed.\nWould it make sense to mention that \"bootstrap\" is used in place of\n\"load\" in older versions of 'launchctl'?\n\n> +Your user-specific scheduled tasks are stored as XML-formatted `.plist`\n> +files in `~/Library/LaunchAgents/`. You can see the currently-registered\n> +tasks using the following command:\n> +\n> +-----------------------------------------------------------------------\n> +$ ls ~/Library/LaunchAgents/ | grep org.git-scm.git\n\nAlternately (unimportant):\n\n ls ~/Library/LaunchAgents/org.git-scm.git.*\n\nalthough that would emit \"No such file\" if you don't have any\nregistered, which might suggest:\n\n find ~/Library/LaunchAgents -name 'org.git-scm.git.*'\n\n> +To create more advanced customizations to your background tasks, see\n> +https://developer.apple.com/library/archive/documentation/MacOSX/Conceptual/BPSystemStartup/Chapters/CreatingLaunchdJobs.html#//apple_ref/doc/uid/TP40001762-104142[the `launchctl` documentation]\n> +for more information.\n\nI really worry about this sort of URL becoming outdated. Would it make\nsense instead to just point the user at the man page,\nlaunchd.plist(5)? It's not quite the same, as it doesn't provide the\nrange of examples as the URL you cite, but it should get the user\nstarted.\n\n> diff --git a/builtin/gc.c b/builtin/gc.c\n> @@ -1491,6 +1491,214 @@ static int maintenance_unregister(void)\n> +static int remove_plist(enum schedule_priority schedule)\n> +{\n> + const char *frequency = get_frequency(schedule);\n> + char *name = get_service_name(frequency);\n> + char *filename = get_service_filename(name);\n> + int result = bootout(filename);\n> + free(filename);\n> + free(name);\n> + return result;\n> +}\n>\n> +static int remove_plists(void)\n> +{\n> + return remove_plist(SCHEDULE_HOURLY) ||\n> + remove_plist(SCHEDULE_DAILY) ||\n> + remove_plist(SCHEDULE_WEEKLY);\n> +}\n\nThe new documentation you added says that the plist files will be\ndeleted after they are deregistered using launchctl, but I don't see\nanything actually deleting them. Am I missing something obvious?\n\n> +static int schedule_plist(const char *exec_path, enum schedule_priority schedule)\n> +{\n> + plist = fopen(filename, \"w\");\n> + if (!plist)\n> + die(_(\"failed to open '%s'\"), filename);\n\nAs mentioned previously, these could be replaced with a simple xfopen().\n\nIn fact, I'm having trouble seeing changes in this re-roll which you\nhad planned on making, such as consolidating the repeated code in\nbootout() and bootstrap(), and ensuring that bootout() doesn't\ncomplain if the plist files are already missing, and so forth. Did you\nopt to not make those changes? (Which would be fine; they were minor\nsuggestions.)\n\n> + preamble = \"\\n\"\n> + \"\\n\"\n> + \"\"\n> + \"\\n\"\n> + \"Label%s\\n\"\n> + \"ProgramArguments\\n\"\n> + \"\\n\"\n> + \"%s/git\\n\"\n> + \"--exec-path=%s\\n\"\n> + \"for-each-repo\\n\"\n> + \"--config=maintenance.repo\\n\"\n> + \"maintenance\\n\"\n> + \"run\\n\"\n> + \"--schedule=%s\\n\"\n> + \"\\n\"\n> + \"StartCalendarInterval\\n\"\n> + \"\\n\";\n> + fprintf(plist, preamble, name, exec_path, exec_path, frequency);\n\nThe Git test framework ensures that this will be written into the test\ndirectory rather than the user's actual ~/Library/LaunchAgents\ndirectory during testing. Okay.\n\n> +test_expect_success MACOS_MAINTENANCE 'start and stop macOS maintenance' '\n> + echo \"#!/bin/sh\\necho \\$@ >>args\" >print-args &&\n> + chmod a+x print-args &&\n\nEarlier review already mentioned write_script() and \"$@\". (Not\nnecessarily worth a re-roll.)\n\n> + for frequency in hourly daily weekly\n> + do\n> + PLIST=\"$HOME/Library/LaunchAgents/org.git-scm.git.$frequency.plist\" &&\n> + xmllint \"$PLIST\" >/dev/null &&\n\nDo we really need to suppress xmllint's stdout?\n\n> + grep schedule=$frequency \"$PLIST\" &&\n> + echo \"bootout gui/$UID $PLIST\" >>expect &&\n> + echo \"bootstrap gui/$UID $PLIST\" >>expect || return 1\n> + done &&\n> + test_cmp expect args &&\n> +\n> + rm -f args &&\n> + GIT_TEST_CRONTAB=\"./print-args\" git maintenance stop &&\n\nThere is still an extra space between the closing quote and git\ncommand (mentioned previously).\n\n> + # stop does not unregister the repo\n> + git config --get --global maintenance.repo \"$(pwd)\" &&\n> +\n> + # stop does not remove plist files, but boots them out\n\nDocumentation added in this re-roll claims that the plist files do get deleted.\n"},{"id":"409592","messageId":"CAPig+cRRQc=RNd=zGFvi2yiLD6PoLLBJQtFxkZ+QaVVA9MP1DA@mail.gmail.com","threadId":"54562","inReplyTo":"84eb44de31f04b2a94f57ee11d70be81f5bbeee2.1604520368.git.gitgitgadget@gmail.com","subject":"Re: [PATCH v2 4/4] maintenance: use Windows scheduled tasks","fromName":"Eric Sunshine","fromEmail":"sunshine@sunshineco.com","sentAt":"2020-11-11T08:59:35Z","receivedAt":"2020-11-11T08:59:50Z","isPatch":true,"sender":{"key":"sunshine@sunshineco.com","avatar":"https://avatars.githubusercontent.com/u/163641?v=4"},"body":"On Wed, Nov 4, 2020 at 3:06 PM Derrick Stolee via GitGitGadget\n wrote:\n> Git's background maintenance uses cron by default, but this is not\n> available on Windows. Instead, integrate with Task Scheduler.\n> [...]\n> Signed-off-by: Derrick Stolee \n> ---\n> diff --git a/builtin/gc.c b/builtin/gc.c\n> @@ -1698,6 +1698,187 @@ static int platform_update_schedule(int run_maintenance, int fd)\n> +static int schedule_task(const char *exec_path, enum schedule_priority schedule)\n> +{\n> + xmlpath = xstrfmt(\"%s/schedule-%s.xml\",\n> + the_repository->objects->odb->path,\n> + frequency);\n\nAm I reading correctly that it is writing this throwaway XML file into\nthe Git object directory? Would writing to a temporary directory make\nmore sense? (Not worth a re-roll.)\n\n> + xmlfp = fopen(xmlpath, \"w\");\n> + if (!xmlfp)\n> + die(_(\"failed to open '%s'\"), xmlpath);\n\nCould use xfopen() as mentioned previously.\n\n> diff --git a/t/t7900-maintenance.sh b/t/t7900-maintenance.sh\n> @@ -442,6 +442,36 @@ test_expect_success MACOS_MAINTENANCE 'start and stop macOS maintenance' '\n> +test_expect_success MINGW 'start and stop Windows maintenance' '\n> + for frequency in hourly daily weekly\n> + do\n> + printf \"/create /tn Git Maintenance (%s) /f /xml .git/objects/schedule-%s.xml\\n\" \\\n> + $frequency $frequency\n\nNit: You lost the `|| return 1` which was present in the previous\nversion. True, it's very unlikely that `printf` could fail, but having\nthe `|| return 1` there makes it easier for the reader's eye to glide\nover the code without having to worry about whether it is handling\nerror conditions correctly, thus reduces cognitive load.\n\n> + done >expect &&\n\nRather than a loop, you could just use:\n\n printf \"/create ... (%s) /f /xml ...schedule-%s.xml\\n\" \\\n hourly hourly daily daily weekly weekly >expect &&\n\nthough it's subjective as to whether that is an improvement.\n"},{"id":"409755","messageId":"5e0307b8-8e63-f82d-b417-7007d4a3a5b3@gmail.com","threadId":"54562","inReplyTo":"CAPig+cT=DytbMH6KkC6ipD3jbWNa7jgW9G0Q76rwJoEsLGn_ow@mail.gmail.com","subject":"Re: [PATCH v2 3/4] maintenance: use launchctl on macOS","fromName":"Derrick Stolee","fromEmail":"stolee@gmail.com","sentAt":"2020-11-12T13:42:52Z","receivedAt":"2020-11-12T13:42:58Z","isPatch":true,"sender":{"key":"stolee@gmail.com","avatar":"https://avatars.githubusercontent.com/u/570044?v=4"},"body":"On 11/11/2020 3:12 AM, Eric Sunshine wrote:\n> On Wed, Nov 4, 2020 at 3:06 PM Derrick Stolee via GitGitGadget\n> wrote:\n>> [...]\n>> The solution is to switch from cron to the Apple-recommended [1]\n>> 'launchd' tool.\n>> [...]\n>> Signed-off-by: Derrick Stolee \n>> ---\n>> diff --git a/Documentation/git-maintenance.txt b/Documentation/git-maintenance.txt\n>> +While macOS technically supports `cron`, using `crontab -e` requires\n>> +elevated privileges and the executed process do not have a full user\n> \n> Either s/process/processes/ or s/do/does/\n> \n>> +context. Without a full user context, Git and its credential helpers\n>> +cannot access stored credentials, so some maintenance tasks are not\n>> +functional.\n> \n> Nicely explained.\n> \n>> +Instead, `git maintenance start` interacts with the `launchctl` tool,\n>> +which is the recommended way to\n>> +https://developer.apple.com/library/archive/documentation/MacOSX/Conceptual/BPSystemStartup/Chapters/ScheduledJobs.html[schedule timed jobs in macOS].\n> \n> Nit: I worry a bit about links to Apple documentation becoming\n> outdated. It might not hurt to omit this link altogether, or perhaps\n> demote it to a footnote (which might allow it to be somewhat usable\n> even when Git documentation is rendered into something other than\n> HTML).\n> \n>> +Scheduling maintenance through `git maintenance (start|stop)` requires\n>> +some `launchctl` features available only in macOS 10.11 or later.\n> \n> Nit: This leaves the reader wondering what modern features are needed.\n> Would it make sense to mention that \"bootstrap\" is used in place of\n> \"load\" in older versions of 'launchctl'?\n> \n>> +Your user-specific scheduled tasks are stored as XML-formatted `.plist`\n>> +files in `~/Library/LaunchAgents/`. You can see the currently-registered\n>> +tasks using the following command:\n>> +\n>> +-----------------------------------------------------------------------\n>> +$ ls ~/Library/LaunchAgents/ | grep org.git-scm.git\n> \n> Alternately (unimportant):\n> \n> ls ~/Library/LaunchAgents/org.git-scm.git.*\n> \n> although that would emit \"No such file\" if you don't have any\n> registered, which might suggest:\n> \n> find ~/Library/LaunchAgents -name 'org.git-scm.git.*'\n> \n>> +To create more advanced customizations to your background tasks, see\n>> +https://developer.apple.com/library/archive/documentation/MacOSX/Conceptual/BPSystemStartup/Chapters/CreatingLaunchdJobs.html#//apple_ref/doc/uid/TP40001762-104142[the `launchctl` documentation]\n>> +for more information.\n> \n> I really worry about this sort of URL becoming outdated. Would it make\n> sense instead to just point the user at the man page,\n> launchd.plist(5)? It's not quite the same, as it doesn't provide the\n> range of examples as the URL you cite, but it should get the user\n> started.\n\nI shared similar concerns. I'll use the man page references instead.\nAll of the information should be a short web search away after the\nuser is given the right terminology.\n\n>> diff --git a/builtin/gc.c b/builtin/gc.c\n>> @@ -1491,6 +1491,214 @@ static int maintenance_unregister(void)\n>> +static int remove_plist(enum schedule_priority schedule)\n>> +{\n>> + const char *frequency = get_frequency(schedule);\n>> + char *name = get_service_name(frequency);\n>> + char *filename = get_service_filename(name);\n>> + int result = bootout(filename);\n>> + free(filename);\n>> + free(name);\n>> + return result;\n>> +}\n>>\n>> +static int remove_plists(void)\n>> +{\n>> + return remove_plist(SCHEDULE_HOURLY) ||\n>> + remove_plist(SCHEDULE_DAILY) ||\n>> + remove_plist(SCHEDULE_WEEKLY);\n>> +}\n> \n> The new documentation you added says that the plist files will be\n> deleted after they are deregistered using launchctl, but I don't see\n> anything actually deleting them. Am I missing something obvious?\n\nAs mentioned below, this was a change that I made but somehow lost\nwhile juggling multiple copies of my branch.\n\n>> +static int schedule_plist(const char *exec_path, enum schedule_priority schedule)\n>> +{\n>> + plist = fopen(filename, \"w\");\n>> + if (!plist)\n>> + die(_(\"failed to open '%s'\"), filename);\n> \n> As mentioned previously, these could be replaced with a simple xfopen().\n> \n> In fact, I'm having trouble seeing changes in this re-roll which you\n> had planned on making, such as consolidating the repeated code in\n> bootout() and bootstrap(), and ensuring that bootout() doesn't\n> complain if the plist files are already missing, and so forth. Did you\n> opt to not make those changes? (Which would be fine; they were minor\n> suggestions.)\n\nNo, I definitely made those changes _somewhere_ but I must have\ngotten confused as to which of my machines had those changes. I\nguess that's part of the risk of testing across three platforms.\n\nThank you for noticing, and I'll be more careful from now on.\n\n>> +test_expect_success MACOS_MAINTENANCE 'start and stop macOS maintenance' '\n>> + echo \"#!/bin/sh\\necho \\$@ >>args\" >print-args &&\n>> + chmod a+x print-args &&\n> \n> Earlier review already mentioned write_script() and \"$@\". (Not\n> necessarily worth a re-roll.)\n\nI'm going to go back to all of your earlier comments to make sure\nthey are _actually_ applied in v3.\n \n>> + for frequency in hourly daily weekly\n>> + do\n>> + PLIST=\"$HOME/Library/LaunchAgents/org.git-scm.git.$frequency.plist\" &&\n>> + xmllint \"$PLIST\" >/dev/null &&\n> \n> Do we really need to suppress xmllint's stdout?\n\nIt outputs the XML itself. Maybe there is a command to stop that from\nhappening, but nulling stdout keeps the test log clean.\n\nThanks,\n-Stolee\n"},{"id":"409758","messageId":"d46826a5-0c30-3d71-26a0-f579d997c3a0@gmail.com","threadId":"54562","inReplyTo":"CAPig+cRRQc=RNd=zGFvi2yiLD6PoLLBJQtFxkZ+QaVVA9MP1DA@mail.gmail.com","subject":"Re: [PATCH v2 4/4] maintenance: use Windows scheduled tasks","fromName":"Derrick Stolee","fromEmail":"stolee@gmail.com","sentAt":"2020-11-12T13:56:45Z","receivedAt":"2020-11-12T13:56:50Z","isPatch":true,"sender":{"key":"stolee@gmail.com","avatar":"https://avatars.githubusercontent.com/u/570044?v=4"},"body":"On 11/11/2020 3:59 AM, Eric Sunshine wrote:\n> On Wed, Nov 4, 2020 at 3:06 PM Derrick Stolee via GitGitGadget\n> wrote:\n>> Git's background maintenance uses cron by default, but this is not\n>> available on Windows. Instead, integrate with Task Scheduler.\n>> [...]\n>> Signed-off-by: Derrick Stolee \n>> ---\n>> diff --git a/builtin/gc.c b/builtin/gc.c\n>> @@ -1698,6 +1698,187 @@ static int platform_update_schedule(int run_maintenance, int fd)\n>> +static int schedule_task(const char *exec_path, enum schedule_priority schedule)\n>> +{\n>> + xmlpath = xstrfmt(\"%s/schedule-%s.xml\",\n>> + the_repository->objects->odb->path,\n>> + frequency);\n> \n> Am I reading correctly that it is writing this throwaway XML file into\n> the Git object directory? Would writing to a temporary directory make\n> more sense? (Not worth a re-roll.)\n\nA temp directory is a good idea.\n\n>> + xmlfp = fopen(xmlpath, \"w\");\n>> + if (!xmlfp)\n>> + die(_(\"failed to open '%s'\"), xmlpath);\n> \n> Could use xfopen() as mentioned previously.\n> \n>> diff --git a/t/t7900-maintenance.sh b/t/t7900-maintenance.sh\n>> @@ -442,6 +442,36 @@ test_expect_success MACOS_MAINTENANCE 'start and stop macOS maintenance' '\n>> +test_expect_success MINGW 'start and stop Windows maintenance' '\n>> + for frequency in hourly daily weekly\n>> + do\n>> + printf \"/create /tn Git Maintenance (%s) /f /xml .git/objects/schedule-%s.xml\\n\" \\\n>> + $frequency $frequency\n> \n> Nit: You lost the `|| return 1` which was present in the previous\n> version. True, it's very unlikely that `printf` could fail, but having\n> the `|| return 1` there makes it easier for the reader's eye to glide\n> over the code without having to worry about whether it is handling\n> error conditions correctly, thus reduces cognitive load.\n>\n>> + done >expect &&\n> \n> Rather than a loop, you could just use:\n> \n> printf \"/create ... (%s) /f /xml ...schedule-%s.xml\\n\" \\\n> hourly hourly daily daily weekly weekly >expect &&\n> \n> though it's subjective as to whether that is an improvement.\n\nIt's sufficient, and avoids issues with deep tabbing and\nchaining \"|| return 1\"\".\n\nThanks,\n-Stolee\n\n"},{"id":"409771","messageId":"CAPig+cQzp_nyp9BNc+WZuBWmXnArMGp8iCB4pHM+Fw+XJ8+eCw@mail.gmail.com","threadId":"54562","inReplyTo":"5e0307b8-8e63-f82d-b417-7007d4a3a5b3@gmail.com","subject":"Re: [PATCH v2 3/4] maintenance: use launchctl on macOS","fromName":"Eric Sunshine","fromEmail":"sunshine@sunshineco.com","sentAt":"2020-11-12T16:43:57Z","receivedAt":"2020-11-12T16:44:14Z","isPatch":true,"sender":{"key":"sunshine@sunshineco.com","avatar":"https://avatars.githubusercontent.com/u/163641?v=4"},"body":"On Thu, Nov 12, 2020 at 8:43 AM Derrick Stolee wrote:\n> On 11/11/2020 3:12 AM, Eric Sunshine wrote:\n> > On Wed, Nov 4, 2020 at 3:06 PM Derrick Stolee via GitGitGadget\n> > wrote:\n> >> + xmllint \"$PLIST\" >/dev/null &&\n> >\n> > Do we really need to suppress xmllint's stdout?\n>\n> It outputs the XML itself. Maybe there is a command to stop that from\n> happening, but nulling stdout keeps the test log clean.\n\nxmllint's --noout option should do the trick.\n"},{"id":"409894","messageId":"d35f1aa162097d4dfe468b3ac1717bca92a41cfa.1605276024.git.gitgitgadget@gmail.com","threadId":"54562","inReplyTo":"pull.776.v3.git.1605276024.gitgitgadget@gmail.com","subject":"[PATCH v3 1/4] maintenance: extract platform-specific scheduling","fromName":"Derrick Stolee via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2020-11-13T14:00:21Z","receivedAt":"2020-11-13T14:00:30Z","isPatch":true,"sender":{"key":"stolee@gmail.com","avatar":"https://avatars.githubusercontent.com/u/570044?v=4"},"body":"From: Derrick Stolee \n\nThe existing schedule mechanism using 'cron' is supported by POSIX\nplatforms, but not Windows. It also works slightly differently on\nmacOS to significant detriment of the user experience. To allow for\nnew implementations on these platforms, extract a method that\nperforms the platform-specific scheduling mechanism. This will be\nswapped at compile time with new implementations on specialized\nplatforms.\n\nSigned-off-by: Derrick Stolee \n---\n builtin/gc.c | 38 +++++++++++++++++++++-----------------\n 1 file changed, 21 insertions(+), 17 deletions(-)\n\ndiff --git a/builtin/gc.c b/builtin/gc.c\nindex e3098ef6a1..c1f7d9bdc2 100644\n--- a/builtin/gc.c\n+++ b/builtin/gc.c\n@@ -1494,7 +1494,7 @@ static int maintenance_unregister(void)\n #define BEGIN_LINE \"# BEGIN GIT MAINTENANCE SCHEDULE\"\n #define END_LINE \"# END GIT MAINTENANCE SCHEDULE\"\n \n-static int update_background_schedule(int run_maintenance)\n+static int platform_update_schedule(int run_maintenance, int fd)\n {\n \tint result = 0;\n \tint in_old_region = 0;\n@@ -1503,11 +1503,6 @@ static int update_background_schedule(int run_maintenance)\n \tFILE *cron_list, *cron_in;\n \tconst char *crontab_name;\n \tstruct strbuf line = STRBUF_INIT;\n-\tstruct lock_file lk;\n-\tchar *lock_path = xstrfmt(\"%s/schedule\", the_repository->objects->odb->path);\n-\n-\tif (hold_lock_file_for_update(&lk, lock_path, LOCK_NO_DEREF) < 0)\n-\t\treturn error(_(\"another process is scheduling background maintenance\"));\n \n \tcrontab_name = getenv(\"GIT_TEST_CRONTAB\");\n \tif (!crontab_name)\n@@ -1516,12 +1511,11 @@ static int update_background_schedule(int run_maintenance)\n \tstrvec_split(&crontab_list.args, crontab_name);\n \tstrvec_push(&crontab_list.args, \"-l\");\n \tcrontab_list.in = -1;\n-\tcrontab_list.out = dup(lk.tempfile->fd);\n+\tcrontab_list.out = dup(fd);\n \tcrontab_list.git_cmd = 0;\n \n \tif (start_command(&crontab_list)) {\n-\t\tresult = error(_(\"failed to run 'crontab -l'; your system might not support 'cron'\"));\n-\t\tgoto cleanup;\n+\t\treturn error(_(\"failed to run 'crontab -l'; your system might not support 'cron'\"));\n \t}\n \n \t/* Ignore exit code, as an empty crontab will return error. */\n@@ -1531,7 +1525,7 @@ static int update_background_schedule(int run_maintenance)\n \t * Read from the .lock file, filtering out the old\n \t * schedule while appending the new schedule.\n \t */\n-\tcron_list = fdopen(lk.tempfile->fd, \"r\");\n+\tcron_list = fdopen(fd, \"r\");\n \trewind(cron_list);\n \n \tstrvec_split(&crontab_edit.args, crontab_name);\n@@ -1539,8 +1533,7 @@ static int update_background_schedule(int run_maintenance)\n \tcrontab_edit.git_cmd = 0;\n \n \tif (start_command(&crontab_edit)) {\n-\t\tresult = error(_(\"failed to run 'crontab'; your system might not support 'cron'\"));\n-\t\tgoto cleanup;\n+\t\treturn error(_(\"failed to run 'crontab'; your system might not support 'cron'\"));\n \t}\n \n \tcron_in = fdopen(crontab_edit.in, \"w\");\n@@ -1586,13 +1579,24 @@ static int update_background_schedule(int run_maintenance)\n \tclose(crontab_edit.in);\n \n done_editing:\n-\tif (finish_command(&crontab_edit)) {\n+\tif (finish_command(&crontab_edit))\n \t\tresult = error(_(\"'crontab' died\"));\n-\t\tgoto cleanup;\n-\t}\n-\tfclose(cron_list);\n+\telse\n+\t\tfclose(cron_list);\n+\treturn result;\n+}\n+\n+static int update_background_schedule(int run_maintenance)\n+{\n+\tint result;\n+\tstruct lock_file lk;\n+\tchar *lock_path = xstrfmt(\"%s/schedule\", the_repository->objects->odb->path);\n+\n+\tif (hold_lock_file_for_update(&lk, lock_path, LOCK_NO_DEREF) < 0)\n+\t\treturn error(_(\"another process is scheduling background maintenance\"));\n+\n+\tresult = platform_update_schedule(run_maintenance, lk.tempfile->fd);\n \n-cleanup:\n \trollback_lock_file(&lk);\n \treturn result;\n }\n-- \ngitgitgadget\n\n"},{"id":"409895","messageId":"0dfe53092ea507e74260f2638c46c618fa554c94.1605276024.git.gitgitgadget@gmail.com","threadId":"54562","inReplyTo":"pull.776.v3.git.1605276024.gitgitgadget@gmail.com","subject":"[PATCH v3 2/4] maintenance: include 'cron' details in docs","fromName":"Derrick Stolee via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2020-11-13T14:00:22Z","receivedAt":"2020-11-13T14:00:36Z","isPatch":true,"sender":{"key":"stolee@gmail.com","avatar":"https://avatars.githubusercontent.com/u/570044?v=4"},"body":"From: Derrick Stolee \n\nAdvanced and expert users may want to know how 'git maintenance start'\nschedules background maintenance in order to customize their own\nschedules beyond what the maintenance.* config values allow. Start a new\nset of sections in git-maintenance.txt that describe how 'cron' is used\nto run these tasks.\n\nThis is particularly valuable for users who want to inspect what Git is\ndoing or for users who want to customize the schedule further. Having a\nbaseline can provide a way forward for users who have never worked with\ncron schedules.\n\nHelped-by: Eric Sunshine \nSigned-off-by: Derrick Stolee \n---\n Documentation/git-maintenance.txt | 54 +++++++++++++++++++++++++++++++\n 1 file changed, 54 insertions(+)\n\ndiff --git a/Documentation/git-maintenance.txt b/Documentation/git-maintenance.txt\nindex 6fec1eb8dc..1aa1112418 100644\n--- a/Documentation/git-maintenance.txt\n+++ b/Documentation/git-maintenance.txt\n@@ -218,6 +218,60 @@ Further, the `git gc` command should not be combined with\n but does not take the lock in the same way as `git maintenance run`. If\n possible, use `git maintenance run --task=gc` instead of `git gc`.\n \n+The following sections describe the mechanisms put in place to run\n+background maintenance by `git maintenance start` and how to customize\n+them.\n+\n+BACKGROUND MAINTENANCE ON POSIX SYSTEMS\n+---------------------------------------\n+\n+The standard mechanism for scheduling background tasks on POSIX systems\n+is cron(8). This tool executes commands based on a given schedule. The\n+current list of user-scheduled tasks can be found by running `crontab -l`.\n+The schedule written by `git maintenance start` is similar to this:\n+\n+-----------------------------------------------------------------------\n+# BEGIN GIT MAINTENANCE SCHEDULE\n+# The following schedule was created by Git\n+# Any edits made in this region might be\n+# replaced in the future by a Git command.\n+\n+0 1-23 * * * \"//git\" --exec-path=\"/\" for-each-repo --config=maintenance.repo maintenance run --schedule=hourly\n+0 0 * * 1-6 \"//git\" --exec-path=\"/\" for-each-repo --config=maintenance.repo maintenance run --schedule=daily\n+0 0 * * 0 \"//git\" --exec-path=\"/\" for-each-repo --config=maintenance.repo maintenance run --schedule=weekly\n+\n+# END GIT MAINTENANCE SCHEDULE\n+-----------------------------------------------------------------------\n+\n+The comments are used as a region to mark the schedule as written by Git.\n+Any modifications within this region will be completely deleted by\n+`git maintenance stop` or overwritten by `git maintenance start`.\n+\n+The `crontab` entry specifies the full path of the `git` executable to\n+ensure that the executed `git` command is the same one with which\n+`git maintenance start` was issued independent of `PATH`. If the same user\n+runs `git maintenance start` with multiple Git executables, then only the\n+latest executable is used.\n+\n+These commands use `git for-each-repo --config=maintenance.repo` to run\n+`git maintenance run --schedule=` on each repository listed in\n+the multi-valued `maintenance.repo` config option. These are typically\n+loaded from the user-specific global config. The `git maintenance` process\n+then determines which maintenance tasks are configured to run on each\n+repository with each `` using the `maintenance..schedule`\n+config options. These values are loaded from the global or repository\n+config values.\n+\n+If the config values are insufficient to achieve your desired background\n+maintenance schedule, then you can create your own schedule. If you run\n+`crontab -e`, then an editor will load with your user-specific `cron`\n+schedule. In that editor, you can add your own schedule lines. You could\n+start by adapting the default schedule listed earlier, or you could read\n+the crontab(5) documentation for advanced scheduling techniques. Please\n+do use the full path and `--exec-path` techniques from the default\n+schedule to ensure you are executing the correct binaries in your\n+schedule.\n+\n \n GIT\n ---\n-- \ngitgitgadget\n\n"},{"id":"409896","messageId":"1629bcfcf82dbc2ed9889a0e9ea2d08427901c4e.1605276024.git.gitgitgadget@gmail.com","threadId":"54562","inReplyTo":"pull.776.v3.git.1605276024.gitgitgadget@gmail.com","subject":"[PATCH v3 3/4] maintenance: use launchctl on macOS","fromName":"Derrick Stolee via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2020-11-13T14:00:23Z","receivedAt":"2020-11-13T14:00:37Z","isPatch":true,"sender":{"key":"stolee@gmail.com","avatar":"https://avatars.githubusercontent.com/u/570044?v=4"},"body":"From: Derrick Stolee \n\nThe existing mechanism for scheduling background maintenance is done\nthrough cron. The 'crontab -e' command allows updating the schedule\nwhile cron itself runs those commands. While this is technically\nsupported by macOS, it has some significant deficiencies:\n\n1. Every run of 'crontab -e' must request elevated privileges through\n the user interface. When running 'git maintenance start' from the\n Terminal app, it presents a dialog box saying \"Terminal.app would\n like to administer your computer. Administration can include\n modifying passwords, networking, and system settings.\" This is more\n alarming than what we are hoping to achieve. If this alert had some\n information about how \"git\" is trying to run \"crontab\" then we would\n have some reason to believe that this dialog might be fine. However,\n it also doesn't help that some scenarios just leave Git waiting for\n a response without presenting anything to the user. I experienced\n this when executing the command from a Bash terminal view inside\n Visual Studio Code.\n\n2. While cron initializes a user environment enough for \"git config\n --global --show-origin\" to show the correct config file information,\n it does not set up the environment enough for Git Credential Manager\n Core to load credentials during a 'prefetch' task. My prefetches\n against private repositories required re-authenticating through UI\n pop-ups in a way that should not be required.\n\nThe solution is to switch from cron to the Apple-recommended [1]\n'launchd' tool.\n\n[1] https://developer.apple.com/library/archive/documentation/MacOSX/Conceptual/BPSystemStartup/Chapters/ScheduledJobs.html\n\nThe basics of this tool is that we need to create XML-formatted\n\"plist\" files inside \"~/Library/LaunchAgents/\" and then use the\n'launchctl' tool to make launchd aware of them. The plist files\ninclude all of the scheduling information, along with the command-line\narguments split across an array of tags.\n\nFor example, here is my plist file for the weekly scheduled tasks:\n\n\n\n\nLabelorg.git-scm.git.weekly\nProgramArguments\n\n/usr/local/libexec/git-core/git\n--exec-path=/usr/local/libexec/git-core\nfor-each-repo\n--config=maintenance.repo\nmaintenance\nrun\n--schedule=weekly\n\nStartCalendarInterval\n\n\nDay0\nHour0\nMinute0\n\n\n\n\n\nThe schedules for the daily and hourly tasks are more complicated\nsince we need to use an array for the StartCalendarInterval with\nan entry for each of the six days other than the 0th day (to avoid\ncolliding with the weekly task), and each of the 23 hours other\nthan the 0th hour (to avoid colliding with the daily task).\n\nThe \"Label\" value is currently filled with \"org.git-scm.git.X\"\nwhere X is the frequency. We need a different plist file for each\nfrequency.\n\nThe launchctl command needs to be aligned with a user id in order\nto initialize the command environment. This must be done using\nthe 'launchctl bootstrap' subcommand. This subcommand is new as\nof macOS 10.11, which was released in September 2015. Before that\nrelease the 'launchctl load' subcommand was recommended. The best\nsource of information on this transition I have seen is available\nat [2]. The current design does not preclude a future version that\ndetects the available fatures of 'launchctl' to use the older\ncommands. However, it is best to rely on the newest version since\nApple might completely remove the deprecated version on short\nnotice.\n\n[2] https://babodee.wordpress.com/2016/04/09/launchctl-2-0-syntax/\n\nTo remove a schedule, we must run 'launchctl bootout' with a valid\nplist file. We also need to 'bootout' a task before the 'bootstrap'\nsubcommand will succeed, if such a task already exists.\n\nWe can verify the commands that were run by 'git maintenance start'\nand 'git maintenance stop' by injecting a script that writes the\ncommand-line arguments into GIT_TEST_CRONTAB.\n\nAn earlier version of this patch accidentally had an opening\n\"\" tag when it should have had a closing \"\" tag. This\nwas caught during manual testing with actual 'launchctl' commands,\nbut we do not want to update developers' tasks when running tests.\nIt appears that macOS includes the \"xmllint\" tool which can verify\nthe XML format, so call it from the macOS-specific tests to ensure\nthe .plist files are well-formatted.\n\nHelped-by: Eric Sunshine \nSigned-off-by: Derrick Stolee \n---\n Documentation/git-maintenance.txt | 40 ++++++\n builtin/gc.c | 195 ++++++++++++++++++++++++++++++\n t/t7900-maintenance.sh | 48 +++++++-\n t/test-lib.sh | 4 +\n 4 files changed, 284 insertions(+), 3 deletions(-)\n\ndiff --git a/Documentation/git-maintenance.txt b/Documentation/git-maintenance.txt\nindex 1aa1112418..5f8f63f098 100644\n--- a/Documentation/git-maintenance.txt\n+++ b/Documentation/git-maintenance.txt\n@@ -273,6 +273,46 @@ schedule to ensure you are executing the correct binaries in your\n schedule.\n \n \n+BACKGROUND MAINTENANCE ON MACOS SYSTEMS\n+---------------------------------------\n+\n+While macOS technically supports `cron`, using `crontab -e` requires\n+elevated privileges and the executed process does not have a full user\n+context. Without a full user context, Git and its credential helpers\n+cannot access stored credentials, so some maintenance tasks are not\n+functional.\n+\n+Instead, `git maintenance start` interacts with the `launchctl` tool,\n+which is the recommended way to schedule timed jobs in macOS. Scheduling\n+maintenance through `git maintenance (start|stop)` requires some\n+`launchctl` features available only in macOS 10.11 or later.\n+\n+Your user-specific scheduled tasks are stored as XML-formatted `.plist`\n+files in `~/Library/LaunchAgents/`. You can see the currently-registered\n+tasks using the following command:\n+\n+-----------------------------------------------------------------------\n+$ ls ~/Library/LaunchAgents/org.git-scm.git*\n+org.git-scm.git.daily.plist\n+org.git-scm.git.hourly.plist\n+org.git-scm.git.weekly.plist\n+-----------------------------------------------------------------------\n+\n+One task is registered for each `--schedule=` option. To\n+inspect how the XML format describes each schedule, open one of these\n+`.plist` files in an editor and inspect the `` element following\n+the `StartCalendarInterval` element.\n+\n+`git maintenance start` will overwrite these files and register the\n+tasks again with `launchctl`, so any customizations should be done by\n+creating your own `.plist` files with distinct names. Similarly, the\n+`git maintenance stop` command will unregister the tasks with `launchctl`\n+and delete the `.plist` files.\n+\n+To create more advanced customizations to your background tasks, see\n+launchctl.plist(5) for more information.\n+\n+\n GIT\n ---\n Part of the linkgit:git[1] suite\ndiff --git a/builtin/gc.c b/builtin/gc.c\nindex c1f7d9bdc2..da2c892f68 100644\n--- a/builtin/gc.c\n+++ b/builtin/gc.c\n@@ -1491,6 +1491,200 @@ static int maintenance_unregister(void)\n \treturn run_command(&config_unset);\n }\n \n+#if defined(__APPLE__)\n+\n+static char *get_service_name(const char *frequency)\n+{\n+\tstruct strbuf label = STRBUF_INIT;\n+\tstrbuf_addf(&label, \"org.git-scm.git.%s\", frequency);\n+\treturn strbuf_detach(&label, NULL);\n+}\n+\n+static char *get_service_filename(const char *name)\n+{\n+\tchar *expanded;\n+\tstruct strbuf filename = STRBUF_INIT;\n+\tstrbuf_addf(&filename, \"~/Library/LaunchAgents/%s.plist\", name);\n+\n+\texpanded = expand_user_path(filename.buf, 1);\n+\tif (!expanded)\n+\t\tdie(_(\"failed to expand path '%s'\"), filename.buf);\n+\n+\tstrbuf_release(&filename);\n+\treturn expanded;\n+}\n+\n+static const char *get_frequency(enum schedule_priority schedule)\n+{\n+\tswitch (schedule) {\n+\tcase SCHEDULE_HOURLY:\n+\t\treturn \"hourly\";\n+\tcase SCHEDULE_DAILY:\n+\t\treturn \"daily\";\n+\tcase SCHEDULE_WEEKLY:\n+\t\treturn \"weekly\";\n+\tdefault:\n+\t\tBUG(\"invalid schedule %d\", schedule);\n+\t}\n+}\n+\n+static char *get_uid(void)\n+{\n+\tstruct strbuf output = STRBUF_INIT;\n+\tstruct child_process id = CHILD_PROCESS_INIT;\n+\n+\tstrvec_pushl(&id.args, \"/usr/bin/id\", \"-u\", NULL);\n+\tif (capture_command(&id, &output, 0))\n+\t\tdie(_(\"failed to discover user id\"));\n+\n+\tstrbuf_trim_trailing_newline(&output);\n+\treturn strbuf_detach(&output, NULL);\n+}\n+\n+static int boot_plist(int enable, const char *filename)\n+{\n+\tint result;\n+\tstruct child_process child = CHILD_PROCESS_INIT;\n+\tchar *uid = get_uid();\n+\tconst char *launchctl = getenv(\"GIT_TEST_CRONTAB\");\n+\tif (!launchctl)\n+\t\tlaunchctl = \"/bin/launchctl\";\n+\n+\tstrvec_split(&child.args, launchctl);\n+\n+\tif (enable)\n+\t\tstrvec_push(&child.args, \"bootstrap\");\n+\telse\n+\t\tstrvec_push(&child.args, \"bootout\");\n+\tstrvec_pushf(&child.args, \"gui/%s\", uid);\n+\tstrvec_push(&child.args, filename);\n+\n+\tchild.no_stderr = 1;\n+\tchild.no_stdout = 1;\n+\n+\tif (start_command(&child))\n+\t\tdie(_(\"failed to start launchctl\"));\n+\n+\tresult = finish_command(&child);\n+\n+\tfree(uid);\n+\treturn result;\n+}\n+\n+static int remove_plist(enum schedule_priority schedule)\n+{\n+\tconst char *frequency = get_frequency(schedule);\n+\tchar *name = get_service_name(frequency);\n+\tchar *filename = get_service_filename(name);\n+\tint result = boot_plist(0, filename);\n+\tunlink(filename);\n+\tfree(filename);\n+\tfree(name);\n+\treturn result;\n+}\n+\n+static int remove_plists(void)\n+{\n+\treturn remove_plist(SCHEDULE_HOURLY) ||\n+\t\tremove_plist(SCHEDULE_DAILY) ||\n+\t\tremove_plist(SCHEDULE_WEEKLY);\n+}\n+\n+static int schedule_plist(const char *exec_path, enum schedule_priority schedule)\n+{\n+\tFILE *plist;\n+\tint i;\n+\tconst char *preamble, *repeat;\n+\tconst char *frequency = get_frequency(schedule);\n+\tchar *name = get_service_name(frequency);\n+\tchar *filename = get_service_filename(name);\n+\n+\tif (safe_create_leading_directories(filename))\n+\t\tdie(_(\"failed to create directories for '%s'\"), filename);\n+\tplist = xfopen(filename, \"w\");\n+\n+\tpreamble = \"\\n\"\n+\t\t \"\\n\"\n+\t\t \"\"\n+\t\t \"\\n\"\n+\t\t \"Label%s\\n\"\n+\t\t \"ProgramArguments\\n\"\n+\t\t \"\\n\"\n+\t\t \"%s/git\\n\"\n+\t\t \"--exec-path=%s\\n\"\n+\t\t \"for-each-repo\\n\"\n+\t\t \"--config=maintenance.repo\\n\"\n+\t\t \"maintenance\\n\"\n+\t\t \"run\\n\"\n+\t\t \"--schedule=%s\\n\"\n+\t\t \"\\n\"\n+\t\t \"StartCalendarInterval\\n\"\n+\t\t \"\\n\";\n+\tfprintf(plist, preamble, name, exec_path, exec_path, frequency);\n+\n+\tswitch (schedule) {\n+\tcase SCHEDULE_HOURLY:\n+\t\trepeat = \"\\n\"\n+\t\t\t \"Hour%d\\n\"\n+\t\t\t \"Minute0\\n\"\n+\t\t\t \"\\n\";\n+\t\tfor (i = 1; i <= 23; i++)\n+\t\t\tfprintf(plist, repeat, i);\n+\t\tbreak;\n+\n+\tcase SCHEDULE_DAILY:\n+\t\trepeat = \"\\n\"\n+\t\t\t \"Day%d\\n\"\n+\t\t\t \"Hour0\\n\"\n+\t\t\t \"Minute0\\n\"\n+\t\t\t \"\\n\";\n+\t\tfor (i = 1; i <= 6; i++)\n+\t\t\tfprintf(plist, repeat, i);\n+\t\tbreak;\n+\n+\tcase SCHEDULE_WEEKLY:\n+\t\tfprintf(plist,\n+\t\t\t\"\\n\"\n+\t\t\t\"Day0\\n\"\n+\t\t\t\"Hour0\\n\"\n+\t\t\t\"Minute0\\n\"\n+\t\t\t\"\\n\");\n+\t\tbreak;\n+\n+\tdefault:\n+\t\t/* unreachable */\n+\t\tbreak;\n+\t}\n+\tfprintf(plist, \"\\n\\n\\n\");\n+\n+\t/* bootout might fail if not already running, so ignore */\n+\tboot_plist(0, filename);\n+\tif (boot_plist(1, filename))\n+\t\tdie(_(\"failed to bootstrap service %s\"), filename);\n+\n+\tfclose(plist);\n+\tfree(filename);\n+\tfree(name);\n+\treturn 0;\n+}\n+\n+static int add_plists(void)\n+{\n+\tconst char *exec_path = git_exec_path();\n+\n+\treturn schedule_plist(exec_path, SCHEDULE_HOURLY) ||\n+\t\tschedule_plist(exec_path, SCHEDULE_DAILY) ||\n+\t\tschedule_plist(exec_path, SCHEDULE_WEEKLY);\n+}\n+\n+static int platform_update_schedule(int run_maintenance, int fd)\n+{\n+\tif (run_maintenance)\n+\t\treturn add_plists();\n+\telse\n+\t\treturn remove_plists();\n+}\n+#else\n #define BEGIN_LINE \"# BEGIN GIT MAINTENANCE SCHEDULE\"\n #define END_LINE \"# END GIT MAINTENANCE SCHEDULE\"\n \n@@ -1585,6 +1779,7 @@ static int platform_update_schedule(int run_maintenance, int fd)\n \t\tfclose(cron_list);\n \treturn result;\n }\n+#endif\n \n static int update_background_schedule(int run_maintenance)\n {\ndiff --git a/t/t7900-maintenance.sh b/t/t7900-maintenance.sh\nindex 20184e96e1..29d340a828 100755\n--- a/t/t7900-maintenance.sh\n+++ b/t/t7900-maintenance.sh\n@@ -367,7 +367,7 @@ test_expect_success 'register and unregister' '\n \ttest_cmp before actual\n '\n \n-test_expect_success 'start from empty cron table' '\n+test_expect_success !MACOS_MAINTENANCE 'start from empty cron table' '\n \tGIT_TEST_CRONTAB=\"test-tool crontab cron.txt\" git maintenance start &&\n \n \t# start registers the repo\n@@ -378,7 +378,7 @@ test_expect_success 'start from empty cron table' '\n \tgrep \"for-each-repo --config=maintenance.repo maintenance run --schedule=weekly\" cron.txt\n '\n \n-test_expect_success 'stop from existing schedule' '\n+test_expect_success !MACOS_MAINTENANCE 'stop from existing schedule' '\n \tGIT_TEST_CRONTAB=\"test-tool crontab cron.txt\" git maintenance stop &&\n \n \t# stop does not unregister the repo\n@@ -389,12 +389,54 @@ test_expect_success 'stop from existing schedule' '\n \ttest_must_be_empty cron.txt\n '\n \n-test_expect_success 'start preserves existing schedule' '\n+test_expect_success !MACOS_MAINTENANCE 'start preserves existing schedule' '\n \techo \"Important information!\" >cron.txt &&\n \tGIT_TEST_CRONTAB=\"test-tool crontab cron.txt\" git maintenance start &&\n \tgrep \"Important information!\" cron.txt\n '\n \n+test_expect_success MACOS_MAINTENANCE 'start and stop macOS maintenance' '\n+\twrite_script print-args \"#!/bin/sh\\necho \\$* >>args\" &&\n+\n+\trm -f args &&\n+\tGIT_TEST_CRONTAB=\"./print-args\" git maintenance start &&\n+\n+\t# start registers the repo\n+\tgit config --get --global maintenance.repo \"$(pwd)\" &&\n+\n+\t# ~/Library/LaunchAgents\n+\tls \"$HOME/Library/LaunchAgents\" >actual &&\n+\tcat >expect <<-\\EOF &&\n+\torg.git-scm.git.daily.plist\n+\torg.git-scm.git.hourly.plist\n+\torg.git-scm.git.weekly.plist\n+\tEOF\n+\ttest_cmp expect actual &&\n+\n+\trm expect &&\n+\tfor frequency in hourly daily weekly\n+\tdo\n+\t\tPLIST=\"$HOME/Library/LaunchAgents/org.git-scm.git.$frequency.plist\" &&\n+\t\txmllint --noout \"$PLIST\" &&\n+\t\tgrep schedule=$frequency \"$PLIST\" &&\n+\t\techo \"bootout gui/$UID $PLIST\" >>expect &&\n+\t\techo \"bootstrap gui/$UID $PLIST\" >>expect || return 1\n+\tdone &&\n+\ttest_cmp expect args &&\n+\n+\trm -f args &&\n+\tGIT_TEST_CRONTAB=\"./print-args\" git maintenance stop &&\n+\n+\t# stop does not unregister the repo\n+\tgit config --get --global maintenance.repo \"$(pwd)\" &&\n+\n+\tprintf \"bootout gui/$UID $HOME/Library/LaunchAgents/org.git-scm.git.%s.plist\\n\" \\\n+\t\thourly daily weekly >expect &&\n+\ttest_cmp expect args &&\n+\tls \"$HOME/Library/LaunchAgents\" >actual &&\n+\ttest_line_count = 0 actual\n+'\n+\n test_expect_success 'register preserves existing strategy' '\n \tgit config maintenance.strategy none &&\n \tgit maintenance register &&\ndiff --git a/t/test-lib.sh b/t/test-lib.sh\nindex 4a60d1ed76..620ffbf3af 100644\n--- a/t/test-lib.sh\n+++ b/t/test-lib.sh\n@@ -1703,6 +1703,10 @@ test_lazy_prereq REBASE_P '\n \ttest -z \"$GIT_TEST_SKIP_REBASE_P\"\n '\n \n+test_lazy_prereq MACOS_MAINTENANCE '\n+\tlaunchctl list\n+'\n+\n # Ensure that no test accidentally triggers a Git command\n # that runs 'crontab', affecting a user's cron schedule.\n # Tests that verify the cron integration must set this locally\n-- \ngitgitgadget\n\n"},{"id":"409897","messageId":"ed7a61978fe9dce26ca459b5a86490c15e470698.1605276024.git.gitgitgadget@gmail.com","threadId":"54562","inReplyTo":"pull.776.v3.git.1605276024.gitgitgadget@gmail.com","subject":"[PATCH v3 4/4] maintenance: use Windows scheduled tasks","fromName":"Derrick Stolee via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2020-11-13T14:00:24Z","receivedAt":"2020-11-13T14:00:39Z","isPatch":true,"sender":{"key":"stolee@gmail.com","avatar":"https://avatars.githubusercontent.com/u/570044?v=4"},"body":"From: Derrick Stolee \n\nGit's background maintenance uses cron by default, but this is not\navailable on Windows. Instead, integrate with Task Scheduler.\n\nTasks can be scheduled using the 'schtasks' command. There are several\ncommand-line options that can allow for some advanced scheduling, but\nunfortunately these seem to all require authenticating using a password.\n\nInstead, use the \"/xml\" option to pass an XML file that contains the\nconfiguration for the necessary schedule. These XML files are based on\nsome that I exported after constructing a schedule in the Task Scheduler\nGUI. These options only run background maintenance when the user is\nlogged in, and more fields are populated with the current username and\nSID at run-time by 'schtasks'.\n\nThere is a deficiency in the current design. Windows has two kinds of\napplications: GUI applications that start by \"winmain()\" and console\napplications that start by \"main()\". Console applications are attached\nto a new Console window if they are not already associated with a GUI\napplication. This means that every hour the scheudled task launches a\ncommand window for the scheduled tasks. Not only is this visually\nobtrusive, but it also takes focus from whatever else the user is\ndoing!\n\nA simple fix would be to insert a GUI application that acts as a shim\nbetween the scheduled task and Git. This is currently possible in Git\nfor Windows by setting the tag equal to\n\n C:\\Program Files\\Git\\git-bash.exe\n\nwith options \"--hide --no-needs-console --command=cmd\\git.exe\"\nfollowed by the arguments currently used. Since git-bash.exe is not\nincluded in Windows builds of core Git, I chose to leave out this\nfeature. My plan is to submit a small patch to Git for Windows that\nconverts the use of git.exe with this use of git-bash.exe in the\nshort term. In the long term, we can consider creating this GUI\nshim application within core Git, perhaps in contrib/.\n\nHelped-by: Eric Sunshine \nSigned-off-by: Derrick Stolee \n---\n Documentation/git-maintenance.txt | 22 ++++\n builtin/gc.c | 184 ++++++++++++++++++++++++++++++\n t/t7900-maintenance.sh | 33 +++++-\n 3 files changed, 236 insertions(+), 3 deletions(-)\n\ndiff --git a/Documentation/git-maintenance.txt b/Documentation/git-maintenance.txt\nindex 5f8f63f098..6970f2b898 100644\n--- a/Documentation/git-maintenance.txt\n+++ b/Documentation/git-maintenance.txt\n@@ -313,6 +313,28 @@ To create more advanced customizations to your background tasks, see\n launchctl.plist(5) for more information.\n \n \n+BACKGROUND MAINTENANCE ON WINDOWS SYSTEMS\n+-----------------------------------------\n+\n+Windows does not support `cron` and instead has its own system for\n+scheduling background tasks. The `git maintenance start` command uses\n+the `schtasks` command to submit tasks to this system. You can inspect\n+all background tasks using the Task Scheduler application. The tasks\n+added by Git have names of the form `Git Maintenance ()`.\n+The Task Scheduler GUI has ways to inspect these tasks, but you can also\n+export the tasks to XML files and view the details there.\n+\n+Note that since Git is a console application, these background tasks\n+create a console window visible to the current user. This can be changed\n+manually by selecting the \"Run whether user is logged in or not\" option\n+in Task Scheduler. This change requires a password input, which is why\n+`git maintenance start` does not select it by default.\n+\n+If you want to customize the background tasks, please rename the tasks\n+so future calls to `git maintenance (start|stop)` do not overwrite your\n+custom tasks.\n+\n+\n GIT\n ---\n Part of the linkgit:git[1] suite\ndiff --git a/builtin/gc.c b/builtin/gc.c\nindex da2c892f68..76a3afa20a 100644\n--- a/builtin/gc.c\n+++ b/builtin/gc.c\n@@ -1684,6 +1684,190 @@ static int platform_update_schedule(int run_maintenance, int fd)\n \telse\n \t\treturn remove_plists();\n }\n+\n+#elif defined(GIT_WINDOWS_NATIVE)\n+\n+static const char *get_frequency(enum schedule_priority schedule)\n+{\n+\tswitch (schedule) {\n+\tcase SCHEDULE_HOURLY:\n+\t\treturn \"hourly\";\n+\tcase SCHEDULE_DAILY:\n+\t\treturn \"daily\";\n+\tcase SCHEDULE_WEEKLY:\n+\t\treturn \"weekly\";\n+\tdefault:\n+\t\tBUG(\"invalid schedule %d\", schedule);\n+\t}\n+}\n+\n+static char *get_task_name(const char *frequency)\n+{\n+\tstruct strbuf label = STRBUF_INIT;\n+\tstrbuf_addf(&label, \"Git Maintenance (%s)\", frequency);\n+\treturn strbuf_detach(&label, NULL);\n+}\n+\n+static int remove_task(enum schedule_priority schedule)\n+{\n+\tint result;\n+\tstruct strvec args = STRVEC_INIT;\n+\tconst char *frequency = get_frequency(schedule);\n+\tchar *name = get_task_name(frequency);\n+\tconst char *schtasks = getenv(\"GIT_TEST_CRONTAB\");\n+\tif (!schtasks)\n+\t\tschtasks = \"schtasks\";\n+\n+\tstrvec_split(&args, schtasks);\n+\tstrvec_pushl(&args, \"/delete\", \"/tn\", name, \"/f\", NULL);\n+\n+\tresult = run_command_v_opt(args.v, 0);\n+\n+\tstrvec_clear(&args);\n+\tfree(name);\n+\treturn result;\n+}\n+\n+static int remove_scheduled_tasks(void)\n+{\n+\treturn remove_task(SCHEDULE_HOURLY) ||\n+\t\tremove_task(SCHEDULE_DAILY) ||\n+\t\tremove_task(SCHEDULE_WEEKLY);\n+}\n+\n+static int schedule_task(const char *exec_path, enum schedule_priority schedule)\n+{\n+\tint result;\n+\tstruct child_process child = CHILD_PROCESS_INIT;\n+\tconst char *xml, *schtasks;\n+\tchar *xmlpath, *tempDir;\n+\tFILE *xmlfp;\n+\tconst char *frequency = get_frequency(schedule);\n+\tchar *name = get_task_name(frequency);\n+\n+\ttempDir = xstrfmt(\"%s/temp\", the_repository->objects->odb->path);\n+\txmlpath = xstrfmt(\"%s/schedule-%s.xml\", tempDir, frequency);\n+\tsafe_create_leading_directories(xmlpath);\n+\txmlfp = xfopen(xmlpath, \"w\");\n+\n+\txml = \"\\n\"\n+\t \"\\n\"\n+\t \"\\n\"\n+\t \"\\n\";\n+\tfprintf(xmlfp, xml);\n+\n+\tswitch (schedule) {\n+\tcase SCHEDULE_HOURLY:\n+\t\tfprintf(xmlfp,\n+\t\t\t\"2020-01-01T01:00:00\\n\"\n+\t\t\t\"true\\n\"\n+\t\t\t\"\\n\"\n+\t\t\t\"1\\n\"\n+\t\t\t\"\\n\"\n+\t\t\t\"\\n\"\n+\t\t\t\"PT1H\\n\"\n+\t\t\t\"PT23H\\n\"\n+\t\t\t\"false\\n\"\n+\t\t\t\"\\n\");\n+\t\tbreak;\n+\n+\tcase SCHEDULE_DAILY:\n+\t\tfprintf(xmlfp,\n+\t\t\t\"2020-01-01T00:00:00\\n\"\n+\t\t\t\"true\\n\"\n+\t\t\t\"\\n\"\n+\t\t\t\"\\n\"\n+\t\t\t\"\\n\"\n+\t\t\t\"\\n\"\n+\t\t\t\"\\n\"\n+\t\t\t\"\\n\"\n+\t\t\t\"\\n\"\n+\t\t\t\"\\n\"\n+\t\t\t\"\\n\"\n+\t\t\t\"1\\n\"\n+\t\t\t\"\\n\");\n+\t\tbreak;\n+\n+\tcase SCHEDULE_WEEKLY:\n+\t\tfprintf(xmlfp,\n+\t\t\t\"2020-01-01T00:00:00\\n\"\n+\t\t\t\"true\\n\"\n+\t\t\t\"\\n\"\n+\t\t\t\"\\n\"\n+\t\t\t\"\\n\"\n+\t\t\t\"\\n\"\n+\t\t\t\"1\\n\"\n+\t\t\t\"\\n\");\n+\t\tbreak;\n+\n+\tdefault:\n+\t\tbreak;\n+\t}\n+\n+\txml= \"\\n\"\n+\t \"\\n\"\n+\t \"\\n\"\n+\t \"\\n\"\n+\t \"InteractiveToken\\n\"\n+\t \"LeastPrivilege\\n\"\n+\t \"\\n\"\n+\t \"\\n\"\n+\t \"\\n\"\n+\t \"IgnoreNew\\n\"\n+\t \"true\\n\"\n+\t \"true\\n\"\n+\t \"true\\n\"\n+\t \"false\\n\"\n+\t \"PT72H\\n\"\n+\t \"7\\n\"\n+\t \"\\n\"\n+\t \"\\n\"\n+\t \"\\n\"\n+\t \"\\\"%s\\\\git.exe\\\"\\n\"\n+\t \"--exec-path=\\\"%s\\\" for-each-repo --config=maintenance.repo maintenance run --schedule=%s\\n\"\n+\t \"\\n\"\n+\t \"\\n\"\n+\t \"\\n\";\n+\tfprintf(xmlfp, xml, exec_path, exec_path, frequency);\n+\tfclose(xmlfp);\n+\n+\tschtasks = getenv(\"GIT_TEST_CRONTAB\");\n+\tif (!schtasks)\n+\t\tschtasks = \"schtasks\";\n+\tstrvec_split(&child.args, schtasks);\n+\tstrvec_pushl(&child.args, \"/create\", \"/tn\", name, \"/f\", \"/xml\", xmlpath, NULL);\n+\n+\tchild.no_stdout = 1;\n+\tchild.no_stderr = 1;\n+\n+\tif (start_command(&child))\n+\t\tdie(_(\"failed to start schtasks\"));\n+\tresult = finish_command(&child);\n+\n+\tunlink(xmlpath);\n+\trmdir(tempDir);\n+\tfree(xmlpath);\n+\tfree(name);\n+\treturn result;\n+}\n+\n+static int add_scheduled_tasks(void)\n+{\n+\tconst char *exec_path = git_exec_path();\n+\n+\treturn schedule_task(exec_path, SCHEDULE_HOURLY) ||\n+\t\tschedule_task(exec_path, SCHEDULE_DAILY) ||\n+\t\tschedule_task(exec_path, SCHEDULE_WEEKLY);\n+}\n+\n+static int platform_update_schedule(int run_maintenance, int fd)\n+{\n+\tif (run_maintenance)\n+\t\treturn add_scheduled_tasks();\n+\telse\n+\t\treturn remove_scheduled_tasks();\n+}\n+\n #else\n #define BEGIN_LINE \"# BEGIN GIT MAINTENANCE SCHEDULE\"\n #define END_LINE \"# END GIT MAINTENANCE SCHEDULE\"\ndiff --git a/t/t7900-maintenance.sh b/t/t7900-maintenance.sh\nindex 29d340a828..0dc2479117 100755\n--- a/t/t7900-maintenance.sh\n+++ b/t/t7900-maintenance.sh\n@@ -367,7 +367,7 @@ test_expect_success 'register and unregister' '\n \ttest_cmp before actual\n '\n \n-test_expect_success !MACOS_MAINTENANCE 'start from empty cron table' '\n+test_expect_success !MACOS_MAINTENANCE,!MINGW 'start from empty cron table' '\n \tGIT_TEST_CRONTAB=\"test-tool crontab cron.txt\" git maintenance start &&\n \n \t# start registers the repo\n@@ -378,7 +378,7 @@ test_expect_success !MACOS_MAINTENANCE 'start from empty cron table' '\n \tgrep \"for-each-repo --config=maintenance.repo maintenance run --schedule=weekly\" cron.txt\n '\n \n-test_expect_success !MACOS_MAINTENANCE 'stop from existing schedule' '\n+test_expect_success !MACOS_MAINTENANCE,!MINGW 'stop from existing schedule' '\n \tGIT_TEST_CRONTAB=\"test-tool crontab cron.txt\" git maintenance stop &&\n \n \t# stop does not unregister the repo\n@@ -389,7 +389,7 @@ test_expect_success !MACOS_MAINTENANCE 'stop from existing schedule' '\n \ttest_must_be_empty cron.txt\n '\n \n-test_expect_success !MACOS_MAINTENANCE 'start preserves existing schedule' '\n+test_expect_success !MACOS_MAINTENANCE,!MINGW 'start preserves existing schedule' '\n \techo \"Important information!\" >cron.txt &&\n \tGIT_TEST_CRONTAB=\"test-tool crontab cron.txt\" git maintenance start &&\n \tgrep \"Important information!\" cron.txt\n@@ -437,6 +437,33 @@ test_expect_success MACOS_MAINTENANCE 'start and stop macOS maintenance' '\n \ttest_line_count = 0 actual\n '\n \n+test_expect_success MINGW 'start and stop Windows maintenance' '\n+\twrite_script print-args <<-\\EOF &&\n+\techo $* >>args\n+\tEOF\n+\n+\trm -f args &&\n+\tGIT_TEST_CRONTAB=\"/bin/sh print-args\" git maintenance start &&\n+\n+\t# start registers the repo\n+\tgit config --get --global maintenance.repo \"$(pwd)\" &&\n+\n+\tprintf \"/create /tn Git Maintenance (%s) /f /xml .git/objects/temp/schedule-%s.xml\\n\" \\\n+\t\thourly hourly daily daily weekly weekly >expect &&\n+\ttest_cmp expect args &&\n+\n+\trm -f args &&\n+\tGIT_TEST_CRONTAB=\"/bin/sh print-args\" git maintenance stop &&\n+\n+\t# stop does not unregister the repo\n+\tgit config --get --global maintenance.repo \"$(pwd)\" &&\n+\n+\trm expect &&\n+\tprintf \"/delete /tn Git Maintenance (%s) /f\\n\" \\\n+\t\thourly daily weekly >expect &&\n+\ttest_cmp expect args\n+'\n+\n test_expect_success 'register preserves existing strategy' '\n \tgit config maintenance.strategy none &&\n \tgit maintenance register &&\n-- \ngitgitgadget\n"},{"id":"409898","messageId":"pull.776.v3.git.1605276024.gitgitgadget@gmail.com","threadId":"54562","inReplyTo":"pull.776.v2.git.1604520368.gitgitgadget@gmail.com","subject":"[PATCH v3 0/4] Maintenance IV: Platform-specific background maintenance","fromName":"Derrick Stolee via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2020-11-13T14:00:20Z","receivedAt":"2020-11-13T14:00:49Z","isPatch":true,"sender":{"key":"stolee@gmail.com","avatar":"https://avatars.githubusercontent.com/u/570044?v=4"},"body":"This is based on ds/maintenance-part-3.\n\nAfter sitting with the background maintenance as it has been cooking, I\nwanted to come back around and implement the background maintenance for\nWindows. However, I noticed that there were some things bothering me with\nbackground maintenance on my macOS machine. These are detailed in PATCH 3,\nbut the tl;dr is that 'cron' is not recommended by Apple and instead\n'launchd' satisfies our needs.\n\nThis series implements the background scheduling so git maintenance\n(start|stop) works on those platforms. I've been operating with these\nschedules for a while now without the problems described in the patches.\n\nThere is a particularly annoying case about console windows popping up on\nWindows, but PATCH 4 describes a plan to get around that.\n\nUpdates in V3\n=============\n\n * This actually includes the feedback responses I had intended for v2.\n Sorry about that!\n \n \n * One major change is the use of a 'struct child_process' instead of just\n run_command_v_opt() so we can suppress error messages from the schedule\n helpers. We will rely on exit code and present our own error messages, as\n necessary.\n \n \n * Some doc and test fixes.\n \n \n\nUpdates in V2\n=============\n\n * This is a faster turnaround for a v2 than I would normally like, but Eric\n inspired extra documentation about how to customize background schedules.\n \n \n * New extensions to git-maintenance.txt include guidelines for inspecting\n what git maintenance start does and how to customize beyond that. This\n includes a new PATCH 2 that includes documentation for 'cron' on\n non-macOS non-Windows systems.\n \n \n * Several improvements, especially in the tests, are included.\n \n \n * While testing manually, I noticed that somehow I had incorrectly had an\n opening tag instead of a closing tag in the hourly format\n on macOS. I found that the xmllint tool can verify the XML format of a\n file, which catches the bug. This seems like a good approach since the\n test is macOS-only. Does anyone have concerns about adding this\n dependency?\n \n \n\nThanks, -Stolee\n\ncc: jrnieder@gmail.com [jrnieder@gmail.com], jonathantanmy@google.com\n[jonathantanmy@google.com], sluongng@gmail.com [sluongng@gmail.com]cc:\nDerrick Stolee stolee@gmail.com [stolee@gmail.com]cc: Đoàn Trần Công Danh \ncongdanhqx@gmail.com [congdanhqx@gmail.com]cc: Martin Ågren \nmartin.agren@gmail.com [martin.agren@gmail.com]cc: Eric Sunshine \nsunshine@sunshineco.com [sunshine@sunshineco.com]cc: Derrick Stolee \nstolee@gmail.com [stolee@gmail.com]\n\nDerrick Stolee (4):\n maintenance: extract platform-specific scheduling\n maintenance: include 'cron' details in docs\n maintenance: use launchctl on macOS\n maintenance: use Windows scheduled tasks\n\n Documentation/git-maintenance.txt | 116 +++++++++\n builtin/gc.c | 417 ++++++++++++++++++++++++++++--\n t/t7900-maintenance.sh | 75 +++++-\n t/test-lib.sh | 4 +\n 4 files changed, 592 insertions(+), 20 deletions(-)\n\n\nbase-commit: 0016b618182f642771dc589cf0090289f9fe1b4f\nPublished-As: https://github.com/gitgitgadget/git/releases/tag/pr-776%2Fderrickstolee%2Fmaintenance%2FmacOS-v3\nFetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-776/derrickstolee/maintenance/macOS-v3\nPull-Request: https://github.com/gitgitgadget/git/pull/776\n\nRange-diff vs v2:\n\n 1: d35f1aa162 = 1: d35f1aa162 maintenance: extract platform-specific scheduling\n 2: 709a173720 ! 2: 0dfe53092e maintenance: include 'cron' details in docs\n @@ Commit message\n baseline can provide a way forward for users who have never worked with\n cron schedules.\n \n + Helped-by: Eric Sunshine \n Signed-off-by: Derrick Stolee \n \n ## Documentation/git-maintenance.txt ##\n @@ Documentation/git-maintenance.txt: Further, the `git gc` command should not be c\n +---------------------------------------\n +\n +The standard mechanism for scheduling background tasks on POSIX systems\n -+is `cron`. This tool executes commands based on a given schedule. The\n ++is cron(8). This tool executes commands based on a given schedule. The\n +current list of user-scheduled tasks can be found by running `crontab -l`.\n +The schedule written by `git maintenance start` is similar to this:\n +\n @@ Documentation/git-maintenance.txt: Further, the `git gc` command should not be c\n +Any modifications within this region will be completely deleted by\n +`git maintenance stop` or overwritten by `git maintenance start`.\n +\n -+The `` string is loaded to specifically use the location for the\n -+`git` executable used in the `git maintenance start` command. This allows\n -+for multiple versions to be compatible. However, if the same user runs\n -+`git maintenance start` with multiple Git executables, then only the\n -+latest executable will be used.\n ++The `crontab` entry specifies the full path of the `git` executable to\n ++ensure that the executed `git` command is the same one with which\n ++`git maintenance start` was issued independent of `PATH`. If the same user\n ++runs `git maintenance start` with multiple Git executables, then only the\n ++latest executable is used.\n +\n +These commands use `git for-each-repo --config=maintenance.repo` to run\n +`git maintenance run --schedule=` on each repository listed in\n +the multi-valued `maintenance.repo` config option. These are typically\n -+loaded from the user-specific global config located at `~/.gitconfig`.\n -+The `git maintenance` process then determines which maintenance tasks\n -+are configured to run on each repository with each `` using\n -+the `maintenance..schedule` config options. These values are loaded\n -+from the global or repository config values.\n ++loaded from the user-specific global config. The `git maintenance` process\n ++then determines which maintenance tasks are configured to run on each\n ++repository with each `` using the `maintenance..schedule`\n ++config options. These values are loaded from the global or repository\n ++config values.\n +\n +If the config values are insufficient to achieve your desired background\n +maintenance schedule, then you can create your own schedule. If you run\n +`crontab -e`, then an editor will load with your user-specific `cron`\n +schedule. In that editor, you can add your own schedule lines. You could\n +start by adapting the default schedule listed earlier, or you could read\n -+https://man7.org/linux/man-pages/man5/crontab.5.html[the `crontab` documentation]\n -+for advanced scheduling techniques. Please do use the full path and\n -+`--exec-path` techniques from the default schedule to ensure you are\n -+executing the correct binaries in your schedule.\n ++the crontab(5) documentation for advanced scheduling techniques. Please\n ++do use the full path and `--exec-path` techniques from the default\n ++schedule to ensure you are executing the correct binaries in your\n ++schedule.\n +\n \n GIT\n 3: 0fafd75d10 ! 3: 1629bcfcf8 maintenance: use launchctl on macOS\n @@ Commit message\n of macOS 10.11, which was released in September 2015. Before that\n release the 'launchctl load' subcommand was recommended. The best\n source of information on this transition I have seen is available\n - at [2].\n + at [2]. The current design does not preclude a future version that\n + detects the available fatures of 'launchctl' to use the older\n + commands. However, it is best to rely on the newest version since\n + Apple might completely remove the deprecated version on short\n + notice.\n \n [2] https://babodee.wordpress.com/2016/04/09/launchctl-2-0-syntax/\n \n @@ Commit message\n Signed-off-by: Derrick Stolee \n \n ## Documentation/git-maintenance.txt ##\n -@@ Documentation/git-maintenance.txt: for advanced scheduling techniques. Please do use the full path and\n - executing the correct binaries in your schedule.\n +@@ Documentation/git-maintenance.txt: schedule to ensure you are executing the correct binaries in your\n + schedule.\n \n \n +BACKGROUND MAINTENANCE ON MACOS SYSTEMS\n +---------------------------------------\n +\n +While macOS technically supports `cron`, using `crontab -e` requires\n -+elevated privileges and the executed process do not have a full user\n ++elevated privileges and the executed process does not have a full user\n +context. Without a full user context, Git and its credential helpers\n +cannot access stored credentials, so some maintenance tasks are not\n +functional.\n +\n +Instead, `git maintenance start` interacts with the `launchctl` tool,\n -+which is the recommended way to\n -+https://developer.apple.com/library/archive/documentation/MacOSX/Conceptual/BPSystemStartup/Chapters/ScheduledJobs.html[schedule timed jobs in macOS].\n -+\n -+Scheduling maintenance through `git maintenance (start|stop)` requires\n -+some `launchctl` features available only in macOS 10.11 or later.\n ++which is the recommended way to schedule timed jobs in macOS. Scheduling\n ++maintenance through `git maintenance (start|stop)` requires some\n ++`launchctl` features available only in macOS 10.11 or later.\n +\n +Your user-specific scheduled tasks are stored as XML-formatted `.plist`\n +files in `~/Library/LaunchAgents/`. You can see the currently-registered\n +tasks using the following command:\n +\n +-----------------------------------------------------------------------\n -+$ ls ~/Library/LaunchAgents/ | grep org.git-scm.git\n ++$ ls ~/Library/LaunchAgents/org.git-scm.git*\n +org.git-scm.git.daily.plist\n +org.git-scm.git.hourly.plist\n +org.git-scm.git.weekly.plist\n @@ Documentation/git-maintenance.txt: for advanced scheduling techniques. Please do\n +and delete the `.plist` files.\n +\n +To create more advanced customizations to your background tasks, see\n -+https://developer.apple.com/library/archive/documentation/MacOSX/Conceptual/BPSystemStartup/Chapters/CreatingLaunchdJobs.html#//apple_ref/doc/uid/TP40001762-104142[the `launchctl` documentation]\n -+for more information.\n ++launchctl.plist(5) for more information.\n +\n +\n GIT\n @@ builtin/gc.c: static int maintenance_unregister(void)\n +\treturn strbuf_detach(&output, NULL);\n +}\n +\n -+static int bootout(const char *filename)\n ++static int boot_plist(int enable, const char *filename)\n +{\n +\tint result;\n -+\tstruct strvec args = STRVEC_INIT;\n ++\tstruct child_process child = CHILD_PROCESS_INIT;\n +\tchar *uid = get_uid();\n +\tconst char *launchctl = getenv(\"GIT_TEST_CRONTAB\");\n +\tif (!launchctl)\n +\t\tlaunchctl = \"/bin/launchctl\";\n +\n -+\tstrvec_split(&args, launchctl);\n -+\tstrvec_push(&args, \"bootout\");\n -+\tstrvec_pushf(&args, \"gui/%s\", uid);\n -+\tstrvec_push(&args, filename);\n ++\tstrvec_split(&child.args, launchctl);\n +\n -+\tresult = run_command_v_opt(args.v, 0);\n ++\tif (enable)\n ++\t\tstrvec_push(&child.args, \"bootstrap\");\n ++\telse\n ++\t\tstrvec_push(&child.args, \"bootout\");\n ++\tstrvec_pushf(&child.args, \"gui/%s\", uid);\n ++\tstrvec_push(&child.args, filename);\n +\n -+\tstrvec_clear(&args);\n -+\tfree(uid);\n -+\treturn result;\n -+}\n ++\tchild.no_stderr = 1;\n ++\tchild.no_stdout = 1;\n +\n -+static int bootstrap(const char *filename)\n -+{\n -+\tint result;\n -+\tstruct strvec args = STRVEC_INIT;\n -+\tchar *uid = get_uid();\n -+\tconst char *launchctl = getenv(\"GIT_TEST_CRONTAB\");\n -+\tif (!launchctl)\n -+\t\tlaunchctl = \"/bin/launchctl\";\n ++\tif (start_command(&child))\n ++\t\tdie(_(\"failed to start launchctl\"));\n +\n -+\tstrvec_split(&args, launchctl);\n -+\tstrvec_push(&args, \"bootstrap\");\n -+\tstrvec_pushf(&args, \"gui/%s\", uid);\n -+\tstrvec_push(&args, filename);\n ++\tresult = finish_command(&child);\n +\n -+\tresult = run_command_v_opt(args.v, 0);\n -+\n -+\tstrvec_clear(&args);\n +\tfree(uid);\n +\treturn result;\n +}\n @@ builtin/gc.c: static int maintenance_unregister(void)\n +\tconst char *frequency = get_frequency(schedule);\n +\tchar *name = get_service_name(frequency);\n +\tchar *filename = get_service_filename(name);\n -+\tint result = bootout(filename);\n ++\tint result = boot_plist(0, filename);\n ++\tunlink(filename);\n +\tfree(filename);\n +\tfree(name);\n +\treturn result;\n @@ builtin/gc.c: static int maintenance_unregister(void)\n +\n +\tif (safe_create_leading_directories(filename))\n +\t\tdie(_(\"failed to create directories for '%s'\"), filename);\n -+\tplist = fopen(filename, \"w\");\n -+\n -+\tif (!plist)\n -+\t\tdie(_(\"failed to open '%s'\"), filename);\n ++\tplist = xfopen(filename, \"w\");\n +\n +\tpreamble = \"\\n\"\n +\t\t \"\\n\"\n @@ builtin/gc.c: static int maintenance_unregister(void)\n +\tfprintf(plist, \"\\n\\n\\n\");\n +\n +\t/* bootout might fail if not already running, so ignore */\n -+\tbootout(filename);\n -+\tif (bootstrap(filename))\n ++\tboot_plist(0, filename);\n ++\tif (boot_plist(1, filename))\n +\t\tdie(_(\"failed to bootstrap service %s\"), filename);\n +\n +\tfclose(plist);\n @@ t/t7900-maintenance.sh: test_expect_success 'stop from existing schedule' '\n '\n \n +test_expect_success MACOS_MAINTENANCE 'start and stop macOS maintenance' '\n -+\techo \"#!/bin/sh\\necho \\$@ >>args\" >print-args &&\n -+\tchmod a+x print-args &&\n ++\twrite_script print-args \"#!/bin/sh\\necho \\$* >>args\" &&\n +\n +\trm -f args &&\n +\tGIT_TEST_CRONTAB=\"./print-args\" git maintenance start &&\n @@ t/t7900-maintenance.sh: test_expect_success 'stop from existing schedule' '\n +\tfor frequency in hourly daily weekly\n +\tdo\n +\t\tPLIST=\"$HOME/Library/LaunchAgents/org.git-scm.git.$frequency.plist\" &&\n -+\t\txmllint \"$PLIST\" >/dev/null &&\n ++\t\txmllint --noout \"$PLIST\" &&\n +\t\tgrep schedule=$frequency \"$PLIST\" &&\n +\t\techo \"bootout gui/$UID $PLIST\" >>expect &&\n +\t\techo \"bootstrap gui/$UID $PLIST\" >>expect || return 1\n @@ t/t7900-maintenance.sh: test_expect_success 'stop from existing schedule' '\n +\ttest_cmp expect args &&\n +\n +\trm -f args &&\n -+\tGIT_TEST_CRONTAB=\"./print-args\" git maintenance stop &&\n ++\tGIT_TEST_CRONTAB=\"./print-args\" git maintenance stop &&\n +\n +\t# stop does not unregister the repo\n +\tgit config --get --global maintenance.repo \"$(pwd)\" &&\n +\n -+\t# stop does not remove plist files, but boots them out\n -+\trm expect &&\n -+\tfor frequency in hourly daily weekly\n -+\tdo\n -+\t\tPLIST=\"$HOME/Library/LaunchAgents/org.git-scm.git.$frequency.plist\" &&\n -+\t\tgrep schedule=$frequency \"$PLIST\" &&\n -+\t\techo \"bootout gui/$UID $PLIST\" >>expect || return 1\n -+\tdone &&\n -+\ttest_cmp expect args\n ++\tprintf \"bootout gui/$UID $HOME/Library/LaunchAgents/org.git-scm.git.%s.plist\\n\" \\\n ++\t\thourly daily weekly >expect &&\n ++\ttest_cmp expect args &&\n ++\tls \"$HOME/Library/LaunchAgents\" >actual &&\n ++\ttest_line_count = 0 actual\n +'\n +\n test_expect_success 'register preserves existing strategy' '\n 4: 84eb44de31 ! 4: ed7a61978f maintenance: use Windows scheduled tasks\n @@ Commit message\n Signed-off-by: Derrick Stolee \n \n ## Documentation/git-maintenance.txt ##\n -@@ Documentation/git-maintenance.txt: https://developer.apple.com/library/archive/documentation/MacOSX/Conceptual/BPSy\n - for more information.\n +@@ Documentation/git-maintenance.txt: To create more advanced customizations to your background tasks, see\n + launchctl.plist(5) for more information.\n \n \n +BACKGROUND MAINTENANCE ON WINDOWS SYSTEMS\n @@ builtin/gc.c: static int platform_update_schedule(int run_maintenance, int fd)\n +static int schedule_task(const char *exec_path, enum schedule_priority schedule)\n +{\n +\tint result;\n -+\tstruct strvec args = STRVEC_INIT;\n ++\tstruct child_process child = CHILD_PROCESS_INIT;\n +\tconst char *xml, *schtasks;\n -+\tchar *xmlpath;\n ++\tchar *xmlpath, *tempDir;\n +\tFILE *xmlfp;\n +\tconst char *frequency = get_frequency(schedule);\n +\tchar *name = get_task_name(frequency);\n +\n -+\txmlpath = xstrfmt(\"%s/schedule-%s.xml\",\n -+\t\t\t the_repository->objects->odb->path,\n -+\t\t\t frequency);\n -+\txmlfp = fopen(xmlpath, \"w\");\n -+\tif (!xmlfp)\n -+\t\tdie(_(\"failed to open '%s'\"), xmlpath);\n ++\ttempDir = xstrfmt(\"%s/temp\", the_repository->objects->odb->path);\n ++\txmlpath = xstrfmt(\"%s/schedule-%s.xml\", tempDir, frequency);\n ++\tsafe_create_leading_directories(xmlpath);\n ++\txmlfp = xfopen(xmlpath, \"w\");\n +\n +\txml = \"\\n\"\n +\t \"\\n\"\n @@ builtin/gc.c: static int platform_update_schedule(int run_maintenance, int fd)\n +\tschtasks = getenv(\"GIT_TEST_CRONTAB\");\n +\tif (!schtasks)\n +\t\tschtasks = \"schtasks\";\n -+\tstrvec_split(&args, schtasks);\n -+\tstrvec_pushl(&args, \"/create\", \"/tn\", name, \"/f\", \"/xml\", xmlpath, NULL);\n ++\tstrvec_split(&child.args, schtasks);\n ++\tstrvec_pushl(&child.args, \"/create\", \"/tn\", name, \"/f\", \"/xml\", xmlpath, NULL);\n +\n -+\tresult = run_command_v_opt(args.v, 0);\n ++\tchild.no_stdout = 1;\n ++\tchild.no_stderr = 1;\n ++\n ++\tif (start_command(&child))\n ++\t\tdie(_(\"failed to start schtasks\"));\n ++\tresult = finish_command(&child);\n +\n -+\tstrvec_clear(&args);\n +\tunlink(xmlpath);\n ++\trmdir(tempDir);\n +\tfree(xmlpath);\n +\tfree(name);\n +\treturn result;\n @@ t/t7900-maintenance.sh: test_expect_success !MACOS_MAINTENANCE 'stop from existi\n \tGIT_TEST_CRONTAB=\"test-tool crontab cron.txt\" git maintenance start &&\n \tgrep \"Important information!\" cron.txt\n @@ t/t7900-maintenance.sh: test_expect_success MACOS_MAINTENANCE 'start and stop macOS maintenance' '\n - \ttest_cmp expect args\n + \ttest_line_count = 0 actual\n '\n \n +test_expect_success MINGW 'start and stop Windows maintenance' '\n @@ t/t7900-maintenance.sh: test_expect_success MACOS_MAINTENANCE 'start and stop ma\n +\t# start registers the repo\n +\tgit config --get --global maintenance.repo \"$(pwd)\" &&\n +\n -+\tfor frequency in hourly daily weekly\n -+\tdo\n -+\t\tprintf \"/create /tn Git Maintenance (%s) /f /xml .git/objects/schedule-%s.xml\\n\" \\\n -+\t\t\t$frequency $frequency\n -+\tdone >expect &&\n ++\tprintf \"/create /tn Git Maintenance (%s) /f /xml .git/objects/temp/schedule-%s.xml\\n\" \\\n ++\t\thourly hourly daily daily weekly weekly >expect &&\n +\ttest_cmp expect args &&\n +\n +\trm -f args &&\n\n-- \ngitgitgadget\n"},{"id":"409915","messageId":"CAPig+cQQrucwE-KSndQ5kCjG9Q1MtjxraAmA95+OQe=6bdks-Q@mail.gmail.com","threadId":"54562","inReplyTo":"1629bcfcf82dbc2ed9889a0e9ea2d08427901c4e.1605276024.git.gitgitgadget@gmail.com","subject":"Re: [PATCH v3 3/4] maintenance: use launchctl on macOS","fromName":"Eric Sunshine","fromEmail":"sunshine@sunshineco.com","sentAt":"2020-11-13T20:19:25Z","receivedAt":"2020-11-13T20:19:43Z","isPatch":true,"sender":{"key":"sunshine@sunshineco.com","avatar":"https://avatars.githubusercontent.com/u/163641?v=4"},"body":"On Fri, Nov 13, 2020 at 9:00 AM Derrick Stolee via GitGitGadget\n wrote:\n> [...]\n> The solution is to switch from cron to the Apple-recommended [1]\n> 'launchd' tool.\n> [...]\n> Signed-off-by: Derrick Stolee \n> ---\n> diff --git a/builtin/gc.c b/builtin/gc.c\n> @@ -1491,6 +1491,200 @@ static int maintenance_unregister(void)\n> +static int boot_plist(int enable, const char *filename)\n> +{\n> + struct child_process child = CHILD_PROCESS_INIT;\n> + child.no_stderr = 1;\n> + child.no_stdout = 1;\n> + if (start_command(&child))\n> + die(_(\"failed to start launchctl\"));\n\nNot necessarily worth a re-roll -- in fact, it could be done atop this\nseries to avoid holding this series up -- but this too-succinct error\nreporting won't help users diagnose the failure. An alternative would\nbe to capture stdout and stderr and only print them if the command\nfails. Perhaps something like this:\n\n struct strbuf out = STRBUF_INIT;\n struct strbuf err = STRBUF_INIT;\n ...\n if (pipe_command(child, NULL, 0, &out, 0, &err, 0) {\n if (out.len && err.len)\n strbuf_addstr(&out, \"; \");\n strbuf_addbuf(&out, &err);\n die(_(\"launchctl failed: %s\"), out.buf);\n }\n\nBy the way, won't this die() be a problem when schedule_plist() calls\nboot_plist() to remove the old scheduled tasks before calling it again\nto register the new ones? If the old ones don't exist, then it will\ndie() unnecessarily and never register the new ones. Or am I\nmisunderstanding? (I'm guessing that I must be misunderstanding since\nthe test script presumably passes.)\n\n> diff --git a/t/t7900-maintenance.sh b/t/t7900-maintenance.sh\n> @@ -389,12 +389,54 @@ test_expect_success 'stop from existing schedule' '\n> +test_expect_success MACOS_MAINTENANCE 'start and stop macOS maintenance' '\n> + write_script print-args \"#!/bin/sh\\necho \\$* >>args\" &&\n\nwrite_script() takes the script body as stdin, not as an argument, and\nyou don't need to specify /bin/sh. What you have here works by\naccident only because write_script() takes an optional second argument\nspecifying the shell to use in place of the default /bin/sh.\nNevertheless, it should really be written:\n\n write_script print-args <<-\\EOF\n echo $*\n EOF\n\nPatch [4/4] uses write_script() correctly.\n\n> + rm -f args &&\n> + GIT_TEST_CRONTAB=\"./print-args\" git maintenance start &&\n> +\n> + # start registers the repo\n> + git config --get --global maintenance.repo \"$(pwd)\" &&\n> +\n> + # ~/Library/LaunchAgents\n> + ls \"$HOME/Library/LaunchAgents\" >actual &&\n\nNot sure what the comment above the `ls` is meant to be conveying.\nCould be dropped but not itself worth a re-roll.\n\n> + cat >expect <<-\\EOF &&\n> + org.git-scm.git.daily.plist\n> + org.git-scm.git.hourly.plist\n> + org.git-scm.git.weekly.plist\n> + EOF\n> + test_cmp expect actual &&\n> +\n> + rm expect &&\n> + for frequency in hourly daily weekly\n> + do\n> + PLIST=\"$HOME/Library/LaunchAgents/org.git-scm.git.$frequency.plist\" &&\n> + xmllint --noout \"$PLIST\" &&\n> + grep schedule=$frequency \"$PLIST\" &&\n> + echo \"bootout gui/$UID $PLIST\" >>expect &&\n> + echo \"bootstrap gui/$UID $PLIST\" >>expect || return 1\n> + done &&\n\nThis is still relying upon $UID picked up from the users environment\n(as far as I can tell), which seems fragile. As mentioned in my first\nreview, it probably would be more robust to compute UID manually the\nsame way git-maintenance itself does.\n\n> + test_cmp expect args &&\n> +\n> + rm -f args &&\n> + GIT_TEST_CRONTAB=\"./print-args\" git maintenance stop &&\n\nMinor: No need for the quotes around ./print-args (though they don't\nhurt either, and certainly not worth re-rolling just to drop them, and\nit's subjective so don't drop them just for my sake).\n\n> + # stop does not unregister the repo\n> + git config --get --global maintenance.repo \"$(pwd)\" &&\n> +\n> + printf \"bootout gui/$UID $HOME/Library/LaunchAgents/org.git-scm.git.%s.plist\\n\" \\\n> + hourly daily weekly >expect &&\n> + test_cmp expect args &&\n> + ls \"$HOME/Library/LaunchAgents\" >actual &&\n> + test_line_count = 0 actual\n> +'\n"},{"id":"409917","messageId":"d9c51446-c223-dce9-692a-959f841d506f@gmail.com","threadId":"54562","inReplyTo":"CAPig+cQQrucwE-KSndQ5kCjG9Q1MtjxraAmA95+OQe=6bdks-Q@mail.gmail.com","subject":"Re: [PATCH v3 3/4] maintenance: use launchctl on macOS","fromName":"Derrick Stolee","fromEmail":"stolee@gmail.com","sentAt":"2020-11-13T20:42:03Z","receivedAt":"2020-11-13T20:42:10Z","isPatch":true,"sender":{"key":"stolee@gmail.com","avatar":"https://avatars.githubusercontent.com/u/570044?v=4"},"body":"On 11/13/2020 3:19 PM, Eric Sunshine wrote:\n> On Fri, Nov 13, 2020 at 9:00 AM Derrick Stolee via GitGitGadget\n> wrote:\n>> [...]\n>> The solution is to switch from cron to the Apple-recommended [1]\n>> 'launchd' tool.\n>> [...]\n>> Signed-off-by: Derrick Stolee \n>> ---\n>> diff --git a/builtin/gc.c b/builtin/gc.c\n>> @@ -1491,6 +1491,200 @@ static int maintenance_unregister(void)\n>> +static int boot_plist(int enable, const char *filename)\n>> +{\n>> + struct child_process child = CHILD_PROCESS_INIT;\n>> + child.no_stderr = 1;\n>> + child.no_stdout = 1;\n>> + if (start_command(&child))\n>> + die(_(\"failed to start launchctl\"));\n> \n> Not necessarily worth a re-roll -- in fact, it could be done atop this\n> series to avoid holding this series up -- but this too-succinct error\n> reporting won't help users diagnose the failure. An alternative would\n> be to capture stdout and stderr and only print them if the command\n> fails. Perhaps something like this:\n> \n> struct strbuf out = STRBUF_INIT;\n> struct strbuf err = STRBUF_INIT;\n> ...\n> if (pipe_command(child, NULL, 0, &out, 0, &err, 0) {\n> if (out.len && err.len)\n> strbuf_addstr(&out, \"; \");\n> strbuf_addbuf(&out, &err);\n> die(_(\"launchctl failed: %s\"), out.buf);\n> }\n\nWe would also want to pass a \"die_on_failure\" into the method, since\nin the 'git maintenance start' case we don't want to report a failure\nwhen 'launchctl bootout' fails before we call 'launchctl bootstrap'.\n\n> By the way, won't this die() be a problem when schedule_plist() calls\n> boot_plist() to remove the old scheduled tasks before calling it again\n> to register the new ones? If the old ones don't exist, then it will\n> die() unnecessarily and never register the new ones. Or am I\n> misunderstanding? (I'm guessing that I must be misunderstanding since\n> the test script presumably passes.)\n\nThis die() is only if the process cannot _start_, for example due to\nlaunchctl not existing on $PATH. The result from finish_command()\nwould be non-zero when we bootout a plist that doesn't exist.\n\n>> diff --git a/t/t7900-maintenance.sh b/t/t7900-maintenance.sh\n>> @@ -389,12 +389,54 @@ test_expect_success 'stop from existing schedule' '\n>> +test_expect_success MACOS_MAINTENANCE 'start and stop macOS maintenance' '\n>> + write_script print-args \"#!/bin/sh\\necho \\$* >>args\" &&\n> \n> write_script() takes the script body as stdin, not as an argument, and\n> you don't need to specify /bin/sh. What you have here works by\n> accident only because write_script() takes an optional second argument\n> specifying the shell to use in place of the default /bin/sh.\n> Nevertheless, it should really be written:\n> \n> write_script print-args <<-\\EOF\n> echo $*\n> EOF\n> \n> Patch [4/4] uses write_script() correctly.\n\nAh. Sorry for misunderstanding. That explains why it works this way\non macOS but it did _not_ work that way on Windows.\n\n>> + rm -f args &&\n>> + GIT_TEST_CRONTAB=\"./print-args\" git maintenance start &&\n>> +\n>> + # start registers the repo\n>> + git config --get --global maintenance.repo \"$(pwd)\" &&\n>> +\n>> + # ~/Library/LaunchAgents\n>> + ls \"$HOME/Library/LaunchAgents\" >actual &&\n> \n> Not sure what the comment above the `ls` is meant to be conveying.\n> Could be dropped but not itself worth a re-roll.\n> \n>> + cat >expect <<-\\EOF &&\n>> + org.git-scm.git.daily.plist\n>> + org.git-scm.git.hourly.plist\n>> + org.git-scm.git.weekly.plist\n>> + EOF\n>> + test_cmp expect actual &&\n>> +\n>> + rm expect &&\n>> + for frequency in hourly daily weekly\n>> + do\n>> + PLIST=\"$HOME/Library/LaunchAgents/org.git-scm.git.$frequency.plist\" &&\n>> + xmllint --noout \"$PLIST\" &&\n>> + grep schedule=$frequency \"$PLIST\" &&\n>> + echo \"bootout gui/$UID $PLIST\" >>expect &&\n>> + echo \"bootstrap gui/$UID $PLIST\" >>expect || return 1\n>> + done &&\n> \n> This is still relying upon $UID picked up from the users environment\n> (as far as I can tell), which seems fragile. As mentioned in my first\n> review, it probably would be more robust to compute UID manually the\n> same way git-maintenance itself does.\n\nSorry, I missed this comment from v1 when reapplying the changes for v3.\n\n>> + test_cmp expect args &&\n>> +\n>> + rm -f args &&\n>> + GIT_TEST_CRONTAB=\"./print-args\" git maintenance stop &&\n> \n> Minor: No need for the quotes around ./print-args (though they don't\n> hurt either, and certainly not worth re-rolling just to drop them, and\n> it's subjective so don't drop them just for my sake).\n\nThank you for your continued attention and patience.\n-Stolee\n"},{"id":"409918","messageId":"CAPig+cS3YXH=SmTsRTOa5Cdppt9L=MbFRT+LWn9QE7aEqRiDqQ@mail.gmail.com","threadId":"54562","inReplyTo":"ed7a61978fe9dce26ca459b5a86490c15e470698.1605276024.git.gitgitgadget@gmail.com","subject":"Re: [PATCH v3 4/4] maintenance: use Windows scheduled tasks","fromName":"Eric Sunshine","fromEmail":"sunshine@sunshineco.com","sentAt":"2020-11-13T20:44:16Z","receivedAt":"2020-11-13T20:44:32Z","isPatch":true,"sender":{"key":"sunshine@sunshineco.com","avatar":"https://avatars.githubusercontent.com/u/163641?v=4"},"body":"On Fri, Nov 13, 2020 at 9:00 AM Derrick Stolee via GitGitGadget\n wrote:\n> Git's background maintenance uses cron by default, but this is not\n> available on Windows. Instead, integrate with Task Scheduler.\n> [...]\n> Signed-off-by: Derrick Stolee \n> ---\n> diff --git a/builtin/gc.c b/builtin/gc.c\n> @@ -1684,6 +1684,190 @@ static int platform_update_schedule(int run_maintenance, int fd)\n> +static int schedule_task(const char *exec_path, enum schedule_priority schedule)\n> +{\n> + char *xmlpath, *tempDir;\n> + tempDir = xstrfmt(\"%s/temp\", the_repository->objects->odb->path);\n> + xmlpath = xstrfmt(\"%s/schedule-%s.xml\", tempDir, frequency);\n\nWhen I wondered aloud in my previous review whether writing these\nthrowaway files to a temporary directory would make sense, I was\nthinking more along the lines of /tmp or $TEMP. More specifically, we\nhave xmkstemp() in wrapper.c which is good for this sort of thing (or\none of the other temporary-file-making functions in there). We also\nhave a more full-featured temporary-file API in tempfile.h which would\nensure that these throwaway files actually get thrown away when the\ncommand finishes.\n\nThis is not necessarily worth a re-roll.\n\n> + if (start_command(&child))\n> + die(_(\"failed to start schtasks\"));\n> + result = finish_command(&child);\n> +\n> + unlink(xmlpath);\n> + rmdir(tempDir);\n\nNeither xmlpath and tempDir get cleaned up from the filesystem if the\npreceding die() is triggered (which may or may not make sense --\nperhaps you want to keep them around if it helps with the diagnosis of\nthe failure). The functions in tempfile.h would ensure the temporary\nfile is cleaned up even if the program die()s, or you could manually\nremove the temporary file before die()ing.\n\n> diff --git a/t/t7900-maintenance.sh b/t/t7900-maintenance.sh\n> @@ -437,6 +437,33 @@ test_expect_success MACOS_MAINTENANCE 'start and stop macOS maintenance' '\n> +test_expect_success MINGW 'start and stop Windows maintenance' '\n> + write_script print-args <<-\\EOF &&\n> + echo $* >>args\n> + EOF\n\nUsing `>>` here makes it harder to reason about the test than using\n`>` would, especially since `>>` seems to be unnecessary in this case.\n\n> + rm -f args &&\n> + GIT_TEST_CRONTAB=\"/bin/sh print-args\" git maintenance start &&\n\nIs it a requirement on Windows to mention /bin/sh here? Specifically,\nI'm wondering why a simple ./print-args doesn't work. (It's especially\nunclear since write_script() is used heavily in the test suite and it\nseems to work well enough on Windows without specifying /bin/sh.)\n"},{"id":"409919","messageId":"CAPig+cRPKZGR0XKv4vwtP-xpzYDpB6PL-=Q7weeWtTbDG7o3vQ@mail.gmail.com","threadId":"54562","inReplyTo":"pull.776.v3.git.1605276024.gitgitgadget@gmail.com","subject":"Re: [PATCH v3 0/4] Maintenance IV: Platform-specific background maintenance","fromName":"Eric Sunshine","fromEmail":"sunshine@sunshineco.com","sentAt":"2020-11-13T20:47:15Z","receivedAt":"2020-11-13T20:47:30Z","isPatch":true,"sender":{"key":"sunshine@sunshineco.com","avatar":"https://avatars.githubusercontent.com/u/163641?v=4"},"body":"On Fri, Nov 13, 2020 at 9:00 AM Derrick Stolee via GitGitGadget\n wrote:\n> * This actually includes the feedback responses I had intended for v2.\n> Sorry about that!\n\nI forgot to mention a couple things when reviewing the patches\nindividually, so I'll point them out here...\n\n> + at [2]. The current design does not preclude a future version that\n> + detects the available fatures of 'launchctl' to use the older\n\ns/fatures/features/\n\n> -+ test_cmp expect args\n> ++ test_line_count = 0 actual\n\nThese days, we usually say:\n\n test_must_be_empty actual\n"},{"id":"409920","messageId":"CAPig+cTRJb-fn2R6rJO1hkeCc_ehVhkNufO4=LhtPQudVeonnA@mail.gmail.com","threadId":"54562","inReplyTo":"d9c51446-c223-dce9-692a-959f841d506f@gmail.com","subject":"Re: [PATCH v3 3/4] maintenance: use launchctl on macOS","fromName":"Eric Sunshine","fromEmail":"sunshine@sunshineco.com","sentAt":"2020-11-13T20:53:57Z","receivedAt":"2020-11-13T20:54:12Z","isPatch":true,"sender":{"key":"sunshine@sunshineco.com","avatar":"https://avatars.githubusercontent.com/u/163641?v=4"},"body":"On Fri, Nov 13, 2020 at 3:42 PM Derrick Stolee wrote:\n> On 11/13/2020 3:19 PM, Eric Sunshine wrote:\n> > if (pipe_command(child, NULL, 0, &out, 0, &err, 0) {\n> > if (out.len && err.len)\n> > strbuf_addstr(&out, \"; \");\n> > strbuf_addbuf(&out, &err);\n> > die(_(\"launchctl failed: %s\"), out.buf);\n> > }\n>\n> We would also want to pass a \"die_on_failure\" into the method, since\n> in the 'git maintenance start' case we don't want to report a failure\n> when 'launchctl bootout' fails before we call 'launchctl bootstrap'.\n\nRight. I started writing that we'd also need a `die_one_failure` flag\nbut deleted the comment since I decided to wait until I got an\nanswer...\n\n> > By the way, won't this die() be a problem when schedule_plist() calls\n> > boot_plist() to remove the old scheduled tasks before calling it again\n> > to register the new ones? If the old ones don't exist, then it will\n> > die() unnecessarily and never register the new ones. Or am I\n> > misunderstanding? (I'm guessing that I must be misunderstanding since\n> > the test script presumably passes.)\n>\n> This die() is only if the process cannot _start_, for example due to\n> launchctl not existing on $PATH. The result from finish_command()\n> would be non-zero when we bootout a plist that doesn't exist.\n\n... to this question.\n\nAnother thought I had was simply checking for the presence of the file\nand skipping `bootout` altogether if it doesn't exist. That would, I\nthink, obviate the need for mucking with stdout/stderr oppression.\n\n> > write_script() takes the script body as stdin, not as an argument, and\n> > you don't need to specify /bin/sh. What you have here works by\n> > accident only because write_script() takes an optional second argument\n> > specifying the shell to use in place of the default /bin/sh.\n> > Nevertheless, it should really be written:\n> >\n> > write_script print-args <<-\\EOF\n> > echo $*\n> > EOF\n> >\n> > Patch [4/4] uses write_script() correctly.\n>\n> Ah. Sorry for misunderstanding. That explains why it works this way\n> on macOS but it did _not_ work that way on Windows.\n\nSorry on my part too. I missed the `args` redirect in my example. It should be:\n\n write_script print-args <<-\\EOF\n echo $* >args\n EOF\n"},{"id":"409921","messageId":"CAPig+cR1QnqnZjWFSTP5NuzaeedRThxnvKbqU_U_=d+p2DOOyQ@mail.gmail.com","threadId":"54562","inReplyTo":"CAPig+cTRJb-fn2R6rJO1hkeCc_ehVhkNufO4=LhtPQudVeonnA@mail.gmail.com","subject":"Re: [PATCH v3 3/4] maintenance: use launchctl on macOS","fromName":"Eric Sunshine","fromEmail":"sunshine@sunshineco.com","sentAt":"2020-11-13T20:56:47Z","receivedAt":"2020-11-13T20:57:01Z","isPatch":true,"sender":{"key":"sunshine@sunshineco.com","avatar":"https://avatars.githubusercontent.com/u/163641?v=4"},"body":"On Fri, Nov 13, 2020 at 3:53 PM Eric Sunshine wrote:\n> Another thought I had was simply checking for the presence of the file\n> and skipping `bootout` altogether if it doesn't exist. That would, I\n> think, obviate the need for mucking with stdout/stderr oppression.\n\nErm, s/oppression/suppression/.\n"},{"id":"409923","messageId":"73c902db-40f5-2693-c350-9c04ba2751cf@gmail.com","threadId":"54562","inReplyTo":"CAPig+cS3YXH=SmTsRTOa5Cdppt9L=MbFRT+LWn9QE7aEqRiDqQ@mail.gmail.com","subject":"Re: [PATCH v3 4/4] maintenance: use Windows scheduled tasks","fromName":"Derrick Stolee","fromEmail":"stolee@gmail.com","sentAt":"2020-11-13T21:32:13Z","receivedAt":"2020-11-13T21:32:21Z","isPatch":true,"sender":{"key":"stolee@gmail.com","avatar":"https://avatars.githubusercontent.com/u/570044?v=4"},"body":"On 11/13/2020 3:44 PM, Eric Sunshine wrote:\n> On Fri, Nov 13, 2020 at 9:00 AM Derrick Stolee via GitGitGadget\n> wrote:\n>> Git's background maintenance uses cron by default, but this is not\n>> available on Windows. Instead, integrate with Task Scheduler.\n>> [...]\n>> Signed-off-by: Derrick Stolee \n>> ---\n>> diff --git a/builtin/gc.c b/builtin/gc.c\n>> @@ -1684,6 +1684,190 @@ static int platform_update_schedule(int run_maintenance, int fd)\n>> +static int schedule_task(const char *exec_path, enum schedule_priority schedule)\n>> +{\n>> + char *xmlpath, *tempDir;\n>> + tempDir = xstrfmt(\"%s/temp\", the_repository->objects->odb->path);\n>> + xmlpath = xstrfmt(\"%s/schedule-%s.xml\", tempDir, frequency);\n> \n> When I wondered aloud in my previous review whether writing these\n> throwaway files to a temporary directory would make sense, I was\n> thinking more along the lines of /tmp or $TEMP. More specifically, we\n> have xmkstemp() in wrapper.c which is good for this sort of thing (or\n> one of the other temporary-file-making functions in there). We also\n> have a more full-featured temporary-file API in tempfile.h which would\n> ensure that these throwaway files actually get thrown away when the\n> command finishes.\n> \n> This is not necessarily worth a re-roll.\n> \n>> + if (start_command(&child))\n>> + die(_(\"failed to start schtasks\"));\n>> + result = finish_command(&child);\n>> +\n>> + unlink(xmlpath);\n>> + rmdir(tempDir);\n> \n> Neither xmlpath and tempDir get cleaned up from the filesystem if the\n> preceding die() is triggered (which may or may not make sense --\n> perhaps you want to keep them around if it helps with the diagnosis of\n> the failure). The functions in tempfile.h would ensure the temporary\n> file is cleaned up even if the program die()s, or you could manually\n> remove the temporary file before die()ing.\n\nWhile I do like to have access to the data when trying to resolve\nan issue, it's probably better to use the tempfile library.\n\n>> diff --git a/t/t7900-maintenance.sh b/t/t7900-maintenance.sh\n>> @@ -437,6 +437,33 @@ test_expect_success MACOS_MAINTENANCE 'start and stop macOS maintenance' '\n>> +test_expect_success MINGW 'start and stop Windows maintenance' '\n>> + write_script print-args <<-\\EOF &&\n>> + echo $* >>args\n>> + EOF\n> \n> Using `>>` here makes it harder to reason about the test than using\n> `>` would, especially since `>>` seems to be unnecessary in this case.\n\nSince we execute the GIT_TEST_CRONTAB executable multiple times, we\nneed to use >> to log all three instances (and their order). Using \">args\"\nwould only capture the final call for the weekly schedule.\n\nOn macOS, there are as many as six calls (three bootouts, three bootstraps).\n\n>> + rm -f args &&\n>> + GIT_TEST_CRONTAB=\"/bin/sh print-args\" git maintenance start &&\n> \n> Is it a requirement on Windows to mention /bin/sh here? Specifically,\n> I'm wondering why a simple ./print-args doesn't work. (It's especially\n> unclear since write_script() is used heavily in the test suite and it\n> seems to work well enough on Windows without specifying /bin/sh.)\n\nI landed on this after trying several attempts to get this to work,\nincluding \"$(pwd)/print-args\" and I'm not sure why it doesn't work\nin the Windows case. It is something to do with how I am executing\nthe subcommand from within Git. I'm pretty sure this idea of \"mocking\"\nan executable through Git is relatively new, or at least rare\n\nThanks,\n-Stolee\n"},{"id":"409925","messageId":"CAPig+cRv_Swd4WcZDnyDxg4E1HdwdkfybjWiZsYwPFSxkhUz0w@mail.gmail.com","threadId":"54562","inReplyTo":"73c902db-40f5-2693-c350-9c04ba2751cf@gmail.com","subject":"Re: [PATCH v3 4/4] maintenance: use Windows scheduled tasks","fromName":"Eric Sunshine","fromEmail":"sunshine@sunshineco.com","sentAt":"2020-11-13T21:40:40Z","receivedAt":"2020-11-13T21:41:02Z","isPatch":true,"sender":{"key":"sunshine@sunshineco.com","avatar":"https://avatars.githubusercontent.com/u/163641?v=4"},"body":"On Fri, Nov 13, 2020 at 4:32 PM Derrick Stolee wrote:\n> On 11/13/2020 3:44 PM, Eric Sunshine wrote:\n> > On Fri, Nov 13, 2020 at 9:00 AM Derrick Stolee via GitGitGadget\n> > wrote:\n> >> +test_expect_success MINGW 'start and stop Windows maintenance' '\n> >> + write_script print-args <<-\\EOF &&\n> >> + echo $* >>args\n> >> + EOF\n> >\n> > Using `>>` here makes it harder to reason about the test than using\n> > `>` would, especially since `>>` seems to be unnecessary in this case.\n>\n> Since we execute the GIT_TEST_CRONTAB executable multiple times, we\n> need to use >> to log all three instances (and their order). Using \">args\"\n> would only capture the final call for the weekly schedule.\n>\n> On macOS, there are as many as six calls (three bootouts, three bootstraps).\n\nMakes sense. Thanks.\n\n> >> + GIT_TEST_CRONTAB=\"/bin/sh print-args\" git maintenance start &&\n> >\n> > Is it a requirement on Windows to mention /bin/sh here? Specifically,\n> > I'm wondering why a simple ./print-args doesn't work. (It's especially\n> > unclear since write_script() is used heavily in the test suite and it\n> > seems to work well enough on Windows without specifying /bin/sh.)\n>\n> I landed on this after trying several attempts to get this to work,\n> including \"$(pwd)/print-args\" and I'm not sure why it doesn't work\n> in the Windows case. It is something to do with how I am executing\n> the subcommand from within Git. I'm pretty sure this idea of \"mocking\"\n> an executable through Git is relatively new, or at least rare\n\nJust for clarification... You mentioned in response to my [3/4] review\nthat your accidentally-working write_script() only worked as expected\non Mac but not on Windows. When you arrived at this solution of\nGIT_TEST_CRONTAB=\"/bin/sh ...\" here, was that before or after you\nfixed write_script() to take the script body from stdin?\n"},{"id":"409948","messageId":"X6+iJNYEbpQjHCb0@flurp.local","threadId":"54562","inReplyTo":"CAPig+cRPKZGR0XKv4vwtP-xpzYDpB6PL-=Q7weeWtTbDG7o3vQ@mail.gmail.com","subject":"Re: [PATCH v3 0/4] Maintenance IV: Platform-specific background maintenance","fromName":"Eric Sunshine","fromEmail":"sunshine@sunshineco.com","sentAt":"2020-11-14T09:23:48Z","receivedAt":"2020-11-14T09:24:19Z","isPatch":true,"sender":{"key":"sunshine@sunshineco.com","avatar":"https://avatars.githubusercontent.com/u/163641?v=4"},"body":"On Fri, Nov 13, 2020 at 03:47:15PM -0500, Eric Sunshine wrote:\n> I forgot to mention a couple things when reviewing the patches\n> individually, so I'll point them out here...\n\nIn v2, you added an `xmllint` check on MacOS after discovering that\ngc.c was generating a malformed .plist file on that platform. That got\nme thinking that it would have been nice to have caught the problem\nearlier, if possible, even without having access to MacOS. Since none\nof the code added to gc.c has a hard platform dependency, it should be\npossible to perform all the tests on any platform rather than\nrestricting them to specific platforms via test prerequisites. The\npatch below, which is built atop v3, does just that. It removes the\nconditional compilation directives from gc.c and the prerequisites\nfrom the test script so that all scheduler-specific code in gc.c is\ntested on all platform.\n\nThe changes made by the patch are intended to be folded into each of\nyour patches where appropriate (rather than existing atop your series,\nwhich, though possible, would be ugly). If you're interested in\nincorporating any of these improvements into v4, you can have my\n\"Signed-off-by: Eric Sunshine \" in addition\nto the Helped-by: you already added.\n\nA few more notes...\n\nIn addition to making it possible to test all platform-specific\nschedulers on each platform, I also made a few other\nchanges/enhancements:\n\n* simplified UID retrieval and eliminated platform-specific\n dependencies (though this may need some additional tweaking on\n Windows, for which I did not test); also fixed the $UID issue\n mentioned in review\n\n* extended xmllint testing to the XML files generated for `schtasks`\n on Windows too; this required a small modification to the XML header\n boilerplate to specify the correct file encoding since `xmllint`\n complains when the file is UTF-8 but claims to be UTF-16; now that\n the test script captures the generated `schtasks` XML file for\n checking against `xmllint`, you have the opportunity to perform\n other sorts of validation checks on the XML too, such as you do in\n the MacOS `launchctl` test (though I did not add any additional\n checks)\n\n* fixed a potentially crashable `fprintf(xmlfp, xml)` by changing it\n to `fputs(xml, xmlfp)` since the compiler complains about the former\n because it can crash if `xml` contains a \"%\"\n\n* fixed the malformed write_script() issue for the MacOS test\n mentioned in review\n\n--- >8 ---\nFrom 016887b9fa4269bd4df46bea1d7849c08aba6ad6 Mon Sep 17 00:00:00 2001\nFrom: Eric Sunshine \nDate: Sat, 14 Nov 2020 02:39:05 -0500\nSubject: [PATCH] maintenance: test start/stop on all platforms from any\n platform\n\nSigned-off-by: Eric Sunshine \n---\n builtin/gc.c | 204 +++++++++++++++++++----------------------\n t/t7900-maintenance.sh | 66 +++++++++----\n t/test-lib.sh | 4 -\n 3 files changed, 143 insertions(+), 131 deletions(-)\n\ndiff --git a/builtin/gc.c b/builtin/gc.c\nindex 76a3afa20a..955d4b3baf 100644\n--- a/builtin/gc.c\n+++ b/builtin/gc.c\n@@ -1491,16 +1491,28 @@ static int maintenance_unregister(void)\n \treturn run_command(&config_unset);\n }\n \n-#if defined(__APPLE__)\n+static const char *get_frequency(enum schedule_priority schedule)\n+{\n+\tswitch (schedule) {\n+\tcase SCHEDULE_HOURLY:\n+\t\treturn \"hourly\";\n+\tcase SCHEDULE_DAILY:\n+\t\treturn \"daily\";\n+\tcase SCHEDULE_WEEKLY:\n+\t\treturn \"weekly\";\n+\tdefault:\n+\t\tBUG(\"invalid schedule %d\", schedule);\n+\t}\n+}\n \n-static char *get_service_name(const char *frequency)\n+static char *launchctl_service_name(const char *frequency)\n {\n \tstruct strbuf label = STRBUF_INIT;\n \tstrbuf_addf(&label, \"org.git-scm.git.%s\", frequency);\n \treturn strbuf_detach(&label, NULL);\n }\n \n-static char *get_service_filename(const char *name)\n+static char *launchctl_service_filename(const char *name)\n {\n \tchar *expanded;\n \tstruct strbuf filename = STRBUF_INIT;\n@@ -1514,49 +1526,23 @@ static char *get_service_filename(const char *name)\n \treturn expanded;\n }\n \n-static const char *get_frequency(enum schedule_priority schedule)\n-{\n-\tswitch (schedule) {\n-\tcase SCHEDULE_HOURLY:\n-\t\treturn \"hourly\";\n-\tcase SCHEDULE_DAILY:\n-\t\treturn \"daily\";\n-\tcase SCHEDULE_WEEKLY:\n-\t\treturn \"weekly\";\n-\tdefault:\n-\t\tBUG(\"invalid schedule %d\", schedule);\n-\t}\n-}\n-\n-static char *get_uid(void)\n+static char *launchctl_get_uid(void)\n {\n-\tstruct strbuf output = STRBUF_INIT;\n-\tstruct child_process id = CHILD_PROCESS_INIT;\n-\n-\tstrvec_pushl(&id.args, \"/usr/bin/id\", \"-u\", NULL);\n-\tif (capture_command(&id, &output, 0))\n-\t\tdie(_(\"failed to discover user id\"));\n-\n-\tstrbuf_trim_trailing_newline(&output);\n-\treturn strbuf_detach(&output, NULL);\n+\treturn xstrfmt(\"gui/%d\", getuid());\n }\n \n-static int boot_plist(int enable, const char *filename)\n+static int launchctl_boot_plist(int enable, const char *filename, const char *cmd)\n {\n \tint result;\n \tstruct child_process child = CHILD_PROCESS_INIT;\n-\tchar *uid = get_uid();\n-\tconst char *launchctl = getenv(\"GIT_TEST_CRONTAB\");\n-\tif (!launchctl)\n-\t\tlaunchctl = \"/bin/launchctl\";\n-\n-\tstrvec_split(&child.args, launchctl);\n+\tchar *uid = launchctl_get_uid();\n \n+\tstrvec_split(&child.args, cmd);\n \tif (enable)\n \t\tstrvec_push(&child.args, \"bootstrap\");\n \telse\n \t\tstrvec_push(&child.args, \"bootout\");\n-\tstrvec_pushf(&child.args, \"gui/%s\", uid);\n+\tstrvec_push(&child.args, uid);\n \tstrvec_push(&child.args, filename);\n \n \tchild.no_stderr = 1;\n@@ -1571,33 +1557,33 @@ static int boot_plist(int enable, const char *filename)\n \treturn result;\n }\n \n-static int remove_plist(enum schedule_priority schedule)\n+static int launchctl_remove_plist(enum schedule_priority schedule, const char *cmd)\n {\n \tconst char *frequency = get_frequency(schedule);\n-\tchar *name = get_service_name(frequency);\n-\tchar *filename = get_service_filename(name);\n-\tint result = boot_plist(0, filename);\n+\tchar *name = launchctl_service_name(frequency);\n+\tchar *filename = launchctl_service_filename(name);\n+\tint result = launchctl_boot_plist(0, filename, cmd);\n \tunlink(filename);\n \tfree(filename);\n \tfree(name);\n \treturn result;\n }\n \n-static int remove_plists(void)\n+static int launchctl_remove_plists(const char *cmd)\n {\n-\treturn remove_plist(SCHEDULE_HOURLY) ||\n-\t\tremove_plist(SCHEDULE_DAILY) ||\n-\t\tremove_plist(SCHEDULE_WEEKLY);\n+\treturn launchctl_remove_plist(SCHEDULE_HOURLY, cmd) ||\n+\t\tlaunchctl_remove_plist(SCHEDULE_DAILY, cmd) ||\n+\t\tlaunchctl_remove_plist(SCHEDULE_WEEKLY, cmd);\n }\n \n-static int schedule_plist(const char *exec_path, enum schedule_priority schedule)\n+static int launchctl_schedule_plist(const char *exec_path, enum schedule_priority schedule, const char *cmd)\n {\n \tFILE *plist;\n \tint i;\n \tconst char *preamble, *repeat;\n \tconst char *frequency = get_frequency(schedule);\n-\tchar *name = get_service_name(frequency);\n-\tchar *filename = get_service_filename(name);\n+\tchar *name = launchctl_service_name(frequency);\n+\tchar *filename = launchctl_service_filename(name);\n \n \tif (safe_create_leading_directories(filename))\n \t\tdie(_(\"failed to create directories for '%s'\"), filename);\n@@ -1658,8 +1644,8 @@ static int schedule_plist(const char *exec_path, enum schedule_priority schedule\n \tfprintf(plist, \"\\n\\n\\n\");\n \n \t/* bootout might fail if not already running, so ignore */\n-\tboot_plist(0, filename);\n-\tif (boot_plist(1, filename))\n+\tlaunchctl_boot_plist(0, filename, cmd);\n+\tif (launchctl_boot_plist(1, filename, cmd))\n \t\tdie(_(\"failed to bootstrap service %s\"), filename);\n \n \tfclose(plist);\n@@ -1668,57 +1654,38 @@ static int schedule_plist(const char *exec_path, enum schedule_priority schedule\n \treturn 0;\n }\n \n-static int add_plists(void)\n+static int launchctl_add_plists(const char *cmd)\n {\n \tconst char *exec_path = git_exec_path();\n \n-\treturn schedule_plist(exec_path, SCHEDULE_HOURLY) ||\n-\t\tschedule_plist(exec_path, SCHEDULE_DAILY) ||\n-\t\tschedule_plist(exec_path, SCHEDULE_WEEKLY);\n+\treturn launchctl_schedule_plist(exec_path, SCHEDULE_HOURLY, cmd) ||\n+\t\tlaunchctl_schedule_plist(exec_path, SCHEDULE_DAILY, cmd) ||\n+\t\tlaunchctl_schedule_plist(exec_path, SCHEDULE_WEEKLY, cmd);\n }\n \n-static int platform_update_schedule(int run_maintenance, int fd)\n+static int launchctl_update_schedule(int run_maintenance, int fd, const char *cmd)\n {\n \tif (run_maintenance)\n-\t\treturn add_plists();\n+\t\treturn launchctl_add_plists(cmd);\n \telse\n-\t\treturn remove_plists();\n-}\n-\n-#elif defined(GIT_WINDOWS_NATIVE)\n-\n-static const char *get_frequency(enum schedule_priority schedule)\n-{\n-\tswitch (schedule) {\n-\tcase SCHEDULE_HOURLY:\n-\t\treturn \"hourly\";\n-\tcase SCHEDULE_DAILY:\n-\t\treturn \"daily\";\n-\tcase SCHEDULE_WEEKLY:\n-\t\treturn \"weekly\";\n-\tdefault:\n-\t\tBUG(\"invalid schedule %d\", schedule);\n-\t}\n+\t\treturn launchctl_remove_plists(cmd);\n }\n \n-static char *get_task_name(const char *frequency)\n+static char *schtasks_task_name(const char *frequency)\n {\n \tstruct strbuf label = STRBUF_INIT;\n \tstrbuf_addf(&label, \"Git Maintenance (%s)\", frequency);\n \treturn strbuf_detach(&label, NULL);\n }\n \n-static int remove_task(enum schedule_priority schedule)\n+static int schtasks_remove_task(enum schedule_priority schedule, const char *cmd)\n {\n \tint result;\n \tstruct strvec args = STRVEC_INIT;\n \tconst char *frequency = get_frequency(schedule);\n-\tchar *name = get_task_name(frequency);\n-\tconst char *schtasks = getenv(\"GIT_TEST_CRONTAB\");\n-\tif (!schtasks)\n-\t\tschtasks = \"schtasks\";\n+\tchar *name = schtasks_task_name(frequency);\n \n-\tstrvec_split(&args, schtasks);\n+\tstrvec_split(&args, cmd);\n \tstrvec_pushl(&args, \"/delete\", \"/tn\", name, \"/f\", NULL);\n \n \tresult = run_command_v_opt(args.v, 0);\n@@ -1728,33 +1695,33 @@ static int remove_task(enum schedule_priority schedule)\n \treturn result;\n }\n \n-static int remove_scheduled_tasks(void)\n+static int schtasks_remove_tasks(const char *cmd)\n {\n-\treturn remove_task(SCHEDULE_HOURLY) ||\n-\t\tremove_task(SCHEDULE_DAILY) ||\n-\t\tremove_task(SCHEDULE_WEEKLY);\n+\treturn schtasks_remove_task(SCHEDULE_HOURLY, cmd) ||\n+\t\tschtasks_remove_task(SCHEDULE_DAILY, cmd) ||\n+\t\tschtasks_remove_task(SCHEDULE_WEEKLY, cmd);\n }\n \n-static int schedule_task(const char *exec_path, enum schedule_priority schedule)\n+static int schtasks_schedule_task(const char *exec_path, enum schedule_priority schedule, const char *cmd)\n {\n \tint result;\n \tstruct child_process child = CHILD_PROCESS_INIT;\n-\tconst char *xml, *schtasks;\n+\tconst char *xml;\n \tchar *xmlpath, *tempDir;\n \tFILE *xmlfp;\n \tconst char *frequency = get_frequency(schedule);\n-\tchar *name = get_task_name(frequency);\n+\tchar *name = schtasks_task_name(frequency);\n \n \ttempDir = xstrfmt(\"%s/temp\", the_repository->objects->odb->path);\n \txmlpath = xstrfmt(\"%s/schedule-%s.xml\", tempDir, frequency);\n \tsafe_create_leading_directories(xmlpath);\n \txmlfp = xfopen(xmlpath, \"w\");\n \n-\txml = \"\\n\"\n+\txml = \"\\n\"\n \t \"\\n\"\n \t \"\\n\"\n \t \"\\n\";\n-\tfprintf(xmlfp, xml);\n+\tfputs(xml, xmlfp);\n \n \tswitch (schedule) {\n \tcase SCHEDULE_HOURLY:\n@@ -1831,10 +1798,7 @@ static int schedule_task(const char *exec_path, enum schedule_priority schedule)\n \tfprintf(xmlfp, xml, exec_path, exec_path, frequency);\n \tfclose(xmlfp);\n \n-\tschtasks = getenv(\"GIT_TEST_CRONTAB\");\n-\tif (!schtasks)\n-\t\tschtasks = \"schtasks\";\n-\tstrvec_split(&child.args, schtasks);\n+\tstrvec_split(&child.args, cmd);\n \tstrvec_pushl(&child.args, \"/create\", \"/tn\", name, \"/f\", \"/xml\", xmlpath, NULL);\n \n \tchild.no_stdout = 1;\n@@ -1851,42 +1815,36 @@ static int schedule_task(const char *exec_path, enum schedule_priority schedule)\n \treturn result;\n }\n \n-static int add_scheduled_tasks(void)\n+static int schtasks_schedule_tasks(const char *cmd)\n {\n \tconst char *exec_path = git_exec_path();\n \n-\treturn schedule_task(exec_path, SCHEDULE_HOURLY) ||\n-\t\tschedule_task(exec_path, SCHEDULE_DAILY) ||\n-\t\tschedule_task(exec_path, SCHEDULE_WEEKLY);\n+\treturn schtasks_schedule_task(exec_path, SCHEDULE_HOURLY, cmd) ||\n+\t\tschtasks_schedule_task(exec_path, SCHEDULE_DAILY, cmd) ||\n+\t\tschtasks_schedule_task(exec_path, SCHEDULE_WEEKLY, cmd);\n }\n \n-static int platform_update_schedule(int run_maintenance, int fd)\n+static int schtasks_update_schedule(int run_maintenance, int fd, const char *cmd)\n {\n \tif (run_maintenance)\n-\t\treturn add_scheduled_tasks();\n+\t\treturn schtasks_schedule_tasks(cmd);\n \telse\n-\t\treturn remove_scheduled_tasks();\n+\t\treturn schtasks_remove_tasks(cmd);\n }\n \n-#else\n #define BEGIN_LINE \"# BEGIN GIT MAINTENANCE SCHEDULE\"\n #define END_LINE \"# END GIT MAINTENANCE SCHEDULE\"\n \n-static int platform_update_schedule(int run_maintenance, int fd)\n+static int crontab_update_schedule(int run_maintenance, int fd, const char *cmd)\n {\n \tint result = 0;\n \tint in_old_region = 0;\n \tstruct child_process crontab_list = CHILD_PROCESS_INIT;\n \tstruct child_process crontab_edit = CHILD_PROCESS_INIT;\n \tFILE *cron_list, *cron_in;\n-\tconst char *crontab_name;\n \tstruct strbuf line = STRBUF_INIT;\n \n-\tcrontab_name = getenv(\"GIT_TEST_CRONTAB\");\n-\tif (!crontab_name)\n-\t\tcrontab_name = \"crontab\";\n-\n-\tstrvec_split(&crontab_list.args, crontab_name);\n+\tstrvec_split(&crontab_list.args, cmd);\n \tstrvec_push(&crontab_list.args, \"-l\");\n \tcrontab_list.in = -1;\n \tcrontab_list.out = dup(fd);\n@@ -1906,7 +1864,7 @@ static int platform_update_schedule(int run_maintenance, int fd)\n \tcron_list = fdopen(fd, \"r\");\n \trewind(cron_list);\n \n-\tstrvec_split(&crontab_edit.args, crontab_name);\n+\tstrvec_split(&crontab_edit.args, cmd);\n \tcrontab_edit.in = -1;\n \tcrontab_edit.git_cmd = 0;\n \n@@ -1963,20 +1921,48 @@ static int platform_update_schedule(int run_maintenance, int fd)\n \t\tfclose(cron_list);\n \treturn result;\n }\n+\n+#if defined(__APPLE__)\n+static const char platform_scheduler[] = \"launchctl\";\n+#elif defined(GIT_WINDOWS_NATIVE)\n+static const char platform_scheduler[] = \"schtasks\";\n+#else\n+static const char platform_scheduler[] = \"crontab\";\n #endif\n \n-static int update_background_schedule(int run_maintenance)\n+static int update_background_schedule(int enable)\n {\n \tint result;\n+\tconst char *scheduler = platform_scheduler;\n+\tconst char *cmd = scheduler;\n+\tchar *testing;\n \tstruct lock_file lk;\n \tchar *lock_path = xstrfmt(\"%s/schedule\", the_repository->objects->odb->path);\n \n+\ttesting = xstrdup_or_null(getenv(\"GIT_TEST_MAINT_SCHEDULER\"));\n+\tif (testing) {\n+\t\tchar *sep = strchr(testing, ':');\n+\t\tif (!sep)\n+\t\t\tdie(\"GIT_TEST_MAINT_SCHEDULER unparseable: %s\", testing);\n+\t\t*sep = '\\0';\n+\t\tscheduler = testing;\n+\t\tcmd = sep + 1;\n+\t}\n+\n \tif (hold_lock_file_for_update(&lk, lock_path, LOCK_NO_DEREF) < 0)\n \t\treturn error(_(\"another process is scheduling background maintenance\"));\n \n-\tresult = platform_update_schedule(run_maintenance, lk.tempfile->fd);\n+\tif (!strcmp(scheduler, \"launchctl\"))\n+\t\tresult = launchctl_update_schedule(enable, lk.tempfile->fd, cmd);\n+\telse if (!strcmp(scheduler, \"schtasks\"))\n+\t\tresult = schtasks_update_schedule(enable, lk.tempfile->fd, cmd);\n+\telse if (!strcmp(scheduler, \"crontab\"))\n+\t\tresult = crontab_update_schedule(enable, lk.tempfile->fd, cmd);\n+\telse\n+\t\tdie(\"unknown background scheduler: %s\", scheduler);\n \n \trollback_lock_file(&lk);\n+\tfree(testing);\n \treturn result;\n }\n \ndiff --git a/t/t7900-maintenance.sh b/t/t7900-maintenance.sh\nindex 0dc2479117..e92946c10a 100755\n--- a/t/t7900-maintenance.sh\n+++ b/t/t7900-maintenance.sh\n@@ -7,6 +7,19 @@ test_description='git maintenance builtin'\n GIT_TEST_COMMIT_GRAPH=0\n GIT_TEST_MULTI_PACK_INDEX=0\n \n+test_lazy_prereq XMLLINT '\n+\txmllint --version\n+'\n+\n+test_xmllint () {\n+\tif test_have_prereq XMLLINT\n+\tthen\n+\t\txmllint --noout \"$@\"\n+\telse\n+\t\ttrue\n+\tfi\n+}\n+\n test_expect_success 'help text' '\n \ttest_expect_code 129 git maintenance -h 2>err &&\n \ttest_i18ngrep \"usage: git maintenance \" err &&\n@@ -367,8 +380,8 @@ test_expect_success 'register and unregister' '\n \ttest_cmp before actual\n '\n \n-test_expect_success !MACOS_MAINTENANCE,!MINGW 'start from empty cron table' '\n-\tGIT_TEST_CRONTAB=\"test-tool crontab cron.txt\" git maintenance start &&\n+test_expect_success 'start from empty cron table' '\n+\tGIT_TEST_MAINT_SCHEDULER=\"crontab:test-tool crontab cron.txt\" git maintenance start &&\n \n \t# start registers the repo\n \tgit config --get --global maintenance.repo \"$(pwd)\" &&\n@@ -378,28 +391,32 @@ test_expect_success !MACOS_MAINTENANCE,!MINGW 'start from empty cron table' '\n \tgrep \"for-each-repo --config=maintenance.repo maintenance run --schedule=weekly\" cron.txt\n '\n \n-test_expect_success !MACOS_MAINTENANCE,!MINGW 'stop from existing schedule' '\n-\tGIT_TEST_CRONTAB=\"test-tool crontab cron.txt\" git maintenance stop &&\n+test_expect_success 'stop from existing schedule' '\n+\tGIT_TEST_MAINT_SCHEDULER=\"crontab:test-tool crontab cron.txt\" git maintenance stop &&\n \n \t# stop does not unregister the repo\n \tgit config --get --global maintenance.repo \"$(pwd)\" &&\n \n \t# Operation is idempotent\n-\tGIT_TEST_CRONTAB=\"test-tool crontab cron.txt\" git maintenance stop &&\n+\tGIT_TEST_MAINT_SCHEDULER=\"crontab:test-tool crontab cron.txt\" git maintenance stop &&\n \ttest_must_be_empty cron.txt\n '\n \n-test_expect_success !MACOS_MAINTENANCE,!MINGW 'start preserves existing schedule' '\n+test_expect_success 'start preserves existing schedule' '\n \techo \"Important information!\" >cron.txt &&\n-\tGIT_TEST_CRONTAB=\"test-tool crontab cron.txt\" git maintenance start &&\n+\tGIT_TEST_MAINT_SCHEDULER=\"crontab:test-tool crontab cron.txt\" git maintenance start &&\n \tgrep \"Important information!\" cron.txt\n '\n \n-test_expect_success MACOS_MAINTENANCE 'start and stop macOS maintenance' '\n-\twrite_script print-args \"#!/bin/sh\\necho \\$* >>args\" &&\n+test_expect_success 'start and stop macOS maintenance' '\n+\tuid=$(id -u) &&\n+\n+\twrite_script print-args <<-\\EOF &&\n+\techo $* >>args\n+\tEOF\n \n \trm -f args &&\n-\tGIT_TEST_CRONTAB=\"./print-args\" git maintenance start &&\n+\tGIT_TEST_MAINT_SCHEDULER=launchctl:./print-args git maintenance start &&\n \n \t# start registers the repo\n \tgit config --get --global maintenance.repo \"$(pwd)\" &&\n@@ -417,33 +434,41 @@ test_expect_success MACOS_MAINTENANCE 'start and stop macOS maintenance' '\n \tfor frequency in hourly daily weekly\n \tdo\n \t\tPLIST=\"$HOME/Library/LaunchAgents/org.git-scm.git.$frequency.plist\" &&\n-\t\txmllint --noout \"$PLIST\" &&\n+\t\ttest_xmllint \"$PLIST\" &&\n \t\tgrep schedule=$frequency \"$PLIST\" &&\n-\t\techo \"bootout gui/$UID $PLIST\" >>expect &&\n-\t\techo \"bootstrap gui/$UID $PLIST\" >>expect || return 1\n+\t\techo \"bootout gui/$uid $PLIST\" >>expect &&\n+\t\techo \"bootstrap gui/$uid $PLIST\" >>expect || return 1\n \tdone &&\n \ttest_cmp expect args &&\n \n \trm -f args &&\n-\tGIT_TEST_CRONTAB=\"./print-args\" git maintenance stop &&\n+\tGIT_TEST_MAINT_SCHEDULER=launchctl:./print-args git maintenance stop &&\n \n \t# stop does not unregister the repo\n \tgit config --get --global maintenance.repo \"$(pwd)\" &&\n \n-\tprintf \"bootout gui/$UID $HOME/Library/LaunchAgents/org.git-scm.git.%s.plist\\n\" \\\n+\tprintf \"bootout gui/$uid $HOME/Library/LaunchAgents/org.git-scm.git.%s.plist\\n\" \\\n \t\thourly daily weekly >expect &&\n \ttest_cmp expect args &&\n \tls \"$HOME/Library/LaunchAgents\" >actual &&\n \ttest_line_count = 0 actual\n '\n \n-test_expect_success MINGW 'start and stop Windows maintenance' '\n+test_expect_success 'start and stop Windows maintenance' '\n \twrite_script print-args <<-\\EOF &&\n \techo $* >>args\n+\twhile test $# -gt 0\n+\tdo\n+\t\tcase \"$1\" in\n+\t\t/xml) shift; xmlfile=$1; break ;;\n+\t\t*) shift ;;\n+\t\tesac\n+\tdone\n+\ttest -z \"$xmlfile\" || cp \"$xmlfile\" .\n \tEOF\n \n \trm -f args &&\n-\tGIT_TEST_CRONTAB=\"/bin/sh print-args\" git maintenance start &&\n+\tGIT_TEST_MAINT_SCHEDULER=\"schtasks:/bin/sh print-args\" git maintenance start &&\n \n \t# start registers the repo\n \tgit config --get --global maintenance.repo \"$(pwd)\" &&\n@@ -452,8 +477,13 @@ test_expect_success MINGW 'start and stop Windows maintenance' '\n \t\thourly hourly daily daily weekly weekly >expect &&\n \ttest_cmp expect args &&\n \n+\tfor frequency in hourly daily weekly\n+\tdo\n+\t\ttest_xmllint \"schedule-$frequency.xml\"\n+\tdone &&\n+\n \trm -f args &&\n-\tGIT_TEST_CRONTAB=\"/bin/sh print-args\" git maintenance stop &&\n+\tGIT_TEST_MAINT_SCHEDULER=\"schtasks:/bin/sh print-args\" git maintenance stop &&\n \n \t# stop does not unregister the repo\n \tgit config --get --global maintenance.repo \"$(pwd)\" &&\ndiff --git a/t/test-lib.sh b/t/test-lib.sh\nindex 620ffbf3af..4a60d1ed76 100644\n--- a/t/test-lib.sh\n+++ b/t/test-lib.sh\n@@ -1703,10 +1703,6 @@ test_lazy_prereq REBASE_P '\n \ttest -z \"$GIT_TEST_SKIP_REBASE_P\"\n '\n \n-test_lazy_prereq MACOS_MAINTENANCE '\n-\tlaunchctl list\n-'\n-\n # Ensure that no test accidentally triggers a Git command\n # that runs 'crontab', affecting a user's cron schedule.\n # Tests that verify the cron integration must set this locally\n-- \n2.29.2.454.gaff20da3a2\n\n"},{"id":"410008","messageId":"b24ea2df-e937-b8f5-07db-3a1f365035ad@gmail.com","threadId":"54562","inReplyTo":"CAPig+cRv_Swd4WcZDnyDxg4E1HdwdkfybjWiZsYwPFSxkhUz0w@mail.gmail.com","subject":"Re: [PATCH v3 4/4] maintenance: use Windows scheduled tasks","fromName":"Derrick Stolee","fromEmail":"stolee@gmail.com","sentAt":"2020-11-16T13:13:17Z","receivedAt":"2020-11-16T13:13:43Z","isPatch":true,"sender":{"key":"stolee@gmail.com","avatar":"https://avatars.githubusercontent.com/u/570044?v=4"},"body":"On 11/13/2020 4:40 PM, Eric Sunshine wrote:\n> On Fri, Nov 13, 2020 at 4:32 PM Derrick Stolee wrote:\n>> On 11/13/2020 3:44 PM, Eric Sunshine wrote:\n>>> On Fri, Nov 13, 2020 at 9:00 AM Derrick Stolee via GitGitGadget\n>>>> + GIT_TEST_CRONTAB=\"/bin/sh print-args\" git maintenance start &&\n>>>\n>>> Is it a requirement on Windows to mention /bin/sh here? Specifically,\n>>> I'm wondering why a simple ./print-args doesn't work. (It's especially\n>>> unclear since write_script() is used heavily in the test suite and it\n>>> seems to work well enough on Windows without specifying /bin/sh.)\n>>\n>> I landed on this after trying several attempts to get this to work,\n>> including \"$(pwd)/print-args\" and I'm not sure why it doesn't work\n>> in the Windows case. It is something to do with how I am executing\n>> the subcommand from within Git. I'm pretty sure this idea of \"mocking\"\n>> an executable through Git is relatively new, or at least rare\n> \n> Just for clarification... You mentioned in response to my [3/4] review\n> that your accidentally-working write_script() only worked as expected\n> on Mac but not on Windows. When you arrived at this solution of\n> GIT_TEST_CRONTAB=\"/bin/sh ...\" here, was that before or after you\n> fixed write_script() to take the script body from stdin?\n\nYou're right. That was necessary only for the old way that I was\ncreating the script. The correct way works with GIT_TEST_CRONTAB\nequal to ./print-args.\n\n-Stolee\n"},{"id":"410009","messageId":"7e7f0756-9152-4637-753f-2c3eaa562cdb@gmail.com","threadId":"54562","inReplyTo":"X6+iJNYEbpQjHCb0@flurp.local","subject":"Re: [PATCH v3 0/4] Maintenance IV: Platform-specific background maintenance","fromName":"Derrick Stolee","fromEmail":"stolee@gmail.com","sentAt":"2020-11-16T13:17:23Z","receivedAt":"2020-11-16T13:17:28Z","isPatch":true,"sender":{"key":"stolee@gmail.com","avatar":"https://avatars.githubusercontent.com/u/570044?v=4"},"body":"On 11/14/2020 4:23 AM, Eric Sunshine wrote:\n> On Fri, Nov 13, 2020 at 03:47:15PM -0500, Eric Sunshine wrote:\n>> I forgot to mention a couple things when reviewing the patches\n>> individually, so I'll point them out here...\n> \n> In v2, you added an `xmllint` check on MacOS after discovering that\n> gc.c was generating a malformed .plist file on that platform. That got\n> me thinking that it would have been nice to have caught the problem\n> earlier, if possible, even without having access to MacOS. Since none\n> of the code added to gc.c has a hard platform dependency, it should be\n> possible to perform all the tests on any platform rather than\n> restricting them to specific platforms via test prerequisites. The\n> patch below, which is built atop v3, does just that. It removes the\n> conditional compilation directives from gc.c and the prerequisites\n> from the test script so that all scheduler-specific code in gc.c is\n> tested on all platform.\n> \n> The changes made by the patch are intended to be folded into each of\n> your patches where appropriate (rather than existing atop your series,\n> which, though possible, would be ugly). If you're interested in\n> incorporating any of these improvements into v4, you can have my\n> \"Signed-off-by: Eric Sunshine \" in addition\n> to the Helped-by: you already added.\n\nThis approach is fascinating. I will tease it apart to appropriately\nincorporate it into my series. Thank you for your sign-off, since\nthis elevates the patches from \"Helped-by\" to \"Co-authored by\".\n\nThanks,\n-Stolee\n"},{"id":"410130","messageId":"4807342b0019be29bb369ed3403a485f0ce9c15d.1605647598.git.gitgitgadget@gmail.com","threadId":"54562","inReplyTo":"pull.776.v4.git.1605647598.gitgitgadget@gmail.com","subject":"[PATCH v4 1/4] maintenance: extract platform-specific scheduling","fromName":"Derrick Stolee via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2020-11-17T21:13:15Z","receivedAt":"2020-11-17T21:13:45Z","isPatch":true,"sender":{"key":"stolee@gmail.com","avatar":"https://avatars.githubusercontent.com/u/570044?v=4"},"body":"From: Derrick Stolee \n\nThe existing schedule mechanism using 'cron' is supported by POSIX\nplatforms, but not Windows. It also works slightly differently on\nmacOS to significant detriment of the user experience. To allow for\nnew implementations on these platforms, extract a method that\nperforms the platform-specific scheduling mechanism. This will be\nswapped at compile time with new implementations on specialized\nplatforms.\n\nAs we add this generality, rename GIT_TEST_CRONTAB to\nGIT_TEST_MAINT_SCHEDULER. Further, this variable is now parsed as\n\":\" so we can test platform-specific scheduling\nlogic even when not on the correct platform. By specifying the\n in this string, we will be able to test all three sets of\nGit logic from a Linux machine.\n\nCo-authored-by: Eric Sunshine \nSigned-off-by: Eric Sunshine \nSigned-off-by: Derrick Stolee \n---\n builtin/gc.c | 70 ++++++++++++++++++++++++++----------------\n t/t7900-maintenance.sh | 8 ++---\n t/test-lib.sh | 7 +++--\n 3 files changed, 51 insertions(+), 34 deletions(-)\n\ndiff --git a/builtin/gc.c b/builtin/gc.c\nindex e3098ef6a1..18ae7f7138 100644\n--- a/builtin/gc.c\n+++ b/builtin/gc.c\n@@ -1494,35 +1494,23 @@ static int maintenance_unregister(void)\n #define BEGIN_LINE \"# BEGIN GIT MAINTENANCE SCHEDULE\"\n #define END_LINE \"# END GIT MAINTENANCE SCHEDULE\"\n \n-static int update_background_schedule(int run_maintenance)\n+static int crontab_update_schedule(int run_maintenance, int fd, const char *cmd)\n {\n \tint result = 0;\n \tint in_old_region = 0;\n \tstruct child_process crontab_list = CHILD_PROCESS_INIT;\n \tstruct child_process crontab_edit = CHILD_PROCESS_INIT;\n \tFILE *cron_list, *cron_in;\n-\tconst char *crontab_name;\n \tstruct strbuf line = STRBUF_INIT;\n-\tstruct lock_file lk;\n-\tchar *lock_path = xstrfmt(\"%s/schedule\", the_repository->objects->odb->path);\n \n-\tif (hold_lock_file_for_update(&lk, lock_path, LOCK_NO_DEREF) < 0)\n-\t\treturn error(_(\"another process is scheduling background maintenance\"));\n-\n-\tcrontab_name = getenv(\"GIT_TEST_CRONTAB\");\n-\tif (!crontab_name)\n-\t\tcrontab_name = \"crontab\";\n-\n-\tstrvec_split(&crontab_list.args, crontab_name);\n+\tstrvec_split(&crontab_list.args, cmd);\n \tstrvec_push(&crontab_list.args, \"-l\");\n \tcrontab_list.in = -1;\n-\tcrontab_list.out = dup(lk.tempfile->fd);\n+\tcrontab_list.out = dup(fd);\n \tcrontab_list.git_cmd = 0;\n \n-\tif (start_command(&crontab_list)) {\n-\t\tresult = error(_(\"failed to run 'crontab -l'; your system might not support 'cron'\"));\n-\t\tgoto cleanup;\n-\t}\n+\tif (start_command(&crontab_list))\n+\t\treturn error(_(\"failed to run 'crontab -l'; your system might not support 'cron'\"));\n \n \t/* Ignore exit code, as an empty crontab will return error. */\n \tfinish_command(&crontab_list);\n@@ -1531,17 +1519,15 @@ static int update_background_schedule(int run_maintenance)\n \t * Read from the .lock file, filtering out the old\n \t * schedule while appending the new schedule.\n \t */\n-\tcron_list = fdopen(lk.tempfile->fd, \"r\");\n+\tcron_list = fdopen(fd, \"r\");\n \trewind(cron_list);\n \n-\tstrvec_split(&crontab_edit.args, crontab_name);\n+\tstrvec_split(&crontab_edit.args, cmd);\n \tcrontab_edit.in = -1;\n \tcrontab_edit.git_cmd = 0;\n \n-\tif (start_command(&crontab_edit)) {\n-\t\tresult = error(_(\"failed to run 'crontab'; your system might not support 'cron'\"));\n-\t\tgoto cleanup;\n-\t}\n+\tif (start_command(&crontab_edit))\n+\t\treturn error(_(\"failed to run 'crontab'; your system might not support 'cron'\"));\n \n \tcron_in = fdopen(crontab_edit.in, \"w\");\n \tif (!cron_in) {\n@@ -1586,14 +1572,44 @@ static int update_background_schedule(int run_maintenance)\n \tclose(crontab_edit.in);\n \n done_editing:\n-\tif (finish_command(&crontab_edit)) {\n+\tif (finish_command(&crontab_edit))\n \t\tresult = error(_(\"'crontab' died\"));\n-\t\tgoto cleanup;\n+\telse\n+\t\tfclose(cron_list);\n+\treturn result;\n+}\n+\n+static const char platform_scheduler[] = \"crontab\";\n+\n+static int update_background_schedule(int enable)\n+{\n+\tint result;\n+\tconst char *scheduler = platform_scheduler;\n+\tconst char *cmd = scheduler;\n+\tchar *testing;\n+\tstruct lock_file lk;\n+\tchar *lock_path = xstrfmt(\"%s/schedule\", the_repository->objects->odb->path);\n+\n+\ttesting = xstrdup_or_null(getenv(\"GIT_TEST_MAINT_SCHEDULER\"));\n+\tif (testing) {\n+\t\tchar *sep = strchr(testing, ':');\n+\t\tif (!sep)\n+\t\t\tdie(\"GIT_TEST_MAINT_SCHEDULER unparseable: %s\", testing);\n+\t\t*sep = '\\0';\n+\t\tscheduler = testing;\n+\t\tcmd = sep + 1;\n \t}\n-\tfclose(cron_list);\n \n-cleanup:\n+\tif (hold_lock_file_for_update(&lk, lock_path, LOCK_NO_DEREF) < 0)\n+\t\treturn error(_(\"another process is scheduling background maintenance\"));\n+\n+\tif (!strcmp(scheduler, \"crontab\"))\n+\t\tresult = crontab_update_schedule(enable, lk.tempfile->fd, cmd);\n+\telse\n+\t\tdie(\"unknown background scheduler: %s\", scheduler);\n+\n \trollback_lock_file(&lk);\n+\tfree(testing);\n \treturn result;\n }\n \ndiff --git a/t/t7900-maintenance.sh b/t/t7900-maintenance.sh\nindex 20184e96e1..eeb939168d 100755\n--- a/t/t7900-maintenance.sh\n+++ b/t/t7900-maintenance.sh\n@@ -368,7 +368,7 @@ test_expect_success 'register and unregister' '\n '\n \n test_expect_success 'start from empty cron table' '\n-\tGIT_TEST_CRONTAB=\"test-tool crontab cron.txt\" git maintenance start &&\n+\tGIT_TEST_MAINT_SCHEDULER=\"crontab:test-tool crontab cron.txt\" git maintenance start &&\n \n \t# start registers the repo\n \tgit config --get --global maintenance.repo \"$(pwd)\" &&\n@@ -379,19 +379,19 @@ test_expect_success 'start from empty cron table' '\n '\n \n test_expect_success 'stop from existing schedule' '\n-\tGIT_TEST_CRONTAB=\"test-tool crontab cron.txt\" git maintenance stop &&\n+\tGIT_TEST_MAINT_SCHEDULER=\"crontab:test-tool crontab cron.txt\" git maintenance stop &&\n \n \t# stop does not unregister the repo\n \tgit config --get --global maintenance.repo \"$(pwd)\" &&\n \n \t# Operation is idempotent\n-\tGIT_TEST_CRONTAB=\"test-tool crontab cron.txt\" git maintenance stop &&\n+\tGIT_TEST_MAINT_SCHEDULER=\"crontab:test-tool crontab cron.txt\" git maintenance stop &&\n \ttest_must_be_empty cron.txt\n '\n \n test_expect_success 'start preserves existing schedule' '\n \techo \"Important information!\" >cron.txt &&\n-\tGIT_TEST_CRONTAB=\"test-tool crontab cron.txt\" git maintenance start &&\n+\tGIT_TEST_MAINT_SCHEDULER=\"crontab:test-tool crontab cron.txt\" git maintenance start &&\n \tgrep \"Important information!\" cron.txt\n '\n \ndiff --git a/t/test-lib.sh b/t/test-lib.sh\nindex 4a60d1ed76..ddbeee1f5e 100644\n--- a/t/test-lib.sh\n+++ b/t/test-lib.sh\n@@ -1704,7 +1704,8 @@ test_lazy_prereq REBASE_P '\n '\n \n # Ensure that no test accidentally triggers a Git command\n-# that runs 'crontab', affecting a user's cron schedule.\n-# Tests that verify the cron integration must set this locally\n+# that runs the actual maintenance scheduler, affecting a user's\n+# system permanently.\n+# Tests that verify the scheduler integration must set this locally\n # to avoid errors.\n-GIT_TEST_CRONTAB=\"exit 1\"\n+GIT_TEST_MAINT_SCHEDULER=\"none:exit 1\"\n-- \ngitgitgadget\n\n"},{"id":"410131","messageId":"99170df4626544c1dc26d2e188b215a776140a32.1605647598.git.gitgitgadget@gmail.com","threadId":"54562","inReplyTo":"pull.776.v4.git.1605647598.gitgitgadget@gmail.com","subject":"[PATCH v4 2/4] maintenance: include 'cron' details in docs","fromName":"Derrick Stolee via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2020-11-17T21:13:16Z","receivedAt":"2020-11-17T21:13:45Z","isPatch":true,"sender":{"key":"stolee@gmail.com","avatar":"https://avatars.githubusercontent.com/u/570044?v=4"},"body":"From: Derrick Stolee \n\nAdvanced and expert users may want to know how 'git maintenance start'\nschedules background maintenance in order to customize their own\nschedules beyond what the maintenance.* config values allow. Start a new\nset of sections in git-maintenance.txt that describe how 'cron' is used\nto run these tasks.\n\nThis is particularly valuable for users who want to inspect what Git is\ndoing or for users who want to customize the schedule further. Having a\nbaseline can provide a way forward for users who have never worked with\ncron schedules.\n\nHelped-by: Eric Sunshine \nSigned-off-by: Derrick Stolee \n---\n Documentation/git-maintenance.txt | 54 +++++++++++++++++++++++++++++++\n 1 file changed, 54 insertions(+)\n\ndiff --git a/Documentation/git-maintenance.txt b/Documentation/git-maintenance.txt\nindex 6fec1eb8dc..4c7aac877d 100644\n--- a/Documentation/git-maintenance.txt\n+++ b/Documentation/git-maintenance.txt\n@@ -218,6 +218,60 @@ Further, the `git gc` command should not be combined with\n but does not take the lock in the same way as `git maintenance run`. If\n possible, use `git maintenance run --task=gc` instead of `git gc`.\n \n+The following sections describe the mechanisms put in place to run\n+background maintenance by `git maintenance start` and how to customize\n+them.\n+\n+BACKGROUND MAINTENANCE ON POSIX SYSTEMS\n+---------------------------------------\n+\n+The standard mechanism for scheduling background tasks on POSIX systems\n+is `cron`. This tool executes commands based on a given schedule. The\n+current list of user-scheduled tasks can be found by running `crontab -l`.\n+The schedule written by `git maintenance start` is similar to this:\n+\n+-----------------------------------------------------------------------\n+# BEGIN GIT MAINTENANCE SCHEDULE\n+# The following schedule was created by Git\n+# Any edits made in this region might be\n+# replaced in the future by a Git command.\n+\n+0 1-23 * * * \"//git\" --exec-path=\"/\" for-each-repo --config=maintenance.repo maintenance run --schedule=hourly\n+0 0 * * 1-6 \"//git\" --exec-path=\"/\" for-each-repo --config=maintenance.repo maintenance run --schedule=daily\n+0 0 * * 0 \"//git\" --exec-path=\"/\" for-each-repo --config=maintenance.repo maintenance run --schedule=weekly\n+\n+# END GIT MAINTENANCE SCHEDULE\n+-----------------------------------------------------------------------\n+\n+The comments are used as a region to mark the schedule as written by Git.\n+Any modifications within this region will be completely deleted by\n+`git maintenance stop` or overwritten by `git maintenance start`.\n+\n+The `` string is loaded to specifically use the location for the\n+`git` executable used in the `git maintenance start` command. This allows\n+for multiple versions to be compatible. However, if the same user runs\n+`git maintenance start` with multiple Git executables, then only the\n+latest executable will be used.\n+\n+These commands use `git for-each-repo --config=maintenance.repo` to run\n+`git maintenance run --schedule=` on each repository listed in\n+the multi-valued `maintenance.repo` config option. These are typically\n+loaded from the user-specific global config located at `~/.gitconfig`.\n+The `git maintenance` process then determines which maintenance tasks\n+are configured to run on each repository with each `` using\n+the `maintenance..schedule` config options. These values are loaded\n+from the global or repository config values.\n+\n+If the config values are insufficient to achieve your desired background\n+maintenance schedule, then you can create your own schedule. If you run\n+`crontab -e`, then an editor will load with your user-specific `cron`\n+schedule. In that editor, you can add your own schedule lines. You could\n+start by adapting the default schedule listed earlier, or you could read\n+https://man7.org/linux/man-pages/man5/crontab.5.html[the `crontab` documentation]\n+for advanced scheduling techniques. Please do use the full path and\n+`--exec-path` techniques from the default schedule to ensure you are\n+executing the correct binaries in your schedule.\n+\n \n GIT\n ---\n-- \ngitgitgadget\n\n"},{"id":"410133","messageId":"ed0a0011fb0b06780bc6ce2920f17cad46eccdf9.1605647598.git.gitgitgadget@gmail.com","threadId":"54562","inReplyTo":"pull.776.v4.git.1605647598.gitgitgadget@gmail.com","subject":"[PATCH v4 3/4] maintenance: use launchctl on macOS","fromName":"Derrick Stolee via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2020-11-17T21:13:17Z","receivedAt":"2020-11-17T21:13:45Z","isPatch":true,"sender":{"key":"stolee@gmail.com","avatar":"https://avatars.githubusercontent.com/u/570044?v=4"},"body":"From: Derrick Stolee \n\nThe existing mechanism for scheduling background maintenance is done\nthrough cron. The 'crontab -e' command allows updating the schedule\nwhile cron itself runs those commands. While this is technically\nsupported by macOS, it has some significant deficiencies:\n\n1. Every run of 'crontab -e' must request elevated privileges through\n the user interface. When running 'git maintenance start' from the\n Terminal app, it presents a dialog box saying \"Terminal.app would\n like to administer your computer. Administration can include\n modifying passwords, networking, and system settings.\" This is more\n alarming than what we are hoping to achieve. If this alert had some\n information about how \"git\" is trying to run \"crontab\" then we would\n have some reason to believe that this dialog might be fine. However,\n it also doesn't help that some scenarios just leave Git waiting for\n a response without presenting anything to the user. I experienced\n this when executing the command from a Bash terminal view inside\n Visual Studio Code.\n\n2. While cron initializes a user environment enough for \"git config\n --global --show-origin\" to show the correct config file information,\n it does not set up the environment enough for Git Credential Manager\n Core to load credentials during a 'prefetch' task. My prefetches\n against private repositories required re-authenticating through UI\n pop-ups in a way that should not be required.\n\nThe solution is to switch from cron to the Apple-recommended [1]\n'launchd' tool.\n\n[1] https://developer.apple.com/library/archive/documentation/MacOSX/Conceptual/BPSystemStartup/Chapters/ScheduledJobs.html\n\nThe basics of this tool is that we need to create XML-formatted\n\"plist\" files inside \"~/Library/LaunchAgents/\" and then use the\n'launchctl' tool to make launchd aware of them. The plist files\ninclude all of the scheduling information, along with the command-line\narguments split across an array of tags.\n\nFor example, here is my plist file for the weekly scheduled tasks:\n\n\n\n\nLabelorg.git-scm.git.weekly\nProgramArguments\n\n/usr/local/libexec/git-core/git\n--exec-path=/usr/local/libexec/git-core\nfor-each-repo\n--config=maintenance.repo\nmaintenance\nrun\n--schedule=weekly\n\nStartCalendarInterval\n\n\nDay0\nHour0\nMinute0\n\n\n\n\n\nThe schedules for the daily and hourly tasks are more complicated\nsince we need to use an array for the StartCalendarInterval with\nan entry for each of the six days other than the 0th day (to avoid\ncolliding with the weekly task), and each of the 23 hours other\nthan the 0th hour (to avoid colliding with the daily task).\n\nThe \"Label\" value is currently filled with \"org.git-scm.git.X\"\nwhere X is the frequency. We need a different plist file for each\nfrequency.\n\nThe launchctl command needs to be aligned with a user id in order\nto initialize the command environment. This must be done using\nthe 'launchctl bootstrap' subcommand. This subcommand is new as\nof macOS 10.11, which was released in September 2015. Before that\nrelease the 'launchctl load' subcommand was recommended. The best\nsource of information on this transition I have seen is available\nat [2]. The current design does not preclude a future version that\ndetects the available fatures of 'launchctl' to use the older\ncommands. However, it is best to rely on the newest version since\nApple might completely remove the deprecated version on short\nnotice.\n\n[2] https://babodee.wordpress.com/2016/04/09/launchctl-2-0-syntax/\n\nTo remove a schedule, we must run 'launchctl bootout' with a valid\nplist file. We also need to 'bootout' a task before the 'bootstrap'\nsubcommand will succeed, if such a task already exists.\n\nThe need for a user id requires us to run 'id -u' which works on\nPOSIX systems but not Windows. The test therefore has a prerequisite\nthat we are not on Windows. The cross-platform logic still allows us to\ntest the macOS logic on a Linux machine.\n\nWe can verify the commands that were run by 'git maintenance start'\nand 'git maintenance stop' by injecting a script that writes the\ncommand-line arguments into GIT_TEST_MAINT_SCHEDULER.\n\nAn earlier version of this patch accidentally had an opening\n\"\" tag when it should have had a closing \"\" tag. This\nwas caught during manual testing with actual 'launchctl' commands,\nbut we do not want to update developers' tasks when running tests.\nIt appears that macOS includes the \"xmllint\" tool which can verify\nthe XML format. This is useful for any system that might contain\nthe tool, so use it whenever it is available.\n\nCo-authored-by: Eric Sunshine \nSigned-off-by: Eric Sunshine \nSigned-off-by: Derrick Stolee \n---\n Documentation/git-maintenance.txt | 40 +++++++\n builtin/gc.c | 188 +++++++++++++++++++++++++++++-\n t/t7900-maintenance.sh | 58 +++++++++\n 3 files changed, 285 insertions(+), 1 deletion(-)\n\ndiff --git a/Documentation/git-maintenance.txt b/Documentation/git-maintenance.txt\nindex 4c7aac877d..f2d59f2bcc 100644\n--- a/Documentation/git-maintenance.txt\n+++ b/Documentation/git-maintenance.txt\n@@ -273,6 +273,46 @@ for advanced scheduling techniques. Please do use the full path and\n executing the correct binaries in your schedule.\n \n \n+BACKGROUND MAINTENANCE ON MACOS SYSTEMS\n+---------------------------------------\n+\n+While macOS technically supports `cron`, using `crontab -e` requires\n+elevated privileges and the executed process does not have a full user\n+context. Without a full user context, Git and its credential helpers\n+cannot access stored credentials, so some maintenance tasks are not\n+functional.\n+\n+Instead, `git maintenance start` interacts with the `launchctl` tool,\n+which is the recommended way to schedule timed jobs in macOS. Scheduling\n+maintenance through `git maintenance (start|stop)` requires some\n+`launchctl` features available only in macOS 10.11 or later.\n+\n+Your user-specific scheduled tasks are stored as XML-formatted `.plist`\n+files in `~/Library/LaunchAgents/`. You can see the currently-registered\n+tasks using the following command:\n+\n+-----------------------------------------------------------------------\n+$ ls ~/Library/LaunchAgents/org.git-scm.git*\n+org.git-scm.git.daily.plist\n+org.git-scm.git.hourly.plist\n+org.git-scm.git.weekly.plist\n+-----------------------------------------------------------------------\n+\n+One task is registered for each `--schedule=` option. To\n+inspect how the XML format describes each schedule, open one of these\n+`.plist` files in an editor and inspect the `` element following\n+the `StartCalendarInterval` element.\n+\n+`git maintenance start` will overwrite these files and register the\n+tasks again with `launchctl`, so any customizations should be done by\n+creating your own `.plist` files with distinct names. Similarly, the\n+`git maintenance stop` command will unregister the tasks with `launchctl`\n+and delete the `.plist` files.\n+\n+To create more advanced customizations to your background tasks, see\n+launchctl.plist(5) for more information.\n+\n+\n GIT\n ---\n Part of the linkgit:git[1] suite\ndiff --git a/builtin/gc.c b/builtin/gc.c\nindex 18ae7f7138..782769f243 100644\n--- a/builtin/gc.c\n+++ b/builtin/gc.c\n@@ -1491,6 +1491,186 @@ static int maintenance_unregister(void)\n \treturn run_command(&config_unset);\n }\n \n+static const char *get_frequency(enum schedule_priority schedule)\n+{\n+\tswitch (schedule) {\n+\tcase SCHEDULE_HOURLY:\n+\t\treturn \"hourly\";\n+\tcase SCHEDULE_DAILY:\n+\t\treturn \"daily\";\n+\tcase SCHEDULE_WEEKLY:\n+\t\treturn \"weekly\";\n+\tdefault:\n+\t\tBUG(\"invalid schedule %d\", schedule);\n+\t}\n+}\n+\n+static char *launchctl_service_name(const char *frequency)\n+{\n+\tstruct strbuf label = STRBUF_INIT;\n+\tstrbuf_addf(&label, \"org.git-scm.git.%s\", frequency);\n+\treturn strbuf_detach(&label, NULL);\n+}\n+\n+static char *launchctl_service_filename(const char *name)\n+{\n+\tchar *expanded;\n+\tstruct strbuf filename = STRBUF_INIT;\n+\tstrbuf_addf(&filename, \"~/Library/LaunchAgents/%s.plist\", name);\n+\n+\texpanded = expand_user_path(filename.buf, 1);\n+\tif (!expanded)\n+\t\tdie(_(\"failed to expand path '%s'\"), filename.buf);\n+\n+\tstrbuf_release(&filename);\n+\treturn expanded;\n+}\n+\n+static char *launchctl_get_uid(void)\n+{\n+\treturn xstrfmt(\"gui/%d\", getuid());\n+}\n+\n+static int launchctl_boot_plist(int enable, const char *filename, const char *cmd)\n+{\n+\tint result;\n+\tstruct child_process child = CHILD_PROCESS_INIT;\n+\tchar *uid = launchctl_get_uid();\n+\n+\tstrvec_split(&child.args, cmd);\n+\tif (enable)\n+\t\tstrvec_push(&child.args, \"bootstrap\");\n+\telse\n+\t\tstrvec_push(&child.args, \"bootout\");\n+\tstrvec_push(&child.args, uid);\n+\tstrvec_push(&child.args, filename);\n+\n+\tchild.no_stderr = 1;\n+\tchild.no_stdout = 1;\n+\n+\tif (start_command(&child))\n+\t\tdie(_(\"failed to start launchctl\"));\n+\n+\tresult = finish_command(&child);\n+\n+\tfree(uid);\n+\treturn result;\n+}\n+\n+static int launchctl_remove_plist(enum schedule_priority schedule, const char *cmd)\n+{\n+\tconst char *frequency = get_frequency(schedule);\n+\tchar *name = launchctl_service_name(frequency);\n+\tchar *filename = launchctl_service_filename(name);\n+\tint result = launchctl_boot_plist(0, filename, cmd);\n+\tunlink(filename);\n+\tfree(filename);\n+\tfree(name);\n+\treturn result;\n+}\n+\n+static int launchctl_remove_plists(const char *cmd)\n+{\n+\treturn launchctl_remove_plist(SCHEDULE_HOURLY, cmd) ||\n+\t\tlaunchctl_remove_plist(SCHEDULE_DAILY, cmd) ||\n+\t\tlaunchctl_remove_plist(SCHEDULE_WEEKLY, cmd);\n+}\n+\n+static int launchctl_schedule_plist(const char *exec_path, enum schedule_priority schedule, const char *cmd)\n+{\n+\tFILE *plist;\n+\tint i;\n+\tconst char *preamble, *repeat;\n+\tconst char *frequency = get_frequency(schedule);\n+\tchar *name = launchctl_service_name(frequency);\n+\tchar *filename = launchctl_service_filename(name);\n+\n+\tif (safe_create_leading_directories(filename))\n+\t\tdie(_(\"failed to create directories for '%s'\"), filename);\n+\tplist = xfopen(filename, \"w\");\n+\n+\tpreamble = \"\\n\"\n+\t\t \"\\n\"\n+\t\t \"\"\n+\t\t \"\\n\"\n+\t\t \"Label%s\\n\"\n+\t\t \"ProgramArguments\\n\"\n+\t\t \"\\n\"\n+\t\t \"%s/git\\n\"\n+\t\t \"--exec-path=%s\\n\"\n+\t\t \"for-each-repo\\n\"\n+\t\t \"--config=maintenance.repo\\n\"\n+\t\t \"maintenance\\n\"\n+\t\t \"run\\n\"\n+\t\t \"--schedule=%s\\n\"\n+\t\t \"\\n\"\n+\t\t \"StartCalendarInterval\\n\"\n+\t\t \"\\n\";\n+\tfprintf(plist, preamble, name, exec_path, exec_path, frequency);\n+\n+\tswitch (schedule) {\n+\tcase SCHEDULE_HOURLY:\n+\t\trepeat = \"\\n\"\n+\t\t\t \"Hour%d\\n\"\n+\t\t\t \"Minute0\\n\"\n+\t\t\t \"\\n\";\n+\t\tfor (i = 1; i <= 23; i++)\n+\t\t\tfprintf(plist, repeat, i);\n+\t\tbreak;\n+\n+\tcase SCHEDULE_DAILY:\n+\t\trepeat = \"\\n\"\n+\t\t\t \"Day%d\\n\"\n+\t\t\t \"Hour0\\n\"\n+\t\t\t \"Minute0\\n\"\n+\t\t\t \"\\n\";\n+\t\tfor (i = 1; i <= 6; i++)\n+\t\t\tfprintf(plist, repeat, i);\n+\t\tbreak;\n+\n+\tcase SCHEDULE_WEEKLY:\n+\t\tfprintf(plist,\n+\t\t\t\"\\n\"\n+\t\t\t\"Day0\\n\"\n+\t\t\t\"Hour0\\n\"\n+\t\t\t\"Minute0\\n\"\n+\t\t\t\"\\n\");\n+\t\tbreak;\n+\n+\tdefault:\n+\t\t/* unreachable */\n+\t\tbreak;\n+\t}\n+\tfprintf(plist, \"\\n\\n\\n\");\n+\tfclose(plist);\n+\n+\t/* bootout might fail if not already running, so ignore */\n+\tlaunchctl_boot_plist(0, filename, cmd);\n+\tif (launchctl_boot_plist(1, filename, cmd))\n+\t\tdie(_(\"failed to bootstrap service %s\"), filename);\n+\n+\tfree(filename);\n+\tfree(name);\n+\treturn 0;\n+}\n+\n+static int launchctl_add_plists(const char *cmd)\n+{\n+\tconst char *exec_path = git_exec_path();\n+\n+\treturn launchctl_schedule_plist(exec_path, SCHEDULE_HOURLY, cmd) ||\n+\t\tlaunchctl_schedule_plist(exec_path, SCHEDULE_DAILY, cmd) ||\n+\t\tlaunchctl_schedule_plist(exec_path, SCHEDULE_WEEKLY, cmd);\n+}\n+\n+static int launchctl_update_schedule(int run_maintenance, int fd, const char *cmd)\n+{\n+\tif (run_maintenance)\n+\t\treturn launchctl_add_plists(cmd);\n+\telse\n+\t\treturn launchctl_remove_plists(cmd);\n+}\n+\n #define BEGIN_LINE \"# BEGIN GIT MAINTENANCE SCHEDULE\"\n #define END_LINE \"# END GIT MAINTENANCE SCHEDULE\"\n \n@@ -1579,7 +1759,11 @@ static int crontab_update_schedule(int run_maintenance, int fd, const char *cmd)\n \treturn result;\n }\n \n+#if defined(__APPLE__)\n+static const char platform_scheduler[] = \"launchctl\";\n+#else\n static const char platform_scheduler[] = \"crontab\";\n+#endif\n \n static int update_background_schedule(int enable)\n {\n@@ -1603,7 +1787,9 @@ static int update_background_schedule(int enable)\n \tif (hold_lock_file_for_update(&lk, lock_path, LOCK_NO_DEREF) < 0)\n \t\treturn error(_(\"another process is scheduling background maintenance\"));\n \n-\tif (!strcmp(scheduler, \"crontab\"))\n+\tif (!strcmp(scheduler, \"launchctl\"))\n+\t\tresult = launchctl_update_schedule(enable, lk.tempfile->fd, cmd);\n+\telse if (!strcmp(scheduler, \"crontab\"))\n \t\tresult = crontab_update_schedule(enable, lk.tempfile->fd, cmd);\n \telse\n \t\tdie(\"unknown background scheduler: %s\", scheduler);\ndiff --git a/t/t7900-maintenance.sh b/t/t7900-maintenance.sh\nindex eeb939168d..6d37312901 100755\n--- a/t/t7900-maintenance.sh\n+++ b/t/t7900-maintenance.sh\n@@ -7,6 +7,19 @@ test_description='git maintenance builtin'\n GIT_TEST_COMMIT_GRAPH=0\n GIT_TEST_MULTI_PACK_INDEX=0\n \n+test_lazy_prereq XMLLINT '\n+\txmllint --version\n+'\n+\n+test_xmllint () {\n+\tif test_have_prereq XMLLINT\n+\tthen\n+\t\txmllint --noout \"$@\"\n+\telse\n+\t\ttrue\n+\tfi\n+}\n+\n test_expect_success 'help text' '\n \ttest_expect_code 129 git maintenance -h 2>err &&\n \ttest_i18ngrep \"usage: git maintenance \" err &&\n@@ -395,6 +408,51 @@ test_expect_success 'start preserves existing schedule' '\n \tgrep \"Important information!\" cron.txt\n '\n \n+test_expect_success !MINGW 'start and stop macOS maintenance' '\n+\tuid=$(id -u) &&\n+\n+\twrite_script print-args <<-\\EOF &&\n+\techo $* >>args\n+\tEOF\n+\n+\trm -f args &&\n+\tGIT_TEST_MAINT_SCHEDULER=launchctl:./print-args git maintenance start &&\n+\n+\t# start registers the repo\n+\tgit config --get --global maintenance.repo \"$(pwd)\" &&\n+\n+\tls \"$HOME/Library/LaunchAgents\" >actual &&\n+\tcat >expect <<-\\EOF &&\n+\torg.git-scm.git.daily.plist\n+\torg.git-scm.git.hourly.plist\n+\torg.git-scm.git.weekly.plist\n+\tEOF\n+\ttest_cmp expect actual &&\n+\n+\trm expect &&\n+\tfor frequency in hourly daily weekly\n+\tdo\n+\t\tPLIST=\"$HOME/Library/LaunchAgents/org.git-scm.git.$frequency.plist\" &&\n+\t\ttest_xmllint \"$PLIST\" &&\n+\t\tgrep schedule=$frequency \"$PLIST\" &&\n+\t\techo \"bootout gui/$uid $PLIST\" >>expect &&\n+\t\techo \"bootstrap gui/$uid $PLIST\" >>expect || return 1\n+\tdone &&\n+\ttest_cmp expect args &&\n+\n+\trm -f args &&\n+\tGIT_TEST_MAINT_SCHEDULER=launchctl:./print-args git maintenance stop &&\n+\n+\t# stop does not unregister the repo\n+\tgit config --get --global maintenance.repo \"$(pwd)\" &&\n+\n+\tprintf \"bootout gui/$uid $HOME/Library/LaunchAgents/org.git-scm.git.%s.plist\\n\" \\\n+\t\thourly daily weekly >expect &&\n+\ttest_cmp expect args &&\n+\tls \"$HOME/Library/LaunchAgents\" >actual &&\n+\ttest_line_count = 0 actual\n+'\n+\n test_expect_success 'register preserves existing strategy' '\n \tgit config maintenance.strategy none &&\n \tgit maintenance register &&\n-- \ngitgitgadget\n\n"},{"id":"410134","messageId":"pull.776.v4.git.1605647598.gitgitgadget@gmail.com","threadId":"54562","inReplyTo":"pull.776.v3.git.1605276024.gitgitgadget@gmail.com","subject":"[PATCH v4 0/4] Maintenance IV: Platform-specific background maintenance","fromName":"Derrick Stolee via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2020-11-17T21:13:14Z","receivedAt":"2020-11-17T21:13:45Z","isPatch":true,"sender":{"key":"stolee@gmail.com","avatar":"https://avatars.githubusercontent.com/u/570044?v=4"},"body":"This is based on ds/maintenance-part-3.\n\nAfter sitting with the background maintenance as it has been cooking, I\nwanted to come back around and implement the background maintenance for\nWindows. However, I noticed that there were some things bothering me with\nbackground maintenance on my macOS machine. These are detailed in PATCH 3,\nbut the tl;dr is that 'cron' is not recommended by Apple and instead\n'launchd' satisfies our needs.\n\nThis series implements the background scheduling so git maintenance\n(start|stop) works on those platforms. I've been operating with these\nschedules for a while now without the problems described in the patches.\n\nThere is a particularly annoying case about console windows popping up on\nWindows, but PATCH 4 describes a plan to get around that.\n\nUpdates in V4\n=============\n\n * Eric did an excellent job providing a patch that cleans up several parts\n of my series. The most impressive is his mechanism for testing the\n platform-specific Git logic in a way that is (mostly) platform-agnostic.\n \n \n * Windows doesn't have the 'id' command, so we cannot run the macOS\n platform test on Windows.\n \n \n * I noticed far too late that while my example XML files had been edited\n with UTF-8 encoding, Git is actually writing them as US-ASCII. Somehow \n xmllint and launchd are not complaining, but schtasks does complain.\n Unfortunately, I cannot find a way to catch this problem other than to\n install my tip version on all three platforms and go through the entire \n git maintenance start process, and double-check that the processes are\n running on the hour.\n \n \n\nHere is a diff from the tip of v3 + Eric's patch to the tip of v4:\n\ndiff --git a/builtin/gc.c b/builtin/gc.c\nindex 955d4b3baf..1a3725429c 100644\n--- a/builtin/gc.c\n+++ b/builtin/gc.c\n@@ -1642,13 +1642,13 @@ static int launchctl_schedule_plist(const char *exec_path, enum schedule_priorit\n break;\n }\n fprintf(plist, \"\\n\\n\\n\");\n+ fclose(plist);\n\n /* bootout might fail if not already running, so ignore */\n launchctl_boot_plist(0, filename, cmd);\n if (launchctl_boot_plist(1, filename, cmd))\n die(_(\"failed to bootstrap service %s\"), filename);\n\n- fclose(plist);\n free(filename);\n free(name);\n return 0;\n@@ -1707,25 +1707,27 @@ static int schtasks_schedule_task(const char *exec_path, enum schedule_priority\n int result;\n struct child_process child = CHILD_PROCESS_INIT;\n const char *xml;\n- char *xmlpath, *tempDir;\n- FILE *xmlfp;\n+ char *xmlpath;\n+ struct tempfile *tfile;\n const char *frequency = get_frequency(schedule);\n char *name = schtasks_task_name(frequency);\n\n- tempDir = xstrfmt(\"%s/temp\", the_repository->objects->odb->path);\n- xmlpath = xstrfmt(\"%s/schedule-%s.xml\", tempDir, frequency);\n- safe_create_leading_directories(xmlpath);\n- xmlfp = xfopen(xmlpath, \"w\");\n+ xmlpath = xstrfmt(\"%s/schedule-%s.xml\",\n+ the_repository->objects->odb->path,\n+ frequency);\n+ tfile = create_tempfile(xmlpath);\n+ if (!tfile || !fdopen_tempfile(tfile, \"w\"))\n+ die(_(\"failed to create '%s'\"), xmlpath);\n\n- xml = \"\\n\"\n+ xml = \"\\n\"\n \"\\n\"\n \"\\n\"\n \"\\n\";\n- fputs(xml, xmlfp);\n+ fputs(xml, tfile->fp);\n\n switch (schedule) {\n case SCHEDULE_HOURLY:\n- fprintf(xmlfp,\n+ fprintf(tfile->fp,\n \"2020-01-01T01:00:00\\n\"\n \"true\\n\"\n \"\\n\"\n@@ -1739,7 +1741,7 @@ static int schtasks_schedule_task(const char *exec_path, enum schedule_priority\n break;\n\n case SCHEDULE_DAILY:\n- fprintf(xmlfp,\n+ fprintf(tfile->fp,\n \"2020-01-01T00:00:00\\n\"\n \"true\\n\"\n \"\\n\"\n@@ -1756,7 +1758,7 @@ static int schtasks_schedule_task(const char *exec_path, enum schedule_priority\n break;\n\n case SCHEDULE_WEEKLY:\n- fprintf(xmlfp,\n+ fprintf(tfile->fp,\n \"2020-01-01T00:00:00\\n\"\n \"true\\n\"\n \"\\n\"\n@@ -1771,7 +1773,7 @@ static int schtasks_schedule_task(const char *exec_path, enum schedule_priority\n break;\n }\n\n- xml= \"\\n\"\n+ xml = \"\\n\"\n \"\\n\"\n \"\\n\"\n \"\\n\"\n@@ -1795,11 +1797,10 @@ static int schtasks_schedule_task(const char *exec_path, enum schedule_priority\n \"\\n\"\n \"\\n\"\n \"\\n\";\n- fprintf(xmlfp, xml, exec_path, exec_path, frequency);\n- fclose(xmlfp);\n-\n+ fprintf(tfile->fp, xml, exec_path, exec_path, frequency);\n strvec_split(&child.args, cmd);\n strvec_pushl(&child.args, \"/create\", \"/tn\", name, \"/f\", \"/xml\", xmlpath, NULL);\n+ close_tempfile_gently(tfile);\n\n child.no_stdout = 1;\n child.no_stderr = 1;\n@@ -1808,8 +1809,7 @@ static int schtasks_schedule_task(const char *exec_path, enum schedule_priority\n die(_(\"failed to start schtasks\"));\n result = finish_command(&child);\n\n- unlink(xmlpath);\n- rmdir(tempDir);\n+ delete_tempfile(&tfile);\n free(xmlpath);\n free(name);\n return result;\n@@ -1850,9 +1850,8 @@ static int crontab_update_schedule(int run_maintenance, int fd, const char *cmd)\n crontab_list.out = dup(fd);\n crontab_list.git_cmd = 0;\n\n- if (start_command(&crontab_list)) {\n+ if (start_command(&crontab_list))\n return error(_(\"failed to run 'crontab -l'; your system might not support 'cron'\"));\n- }\n\n /* Ignore exit code, as an empty crontab will return error. */\n finish_command(&crontab_list);\n@@ -1868,9 +1867,8 @@ static int crontab_update_schedule(int run_maintenance, int fd, const char *cmd)\n crontab_edit.in = -1;\n crontab_edit.git_cmd = 0;\n\n- if (start_command(&crontab_edit)) {\n+ if (start_command(&crontab_edit))\n return error(_(\"failed to run 'crontab'; your system might not support 'cron'\"));\n- }\n\n cron_in = fdopen(crontab_edit.in, \"w\");\n if (!cron_in) {\ndiff --git a/t/t7900-maintenance.sh b/t/t7900-maintenance.sh\nindex e92946c10a..a26ff22541 100755\n--- a/t/t7900-maintenance.sh\n+++ b/t/t7900-maintenance.sh\n@@ -408,7 +408,7 @@ test_expect_success 'start preserves existing schedule' '\n grep \"Important information!\" cron.txt\n '\n\n-test_expect_success 'start and stop macOS maintenance' '\n+test_expect_success !MINGW 'start and stop macOS maintenance' '\n uid=$(id -u) &&\n\n write_script print-args <<-\\EOF &&\n@@ -421,7 +421,6 @@ test_expect_success 'start and stop macOS maintenance' '\n # start registers the repo\n git config --get --global maintenance.repo \"$(pwd)\" &&\n\n- # ~/Library/LaunchAgents\n ls \"$HOME/Library/LaunchAgents\" >actual &&\n cat >expect <<-\\EOF &&\n org.git-scm.git.daily.plist\n@@ -468,12 +467,12 @@ test_expect_success 'start and stop Windows maintenance' '\n EOF\n\n rm -f args &&\n- GIT_TEST_MAINT_SCHEDULER=\"schtasks:/bin/sh print-args\" git maintenance start &&\n+ GIT_TEST_MAINT_SCHEDULER=\"schtasks:./print-args\" git maintenance start &&\n\n # start registers the repo\n git config --get --global maintenance.repo \"$(pwd)\" &&\n\n- printf \"/create /tn Git Maintenance (%s) /f /xml .git/objects/temp/schedule-%s.xml\\n\" \\\n+ printf \"/create /tn Git Maintenance (%s) /f /xml .git/objects/schedule-%s.xml\\n\" \\\n hourly hourly daily daily weekly weekly >expect &&\n test_cmp expect args &&\n\n@@ -483,7 +482,7 @@ test_expect_success 'start and stop Windows maintenance' '\n done &&\n\n rm -f args &&\n- GIT_TEST_MAINT_SCHEDULER=\"schtasks:/bin/sh print-args\" git maintenance stop &&\n+ GIT_TEST_MAINT_SCHEDULER=\"schtasks:./print-args\" git maintenance stop &&\n\n # stop does not unregister the repo\n git config --get --global maintenance.repo \"$(pwd)\" &&\ndiff --git a/t/test-lib.sh b/t/test-lib.sh\nindex 4a60d1ed76..ddbeee1f5e 100644\n--- a/t/test-lib.sh\n+++ b/t/test-lib.sh\n@@ -1704,7 +1704,8 @@ test_lazy_prereq REBASE_P '\n '\n\n # Ensure that no test accidentally triggers a Git command\n-# that runs 'crontab', affecting a user's cron schedule.\n-# Tests that verify the cron integration must set this locally\n+# that runs the actual maintenance scheduler, affecting a user's\n+# system permanently.\n+# Tests that verify the scheduler integration must set this locally\n # to avoid errors.\n-GIT_TEST_CRONTAB=\"exit 1\"\n+GIT_TEST_MAINT_SCHEDULER=\"none:exit 1\"\n\nThanks, -Stolee\n\ncc: jrnieder@gmail.com [jrnieder@gmail.com], jonathantanmy@google.com\n[jonathantanmy@google.com], sluongng@gmail.com [sluongng@gmail.com]cc:\nDerrick Stolee stolee@gmail.com [stolee@gmail.com]cc: Đoàn Trần Công Danh \ncongdanhqx@gmail.com [congdanhqx@gmail.com]cc: Martin Ågren \nmartin.agren@gmail.com [martin.agren@gmail.com]cc: Eric Sunshine \nsunshine@sunshineco.com [sunshine@sunshineco.com]cc: Derrick Stolee \nstolee@gmail.com [stolee@gmail.com]\n\nDerrick Stolee (4):\n maintenance: extract platform-specific scheduling\n maintenance: include 'cron' details in docs\n maintenance: use launchctl on macOS\n maintenance: use Windows scheduled tasks\n\n Documentation/git-maintenance.txt | 116 ++++++++\n builtin/gc.c | 421 ++++++++++++++++++++++++++++--\n t/t7900-maintenance.sh | 106 +++++++-\n t/test-lib.sh | 7 +-\n 4 files changed, 616 insertions(+), 34 deletions(-)\n\n\nbase-commit: 0016b618182f642771dc589cf0090289f9fe1b4f\nPublished-As: https://github.com/gitgitgadget/git/releases/tag/pr-776%2Fderrickstolee%2Fmaintenance%2FmacOS-v4\nFetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-776/derrickstolee/maintenance/macOS-v4\nPull-Request: https://github.com/gitgitgadget/git/pull/776\n\nRange-diff vs v3:\n\n 1: d35f1aa162 ! 1: 4807342b00 maintenance: extract platform-specific scheduling\n @@ Commit message\n swapped at compile time with new implementations on specialized\n platforms.\n \n + As we add this generality, rename GIT_TEST_CRONTAB to\n + GIT_TEST_MAINT_SCHEDULER. Further, this variable is now parsed as\n + \":\" so we can test platform-specific scheduling\n + logic even when not on the correct platform. By specifying the\n + in this string, we will be able to test all three sets of\n + Git logic from a Linux machine.\n +\n + Co-authored-by: Eric Sunshine \n + Signed-off-by: Eric Sunshine \n Signed-off-by: Derrick Stolee \n \n ## builtin/gc.c ##\n @@ builtin/gc.c: static int maintenance_unregister(void)\n #define END_LINE \"# END GIT MAINTENANCE SCHEDULE\"\n \n -static int update_background_schedule(int run_maintenance)\n -+static int platform_update_schedule(int run_maintenance, int fd)\n ++static int crontab_update_schedule(int run_maintenance, int fd, const char *cmd)\n {\n \tint result = 0;\n \tint in_old_region = 0;\n -@@ builtin/gc.c: static int update_background_schedule(int run_maintenance)\n + \tstruct child_process crontab_list = CHILD_PROCESS_INIT;\n + \tstruct child_process crontab_edit = CHILD_PROCESS_INIT;\n \tFILE *cron_list, *cron_in;\n - \tconst char *crontab_name;\n +-\tconst char *crontab_name;\n \tstruct strbuf line = STRBUF_INIT;\n -\tstruct lock_file lk;\n -\tchar *lock_path = xstrfmt(\"%s/schedule\", the_repository->objects->odb->path);\n --\n + \n -\tif (hold_lock_file_for_update(&lk, lock_path, LOCK_NO_DEREF) < 0)\n -\t\treturn error(_(\"another process is scheduling background maintenance\"));\n - \n - \tcrontab_name = getenv(\"GIT_TEST_CRONTAB\");\n - \tif (!crontab_name)\n -@@ builtin/gc.c: static int update_background_schedule(int run_maintenance)\n - \tstrvec_split(&crontab_list.args, crontab_name);\n +-\n +-\tcrontab_name = getenv(\"GIT_TEST_CRONTAB\");\n +-\tif (!crontab_name)\n +-\t\tcrontab_name = \"crontab\";\n +-\n +-\tstrvec_split(&crontab_list.args, crontab_name);\n ++\tstrvec_split(&crontab_list.args, cmd);\n \tstrvec_push(&crontab_list.args, \"-l\");\n \tcrontab_list.in = -1;\n -\tcrontab_list.out = dup(lk.tempfile->fd);\n +\tcrontab_list.out = dup(fd);\n \tcrontab_list.git_cmd = 0;\n \n - \tif (start_command(&crontab_list)) {\n +-\tif (start_command(&crontab_list)) {\n -\t\tresult = error(_(\"failed to run 'crontab -l'; your system might not support 'cron'\"));\n -\t\tgoto cleanup;\n +-\t}\n ++\tif (start_command(&crontab_list))\n +\t\treturn error(_(\"failed to run 'crontab -l'; your system might not support 'cron'\"));\n - \t}\n \n \t/* Ignore exit code, as an empty crontab will return error. */\n + \tfinish_command(&crontab_list);\n @@ builtin/gc.c: static int update_background_schedule(int run_maintenance)\n \t * Read from the .lock file, filtering out the old\n \t * schedule while appending the new schedule.\n @@ builtin/gc.c: static int update_background_schedule(int run_maintenance)\n +\tcron_list = fdopen(fd, \"r\");\n \trewind(cron_list);\n \n - \tstrvec_split(&crontab_edit.args, crontab_name);\n -@@ builtin/gc.c: static int update_background_schedule(int run_maintenance)\n +-\tstrvec_split(&crontab_edit.args, crontab_name);\n ++\tstrvec_split(&crontab_edit.args, cmd);\n + \tcrontab_edit.in = -1;\n \tcrontab_edit.git_cmd = 0;\n \n - \tif (start_command(&crontab_edit)) {\n +-\tif (start_command(&crontab_edit)) {\n -\t\tresult = error(_(\"failed to run 'crontab'; your system might not support 'cron'\"));\n -\t\tgoto cleanup;\n +-\t}\n ++\tif (start_command(&crontab_edit))\n +\t\treturn error(_(\"failed to run 'crontab'; your system might not support 'cron'\"));\n - \t}\n \n \tcron_in = fdopen(crontab_edit.in, \"w\");\n + \tif (!cron_in) {\n @@ builtin/gc.c: static int update_background_schedule(int run_maintenance)\n \tclose(crontab_edit.in);\n \n @@ builtin/gc.c: static int update_background_schedule(int run_maintenance)\n +\tif (finish_command(&crontab_edit))\n \t\tresult = error(_(\"'crontab' died\"));\n -\t\tgoto cleanup;\n --\t}\n --\tfclose(cron_list);\n +\telse\n +\t\tfclose(cron_list);\n +\treturn result;\n +}\n +\n -+static int update_background_schedule(int run_maintenance)\n ++static const char platform_scheduler[] = \"crontab\";\n ++\n ++static int update_background_schedule(int enable)\n +{\n +\tint result;\n ++\tconst char *scheduler = platform_scheduler;\n ++\tconst char *cmd = scheduler;\n ++\tchar *testing;\n +\tstruct lock_file lk;\n +\tchar *lock_path = xstrfmt(\"%s/schedule\", the_repository->objects->odb->path);\n +\n ++\ttesting = xstrdup_or_null(getenv(\"GIT_TEST_MAINT_SCHEDULER\"));\n ++\tif (testing) {\n ++\t\tchar *sep = strchr(testing, ':');\n ++\t\tif (!sep)\n ++\t\t\tdie(\"GIT_TEST_MAINT_SCHEDULER unparseable: %s\", testing);\n ++\t\t*sep = '\\0';\n ++\t\tscheduler = testing;\n ++\t\tcmd = sep + 1;\n + \t}\n +-\tfclose(cron_list);\n + \n +-cleanup:\n +\tif (hold_lock_file_for_update(&lk, lock_path, LOCK_NO_DEREF) < 0)\n +\t\treturn error(_(\"another process is scheduling background maintenance\"));\n +\n -+\tresult = platform_update_schedule(run_maintenance, lk.tempfile->fd);\n - \n --cleanup:\n ++\tif (!strcmp(scheduler, \"crontab\"))\n ++\t\tresult = crontab_update_schedule(enable, lk.tempfile->fd, cmd);\n ++\telse\n ++\t\tdie(\"unknown background scheduler: %s\", scheduler);\n ++\n \trollback_lock_file(&lk);\n ++\tfree(testing);\n \treturn result;\n }\n + \n +\n + ## t/t7900-maintenance.sh ##\n +@@ t/t7900-maintenance.sh: test_expect_success 'register and unregister' '\n + '\n + \n + test_expect_success 'start from empty cron table' '\n +-\tGIT_TEST_CRONTAB=\"test-tool crontab cron.txt\" git maintenance start &&\n ++\tGIT_TEST_MAINT_SCHEDULER=\"crontab:test-tool crontab cron.txt\" git maintenance start &&\n + \n + \t# start registers the repo\n + \tgit config --get --global maintenance.repo \"$(pwd)\" &&\n +@@ t/t7900-maintenance.sh: test_expect_success 'start from empty cron table' '\n + '\n + \n + test_expect_success 'stop from existing schedule' '\n +-\tGIT_TEST_CRONTAB=\"test-tool crontab cron.txt\" git maintenance stop &&\n ++\tGIT_TEST_MAINT_SCHEDULER=\"crontab:test-tool crontab cron.txt\" git maintenance stop &&\n + \n + \t# stop does not unregister the repo\n + \tgit config --get --global maintenance.repo \"$(pwd)\" &&\n + \n + \t# Operation is idempotent\n +-\tGIT_TEST_CRONTAB=\"test-tool crontab cron.txt\" git maintenance stop &&\n ++\tGIT_TEST_MAINT_SCHEDULER=\"crontab:test-tool crontab cron.txt\" git maintenance stop &&\n + \ttest_must_be_empty cron.txt\n + '\n + \n + test_expect_success 'start preserves existing schedule' '\n + \techo \"Important information!\" >cron.txt &&\n +-\tGIT_TEST_CRONTAB=\"test-tool crontab cron.txt\" git maintenance start &&\n ++\tGIT_TEST_MAINT_SCHEDULER=\"crontab:test-tool crontab cron.txt\" git maintenance start &&\n + \tgrep \"Important information!\" cron.txt\n + '\n + \n +\n + ## t/test-lib.sh ##\n +@@ t/test-lib.sh: test_lazy_prereq REBASE_P '\n + '\n + \n + # Ensure that no test accidentally triggers a Git command\n +-# that runs 'crontab', affecting a user's cron schedule.\n +-# Tests that verify the cron integration must set this locally\n ++# that runs the actual maintenance scheduler, affecting a user's\n ++# system permanently.\n ++# Tests that verify the scheduler integration must set this locally\n + # to avoid errors.\n +-GIT_TEST_CRONTAB=\"exit 1\"\n ++GIT_TEST_MAINT_SCHEDULER=\"none:exit 1\"\n 2: 0dfe53092e ! 2: 99170df462 maintenance: include 'cron' details in docs\n @@ Documentation/git-maintenance.txt: Further, the `git gc` command should not be c\n +---------------------------------------\n +\n +The standard mechanism for scheduling background tasks on POSIX systems\n -+is cron(8). This tool executes commands based on a given schedule. The\n ++is `cron`. This tool executes commands based on a given schedule. The\n +current list of user-scheduled tasks can be found by running `crontab -l`.\n +The schedule written by `git maintenance start` is similar to this:\n +\n @@ Documentation/git-maintenance.txt: Further, the `git gc` command should not be c\n +Any modifications within this region will be completely deleted by\n +`git maintenance stop` or overwritten by `git maintenance start`.\n +\n -+The `crontab` entry specifies the full path of the `git` executable to\n -+ensure that the executed `git` command is the same one with which\n -+`git maintenance start` was issued independent of `PATH`. If the same user\n -+runs `git maintenance start` with multiple Git executables, then only the\n -+latest executable is used.\n ++The `` string is loaded to specifically use the location for the\n ++`git` executable used in the `git maintenance start` command. This allows\n ++for multiple versions to be compatible. However, if the same user runs\n ++`git maintenance start` with multiple Git executables, then only the\n ++latest executable will be used.\n +\n +These commands use `git for-each-repo --config=maintenance.repo` to run\n +`git maintenance run --schedule=` on each repository listed in\n +the multi-valued `maintenance.repo` config option. These are typically\n -+loaded from the user-specific global config. The `git maintenance` process\n -+then determines which maintenance tasks are configured to run on each\n -+repository with each `` using the `maintenance..schedule`\n -+config options. These values are loaded from the global or repository\n -+config values.\n ++loaded from the user-specific global config located at `~/.gitconfig`.\n ++The `git maintenance` process then determines which maintenance tasks\n ++are configured to run on each repository with each `` using\n ++the `maintenance..schedule` config options. These values are loaded\n ++from the global or repository config values.\n +\n +If the config values are insufficient to achieve your desired background\n +maintenance schedule, then you can create your own schedule. If you run\n +`crontab -e`, then an editor will load with your user-specific `cron`\n +schedule. In that editor, you can add your own schedule lines. You could\n +start by adapting the default schedule listed earlier, or you could read\n -+the crontab(5) documentation for advanced scheduling techniques. Please\n -+do use the full path and `--exec-path` techniques from the default\n -+schedule to ensure you are executing the correct binaries in your\n -+schedule.\n ++https://man7.org/linux/man-pages/man5/crontab.5.html[the `crontab` documentation]\n ++for advanced scheduling techniques. Please do use the full path and\n ++`--exec-path` techniques from the default schedule to ensure you are\n ++executing the correct binaries in your schedule.\n +\n \n GIT\n 3: 1629bcfcf8 ! 3: ed0a0011fb maintenance: use launchctl on macOS\n @@ Commit message\n plist file. We also need to 'bootout' a task before the 'bootstrap'\n subcommand will succeed, if such a task already exists.\n \n + The need for a user id requires us to run 'id -u' which works on\n + POSIX systems but not Windows. The test therefore has a prerequisite\n + that we are not on Windows. The cross-platform logic still allows us to\n + test the macOS logic on a Linux machine.\n +\n We can verify the commands that were run by 'git maintenance start'\n and 'git maintenance stop' by injecting a script that writes the\n - command-line arguments into GIT_TEST_CRONTAB.\n + command-line arguments into GIT_TEST_MAINT_SCHEDULER.\n \n An earlier version of this patch accidentally had an opening\n \"\" tag when it should have had a closing \"\" tag. This\n was caught during manual testing with actual 'launchctl' commands,\n but we do not want to update developers' tasks when running tests.\n It appears that macOS includes the \"xmllint\" tool which can verify\n - the XML format, so call it from the macOS-specific tests to ensure\n - the .plist files are well-formatted.\n + the XML format. This is useful for any system that might contain\n + the tool, so use it whenever it is available.\n \n - Helped-by: Eric Sunshine \n + Co-authored-by: Eric Sunshine \n + Signed-off-by: Eric Sunshine \n Signed-off-by: Derrick Stolee \n \n ## Documentation/git-maintenance.txt ##\n -@@ Documentation/git-maintenance.txt: schedule to ensure you are executing the correct binaries in your\n - schedule.\n +@@ Documentation/git-maintenance.txt: for advanced scheduling techniques. Please do use the full path and\n + executing the correct binaries in your schedule.\n \n \n +BACKGROUND MAINTENANCE ON MACOS SYSTEMS\n @@ builtin/gc.c: static int maintenance_unregister(void)\n \treturn run_command(&config_unset);\n }\n \n -+#if defined(__APPLE__)\n ++static const char *get_frequency(enum schedule_priority schedule)\n ++{\n ++\tswitch (schedule) {\n ++\tcase SCHEDULE_HOURLY:\n ++\t\treturn \"hourly\";\n ++\tcase SCHEDULE_DAILY:\n ++\t\treturn \"daily\";\n ++\tcase SCHEDULE_WEEKLY:\n ++\t\treturn \"weekly\";\n ++\tdefault:\n ++\t\tBUG(\"invalid schedule %d\", schedule);\n ++\t}\n ++}\n +\n -+static char *get_service_name(const char *frequency)\n ++static char *launchctl_service_name(const char *frequency)\n +{\n +\tstruct strbuf label = STRBUF_INIT;\n +\tstrbuf_addf(&label, \"org.git-scm.git.%s\", frequency);\n +\treturn strbuf_detach(&label, NULL);\n +}\n +\n -+static char *get_service_filename(const char *name)\n ++static char *launchctl_service_filename(const char *name)\n +{\n +\tchar *expanded;\n +\tstruct strbuf filename = STRBUF_INIT;\n @@ builtin/gc.c: static int maintenance_unregister(void)\n +\treturn expanded;\n +}\n +\n -+static const char *get_frequency(enum schedule_priority schedule)\n ++static char *launchctl_get_uid(void)\n +{\n -+\tswitch (schedule) {\n -+\tcase SCHEDULE_HOURLY:\n -+\t\treturn \"hourly\";\n -+\tcase SCHEDULE_DAILY:\n -+\t\treturn \"daily\";\n -+\tcase SCHEDULE_WEEKLY:\n -+\t\treturn \"weekly\";\n -+\tdefault:\n -+\t\tBUG(\"invalid schedule %d\", schedule);\n -+\t}\n ++\treturn xstrfmt(\"gui/%d\", getuid());\n +}\n +\n -+static char *get_uid(void)\n -+{\n -+\tstruct strbuf output = STRBUF_INIT;\n -+\tstruct child_process id = CHILD_PROCESS_INIT;\n -+\n -+\tstrvec_pushl(&id.args, \"/usr/bin/id\", \"-u\", NULL);\n -+\tif (capture_command(&id, &output, 0))\n -+\t\tdie(_(\"failed to discover user id\"));\n -+\n -+\tstrbuf_trim_trailing_newline(&output);\n -+\treturn strbuf_detach(&output, NULL);\n -+}\n -+\n -+static int boot_plist(int enable, const char *filename)\n ++static int launchctl_boot_plist(int enable, const char *filename, const char *cmd)\n +{\n +\tint result;\n +\tstruct child_process child = CHILD_PROCESS_INIT;\n -+\tchar *uid = get_uid();\n -+\tconst char *launchctl = getenv(\"GIT_TEST_CRONTAB\");\n -+\tif (!launchctl)\n -+\t\tlaunchctl = \"/bin/launchctl\";\n -+\n -+\tstrvec_split(&child.args, launchctl);\n ++\tchar *uid = launchctl_get_uid();\n +\n ++\tstrvec_split(&child.args, cmd);\n +\tif (enable)\n +\t\tstrvec_push(&child.args, \"bootstrap\");\n +\telse\n +\t\tstrvec_push(&child.args, \"bootout\");\n -+\tstrvec_pushf(&child.args, \"gui/%s\", uid);\n ++\tstrvec_push(&child.args, uid);\n +\tstrvec_push(&child.args, filename);\n +\n +\tchild.no_stderr = 1;\n @@ builtin/gc.c: static int maintenance_unregister(void)\n +\treturn result;\n +}\n +\n -+static int remove_plist(enum schedule_priority schedule)\n ++static int launchctl_remove_plist(enum schedule_priority schedule, const char *cmd)\n +{\n +\tconst char *frequency = get_frequency(schedule);\n -+\tchar *name = get_service_name(frequency);\n -+\tchar *filename = get_service_filename(name);\n -+\tint result = boot_plist(0, filename);\n ++\tchar *name = launchctl_service_name(frequency);\n ++\tchar *filename = launchctl_service_filename(name);\n ++\tint result = launchctl_boot_plist(0, filename, cmd);\n +\tunlink(filename);\n +\tfree(filename);\n +\tfree(name);\n +\treturn result;\n +}\n +\n -+static int remove_plists(void)\n ++static int launchctl_remove_plists(const char *cmd)\n +{\n -+\treturn remove_plist(SCHEDULE_HOURLY) ||\n -+\t\tremove_plist(SCHEDULE_DAILY) ||\n -+\t\tremove_plist(SCHEDULE_WEEKLY);\n ++\treturn launchctl_remove_plist(SCHEDULE_HOURLY, cmd) ||\n ++\t\tlaunchctl_remove_plist(SCHEDULE_DAILY, cmd) ||\n ++\t\tlaunchctl_remove_plist(SCHEDULE_WEEKLY, cmd);\n +}\n +\n -+static int schedule_plist(const char *exec_path, enum schedule_priority schedule)\n ++static int launchctl_schedule_plist(const char *exec_path, enum schedule_priority schedule, const char *cmd)\n +{\n +\tFILE *plist;\n +\tint i;\n +\tconst char *preamble, *repeat;\n +\tconst char *frequency = get_frequency(schedule);\n -+\tchar *name = get_service_name(frequency);\n -+\tchar *filename = get_service_filename(name);\n ++\tchar *name = launchctl_service_name(frequency);\n ++\tchar *filename = launchctl_service_filename(name);\n +\n +\tif (safe_create_leading_directories(filename))\n +\t\tdie(_(\"failed to create directories for '%s'\"), filename);\n +\tplist = xfopen(filename, \"w\");\n +\n -+\tpreamble = \"\\n\"\n ++\tpreamble = \"\\n\"\n +\t\t \"\\n\"\n +\t\t \"\"\n +\t\t \"\\n\"\n @@ builtin/gc.c: static int maintenance_unregister(void)\n +\t\tbreak;\n +\t}\n +\tfprintf(plist, \"\\n\\n\\n\");\n ++\tfclose(plist);\n +\n +\t/* bootout might fail if not already running, so ignore */\n -+\tboot_plist(0, filename);\n -+\tif (boot_plist(1, filename))\n ++\tlaunchctl_boot_plist(0, filename, cmd);\n ++\tif (launchctl_boot_plist(1, filename, cmd))\n +\t\tdie(_(\"failed to bootstrap service %s\"), filename);\n +\n -+\tfclose(plist);\n +\tfree(filename);\n +\tfree(name);\n +\treturn 0;\n +}\n +\n -+static int add_plists(void)\n ++static int launchctl_add_plists(const char *cmd)\n +{\n +\tconst char *exec_path = git_exec_path();\n +\n -+\treturn schedule_plist(exec_path, SCHEDULE_HOURLY) ||\n -+\t\tschedule_plist(exec_path, SCHEDULE_DAILY) ||\n -+\t\tschedule_plist(exec_path, SCHEDULE_WEEKLY);\n ++\treturn launchctl_schedule_plist(exec_path, SCHEDULE_HOURLY, cmd) ||\n ++\t\tlaunchctl_schedule_plist(exec_path, SCHEDULE_DAILY, cmd) ||\n ++\t\tlaunchctl_schedule_plist(exec_path, SCHEDULE_WEEKLY, cmd);\n +}\n +\n -+static int platform_update_schedule(int run_maintenance, int fd)\n ++static int launchctl_update_schedule(int run_maintenance, int fd, const char *cmd)\n +{\n +\tif (run_maintenance)\n -+\t\treturn add_plists();\n ++\t\treturn launchctl_add_plists(cmd);\n +\telse\n -+\t\treturn remove_plists();\n ++\t\treturn launchctl_remove_plists(cmd);\n +}\n -+#else\n ++\n #define BEGIN_LINE \"# BEGIN GIT MAINTENANCE SCHEDULE\"\n #define END_LINE \"# END GIT MAINTENANCE SCHEDULE\"\n \n -@@ builtin/gc.c: static int platform_update_schedule(int run_maintenance, int fd)\n - \t\tfclose(cron_list);\n +@@ builtin/gc.c: static int crontab_update_schedule(int run_maintenance, int fd, const char *cmd)\n \treturn result;\n }\n + \n ++#if defined(__APPLE__)\n ++static const char platform_scheduler[] = \"launchctl\";\n ++#else\n + static const char platform_scheduler[] = \"crontab\";\n +#endif\n \n - static int update_background_schedule(int run_maintenance)\n + static int update_background_schedule(int enable)\n {\n +@@ builtin/gc.c: static int update_background_schedule(int enable)\n + \tif (hold_lock_file_for_update(&lk, lock_path, LOCK_NO_DEREF) < 0)\n + \t\treturn error(_(\"another process is scheduling background maintenance\"));\n + \n +-\tif (!strcmp(scheduler, \"crontab\"))\n ++\tif (!strcmp(scheduler, \"launchctl\"))\n ++\t\tresult = launchctl_update_schedule(enable, lk.tempfile->fd, cmd);\n ++\telse if (!strcmp(scheduler, \"crontab\"))\n + \t\tresult = crontab_update_schedule(enable, lk.tempfile->fd, cmd);\n + \telse\n + \t\tdie(\"unknown background scheduler: %s\", scheduler);\n \n ## t/t7900-maintenance.sh ##\n -@@ t/t7900-maintenance.sh: test_expect_success 'register and unregister' '\n - \ttest_cmp before actual\n - '\n - \n --test_expect_success 'start from empty cron table' '\n -+test_expect_success !MACOS_MAINTENANCE 'start from empty cron table' '\n - \tGIT_TEST_CRONTAB=\"test-tool crontab cron.txt\" git maintenance start &&\n - \n - \t# start registers the repo\n -@@ t/t7900-maintenance.sh: test_expect_success 'start from empty cron table' '\n - \tgrep \"for-each-repo --config=maintenance.repo maintenance run --schedule=weekly\" cron.txt\n - '\n - \n --test_expect_success 'stop from existing schedule' '\n -+test_expect_success !MACOS_MAINTENANCE 'stop from existing schedule' '\n - \tGIT_TEST_CRONTAB=\"test-tool crontab cron.txt\" git maintenance stop &&\n +@@ t/t7900-maintenance.sh: test_description='git maintenance builtin'\n + GIT_TEST_COMMIT_GRAPH=0\n + GIT_TEST_MULTI_PACK_INDEX=0\n \n - \t# stop does not unregister the repo\n -@@ t/t7900-maintenance.sh: test_expect_success 'stop from existing schedule' '\n - \ttest_must_be_empty cron.txt\n - '\n - \n --test_expect_success 'start preserves existing schedule' '\n -+test_expect_success !MACOS_MAINTENANCE 'start preserves existing schedule' '\n - \techo \"Important information!\" >cron.txt &&\n - \tGIT_TEST_CRONTAB=\"test-tool crontab cron.txt\" git maintenance start &&\n ++test_lazy_prereq XMLLINT '\n ++\txmllint --version\n ++'\n ++\n ++test_xmllint () {\n ++\tif test_have_prereq XMLLINT\n ++\tthen\n ++\t\txmllint --noout \"$@\"\n ++\telse\n ++\t\ttrue\n ++\tfi\n ++}\n ++\n + test_expect_success 'help text' '\n + \ttest_expect_code 129 git maintenance -h 2>err &&\n + \ttest_i18ngrep \"usage: git maintenance \" err &&\n +@@ t/t7900-maintenance.sh: test_expect_success 'start preserves existing schedule' '\n \tgrep \"Important information!\" cron.txt\n '\n \n -+test_expect_success MACOS_MAINTENANCE 'start and stop macOS maintenance' '\n -+\twrite_script print-args \"#!/bin/sh\\necho \\$* >>args\" &&\n ++test_expect_success !MINGW 'start and stop macOS maintenance' '\n ++\tuid=$(id -u) &&\n ++\n ++\twrite_script print-args <<-\\EOF &&\n ++\techo $* >>args\n ++\tEOF\n +\n +\trm -f args &&\n -+\tGIT_TEST_CRONTAB=\"./print-args\" git maintenance start &&\n ++\tGIT_TEST_MAINT_SCHEDULER=launchctl:./print-args git maintenance start &&\n +\n +\t# start registers the repo\n +\tgit config --get --global maintenance.repo \"$(pwd)\" &&\n +\n -+\t# ~/Library/LaunchAgents\n +\tls \"$HOME/Library/LaunchAgents\" >actual &&\n +\tcat >expect <<-\\EOF &&\n +\torg.git-scm.git.daily.plist\n @@ t/t7900-maintenance.sh: test_expect_success 'stop from existing schedule' '\n +\tfor frequency in hourly daily weekly\n +\tdo\n +\t\tPLIST=\"$HOME/Library/LaunchAgents/org.git-scm.git.$frequency.plist\" &&\n -+\t\txmllint --noout \"$PLIST\" &&\n ++\t\ttest_xmllint \"$PLIST\" &&\n +\t\tgrep schedule=$frequency \"$PLIST\" &&\n -+\t\techo \"bootout gui/$UID $PLIST\" >>expect &&\n -+\t\techo \"bootstrap gui/$UID $PLIST\" >>expect || return 1\n ++\t\techo \"bootout gui/$uid $PLIST\" >>expect &&\n ++\t\techo \"bootstrap gui/$uid $PLIST\" >>expect || return 1\n +\tdone &&\n +\ttest_cmp expect args &&\n +\n +\trm -f args &&\n -+\tGIT_TEST_CRONTAB=\"./print-args\" git maintenance stop &&\n ++\tGIT_TEST_MAINT_SCHEDULER=launchctl:./print-args git maintenance stop &&\n +\n +\t# stop does not unregister the repo\n +\tgit config --get --global maintenance.repo \"$(pwd)\" &&\n +\n -+\tprintf \"bootout gui/$UID $HOME/Library/LaunchAgents/org.git-scm.git.%s.plist\\n\" \\\n ++\tprintf \"bootout gui/$uid $HOME/Library/LaunchAgents/org.git-scm.git.%s.plist\\n\" \\\n +\t\thourly daily weekly >expect &&\n +\ttest_cmp expect args &&\n +\tls \"$HOME/Library/LaunchAgents\" >actual &&\n @@ t/t7900-maintenance.sh: test_expect_success 'stop from existing schedule' '\n test_expect_success 'register preserves existing strategy' '\n \tgit config maintenance.strategy none &&\n \tgit maintenance register &&\n -\n - ## t/test-lib.sh ##\n -@@ t/test-lib.sh: test_lazy_prereq REBASE_P '\n - \ttest -z \"$GIT_TEST_SKIP_REBASE_P\"\n - '\n - \n -+test_lazy_prereq MACOS_MAINTENANCE '\n -+\tlaunchctl list\n -+'\n -+\n - # Ensure that no test accidentally triggers a Git command\n - # that runs 'crontab', affecting a user's cron schedule.\n - # Tests that verify the cron integration must set this locally\n 4: ed7a61978f ! 4: b8d86fb983 maintenance: use Windows scheduled tasks\n @@ Commit message\n logged in, and more fields are populated with the current username and\n SID at run-time by 'schtasks'.\n \n + Since the GIT_TEST_MAINT_SCHEDULER environment variable allows us to\n + specify 'schtasks' as the scheduler, we can test the Windows-specific\n + logic on a macOS platform. Thus, add a check that the XML file written\n + by Git is valid when xmllint exists on the system.\n +\n There is a deficiency in the current design. Windows has two kinds of\n applications: GUI applications that start by \"winmain()\" and console\n applications that start by \"main()\". Console applications are attached\n @@ Commit message\n short term. In the long term, we can consider creating this GUI\n shim application within core Git, perhaps in contrib/.\n \n - Helped-by: Eric Sunshine \n + Co-authored-by: Eric Sunshine \n + Signed-off-by: Eric Sunshine \n Signed-off-by: Derrick Stolee \n \n ## Documentation/git-maintenance.txt ##\n @@ Documentation/git-maintenance.txt: To create more advanced customizations to you\n Part of the linkgit:git[1] suite\n \n ## builtin/gc.c ##\n -@@ builtin/gc.c: static int platform_update_schedule(int run_maintenance, int fd)\n - \telse\n - \t\treturn remove_plists();\n +@@ builtin/gc.c: static int launchctl_update_schedule(int run_maintenance, int fd, const char *cm\n + \t\treturn launchctl_remove_plists(cmd);\n }\n -+\n -+#elif defined(GIT_WINDOWS_NATIVE)\n -+\n -+static const char *get_frequency(enum schedule_priority schedule)\n -+{\n -+\tswitch (schedule) {\n -+\tcase SCHEDULE_HOURLY:\n -+\t\treturn \"hourly\";\n -+\tcase SCHEDULE_DAILY:\n -+\t\treturn \"daily\";\n -+\tcase SCHEDULE_WEEKLY:\n -+\t\treturn \"weekly\";\n -+\tdefault:\n -+\t\tBUG(\"invalid schedule %d\", schedule);\n -+\t}\n -+}\n -+\n -+static char *get_task_name(const char *frequency)\n + \n ++static char *schtasks_task_name(const char *frequency)\n +{\n +\tstruct strbuf label = STRBUF_INIT;\n +\tstrbuf_addf(&label, \"Git Maintenance (%s)\", frequency);\n +\treturn strbuf_detach(&label, NULL);\n +}\n +\n -+static int remove_task(enum schedule_priority schedule)\n ++static int schtasks_remove_task(enum schedule_priority schedule, const char *cmd)\n +{\n +\tint result;\n +\tstruct strvec args = STRVEC_INIT;\n +\tconst char *frequency = get_frequency(schedule);\n -+\tchar *name = get_task_name(frequency);\n -+\tconst char *schtasks = getenv(\"GIT_TEST_CRONTAB\");\n -+\tif (!schtasks)\n -+\t\tschtasks = \"schtasks\";\n ++\tchar *name = schtasks_task_name(frequency);\n +\n -+\tstrvec_split(&args, schtasks);\n ++\tstrvec_split(&args, cmd);\n +\tstrvec_pushl(&args, \"/delete\", \"/tn\", name, \"/f\", NULL);\n +\n +\tresult = run_command_v_opt(args.v, 0);\n @@ builtin/gc.c: static int platform_update_schedule(int run_maintenance, int fd)\n +\treturn result;\n +}\n +\n -+static int remove_scheduled_tasks(void)\n ++static int schtasks_remove_tasks(const char *cmd)\n +{\n -+\treturn remove_task(SCHEDULE_HOURLY) ||\n -+\t\tremove_task(SCHEDULE_DAILY) ||\n -+\t\tremove_task(SCHEDULE_WEEKLY);\n ++\treturn schtasks_remove_task(SCHEDULE_HOURLY, cmd) ||\n ++\t\tschtasks_remove_task(SCHEDULE_DAILY, cmd) ||\n ++\t\tschtasks_remove_task(SCHEDULE_WEEKLY, cmd);\n +}\n +\n -+static int schedule_task(const char *exec_path, enum schedule_priority schedule)\n ++static int schtasks_schedule_task(const char *exec_path, enum schedule_priority schedule, const char *cmd)\n +{\n +\tint result;\n +\tstruct child_process child = CHILD_PROCESS_INIT;\n -+\tconst char *xml, *schtasks;\n -+\tchar *xmlpath, *tempDir;\n -+\tFILE *xmlfp;\n ++\tconst char *xml;\n ++\tchar *xmlpath;\n ++\tstruct tempfile *tfile;\n +\tconst char *frequency = get_frequency(schedule);\n -+\tchar *name = get_task_name(frequency);\n ++\tchar *name = schtasks_task_name(frequency);\n +\n -+\ttempDir = xstrfmt(\"%s/temp\", the_repository->objects->odb->path);\n -+\txmlpath = xstrfmt(\"%s/schedule-%s.xml\", tempDir, frequency);\n -+\tsafe_create_leading_directories(xmlpath);\n -+\txmlfp = xfopen(xmlpath, \"w\");\n ++\txmlpath = xstrfmt(\"%s/schedule-%s.xml\",\n ++\t\t\t the_repository->objects->odb->path,\n ++\t\t\t frequency);\n ++\ttfile = create_tempfile(xmlpath);\n ++\tif (!tfile || !fdopen_tempfile(tfile, \"w\"))\n ++\t\tdie(_(\"failed to create '%s'\"), xmlpath);\n +\n -+\txml = \"\\n\"\n ++\txml = \"\\n\"\n +\t \"\\n\"\n +\t \"\\n\"\n +\t \"\\n\";\n -+\tfprintf(xmlfp, xml);\n ++\tfputs(xml, tfile->fp);\n +\n +\tswitch (schedule) {\n +\tcase SCHEDULE_HOURLY:\n -+\t\tfprintf(xmlfp,\n ++\t\tfprintf(tfile->fp,\n +\t\t\t\"2020-01-01T01:00:00\\n\"\n +\t\t\t\"true\\n\"\n +\t\t\t\"\\n\"\n @@ builtin/gc.c: static int platform_update_schedule(int run_maintenance, int fd)\n +\t\tbreak;\n +\n +\tcase SCHEDULE_DAILY:\n -+\t\tfprintf(xmlfp,\n ++\t\tfprintf(tfile->fp,\n +\t\t\t\"2020-01-01T00:00:00\\n\"\n +\t\t\t\"true\\n\"\n +\t\t\t\"\\n\"\n @@ builtin/gc.c: static int platform_update_schedule(int run_maintenance, int fd)\n +\t\tbreak;\n +\n +\tcase SCHEDULE_WEEKLY:\n -+\t\tfprintf(xmlfp,\n ++\t\tfprintf(tfile->fp,\n +\t\t\t\"2020-01-01T00:00:00\\n\"\n +\t\t\t\"true\\n\"\n +\t\t\t\"\\n\"\n @@ builtin/gc.c: static int platform_update_schedule(int run_maintenance, int fd)\n +\t\tbreak;\n +\t}\n +\n -+\txml= \"\\n\"\n ++\txml = \"\\n\"\n +\t \"\\n\"\n +\t \"\\n\"\n +\t \"\\n\"\n @@ builtin/gc.c: static int platform_update_schedule(int run_maintenance, int fd)\n +\t \"\\n\"\n +\t \"\\n\"\n +\t \"\\n\";\n -+\tfprintf(xmlfp, xml, exec_path, exec_path, frequency);\n -+\tfclose(xmlfp);\n -+\n -+\tschtasks = getenv(\"GIT_TEST_CRONTAB\");\n -+\tif (!schtasks)\n -+\t\tschtasks = \"schtasks\";\n -+\tstrvec_split(&child.args, schtasks);\n ++\tfprintf(tfile->fp, xml, exec_path, exec_path, frequency);\n ++\tstrvec_split(&child.args, cmd);\n +\tstrvec_pushl(&child.args, \"/create\", \"/tn\", name, \"/f\", \"/xml\", xmlpath, NULL);\n ++\tclose_tempfile_gently(tfile);\n +\n +\tchild.no_stdout = 1;\n +\tchild.no_stderr = 1;\n @@ builtin/gc.c: static int platform_update_schedule(int run_maintenance, int fd)\n +\t\tdie(_(\"failed to start schtasks\"));\n +\tresult = finish_command(&child);\n +\n -+\tunlink(xmlpath);\n -+\trmdir(tempDir);\n ++\tdelete_tempfile(&tfile);\n +\tfree(xmlpath);\n +\tfree(name);\n +\treturn result;\n +}\n +\n -+static int add_scheduled_tasks(void)\n ++static int schtasks_schedule_tasks(const char *cmd)\n +{\n +\tconst char *exec_path = git_exec_path();\n +\n -+\treturn schedule_task(exec_path, SCHEDULE_HOURLY) ||\n -+\t\tschedule_task(exec_path, SCHEDULE_DAILY) ||\n -+\t\tschedule_task(exec_path, SCHEDULE_WEEKLY);\n ++\treturn schtasks_schedule_task(exec_path, SCHEDULE_HOURLY, cmd) ||\n ++\t\tschtasks_schedule_task(exec_path, SCHEDULE_DAILY, cmd) ||\n ++\t\tschtasks_schedule_task(exec_path, SCHEDULE_WEEKLY, cmd);\n +}\n +\n -+static int platform_update_schedule(int run_maintenance, int fd)\n ++static int schtasks_update_schedule(int run_maintenance, int fd, const char *cmd)\n +{\n +\tif (run_maintenance)\n -+\t\treturn add_scheduled_tasks();\n ++\t\treturn schtasks_schedule_tasks(cmd);\n +\telse\n -+\t\treturn remove_scheduled_tasks();\n ++\t\treturn schtasks_remove_tasks(cmd);\n +}\n +\n - #else\n #define BEGIN_LINE \"# BEGIN GIT MAINTENANCE SCHEDULE\"\n #define END_LINE \"# END GIT MAINTENANCE SCHEDULE\"\n -\n - ## t/t7900-maintenance.sh ##\n -@@ t/t7900-maintenance.sh: test_expect_success 'register and unregister' '\n - \ttest_cmp before actual\n - '\n - \n --test_expect_success !MACOS_MAINTENANCE 'start from empty cron table' '\n -+test_expect_success !MACOS_MAINTENANCE,!MINGW 'start from empty cron table' '\n - \tGIT_TEST_CRONTAB=\"test-tool crontab cron.txt\" git maintenance start &&\n - \n - \t# start registers the repo\n -@@ t/t7900-maintenance.sh: test_expect_success !MACOS_MAINTENANCE 'start from empty cron table' '\n - \tgrep \"for-each-repo --config=maintenance.repo maintenance run --schedule=weekly\" cron.txt\n - '\n \n --test_expect_success !MACOS_MAINTENANCE 'stop from existing schedule' '\n -+test_expect_success !MACOS_MAINTENANCE,!MINGW 'stop from existing schedule' '\n - \tGIT_TEST_CRONTAB=\"test-tool crontab cron.txt\" git maintenance stop &&\n +@@ builtin/gc.c: static int crontab_update_schedule(int run_maintenance, int fd, const char *cmd)\n \n - \t# stop does not unregister the repo\n -@@ t/t7900-maintenance.sh: test_expect_success !MACOS_MAINTENANCE 'stop from existing schedule' '\n - \ttest_must_be_empty cron.txt\n - '\n + #if defined(__APPLE__)\n + static const char platform_scheduler[] = \"launchctl\";\n ++#elif defined(GIT_WINDOWS_NATIVE)\n ++static const char platform_scheduler[] = \"schtasks\";\n + #else\n + static const char platform_scheduler[] = \"crontab\";\n + #endif\n +@@ builtin/gc.c: static int update_background_schedule(int enable)\n \n --test_expect_success !MACOS_MAINTENANCE 'start preserves existing schedule' '\n -+test_expect_success !MACOS_MAINTENANCE,!MINGW 'start preserves existing schedule' '\n - \techo \"Important information!\" >cron.txt &&\n - \tGIT_TEST_CRONTAB=\"test-tool crontab cron.txt\" git maintenance start &&\n - \tgrep \"Important information!\" cron.txt\n -@@ t/t7900-maintenance.sh: test_expect_success MACOS_MAINTENANCE 'start and stop macOS maintenance' '\n + \tif (!strcmp(scheduler, \"launchctl\"))\n + \t\tresult = launchctl_update_schedule(enable, lk.tempfile->fd, cmd);\n ++\telse if (!strcmp(scheduler, \"schtasks\"))\n ++\t\tresult = schtasks_update_schedule(enable, lk.tempfile->fd, cmd);\n + \telse if (!strcmp(scheduler, \"crontab\"))\n + \t\tresult = crontab_update_schedule(enable, lk.tempfile->fd, cmd);\n + \telse\n +\n + ## t/t7900-maintenance.sh ##\n +@@ t/t7900-maintenance.sh: test_expect_success !MINGW 'start and stop macOS maintenance' '\n \ttest_line_count = 0 actual\n '\n \n -+test_expect_success MINGW 'start and stop Windows maintenance' '\n ++test_expect_success 'start and stop Windows maintenance' '\n +\twrite_script print-args <<-\\EOF &&\n +\techo $* >>args\n ++\twhile test $# -gt 0\n ++\tdo\n ++\t\tcase \"$1\" in\n ++\t\t/xml) shift; xmlfile=$1; break ;;\n ++\t\t*) shift ;;\n ++\t\tesac\n ++\tdone\n ++\ttest -z \"$xmlfile\" || cp \"$xmlfile\" .\n +\tEOF\n +\n +\trm -f args &&\n -+\tGIT_TEST_CRONTAB=\"/bin/sh print-args\" git maintenance start &&\n ++\tGIT_TEST_MAINT_SCHEDULER=\"schtasks:./print-args\" git maintenance start &&\n +\n +\t# start registers the repo\n +\tgit config --get --global maintenance.repo \"$(pwd)\" &&\n +\n -+\tprintf \"/create /tn Git Maintenance (%s) /f /xml .git/objects/temp/schedule-%s.xml\\n\" \\\n ++\tprintf \"/create /tn Git Maintenance (%s) /f /xml .git/objects/schedule-%s.xml\\n\" \\\n +\t\thourly hourly daily daily weekly weekly >expect &&\n +\ttest_cmp expect args &&\n +\n ++\tfor frequency in hourly daily weekly\n ++\tdo\n ++\t\ttest_xmllint \"schedule-$frequency.xml\"\n ++\tdone &&\n ++\n +\trm -f args &&\n -+\tGIT_TEST_CRONTAB=\"/bin/sh print-args\" git maintenance stop &&\n ++\tGIT_TEST_MAINT_SCHEDULER=\"schtasks:./print-args\" git maintenance stop &&\n +\n +\t# stop does not unregister the repo\n +\tgit config --get --global maintenance.repo \"$(pwd)\" &&\n\n-- \ngitgitgadget\n"},{"id":"410132","messageId":"b8d86fb9832b59884d3d38791de252dcb71e3629.1605647598.git.gitgitgadget@gmail.com","threadId":"54562","inReplyTo":"pull.776.v4.git.1605647598.gitgitgadget@gmail.com","subject":"[PATCH v4 4/4] maintenance: use Windows scheduled tasks","fromName":"Derrick Stolee via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2020-11-17T21:13:18Z","receivedAt":"2020-11-17T21:13:46Z","isPatch":true,"sender":{"key":"stolee@gmail.com","avatar":"https://avatars.githubusercontent.com/u/570044?v=4"},"body":"From: Derrick Stolee \n\nGit's background maintenance uses cron by default, but this is not\navailable on Windows. Instead, integrate with Task Scheduler.\n\nTasks can be scheduled using the 'schtasks' command. There are several\ncommand-line options that can allow for some advanced scheduling, but\nunfortunately these seem to all require authenticating using a password.\n\nInstead, use the \"/xml\" option to pass an XML file that contains the\nconfiguration for the necessary schedule. These XML files are based on\nsome that I exported after constructing a schedule in the Task Scheduler\nGUI. These options only run background maintenance when the user is\nlogged in, and more fields are populated with the current username and\nSID at run-time by 'schtasks'.\n\nSince the GIT_TEST_MAINT_SCHEDULER environment variable allows us to\nspecify 'schtasks' as the scheduler, we can test the Windows-specific\nlogic on a macOS platform. Thus, add a check that the XML file written\nby Git is valid when xmllint exists on the system.\n\nThere is a deficiency in the current design. Windows has two kinds of\napplications: GUI applications that start by \"winmain()\" and console\napplications that start by \"main()\". Console applications are attached\nto a new Console window if they are not already associated with a GUI\napplication. This means that every hour the scheudled task launches a\ncommand window for the scheduled tasks. Not only is this visually\nobtrusive, but it also takes focus from whatever else the user is\ndoing!\n\nA simple fix would be to insert a GUI application that acts as a shim\nbetween the scheduled task and Git. This is currently possible in Git\nfor Windows by setting the tag equal to\n\n C:\\Program Files\\Git\\git-bash.exe\n\nwith options \"--hide --no-needs-console --command=cmd\\git.exe\"\nfollowed by the arguments currently used. Since git-bash.exe is not\nincluded in Windows builds of core Git, I chose to leave out this\nfeature. My plan is to submit a small patch to Git for Windows that\nconverts the use of git.exe with this use of git-bash.exe in the\nshort term. In the long term, we can consider creating this GUI\nshim application within core Git, perhaps in contrib/.\n\nCo-authored-by: Eric Sunshine \nSigned-off-by: Eric Sunshine \nSigned-off-by: Derrick Stolee \n---\n Documentation/git-maintenance.txt | 22 ++++\n builtin/gc.c | 165 ++++++++++++++++++++++++++++++\n t/t7900-maintenance.sh | 40 ++++++++\n 3 files changed, 227 insertions(+)\n\ndiff --git a/Documentation/git-maintenance.txt b/Documentation/git-maintenance.txt\nindex f2d59f2bcc..e1adfff6db 100644\n--- a/Documentation/git-maintenance.txt\n+++ b/Documentation/git-maintenance.txt\n@@ -313,6 +313,28 @@ To create more advanced customizations to your background tasks, see\n launchctl.plist(5) for more information.\n \n \n+BACKGROUND MAINTENANCE ON WINDOWS SYSTEMS\n+-----------------------------------------\n+\n+Windows does not support `cron` and instead has its own system for\n+scheduling background tasks. The `git maintenance start` command uses\n+the `schtasks` command to submit tasks to this system. You can inspect\n+all background tasks using the Task Scheduler application. The tasks\n+added by Git have names of the form `Git Maintenance ()`.\n+The Task Scheduler GUI has ways to inspect these tasks, but you can also\n+export the tasks to XML files and view the details there.\n+\n+Note that since Git is a console application, these background tasks\n+create a console window visible to the current user. This can be changed\n+manually by selecting the \"Run whether user is logged in or not\" option\n+in Task Scheduler. This change requires a password input, which is why\n+`git maintenance start` does not select it by default.\n+\n+If you want to customize the background tasks, please rename the tasks\n+so future calls to `git maintenance (start|stop)` do not overwrite your\n+custom tasks.\n+\n+\n GIT\n ---\n Part of the linkgit:git[1] suite\ndiff --git a/builtin/gc.c b/builtin/gc.c\nindex 782769f243..f6c42f96c1 100644\n--- a/builtin/gc.c\n+++ b/builtin/gc.c\n@@ -1671,6 +1671,167 @@ static int launchctl_update_schedule(int run_maintenance, int fd, const char *cm\n \t\treturn launchctl_remove_plists(cmd);\n }\n \n+static char *schtasks_task_name(const char *frequency)\n+{\n+\tstruct strbuf label = STRBUF_INIT;\n+\tstrbuf_addf(&label, \"Git Maintenance (%s)\", frequency);\n+\treturn strbuf_detach(&label, NULL);\n+}\n+\n+static int schtasks_remove_task(enum schedule_priority schedule, const char *cmd)\n+{\n+\tint result;\n+\tstruct strvec args = STRVEC_INIT;\n+\tconst char *frequency = get_frequency(schedule);\n+\tchar *name = schtasks_task_name(frequency);\n+\n+\tstrvec_split(&args, cmd);\n+\tstrvec_pushl(&args, \"/delete\", \"/tn\", name, \"/f\", NULL);\n+\n+\tresult = run_command_v_opt(args.v, 0);\n+\n+\tstrvec_clear(&args);\n+\tfree(name);\n+\treturn result;\n+}\n+\n+static int schtasks_remove_tasks(const char *cmd)\n+{\n+\treturn schtasks_remove_task(SCHEDULE_HOURLY, cmd) ||\n+\t\tschtasks_remove_task(SCHEDULE_DAILY, cmd) ||\n+\t\tschtasks_remove_task(SCHEDULE_WEEKLY, cmd);\n+}\n+\n+static int schtasks_schedule_task(const char *exec_path, enum schedule_priority schedule, const char *cmd)\n+{\n+\tint result;\n+\tstruct child_process child = CHILD_PROCESS_INIT;\n+\tconst char *xml;\n+\tchar *xmlpath;\n+\tstruct tempfile *tfile;\n+\tconst char *frequency = get_frequency(schedule);\n+\tchar *name = schtasks_task_name(frequency);\n+\n+\txmlpath = xstrfmt(\"%s/schedule-%s.xml\",\n+\t\t\t the_repository->objects->odb->path,\n+\t\t\t frequency);\n+\ttfile = create_tempfile(xmlpath);\n+\tif (!tfile || !fdopen_tempfile(tfile, \"w\"))\n+\t\tdie(_(\"failed to create '%s'\"), xmlpath);\n+\n+\txml = \"\\n\"\n+\t \"\\n\"\n+\t \"\\n\"\n+\t \"\\n\";\n+\tfputs(xml, tfile->fp);\n+\n+\tswitch (schedule) {\n+\tcase SCHEDULE_HOURLY:\n+\t\tfprintf(tfile->fp,\n+\t\t\t\"2020-01-01T01:00:00\\n\"\n+\t\t\t\"true\\n\"\n+\t\t\t\"\\n\"\n+\t\t\t\"1\\n\"\n+\t\t\t\"\\n\"\n+\t\t\t\"\\n\"\n+\t\t\t\"PT1H\\n\"\n+\t\t\t\"PT23H\\n\"\n+\t\t\t\"false\\n\"\n+\t\t\t\"\\n\");\n+\t\tbreak;\n+\n+\tcase SCHEDULE_DAILY:\n+\t\tfprintf(tfile->fp,\n+\t\t\t\"2020-01-01T00:00:00\\n\"\n+\t\t\t\"true\\n\"\n+\t\t\t\"\\n\"\n+\t\t\t\"\\n\"\n+\t\t\t\"\\n\"\n+\t\t\t\"\\n\"\n+\t\t\t\"\\n\"\n+\t\t\t\"\\n\"\n+\t\t\t\"\\n\"\n+\t\t\t\"\\n\"\n+\t\t\t\"\\n\"\n+\t\t\t\"1\\n\"\n+\t\t\t\"\\n\");\n+\t\tbreak;\n+\n+\tcase SCHEDULE_WEEKLY:\n+\t\tfprintf(tfile->fp,\n+\t\t\t\"2020-01-01T00:00:00\\n\"\n+\t\t\t\"true\\n\"\n+\t\t\t\"\\n\"\n+\t\t\t\"\\n\"\n+\t\t\t\"\\n\"\n+\t\t\t\"\\n\"\n+\t\t\t\"1\\n\"\n+\t\t\t\"\\n\");\n+\t\tbreak;\n+\n+\tdefault:\n+\t\tbreak;\n+\t}\n+\n+\txml = \"\\n\"\n+\t \"\\n\"\n+\t \"\\n\"\n+\t \"\\n\"\n+\t \"InteractiveToken\\n\"\n+\t \"LeastPrivilege\\n\"\n+\t \"\\n\"\n+\t \"\\n\"\n+\t \"\\n\"\n+\t \"IgnoreNew\\n\"\n+\t \"true\\n\"\n+\t \"true\\n\"\n+\t \"true\\n\"\n+\t \"false\\n\"\n+\t \"PT72H\\n\"\n+\t \"7\\n\"\n+\t \"\\n\"\n+\t \"\\n\"\n+\t \"\\n\"\n+\t \"\\\"%s\\\\git.exe\\\"\\n\"\n+\t \"--exec-path=\\\"%s\\\" for-each-repo --config=maintenance.repo maintenance run --schedule=%s\\n\"\n+\t \"\\n\"\n+\t \"\\n\"\n+\t \"\\n\";\n+\tfprintf(tfile->fp, xml, exec_path, exec_path, frequency);\n+\tstrvec_split(&child.args, cmd);\n+\tstrvec_pushl(&child.args, \"/create\", \"/tn\", name, \"/f\", \"/xml\", xmlpath, NULL);\n+\tclose_tempfile_gently(tfile);\n+\n+\tchild.no_stdout = 1;\n+\tchild.no_stderr = 1;\n+\n+\tif (start_command(&child))\n+\t\tdie(_(\"failed to start schtasks\"));\n+\tresult = finish_command(&child);\n+\n+\tdelete_tempfile(&tfile);\n+\tfree(xmlpath);\n+\tfree(name);\n+\treturn result;\n+}\n+\n+static int schtasks_schedule_tasks(const char *cmd)\n+{\n+\tconst char *exec_path = git_exec_path();\n+\n+\treturn schtasks_schedule_task(exec_path, SCHEDULE_HOURLY, cmd) ||\n+\t\tschtasks_schedule_task(exec_path, SCHEDULE_DAILY, cmd) ||\n+\t\tschtasks_schedule_task(exec_path, SCHEDULE_WEEKLY, cmd);\n+}\n+\n+static int schtasks_update_schedule(int run_maintenance, int fd, const char *cmd)\n+{\n+\tif (run_maintenance)\n+\t\treturn schtasks_schedule_tasks(cmd);\n+\telse\n+\t\treturn schtasks_remove_tasks(cmd);\n+}\n+\n #define BEGIN_LINE \"# BEGIN GIT MAINTENANCE SCHEDULE\"\n #define END_LINE \"# END GIT MAINTENANCE SCHEDULE\"\n \n@@ -1761,6 +1922,8 @@ static int crontab_update_schedule(int run_maintenance, int fd, const char *cmd)\n \n #if defined(__APPLE__)\n static const char platform_scheduler[] = \"launchctl\";\n+#elif defined(GIT_WINDOWS_NATIVE)\n+static const char platform_scheduler[] = \"schtasks\";\n #else\n static const char platform_scheduler[] = \"crontab\";\n #endif\n@@ -1789,6 +1952,8 @@ static int update_background_schedule(int enable)\n \n \tif (!strcmp(scheduler, \"launchctl\"))\n \t\tresult = launchctl_update_schedule(enable, lk.tempfile->fd, cmd);\n+\telse if (!strcmp(scheduler, \"schtasks\"))\n+\t\tresult = schtasks_update_schedule(enable, lk.tempfile->fd, cmd);\n \telse if (!strcmp(scheduler, \"crontab\"))\n \t\tresult = crontab_update_schedule(enable, lk.tempfile->fd, cmd);\n \telse\ndiff --git a/t/t7900-maintenance.sh b/t/t7900-maintenance.sh\nindex 6d37312901..a26ff22541 100755\n--- a/t/t7900-maintenance.sh\n+++ b/t/t7900-maintenance.sh\n@@ -453,6 +453,46 @@ test_expect_success !MINGW 'start and stop macOS maintenance' '\n \ttest_line_count = 0 actual\n '\n \n+test_expect_success 'start and stop Windows maintenance' '\n+\twrite_script print-args <<-\\EOF &&\n+\techo $* >>args\n+\twhile test $# -gt 0\n+\tdo\n+\t\tcase \"$1\" in\n+\t\t/xml) shift; xmlfile=$1; break ;;\n+\t\t*) shift ;;\n+\t\tesac\n+\tdone\n+\ttest -z \"$xmlfile\" || cp \"$xmlfile\" .\n+\tEOF\n+\n+\trm -f args &&\n+\tGIT_TEST_MAINT_SCHEDULER=\"schtasks:./print-args\" git maintenance start &&\n+\n+\t# start registers the repo\n+\tgit config --get --global maintenance.repo \"$(pwd)\" &&\n+\n+\tprintf \"/create /tn Git Maintenance (%s) /f /xml .git/objects/schedule-%s.xml\\n\" \\\n+\t\thourly hourly daily daily weekly weekly >expect &&\n+\ttest_cmp expect args &&\n+\n+\tfor frequency in hourly daily weekly\n+\tdo\n+\t\ttest_xmllint \"schedule-$frequency.xml\"\n+\tdone &&\n+\n+\trm -f args &&\n+\tGIT_TEST_MAINT_SCHEDULER=\"schtasks:./print-args\" git maintenance stop &&\n+\n+\t# stop does not unregister the repo\n+\tgit config --get --global maintenance.repo \"$(pwd)\" &&\n+\n+\trm expect &&\n+\tprintf \"/delete /tn Git Maintenance (%s) /f\\n\" \\\n+\t\thourly daily weekly >expect &&\n+\ttest_cmp expect args\n+'\n+\n test_expect_success 'register preserves existing strategy' '\n \tgit config maintenance.strategy none &&\n \tgit maintenance register &&\n-- \ngitgitgadget\n"},{"id":"410179","messageId":"X7ReZXuwAaAZzMSU@flurp.local","threadId":"54562","inReplyTo":"pull.776.v4.git.1605647598.gitgitgadget@gmail.com","subject":"Re: [PATCH v4 0/4] Maintenance IV: Platform-specific background maintenance","fromName":"Eric Sunshine","fromEmail":"sunshine@sunshineco.com","sentAt":"2020-11-17T23:36:05Z","receivedAt":"2020-11-17T23:36:14Z","isPatch":true,"sender":{"key":"sunshine@sunshineco.com","avatar":"https://avatars.githubusercontent.com/u/163641?v=4"},"body":"On Tue, Nov 17, 2020 at 09:13:14PM +0000, Derrick Stolee via GitGitGadget wrote:\n> Updates in V4\n> =============\n> * Eric did an excellent job providing a patch that cleans up several parts\n> of my series. The most impressive is his mechanism for testing the\n> platform-specific Git logic in a way that is (mostly) platform-agnostic.\n> \n> * Windows doesn't have the 'id' command, so we cannot run the macOS\n> platform test on Windows.\n\nThis is easy to resolve. Drop in the following patch and then replace\nthe `$(id -u)` invocation in the test with `$(test-tool getuid)`.\nThis way, the test should work on any platform since both\nlaunchctl_get_uid() and `test-tool` will retrieve identical values for\nUID.\n\n--- >8 ---\nFrom 84f623bcaec156082c0e7151f40aef18575e6f86 Mon Sep 17 00:00:00 2001\nFrom: Eric Sunshine \nDate: Tue, 17 Nov 2020 18:30:10 -0500\nSubject: [PATCH] test-helper: add `getuid` subcommand\n\nSigned-off-by: Eric Sunshine \n---\n Makefile | 1 +\n t/helper/test-getuid.c | 7 +++++++\n t/helper/test-tool.c | 1 +\n t/helper/test-tool.h | 1 +\n 4 files changed, 10 insertions(+)\n create mode 100644 t/helper/test-getuid.c\n\ndiff --git a/Makefile b/Makefile\nindex 790a883932..230aff5e5c 100644\n--- a/Makefile\n+++ b/Makefile\n@@ -706,6 +706,7 @@ TEST_BUILTINS_OBJS += test-dump-untracked-cache.o\n TEST_BUILTINS_OBJS += test-example-decorate.o\n TEST_BUILTINS_OBJS += test-genrandom.o\n TEST_BUILTINS_OBJS += test-genzeros.o\n+TEST_BUILTINS_OBJS += test-getuid.o\n TEST_BUILTINS_OBJS += test-hash-speed.o\n TEST_BUILTINS_OBJS += test-hash.o\n TEST_BUILTINS_OBJS += test-hashmap.o\ndiff --git a/t/helper/test-getuid.c b/t/helper/test-getuid.c\nnew file mode 100644\nindex 0000000000..d741302461\n--- /dev/null\n+++ b/t/helper/test-getuid.c\n@@ -0,0 +1,7 @@\n+#include \"test-tool.h\"\n+\n+int cmd__getuid(int argc, const char **argv)\n+{\n+\tprintf(\"%d\\n\", getuid());\n+\treturn 0;\n+}\ndiff --git a/t/helper/test-tool.c b/t/helper/test-tool.c\nindex a0d3966b29..ab206541df 100644\n--- a/t/helper/test-tool.c\n+++ b/t/helper/test-tool.c\n@@ -30,6 +30,7 @@ static struct test_cmd cmds[] = {\n \t{ \"example-decorate\", cmd__example_decorate },\n \t{ \"genrandom\", cmd__genrandom },\n \t{ \"genzeros\", cmd__genzeros },\n+\t{ \"getuid\", cmd__getuid },\n \t{ \"hashmap\", cmd__hashmap },\n \t{ \"hash-speed\", cmd__hash_speed },\n \t{ \"index-version\", cmd__index_version },\ndiff --git a/t/helper/test-tool.h b/t/helper/test-tool.h\nindex 07034d3f38..caee0a3667 100644\n--- a/t/helper/test-tool.h\n+++ b/t/helper/test-tool.h\n@@ -20,6 +20,7 @@ int cmd__dump_untracked_cache(int argc, const char **argv);\n int cmd__example_decorate(int argc, const char **argv);\n int cmd__genrandom(int argc, const char **argv);\n int cmd__genzeros(int argc, const char **argv);\n+int cmd__getuid(int argc, const char **argv);\n int cmd__hashmap(int argc, const char **argv);\n int cmd__hash_speed(int argc, const char **argv);\n int cmd__index_version(int argc, const char **argv);\n-- \n2.29.2.454.gaff20da3a2\n\n"},{"id":"410180","messageId":"CAPig+cQCyWKzhm-LUBJ8Xm71KMq=6F+xpeLTCRXNoagtgXAwUg@mail.gmail.com","threadId":"54562","inReplyTo":"pull.776.v4.git.1605647598.gitgitgadget@gmail.com","subject":"Re: [PATCH v4 0/4] Maintenance IV: Platform-specific background maintenance","fromName":"Eric Sunshine","fromEmail":"sunshine@sunshineco.com","sentAt":"2020-11-17T23:54:36Z","receivedAt":"2020-11-17T23:54:59Z","isPatch":true,"sender":{"key":"sunshine@sunshineco.com","avatar":"https://avatars.githubusercontent.com/u/163641?v=4"},"body":"On Tue, Nov 17, 2020 at 4:13 PM Derrick Stolee via GitGitGadget\n wrote:\n> * I noticed far too late that while my example XML files had been edited\n> with UTF-8 encoding, Git is actually writing them as US-ASCII. Somehow\n> xmllint and launchd are not complaining, but schtasks does complain.\n> Unfortunately, I cannot find a way to catch this problem other than to\n> install my tip version on all three platforms and go through the entire\n> git maintenance start process, and double-check that the processes are\n> running on the hour.\n\nI'm having trouble understanding what problem is being described here\nand whether or not it has been solved by v4.\n\nI might guess that you are saying that `schtasks` insists upon seeing\na UTF-8 BOM at the start of the XML file since the XML file declares\nitself as UTF-8, but that Git is (quite naturally) writing out the\nfile without the UTF-8 BOM.\n\n> - xml = \"\\n\"\n> + xml = \"\\n\"\n\nIf the above speculation is correct, and if `schtasks` is happy with\nthe plain text file (lacking UTF-8 BOM) declaring itself as US-ASCII,\nthen this seems a reasonable solution. And it's easy to test that this\ndoesn't get broken. After validating the file with `xmllint`, also\ngrep it for US-ASCII, perhaps like this:\n\n test_xmllint \"schedule-$frequency.xml\" &&\n grep \"encoding=.US-ASCII.\" \"schedule-$frequency.xml\"\n"},{"id":"410185","messageId":"CAPig+cQw=hBqp2jSOOKNCKMe7bUFFLqT2fi0iaBfth-4x7RTgQ@mail.gmail.com","threadId":"54562","inReplyTo":"99170df4626544c1dc26d2e188b215a776140a32.1605647598.git.gitgitgadget@gmail.com","subject":"Re: [PATCH v4 2/4] maintenance: include 'cron' details in docs","fromName":"Eric Sunshine","fromEmail":"sunshine@sunshineco.com","sentAt":"2020-11-18T00:34:35Z","receivedAt":"2020-11-18T00:34:50Z","isPatch":true,"sender":{"key":"sunshine@sunshineco.com","avatar":"https://avatars.githubusercontent.com/u/163641?v=4"},"body":"On Tue, Nov 17, 2020 at 4:13 PM Derrick Stolee via GitGitGadget\n wrote:\n> Advanced and expert users may want to know how 'git maintenance start'\n> schedules background maintenance in order to customize their own\n> schedules beyond what the maintenance.* config values allow. Start a new\n> set of sections in git-maintenance.txt that describe how 'cron' is used\n> to run these tasks.\n>\n> This is particularly valuable for users who want to inspect what Git is\n> doing or for users who want to customize the schedule further. Having a\n> baseline can provide a way forward for users who have never worked with\n> cron schedules.\n>\n> Signed-off-by: Derrick Stolee \n> ---\n> diff --git a/Documentation/git-maintenance.txt b/Documentation/git-maintenance.txt\n> @@ -218,6 +218,60 @@ Further, the `git gc` command should not be combined with\n> +The comments are used as a region to mark the schedule as written by Git.\n> +Any modifications within this region will be completely deleted by\n> +`git maintenance stop` or overwritten by `git maintenance start`.\n> +\n> +The `` string is loaded to specifically use the location for the\n> +`git` executable used in the `git maintenance start` command. This allows\n> +for multiple versions to be compatible. However, if the same user runs\n> +`git maintenance start` with multiple Git executables, then only the\n> +latest executable will be used.\n\nIt looks like this section in v4 got accidentally reverted to the\nwording from v2, whereas v3 had been changed to:\n\n The `crontab` entry specifies the full path of the `git`\n executable to ensure that the executed `git` command is the same\n one with which `git maintenance start` was issued independent of\n `PATH`. If the same user runs `git maintenance start` with\n multiple Git executables, then only the latest executable is used.\n\n> +These commands use `git for-each-repo --config=maintenance.repo` to run\n> +`git maintenance run --schedule=` on each repository listed in\n> +the multi-valued `maintenance.repo` config option. These are typically\n> +loaded from the user-specific global config located at `~/.gitconfig`.\n> +The `git maintenance` process then determines which maintenance tasks\n> +are configured to run on each repository with each `` using\n> +the `maintenance..schedule` config options. These values are loaded\n> +from the global or repository config values.\n\nSame problem here. This wording is from v2, whereas v3 had been\nchanged to say generically \"user-specific global config\" rather than\nmentioning `~/.gitconfig` explicitly.\n\n> +If the config values are insufficient to achieve your desired background\n> +maintenance schedule, then you can create your own schedule. If you run\n> +`crontab -e`, then an editor will load with your user-specific `cron`\n> +schedule. In that editor, you can add your own schedule lines. You could\n> +start by adapting the default schedule listed earlier, or you could read\n> +https://man7.org/linux/man-pages/man5/crontab.5.html[the `crontab` documentation]\n> +for advanced scheduling techniques. Please do use the full path and\n> +`--exec-path` techniques from the default schedule to ensure you are\n> +executing the correct binaries in your schedule.\n\nAnd here too. v3 had updated this to say only \"crontab(5)\" rather than\nproviding an explicit URL.\n"},{"id":"410204","messageId":"CAPig+cRQJ9VpRk_0oz4BP3GF7dtsHtE3Qcd=nGsS26=JZvps7g@mail.gmail.com","threadId":"54562","inReplyTo":"ed0a0011fb0b06780bc6ce2920f17cad46eccdf9.1605647598.git.gitgitgadget@gmail.com","subject":"Re: [PATCH v4 3/4] maintenance: use launchctl on macOS","fromName":"Eric Sunshine","fromEmail":"sunshine@sunshineco.com","sentAt":"2020-11-18T06:45:37Z","receivedAt":"2020-11-18T06:46:12Z","isPatch":true,"sender":{"key":"sunshine@sunshineco.com","avatar":"https://avatars.githubusercontent.com/u/163641?v=4"},"body":"On Tue, Nov 17, 2020 at 4:13 PM Derrick Stolee via GitGitGadget\n wrote:\n> +static int launchctl_boot_plist(int enable, const char *filename, const char *cmd)\n> +{\n> + child.no_stderr = 1;\n> + child.no_stdout = 1;\n> + if (start_command(&child))\n> + die(_(\"failed to start launchctl\"));\n\nDid you have any thoughts on the observation I made in a followup\nresponse[1] during review of v3 in which I suggested that we might be\nable to avoid suppressing stderr (and stdout) here? In particular, the\nidea was that if, in launchctl_schedule_plist(), we do a simple\nexistence check for the .plist file and only call\nlaunchctl_boot_plist(0,...) to `bootout` the .plist file if it exists,\nthen we shouldn't need to muck with stderr/stdout suppression. The\nbenefit is that if `bootout` fails for some reason, then the user\nwould see the (hopefully) meaningful error message emitted `launchctl\nbootout`.\n\nThe same .plist existence check could be done in\nlaunchctl_remove_plist() before trying to `bootout` the file.\n\nAnyhow, such refinement can be done later is desired, so not worth a\nre-roll, but I was curious about your thoughts on the issue.\n\n[1]: https://lore.kernel.org/git/CAPig+cTRJb-fn2R6rJO1hkeCc_ehVhkNufO4=LhtPQudVeonnA@mail.gmail.com/\n"},{"id":"410209","messageId":"CAPig+cRFQfg-NLx5dO+BjQpYduhOYs-_+ZRd=DhO8ebWjGB0iA@mail.gmail.com","threadId":"54562","inReplyTo":"b8d86fb9832b59884d3d38791de252dcb71e3629.1605647598.git.gitgitgadget@gmail.com","subject":"Re: [PATCH v4 4/4] maintenance: use Windows scheduled tasks","fromName":"Eric Sunshine","fromEmail":"sunshine@sunshineco.com","sentAt":"2020-11-18T07:15:04Z","receivedAt":"2020-11-18T07:15:18Z","isPatch":true,"sender":{"key":"sunshine@sunshineco.com","avatar":"https://avatars.githubusercontent.com/u/163641?v=4"},"body":"On Tue, Nov 17, 2020 at 4:13 PM Derrick Stolee via GitGitGadget\n wrote:\n> [...]\n> Since the GIT_TEST_MAINT_SCHEDULER environment variable allows us to\n> specify 'schtasks' as the scheduler, we can test the Windows-specific\n> logic on a macOS platform. Thus, add a check that the XML file written\n> by Git is valid when xmllint exists on the system.\n\nNit: xmllint can be installed on Linux (and likely other platforms),\nas well, so it's not clear why this calls out macOS specially. More\ngenerally, it may not be important to call out xmllint at all in the\ncommit message; it's just _one_ thing being checked by a test which is\nchecking several other things not called out individually by the\ncommit message. Anyhow, this is minor; not worth a re-roll.\n\n> diff --git a/builtin/gc.c b/builtin/gc.c\n> @@ -1671,6 +1671,167 @@ static int launchctl_update_schedule(int run_maintenance, int fd, const char *cm\n> +static int schtasks_schedule_task(const char *exec_path, enum schedule_priority schedule, const char *cmd)\n> +{\n> + xmlpath = xstrfmt(\"%s/schedule-%s.xml\",\n> + the_repository->objects->odb->path,\n> + frequency);\n\nI missed this in the earlier rounds since I wasn't paying close enough\nattention, but placing this XML file within the object database\ndirectory (.git/objects/) feels rather odd, even if it is just a\ntemporary file. Using the .git/ directory itself might be better,\nperhaps like this:\n\n struct strbuf xmlpath = STRBUF_INIT;\n strbuf_git_common_path(&xmlpath, the_repository,\n \"schtasks-%s.xml\", frequency);\n ...\n strbuf_release(&xmlpath);\n"},{"id":"410252","messageId":"3dc00733-e736-9483-4a46-372e1560bcc2@gmail.com","threadId":"54562","inReplyTo":"CAPig+cRQJ9VpRk_0oz4BP3GF7dtsHtE3Qcd=nGsS26=JZvps7g@mail.gmail.com","subject":"Re: [PATCH v4 3/4] maintenance: use launchctl on macOS","fromName":"Derrick Stolee","fromEmail":"stolee@gmail.com","sentAt":"2020-11-18T18:22:46Z","receivedAt":"2020-11-18T18:23:12Z","isPatch":true,"sender":{"key":"stolee@gmail.com","avatar":"https://avatars.githubusercontent.com/u/570044?v=4"},"body":"On 11/18/2020 1:45 AM, Eric Sunshine wrote:\n> On Tue, Nov 17, 2020 at 4:13 PM Derrick Stolee via GitGitGadget\n> wrote:\n>> +static int launchctl_boot_plist(int enable, const char *filename, const char *cmd)\n>> +{\n>> + child.no_stderr = 1;\n>> + child.no_stdout = 1;\n>> + if (start_command(&child))\n>> + die(_(\"failed to start launchctl\"));\n> \n> Did you have any thoughts on the observation I made in a followup\n> response[1] during review of v3 in which I suggested that we might be\n> able to avoid suppressing stderr (and stdout) here? In particular, the\n> idea was that if, in launchctl_schedule_plist(), we do a simple\n> existence check for the .plist file and only call\n> launchctl_boot_plist(0,...) to `bootout` the .plist file if it exists,\n> then we shouldn't need to muck with stderr/stdout suppression. The\n> benefit is that if `bootout` fails for some reason, then the user\n> would see the (hopefully) meaningful error message emitted `launchctl\n> bootout`.\n> \n> The same .plist existence check could be done in\n> launchctl_remove_plist() before trying to `bootout` the file.\n\nIf the file exists but isn't still registered with launchd, then\nthe bootout command will send output\n\n\".plist: Could not find specified service\"\n\nand return a failure code. This output isn't helpful to users,\nsince we still are in the desired state afterwards.\n\n> Anyhow, such refinement can be done later is desired, so not worth a\n> re-roll, but I was curious about your thoughts on the issue.\n\nThis pattern of squashing the output even in the successful case\nis important for Windows where schtasks sends a line of output for\neach task being registered!\n\nI think it would be a reasonable extension to store the error\nmessage for logging or communicating to the user if we actually\nneed it, but I don't believe we should be piping the output\ndirectly.\n\nThanks,\n-Stolee\n"},{"id":"410255","messageId":"e8922076-1ed4-b540-f1c9-9b4a8fa7a085@gmail.com","threadId":"54562","inReplyTo":"CAPig+cRFQfg-NLx5dO+BjQpYduhOYs-_+ZRd=DhO8ebWjGB0iA@mail.gmail.com","subject":"Re: [PATCH v4 4/4] maintenance: use Windows scheduled tasks","fromName":"Derrick Stolee","fromEmail":"stolee@gmail.com","sentAt":"2020-11-18T18:30:15Z","receivedAt":"2020-11-18T18:30:20Z","isPatch":true,"sender":{"key":"stolee@gmail.com","avatar":"https://avatars.githubusercontent.com/u/570044?v=4"},"body":"On 11/18/2020 2:15 AM, Eric Sunshine wrote:\n> On Tue, Nov 17, 2020 at 4:13 PM Derrick Stolee via GitGitGadget\n> wrote:\n>> [...]\n>> Since the GIT_TEST_MAINT_SCHEDULER environment variable allows us to\n>> specify 'schtasks' as the scheduler, we can test the Windows-specific\n>> logic on a macOS platform. Thus, add a check that the XML file written\n>> by Git is valid when xmllint exists on the system.\n> \n> Nit: xmllint can be installed on Linux (and likely other platforms),\n> as well, so it's not clear why this calls out macOS specially. More\n> generally, it may not be important to call out xmllint at all in the\n> commit message; it's just _one_ thing being checked by a test which is\n> checking several other things not called out individually by the\n> commit message. Anyhow, this is minor; not worth a re-roll.\n\nSorry, it should just say \"other platforms\"\n\n>> diff --git a/builtin/gc.c b/builtin/gc.c\n>> @@ -1671,6 +1671,167 @@ static int launchctl_update_schedule(int run_maintenance, int fd, const char *cm\n>> +static int schtasks_schedule_task(const char *exec_path, enum schedule_priority schedule, const char *cmd)\n>> +{\n>> + xmlpath = xstrfmt(\"%s/schedule-%s.xml\",\n>> + the_repository->objects->odb->path,\n>> + frequency);\n> \n> I missed this in the earlier rounds since I wasn't paying close enough\n> attention, but placing this XML file within the object database\n> directory (.git/objects/) feels rather odd, even if it is just a\n> temporary file. Using the .git/ directory itself might be better,\n> perhaps like this:\n> \n> struct strbuf xmlpath = STRBUF_INIT;\n> strbuf_git_common_path(&xmlpath, the_repository,\n> \"schtasks-%s.xml\", frequency);\n> ...\n> strbuf_release(&xmlpath);\n\nIt does look odd, and in this case we could use the .git directory\ninstead. I specifically use the objects directory for the maintenance\nlock in 'git maintenance run' to allow maintenance to run when\nGIT_OBJECT_DIRECTORY points to an alternate, allowing us to maintain\nobject databases that don't have a full .git directory around them.\n\nThanks,\n-Stolee\n"},{"id":"410257","messageId":"6bd1b36f-48fe-c27d-3917-899a9b28eef3@gmail.com","threadId":"54562","inReplyTo":"CAPig+cQw=hBqp2jSOOKNCKMe7bUFFLqT2fi0iaBfth-4x7RTgQ@mail.gmail.com","subject":"Re: [PATCH v4 2/4] maintenance: include 'cron' details in docs","fromName":"Derrick Stolee","fromEmail":"stolee@gmail.com","sentAt":"2020-11-18T18:30:47Z","receivedAt":"2020-11-18T18:30:53Z","isPatch":true,"sender":{"key":"stolee@gmail.com","avatar":"https://avatars.githubusercontent.com/u/570044?v=4"},"body":"On 11/17/2020 7:34 PM, Eric Sunshine wrote:\n> On Tue, Nov 17, 2020 at 4:13 PM Derrick Stolee via GitGitGadget\n> wrote:\n>> Advanced and expert users may want to know how 'git maintenance start'\n>> schedules background maintenance in order to customize their own\n>> schedules beyond what the maintenance.* config values allow. Start a new\n>> set of sections in git-maintenance.txt that describe how 'cron' is used\n>> to run these tasks.\n>>\n>> This is particularly valuable for users who want to inspect what Git is\n>> doing or for users who want to customize the schedule further. Having a\n>> baseline can provide a way forward for users who have never worked with\n>> cron schedules.\n>>\n>> Signed-off-by: Derrick Stolee \n>> ---\n>> diff --git a/Documentation/git-maintenance.txt b/Documentation/git-maintenance.txt\n>> @@ -218,6 +218,60 @@ Further, the `git gc` command should not be combined with\n>> +The comments are used as a region to mark the schedule as written by Git.\n>> +Any modifications within this region will be completely deleted by\n>> +`git maintenance stop` or overwritten by `git maintenance start`.\n>> +\n>> +The `` string is loaded to specifically use the location for the\n>> +`git` executable used in the `git maintenance start` command. This allows\n>> +for multiple versions to be compatible. However, if the same user runs\n>> +`git maintenance start` with multiple Git executables, then only the\n>> +latest executable will be used.\n> \n> It looks like this section in v4 got accidentally reverted to the\n> wording from v2, whereas v3 had been changed to:> \n> The `crontab` entry specifies the full path of the `git`\n> executable to ensure that the executed `git` command is the same\n> one with which `git maintenance start` was issued independent of\n> `PATH`. If the same user runs `git maintenance start` with\n> multiple Git executables, then only the latest executable is used.\n\nEmbarassing. Thanks for the catch.\n\nThanks,\n-Stolee\n"},{"id":"410274","messageId":"CAPig+cQt+RFpikr-8cXfLtsXapS4tK4gufKZa-U1yN-ynJbiow@mail.gmail.com","threadId":"54562","inReplyTo":"e8922076-1ed4-b540-f1c9-9b4a8fa7a085@gmail.com","subject":"Re: [PATCH v4 4/4] maintenance: use Windows scheduled tasks","fromName":"Eric Sunshine","fromEmail":"sunshine@sunshineco.com","sentAt":"2020-11-18T20:54:32Z","receivedAt":"2020-11-18T20:55:06Z","isPatch":true,"sender":{"key":"sunshine@sunshineco.com","avatar":"https://avatars.githubusercontent.com/u/163641?v=4"},"body":"On Wed, Nov 18, 2020 at 1:30 PM Derrick Stolee wrote:\n> On 11/18/2020 2:15 AM, Eric Sunshine wrote:\n> > On Tue, Nov 17, 2020 at 4:13 PM Derrick Stolee via GitGitGadget\n> > wrote:\n> >> + xmlpath = xstrfmt(\"%s/schedule-%s.xml\",\n> >> + the_repository->objects->odb->path,\n> >> + frequency);\n> >\n> > I missed this in the earlier rounds since I wasn't paying close enough\n> > attention, but placing this XML file within the object database\n> > directory (.git/objects/) feels rather odd, even if it is just a\n> > temporary file. Using the .git/ directory itself might be better,\n> > perhaps like this:\n>\n> It does look odd, and in this case we could use the .git directory\n> instead. I specifically use the objects directory for the maintenance\n> lock in 'git maintenance run' to allow maintenance to run when\n> GIT_OBJECT_DIRECTORY points to an alternate, allowing us to maintain\n> object databases that don't have a full .git directory around them.\n\nI guess I'm confused. Won't a Git \"common\" directory exist even for\nsuch a case when GIT_OBJECT_DIRECTORY is pointing elsewhere, whether\nthe \"common\" directory is .git/ or a bare repository, or whatnot?\n\nAnyhow, this brings us back to my original suggestion of creating\nthese temporary files in a genuine temporary directory (/tmp or\n$TMPDIR or $TEMP) instead of arbitrarily choosing a path within the\nrepository itself. An important reason for using a genuine temporary\ndirectory for these temporary XML files is that it makes it less\nconfusing for those who come along later and try to understand this\ncode; they won't have to puzzle out why it is using a repository\nlocation for a file which is clearly temporary.\n\nTo make this really simple, you could use one of the\nx?mks_tempfile_t*() functions from tempfile.h which will automatically\nplace the file in $TMPDIR, thus relieving this code from having to\nmake the choice. Doing so would simplify this code even further since\nyou would replace create_tempfile() with x?mks_tempfile_t*(), and\nwouldn't have to maintain (or free) `xmlpath` manually.\n\nAs for the test script, the `print-args` is already picking up the\npathname of the temporary file specified by the /xml option, so it\nshould be possible to make the rest of the test work with the\ngenerated temporary filename.\n"},{"id":"410277","messageId":"4ab08301-274a-cd38-ab19-5bad55ec4df8@gmail.com","threadId":"54562","inReplyTo":"CAPig+cQt+RFpikr-8cXfLtsXapS4tK4gufKZa-U1yN-ynJbiow@mail.gmail.com","subject":"Re: [PATCH v4 4/4] maintenance: use Windows scheduled tasks","fromName":"Derrick Stolee","fromEmail":"stolee@gmail.com","sentAt":"2020-11-18T21:16:54Z","receivedAt":"2020-11-18T21:16:59Z","isPatch":true,"sender":{"key":"stolee@gmail.com","avatar":"https://avatars.githubusercontent.com/u/570044?v=4"},"body":"On 11/18/2020 3:54 PM, Eric Sunshine wrote:\n> On Wed, Nov 18, 2020 at 1:30 PM Derrick Stolee wrote:\n>> On 11/18/2020 2:15 AM, Eric Sunshine wrote:\n>>> On Tue, Nov 17, 2020 at 4:13 PM Derrick Stolee via GitGitGadget\n>>> wrote:\n>>>> + xmlpath = xstrfmt(\"%s/schedule-%s.xml\",\n>>>> + the_repository->objects->odb->path,\n>>>> + frequency);\n>>>\n>>> I missed this in the earlier rounds since I wasn't paying close enough\n>>> attention, but placing this XML file within the object database\n>>> directory (.git/objects/) feels rather odd, even if it is just a\n>>> temporary file. Using the .git/ directory itself might be better,\n>>> perhaps like this:\n>>\n>> It does look odd, and in this case we could use the .git directory\n>> instead. I specifically use the objects directory for the maintenance\n>> lock in 'git maintenance run' to allow maintenance to run when\n>> GIT_OBJECT_DIRECTORY points to an alternate, allowing us to maintain\n>> object databases that don't have a full .git directory around them.\n> \n> I guess I'm confused. Won't a Git \"common\" directory exist even for\n> such a case when GIT_OBJECT_DIRECTORY is pointing elsewhere, whether\n> the \"common\" directory is .git/ or a bare repository, or whatnot?\n\nThe reason to use the object dir for the 'git maintenance run' lock\nis to avoid multiple enlistments pointing at a common alternate from\nrunning concurrent maintenance on the same object directory.\n\nThat doesn't really apply to the temp files in this patch.\n\n> Anyhow, this brings us back to my original suggestion of creating\n> these temporary files in a genuine temporary directory (/tmp or\n> $TMPDIR or $TEMP) instead of arbitrarily choosing a path within the\n> repository itself. An important reason for using a genuine temporary\n> directory for these temporary XML files is that it makes it less\n> confusing for those who come along later and try to understand this\n> code; they won't have to puzzle out why it is using a repository\n> location for a file which is clearly temporary.\n> \n> To make this really simple, you could use one of the\n> x?mks_tempfile_t*() functions from tempfile.h which will automatically\n> place the file in $TMPDIR, thus relieving this code from having to\n> make the choice. Doing so would simplify this code even further since\n> you would replace create_tempfile() with x?mks_tempfile_t*(), and\n> wouldn't have to maintain (or free) `xmlpath` manually.\n> \n> As for the test script, the `print-args` is already picking up the\n> pathname of the temporary file specified by the /xml option, so it\n> should be possible to make the rest of the test work with the\n> generated temporary filename.\n \nI'll adopt your recommendations here. Thanks.\n-Stolee\n"},{"id":"410619","messageId":"80762efd-c71b-4485-a2bb-f0577d90ff48@gmail.com","threadId":"54562","inReplyTo":"X7ReZXuwAaAZzMSU@flurp.local","subject":"Re: [PATCH v4 0/4] Maintenance IV: Platform-specific background maintenance","fromName":"Derrick Stolee","fromEmail":"stolee@gmail.com","sentAt":"2020-11-24T02:20:46Z","receivedAt":"2020-11-24T02:20:51Z","isPatch":true,"sender":{"key":"stolee@gmail.com","avatar":"https://avatars.githubusercontent.com/u/570044?v=4"},"body":"On 11/17/2020 6:36 PM, Eric Sunshine wrote:\n> On Tue, Nov 17, 2020 at 09:13:14PM +0000, Derrick Stolee via GitGitGadget wrote:\n>> Updates in V4\n>> =============\n>> * Eric did an excellent job providing a patch that cleans up several parts\n>> of my series. The most impressive is his mechanism for testing the\n>> platform-specific Git logic in a way that is (mostly) platform-agnostic.\n>> \n>> * Windows doesn't have the 'id' command, so we cannot run the macOS\n>> platform test on Windows.\n> \n> This is easy to resolve. Drop in the following patch and then replace\n> the `$(id -u)` invocation in the test with `$(test-tool getuid)`.\n> This way, the test should work on any platform since both\n> launchctl_get_uid() and `test-tool` will retrieve identical values for\n> UID.\n\nI was giving your 'test-tool getuid' idea a try, and found that _also_\nthe $HOME environment variable differs from the format we expect in these\nsubcommands:\n\n $HOME: C:\\...\n argument in subcommand: /c/...\n\nSo, there is another reason why these tests don't work on Windows. I'm\nof the opinion that maybe it's not worth _that_ level of cross-platform\ntesting.\n\nUnless I'm missing something simple about a $HOME alternative here, this\nseems to be more work than the resulting value. Personally, I'm happy\nwith the benefit you've already provided in allowing Linux to test all\nplatforms.\n\nThanks,\n-Stolee\n"},{"id":"410626","messageId":"CAPig+cQA35f1RfrvoevTJPeAbxDtoXqdM-N8D_=hw7_ZaNYCAA@mail.gmail.com","threadId":"54562","inReplyTo":"80762efd-c71b-4485-a2bb-f0577d90ff48@gmail.com","subject":"Re: [PATCH v4 0/4] Maintenance IV: Platform-specific background maintenance","fromName":"Eric Sunshine","fromEmail":"sunshine@sunshineco.com","sentAt":"2020-11-24T02:59:14Z","receivedAt":"2020-11-24T02:59:28Z","isPatch":true,"sender":{"key":"sunshine@sunshineco.com","avatar":"https://avatars.githubusercontent.com/u/163641?v=4"},"body":"On Mon, Nov 23, 2020 at 9:20 PM Derrick Stolee wrote:\n> I was giving your 'test-tool getuid' idea a try, and found that _also_\n> the $HOME environment variable differs from the format we expect in these\n> subcommands:\n>\n> $HOME: C:\\...\n> argument in subcommand: /c/...\n\nWhere does this problem crop up exactly? Is the test doing a literal\ncomparison against the value in $HOME?\n\n> So, there is another reason why these tests don't work on Windows. I'm\n> of the opinion that maybe it's not worth _that_ level of cross-platform\n> testing.\n>\n> Unless I'm missing something simple about a $HOME alternative here, this\n> seems to be more work than the resulting value. Personally, I'm happy\n> with the benefit you've already provided in allowing Linux to test all\n> platforms.\n\nIndeed, it's not worth investing a lot of additional time into it. And\nit's certainly not a good reason to hold up the series. Moreover, this\nis the sort of thing which can be refined/handled later if someone\nwants to take a shot at it.\n"},{"id":"410631","messageId":"pull.776.v5.git.1606191405.gitgitgadget@gmail.com","threadId":"54562","inReplyTo":"pull.776.v4.git.1605647598.gitgitgadget@gmail.com","subject":"[PATCH v5 0/4] Maintenance IV: Platform-specific background maintenance","fromName":"Derrick Stolee via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2020-11-24T04:16:41Z","receivedAt":"2020-11-24T04:16:51Z","isPatch":true,"sender":{"key":"stolee@gmail.com","avatar":"https://avatars.githubusercontent.com/u/570044?v=4"},"body":"This is based on ds/maintenance-part-3.\n\nAfter sitting with the background maintenance as it has been cooking, I\nwanted to come back around and implement the background maintenance for\nWindows. However, I noticed that there were some things bothering me with\nbackground maintenance on my macOS machine. These are detailed in PATCH 3,\nbut the tl;dr is that 'cron' is not recommended by Apple and instead\n'launchd' satisfies our needs.\n\nThis series implements the background scheduling so git maintenance\n(start|stop) works on those platforms. I've been operating with these\nschedules for a while now without the problems described in the patches.\n\nThere is a particularly annoying case about console windows popping up on\nWindows, but PATCH 4 describes a plan to get around that.\n\nUpdates in V5\n=============\n\n * Fixed docs from PATCH 2 to match those in v3.\n \n \n * Despite my best efforts, I was unable to make the macOS tests work on\n Windows, so they are still marked with \"!MINGW\". I updated the commit\n message to describe my problems there.\n \n \n * The Windows platform now uses xmks_tempfile() to create the XML files for\n 'schtasks'. This led to some test fallout since the pathnames are no\n longer predictable.\n \n \n\nThanks, -Stolee\n\nDerrick Stolee (4):\n maintenance: extract platform-specific scheduling\n maintenance: include 'cron' details in docs\n maintenance: use launchctl on macOS\n maintenance: use Windows scheduled tasks\n\n Documentation/git-maintenance.txt | 116 +++++++++\n builtin/gc.c | 416 ++++++++++++++++++++++++++++--\n t/t7900-maintenance.sh | 110 +++++++-\n t/test-lib.sh | 7 +-\n 4 files changed, 615 insertions(+), 34 deletions(-)\n\n\nbase-commit: 0016b618182f642771dc589cf0090289f9fe1b4f\nPublished-As: https://github.com/gitgitgadget/git/releases/tag/pr-776%2Fderrickstolee%2Fmaintenance%2FmacOS-v5\nFetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-776/derrickstolee/maintenance/macOS-v5\nPull-Request: https://github.com/gitgitgadget/git/pull/776\n\nRange-diff vs v4:\n\n 1: 4807342b00 = 1: 4807342b00 maintenance: extract platform-specific scheduling\n 2: 99170df462 ! 2: 7cc70a8fe7 maintenance: include 'cron' details in docs\n @@ Documentation/git-maintenance.txt: Further, the `git gc` command should not be c\n +---------------------------------------\n +\n +The standard mechanism for scheduling background tasks on POSIX systems\n -+is `cron`. This tool executes commands based on a given schedule. The\n ++is cron(8). This tool executes commands based on a given schedule. The\n +current list of user-scheduled tasks can be found by running `crontab -l`.\n +The schedule written by `git maintenance start` is similar to this:\n +\n @@ Documentation/git-maintenance.txt: Further, the `git gc` command should not be c\n +Any modifications within this region will be completely deleted by\n +`git maintenance stop` or overwritten by `git maintenance start`.\n +\n -+The `` string is loaded to specifically use the location for the\n -+`git` executable used in the `git maintenance start` command. This allows\n -+for multiple versions to be compatible. However, if the same user runs\n -+`git maintenance start` with multiple Git executables, then only the\n -+latest executable will be used.\n ++The `crontab` entry specifies the full path of the `git` executable to\n ++ensure that the executed `git` command is the same one with which\n ++`git maintenance start` was issued independent of `PATH`. If the same user\n ++runs `git maintenance start` with multiple Git executables, then only the\n ++latest executable is used.\n +\n +These commands use `git for-each-repo --config=maintenance.repo` to run\n +`git maintenance run --schedule=` on each repository listed in\n +the multi-valued `maintenance.repo` config option. These are typically\n -+loaded from the user-specific global config located at `~/.gitconfig`.\n -+The `git maintenance` process then determines which maintenance tasks\n -+are configured to run on each repository with each `` using\n -+the `maintenance..schedule` config options. These values are loaded\n -+from the global or repository config values.\n ++loaded from the user-specific global config. The `git maintenance` process\n ++then determines which maintenance tasks are configured to run on each\n ++repository with each `` using the `maintenance..schedule`\n ++config options. These values are loaded from the global or repository\n ++config values.\n +\n +If the config values are insufficient to achieve your desired background\n +maintenance schedule, then you can create your own schedule. If you run\n +`crontab -e`, then an editor will load with your user-specific `cron`\n +schedule. In that editor, you can add your own schedule lines. You could\n +start by adapting the default schedule listed earlier, or you could read\n -+https://man7.org/linux/man-pages/man5/crontab.5.html[the `crontab` documentation]\n -+for advanced scheduling techniques. Please do use the full path and\n -+`--exec-path` techniques from the default schedule to ensure you are\n -+executing the correct binaries in your schedule.\n ++the crontab(5) documentation for advanced scheduling techniques. Please\n ++do use the full path and `--exec-path` techniques from the default\n ++schedule to ensure you are executing the correct binaries in your\n ++schedule.\n +\n \n GIT\n 3: ed0a0011fb ! 3: cd015a5cbd maintenance: use launchctl on macOS\n @@ Commit message\n subcommand will succeed, if such a task already exists.\n \n The need for a user id requires us to run 'id -u' which works on\n - POSIX systems but not Windows. The test therefore has a prerequisite\n - that we are not on Windows. The cross-platform logic still allows us to\n - test the macOS logic on a Linux machine.\n + POSIX systems but not Windows. Further, the need for fully-qualitifed\n + path names including $HOME behaves differently in the Git internals and\n + the external test suite. The $HOME variable starts with \"C:\\...\" instead\n + of the \"/c/...\" that is provided by Git in these subcommands. The test\n + therefore has a prerequisite that we are not on Windows. The cross-\n + platform logic still allows us to test the macOS logic on a Linux\n + machine.\n \n We can verify the commands that were run by 'git maintenance start'\n and 'git maintenance stop' by injecting a script that writes the\n @@ Commit message\n Signed-off-by: Derrick Stolee \n \n ## Documentation/git-maintenance.txt ##\n -@@ Documentation/git-maintenance.txt: for advanced scheduling techniques. Please do use the full path and\n - executing the correct binaries in your schedule.\n +@@ Documentation/git-maintenance.txt: schedule to ensure you are executing the correct binaries in your\n + schedule.\n \n \n +BACKGROUND MAINTENANCE ON MACOS SYSTEMS\n 4: b8d86fb983 ! 4: ac9a28bea3 maintenance: use Windows scheduled tasks\n @@ Commit message\n \n Since the GIT_TEST_MAINT_SCHEDULER environment variable allows us to\n specify 'schtasks' as the scheduler, we can test the Windows-specific\n - logic on a macOS platform. Thus, add a check that the XML file written\n + logic on other platforms. Thus, add a check that the XML file written\n by Git is valid when xmllint exists on the system.\n \n + Since we use a temporary file for the XML files sent to 'schtasks', we\n + must copy the file to a predictable filename. Use the number of lines in\n + the 'args' file to provide a filename for xmllint. Instead of an exact\n + match on the 'args' file, we 'grep' for the arguments other than the\n + filename.\n +\n There is a deficiency in the current design. Windows has two kinds of\n applications: GUI applications that start by \"winmain()\" and console\n applications that start by \"main()\". Console applications are attached\n @@ builtin/gc.c: static int launchctl_update_schedule(int run_maintenance, int fd,\n +\tint result;\n +\tstruct child_process child = CHILD_PROCESS_INIT;\n +\tconst char *xml;\n -+\tchar *xmlpath;\n +\tstruct tempfile *tfile;\n +\tconst char *frequency = get_frequency(schedule);\n +\tchar *name = schtasks_task_name(frequency);\n +\n -+\txmlpath = xstrfmt(\"%s/schedule-%s.xml\",\n -+\t\t\t the_repository->objects->odb->path,\n -+\t\t\t frequency);\n -+\ttfile = create_tempfile(xmlpath);\n ++\ttfile = xmks_tempfile(\"schedule_XXXXXX\");\n +\tif (!tfile || !fdopen_tempfile(tfile, \"w\"))\n -+\t\tdie(_(\"failed to create '%s'\"), xmlpath);\n ++\t\tdie(_(\"failed to create temp xml file\"));\n +\n +\txml = \"\\n\"\n +\t \"\\n\"\n @@ builtin/gc.c: static int launchctl_update_schedule(int run_maintenance, int fd,\n +\t \"\\n\";\n +\tfprintf(tfile->fp, xml, exec_path, exec_path, frequency);\n +\tstrvec_split(&child.args, cmd);\n -+\tstrvec_pushl(&child.args, \"/create\", \"/tn\", name, \"/f\", \"/xml\", xmlpath, NULL);\n ++\tstrvec_pushl(&child.args, \"/create\", \"/tn\", name, \"/f\", \"/xml\", tfile->filename.buf, NULL);\n +\tclose_tempfile_gently(tfile);\n +\n +\tchild.no_stdout = 1;\n @@ builtin/gc.c: static int launchctl_update_schedule(int run_maintenance, int fd,\n +\tresult = finish_command(&child);\n +\n +\tdelete_tempfile(&tfile);\n -+\tfree(xmlpath);\n +\tfree(name);\n +\treturn result;\n +}\n @@ t/t7900-maintenance.sh: test_expect_success !MINGW 'start and stop macOS mainten\n +\t\t*) shift ;;\n +\t\tesac\n +\tdone\n -+\ttest -z \"$xmlfile\" || cp \"$xmlfile\" .\n ++\tlines=$(wc -l args | awk \"{print \\$1;}\")\n ++\ttest -z \"$xmlfile\" || cp \"$xmlfile\" \"schedule-$lines.xml\"\n +\tEOF\n +\n +\trm -f args &&\n @@ t/t7900-maintenance.sh: test_expect_success !MINGW 'start and stop macOS mainten\n +\t# start registers the repo\n +\tgit config --get --global maintenance.repo \"$(pwd)\" &&\n +\n -+\tprintf \"/create /tn Git Maintenance (%s) /f /xml .git/objects/schedule-%s.xml\\n\" \\\n -+\t\thourly hourly daily daily weekly weekly >expect &&\n -+\ttest_cmp expect args &&\n -+\n +\tfor frequency in hourly daily weekly\n +\tdo\n -+\t\ttest_xmllint \"schedule-$frequency.xml\"\n ++\t\tgrep \"/create /tn Git Maintenance ($frequency) /f /xml\" args \\\n ++\t\t\t|| return 1\n ++\tdone &&\n ++\n ++\tfor i in 1 2 3\n ++\tdo\n ++\t\ttest_xmllint \"schedule-$i.xml\" &&\n ++\t\tgrep \"encoding=.US-ASCII.\" \"schedule-$i.xml\" || return 1\n +\tdone &&\n +\n +\trm -f args &&\n\n-- \ngitgitgadget\n"},{"id":"410634","messageId":"4807342b0019be29bb369ed3403a485f0ce9c15d.1606191405.git.gitgitgadget@gmail.com","threadId":"54562","inReplyTo":"pull.776.v5.git.1606191405.gitgitgadget@gmail.com","subject":"[PATCH v5 1/4] maintenance: extract platform-specific scheduling","fromName":"Derrick Stolee via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2020-11-24T04:16:42Z","receivedAt":"2020-11-24T04:16:52Z","isPatch":true,"sender":{"key":"stolee@gmail.com","avatar":"https://avatars.githubusercontent.com/u/570044?v=4"},"body":"From: Derrick Stolee \n\nThe existing schedule mechanism using 'cron' is supported by POSIX\nplatforms, but not Windows. It also works slightly differently on\nmacOS to significant detriment of the user experience. To allow for\nnew implementations on these platforms, extract a method that\nperforms the platform-specific scheduling mechanism. This will be\nswapped at compile time with new implementations on specialized\nplatforms.\n\nAs we add this generality, rename GIT_TEST_CRONTAB to\nGIT_TEST_MAINT_SCHEDULER. Further, this variable is now parsed as\n\":\" so we can test platform-specific scheduling\nlogic even when not on the correct platform. By specifying the\n in this string, we will be able to test all three sets of\nGit logic from a Linux machine.\n\nCo-authored-by: Eric Sunshine \nSigned-off-by: Eric Sunshine \nSigned-off-by: Derrick Stolee \n---\n builtin/gc.c | 70 ++++++++++++++++++++++++++----------------\n t/t7900-maintenance.sh | 8 ++---\n t/test-lib.sh | 7 +++--\n 3 files changed, 51 insertions(+), 34 deletions(-)\n\ndiff --git a/builtin/gc.c b/builtin/gc.c\nindex e3098ef6a1..18ae7f7138 100644\n--- a/builtin/gc.c\n+++ b/builtin/gc.c\n@@ -1494,35 +1494,23 @@ static int maintenance_unregister(void)\n #define BEGIN_LINE \"# BEGIN GIT MAINTENANCE SCHEDULE\"\n #define END_LINE \"# END GIT MAINTENANCE SCHEDULE\"\n \n-static int update_background_schedule(int run_maintenance)\n+static int crontab_update_schedule(int run_maintenance, int fd, const char *cmd)\n {\n \tint result = 0;\n \tint in_old_region = 0;\n \tstruct child_process crontab_list = CHILD_PROCESS_INIT;\n \tstruct child_process crontab_edit = CHILD_PROCESS_INIT;\n \tFILE *cron_list, *cron_in;\n-\tconst char *crontab_name;\n \tstruct strbuf line = STRBUF_INIT;\n-\tstruct lock_file lk;\n-\tchar *lock_path = xstrfmt(\"%s/schedule\", the_repository->objects->odb->path);\n \n-\tif (hold_lock_file_for_update(&lk, lock_path, LOCK_NO_DEREF) < 0)\n-\t\treturn error(_(\"another process is scheduling background maintenance\"));\n-\n-\tcrontab_name = getenv(\"GIT_TEST_CRONTAB\");\n-\tif (!crontab_name)\n-\t\tcrontab_name = \"crontab\";\n-\n-\tstrvec_split(&crontab_list.args, crontab_name);\n+\tstrvec_split(&crontab_list.args, cmd);\n \tstrvec_push(&crontab_list.args, \"-l\");\n \tcrontab_list.in = -1;\n-\tcrontab_list.out = dup(lk.tempfile->fd);\n+\tcrontab_list.out = dup(fd);\n \tcrontab_list.git_cmd = 0;\n \n-\tif (start_command(&crontab_list)) {\n-\t\tresult = error(_(\"failed to run 'crontab -l'; your system might not support 'cron'\"));\n-\t\tgoto cleanup;\n-\t}\n+\tif (start_command(&crontab_list))\n+\t\treturn error(_(\"failed to run 'crontab -l'; your system might not support 'cron'\"));\n \n \t/* Ignore exit code, as an empty crontab will return error. */\n \tfinish_command(&crontab_list);\n@@ -1531,17 +1519,15 @@ static int update_background_schedule(int run_maintenance)\n \t * Read from the .lock file, filtering out the old\n \t * schedule while appending the new schedule.\n \t */\n-\tcron_list = fdopen(lk.tempfile->fd, \"r\");\n+\tcron_list = fdopen(fd, \"r\");\n \trewind(cron_list);\n \n-\tstrvec_split(&crontab_edit.args, crontab_name);\n+\tstrvec_split(&crontab_edit.args, cmd);\n \tcrontab_edit.in = -1;\n \tcrontab_edit.git_cmd = 0;\n \n-\tif (start_command(&crontab_edit)) {\n-\t\tresult = error(_(\"failed to run 'crontab'; your system might not support 'cron'\"));\n-\t\tgoto cleanup;\n-\t}\n+\tif (start_command(&crontab_edit))\n+\t\treturn error(_(\"failed to run 'crontab'; your system might not support 'cron'\"));\n \n \tcron_in = fdopen(crontab_edit.in, \"w\");\n \tif (!cron_in) {\n@@ -1586,14 +1572,44 @@ static int update_background_schedule(int run_maintenance)\n \tclose(crontab_edit.in);\n \n done_editing:\n-\tif (finish_command(&crontab_edit)) {\n+\tif (finish_command(&crontab_edit))\n \t\tresult = error(_(\"'crontab' died\"));\n-\t\tgoto cleanup;\n+\telse\n+\t\tfclose(cron_list);\n+\treturn result;\n+}\n+\n+static const char platform_scheduler[] = \"crontab\";\n+\n+static int update_background_schedule(int enable)\n+{\n+\tint result;\n+\tconst char *scheduler = platform_scheduler;\n+\tconst char *cmd = scheduler;\n+\tchar *testing;\n+\tstruct lock_file lk;\n+\tchar *lock_path = xstrfmt(\"%s/schedule\", the_repository->objects->odb->path);\n+\n+\ttesting = xstrdup_or_null(getenv(\"GIT_TEST_MAINT_SCHEDULER\"));\n+\tif (testing) {\n+\t\tchar *sep = strchr(testing, ':');\n+\t\tif (!sep)\n+\t\t\tdie(\"GIT_TEST_MAINT_SCHEDULER unparseable: %s\", testing);\n+\t\t*sep = '\\0';\n+\t\tscheduler = testing;\n+\t\tcmd = sep + 1;\n \t}\n-\tfclose(cron_list);\n \n-cleanup:\n+\tif (hold_lock_file_for_update(&lk, lock_path, LOCK_NO_DEREF) < 0)\n+\t\treturn error(_(\"another process is scheduling background maintenance\"));\n+\n+\tif (!strcmp(scheduler, \"crontab\"))\n+\t\tresult = crontab_update_schedule(enable, lk.tempfile->fd, cmd);\n+\telse\n+\t\tdie(\"unknown background scheduler: %s\", scheduler);\n+\n \trollback_lock_file(&lk);\n+\tfree(testing);\n \treturn result;\n }\n \ndiff --git a/t/t7900-maintenance.sh b/t/t7900-maintenance.sh\nindex 20184e96e1..eeb939168d 100755\n--- a/t/t7900-maintenance.sh\n+++ b/t/t7900-maintenance.sh\n@@ -368,7 +368,7 @@ test_expect_success 'register and unregister' '\n '\n \n test_expect_success 'start from empty cron table' '\n-\tGIT_TEST_CRONTAB=\"test-tool crontab cron.txt\" git maintenance start &&\n+\tGIT_TEST_MAINT_SCHEDULER=\"crontab:test-tool crontab cron.txt\" git maintenance start &&\n \n \t# start registers the repo\n \tgit config --get --global maintenance.repo \"$(pwd)\" &&\n@@ -379,19 +379,19 @@ test_expect_success 'start from empty cron table' '\n '\n \n test_expect_success 'stop from existing schedule' '\n-\tGIT_TEST_CRONTAB=\"test-tool crontab cron.txt\" git maintenance stop &&\n+\tGIT_TEST_MAINT_SCHEDULER=\"crontab:test-tool crontab cron.txt\" git maintenance stop &&\n \n \t# stop does not unregister the repo\n \tgit config --get --global maintenance.repo \"$(pwd)\" &&\n \n \t# Operation is idempotent\n-\tGIT_TEST_CRONTAB=\"test-tool crontab cron.txt\" git maintenance stop &&\n+\tGIT_TEST_MAINT_SCHEDULER=\"crontab:test-tool crontab cron.txt\" git maintenance stop &&\n \ttest_must_be_empty cron.txt\n '\n \n test_expect_success 'start preserves existing schedule' '\n \techo \"Important information!\" >cron.txt &&\n-\tGIT_TEST_CRONTAB=\"test-tool crontab cron.txt\" git maintenance start &&\n+\tGIT_TEST_MAINT_SCHEDULER=\"crontab:test-tool crontab cron.txt\" git maintenance start &&\n \tgrep \"Important information!\" cron.txt\n '\n \ndiff --git a/t/test-lib.sh b/t/test-lib.sh\nindex 4a60d1ed76..ddbeee1f5e 100644\n--- a/t/test-lib.sh\n+++ b/t/test-lib.sh\n@@ -1704,7 +1704,8 @@ test_lazy_prereq REBASE_P '\n '\n \n # Ensure that no test accidentally triggers a Git command\n-# that runs 'crontab', affecting a user's cron schedule.\n-# Tests that verify the cron integration must set this locally\n+# that runs the actual maintenance scheduler, affecting a user's\n+# system permanently.\n+# Tests that verify the scheduler integration must set this locally\n # to avoid errors.\n-GIT_TEST_CRONTAB=\"exit 1\"\n+GIT_TEST_MAINT_SCHEDULER=\"none:exit 1\"\n-- \ngitgitgadget\n\n"},{"id":"410632","messageId":"7cc70a8fe7bad92179961b31520147af39d5353c.1606191405.git.gitgitgadget@gmail.com","threadId":"54562","inReplyTo":"pull.776.v5.git.1606191405.gitgitgadget@gmail.com","subject":"[PATCH v5 2/4] maintenance: include 'cron' details in docs","fromName":"Derrick Stolee via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2020-11-24T04:16:43Z","receivedAt":"2020-11-24T04:16:54Z","isPatch":true,"sender":{"key":"stolee@gmail.com","avatar":"https://avatars.githubusercontent.com/u/570044?v=4"},"body":"From: Derrick Stolee \n\nAdvanced and expert users may want to know how 'git maintenance start'\nschedules background maintenance in order to customize their own\nschedules beyond what the maintenance.* config values allow. Start a new\nset of sections in git-maintenance.txt that describe how 'cron' is used\nto run these tasks.\n\nThis is particularly valuable for users who want to inspect what Git is\ndoing or for users who want to customize the schedule further. Having a\nbaseline can provide a way forward for users who have never worked with\ncron schedules.\n\nHelped-by: Eric Sunshine \nSigned-off-by: Derrick Stolee \n---\n Documentation/git-maintenance.txt | 54 +++++++++++++++++++++++++++++++\n 1 file changed, 54 insertions(+)\n\ndiff --git a/Documentation/git-maintenance.txt b/Documentation/git-maintenance.txt\nindex 6fec1eb8dc..1aa1112418 100644\n--- a/Documentation/git-maintenance.txt\n+++ b/Documentation/git-maintenance.txt\n@@ -218,6 +218,60 @@ Further, the `git gc` command should not be combined with\n but does not take the lock in the same way as `git maintenance run`. If\n possible, use `git maintenance run --task=gc` instead of `git gc`.\n \n+The following sections describe the mechanisms put in place to run\n+background maintenance by `git maintenance start` and how to customize\n+them.\n+\n+BACKGROUND MAINTENANCE ON POSIX SYSTEMS\n+---------------------------------------\n+\n+The standard mechanism for scheduling background tasks on POSIX systems\n+is cron(8). This tool executes commands based on a given schedule. The\n+current list of user-scheduled tasks can be found by running `crontab -l`.\n+The schedule written by `git maintenance start` is similar to this:\n+\n+-----------------------------------------------------------------------\n+# BEGIN GIT MAINTENANCE SCHEDULE\n+# The following schedule was created by Git\n+# Any edits made in this region might be\n+# replaced in the future by a Git command.\n+\n+0 1-23 * * * \"//git\" --exec-path=\"/\" for-each-repo --config=maintenance.repo maintenance run --schedule=hourly\n+0 0 * * 1-6 \"//git\" --exec-path=\"/\" for-each-repo --config=maintenance.repo maintenance run --schedule=daily\n+0 0 * * 0 \"//git\" --exec-path=\"/\" for-each-repo --config=maintenance.repo maintenance run --schedule=weekly\n+\n+# END GIT MAINTENANCE SCHEDULE\n+-----------------------------------------------------------------------\n+\n+The comments are used as a region to mark the schedule as written by Git.\n+Any modifications within this region will be completely deleted by\n+`git maintenance stop` or overwritten by `git maintenance start`.\n+\n+The `crontab` entry specifies the full path of the `git` executable to\n+ensure that the executed `git` command is the same one with which\n+`git maintenance start` was issued independent of `PATH`. If the same user\n+runs `git maintenance start` with multiple Git executables, then only the\n+latest executable is used.\n+\n+These commands use `git for-each-repo --config=maintenance.repo` to run\n+`git maintenance run --schedule=` on each repository listed in\n+the multi-valued `maintenance.repo` config option. These are typically\n+loaded from the user-specific global config. The `git maintenance` process\n+then determines which maintenance tasks are configured to run on each\n+repository with each `` using the `maintenance..schedule`\n+config options. These values are loaded from the global or repository\n+config values.\n+\n+If the config values are insufficient to achieve your desired background\n+maintenance schedule, then you can create your own schedule. If you run\n+`crontab -e`, then an editor will load with your user-specific `cron`\n+schedule. In that editor, you can add your own schedule lines. You could\n+start by adapting the default schedule listed earlier, or you could read\n+the crontab(5) documentation for advanced scheduling techniques. Please\n+do use the full path and `--exec-path` techniques from the default\n+schedule to ensure you are executing the correct binaries in your\n+schedule.\n+\n \n GIT\n ---\n-- \ngitgitgadget\n\n"},{"id":"410633","messageId":"ac9a28bea39b4d1897ad6a41331e52770c8c7531.1606191405.git.gitgitgadget@gmail.com","threadId":"54562","inReplyTo":"pull.776.v5.git.1606191405.gitgitgadget@gmail.com","subject":"[PATCH v5 4/4] maintenance: use Windows scheduled tasks","fromName":"Derrick Stolee via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2020-11-24T04:16:45Z","receivedAt":"2020-11-24T04:16:55Z","isPatch":true,"sender":{"key":"stolee@gmail.com","avatar":"https://avatars.githubusercontent.com/u/570044?v=4"},"body":"From: Derrick Stolee \n\nGit's background maintenance uses cron by default, but this is not\navailable on Windows. Instead, integrate with Task Scheduler.\n\nTasks can be scheduled using the 'schtasks' command. There are several\ncommand-line options that can allow for some advanced scheduling, but\nunfortunately these seem to all require authenticating using a password.\n\nInstead, use the \"/xml\" option to pass an XML file that contains the\nconfiguration for the necessary schedule. These XML files are based on\nsome that I exported after constructing a schedule in the Task Scheduler\nGUI. These options only run background maintenance when the user is\nlogged in, and more fields are populated with the current username and\nSID at run-time by 'schtasks'.\n\nSince the GIT_TEST_MAINT_SCHEDULER environment variable allows us to\nspecify 'schtasks' as the scheduler, we can test the Windows-specific\nlogic on other platforms. Thus, add a check that the XML file written\nby Git is valid when xmllint exists on the system.\n\nSince we use a temporary file for the XML files sent to 'schtasks', we\nmust copy the file to a predictable filename. Use the number of lines in\nthe 'args' file to provide a filename for xmllint. Instead of an exact\nmatch on the 'args' file, we 'grep' for the arguments other than the\nfilename.\n\nThere is a deficiency in the current design. Windows has two kinds of\napplications: GUI applications that start by \"winmain()\" and console\napplications that start by \"main()\". Console applications are attached\nto a new Console window if they are not already associated with a GUI\napplication. This means that every hour the scheudled task launches a\ncommand window for the scheduled tasks. Not only is this visually\nobtrusive, but it also takes focus from whatever else the user is\ndoing!\n\nA simple fix would be to insert a GUI application that acts as a shim\nbetween the scheduled task and Git. This is currently possible in Git\nfor Windows by setting the tag equal to\n\n C:\\Program Files\\Git\\git-bash.exe\n\nwith options \"--hide --no-needs-console --command=cmd\\git.exe\"\nfollowed by the arguments currently used. Since git-bash.exe is not\nincluded in Windows builds of core Git, I chose to leave out this\nfeature. My plan is to submit a small patch to Git for Windows that\nconverts the use of git.exe with this use of git-bash.exe in the\nshort term. In the long term, we can consider creating this GUI\nshim application within core Git, perhaps in contrib/.\n\nCo-authored-by: Eric Sunshine \nSigned-off-by: Eric Sunshine \nSigned-off-by: Derrick Stolee \n---\n Documentation/git-maintenance.txt | 22 ++++\n builtin/gc.c | 160 ++++++++++++++++++++++++++++++\n t/t7900-maintenance.sh | 44 ++++++++\n 3 files changed, 226 insertions(+)\n\ndiff --git a/Documentation/git-maintenance.txt b/Documentation/git-maintenance.txt\nindex 5f8f63f098..6970f2b898 100644\n--- a/Documentation/git-maintenance.txt\n+++ b/Documentation/git-maintenance.txt\n@@ -313,6 +313,28 @@ To create more advanced customizations to your background tasks, see\n launchctl.plist(5) for more information.\n \n \n+BACKGROUND MAINTENANCE ON WINDOWS SYSTEMS\n+-----------------------------------------\n+\n+Windows does not support `cron` and instead has its own system for\n+scheduling background tasks. The `git maintenance start` command uses\n+the `schtasks` command to submit tasks to this system. You can inspect\n+all background tasks using the Task Scheduler application. The tasks\n+added by Git have names of the form `Git Maintenance ()`.\n+The Task Scheduler GUI has ways to inspect these tasks, but you can also\n+export the tasks to XML files and view the details there.\n+\n+Note that since Git is a console application, these background tasks\n+create a console window visible to the current user. This can be changed\n+manually by selecting the \"Run whether user is logged in or not\" option\n+in Task Scheduler. This change requires a password input, which is why\n+`git maintenance start` does not select it by default.\n+\n+If you want to customize the background tasks, please rename the tasks\n+so future calls to `git maintenance (start|stop)` do not overwrite your\n+custom tasks.\n+\n+\n GIT\n ---\n Part of the linkgit:git[1] suite\ndiff --git a/builtin/gc.c b/builtin/gc.c\nindex 782769f243..43224e0dec 100644\n--- a/builtin/gc.c\n+++ b/builtin/gc.c\n@@ -1671,6 +1671,162 @@ static int launchctl_update_schedule(int run_maintenance, int fd, const char *cm\n \t\treturn launchctl_remove_plists(cmd);\n }\n \n+static char *schtasks_task_name(const char *frequency)\n+{\n+\tstruct strbuf label = STRBUF_INIT;\n+\tstrbuf_addf(&label, \"Git Maintenance (%s)\", frequency);\n+\treturn strbuf_detach(&label, NULL);\n+}\n+\n+static int schtasks_remove_task(enum schedule_priority schedule, const char *cmd)\n+{\n+\tint result;\n+\tstruct strvec args = STRVEC_INIT;\n+\tconst char *frequency = get_frequency(schedule);\n+\tchar *name = schtasks_task_name(frequency);\n+\n+\tstrvec_split(&args, cmd);\n+\tstrvec_pushl(&args, \"/delete\", \"/tn\", name, \"/f\", NULL);\n+\n+\tresult = run_command_v_opt(args.v, 0);\n+\n+\tstrvec_clear(&args);\n+\tfree(name);\n+\treturn result;\n+}\n+\n+static int schtasks_remove_tasks(const char *cmd)\n+{\n+\treturn schtasks_remove_task(SCHEDULE_HOURLY, cmd) ||\n+\t\tschtasks_remove_task(SCHEDULE_DAILY, cmd) ||\n+\t\tschtasks_remove_task(SCHEDULE_WEEKLY, cmd);\n+}\n+\n+static int schtasks_schedule_task(const char *exec_path, enum schedule_priority schedule, const char *cmd)\n+{\n+\tint result;\n+\tstruct child_process child = CHILD_PROCESS_INIT;\n+\tconst char *xml;\n+\tstruct tempfile *tfile;\n+\tconst char *frequency = get_frequency(schedule);\n+\tchar *name = schtasks_task_name(frequency);\n+\n+\ttfile = xmks_tempfile(\"schedule_XXXXXX\");\n+\tif (!tfile || !fdopen_tempfile(tfile, \"w\"))\n+\t\tdie(_(\"failed to create temp xml file\"));\n+\n+\txml = \"\\n\"\n+\t \"\\n\"\n+\t \"\\n\"\n+\t \"\\n\";\n+\tfputs(xml, tfile->fp);\n+\n+\tswitch (schedule) {\n+\tcase SCHEDULE_HOURLY:\n+\t\tfprintf(tfile->fp,\n+\t\t\t\"2020-01-01T01:00:00\\n\"\n+\t\t\t\"true\\n\"\n+\t\t\t\"\\n\"\n+\t\t\t\"1\\n\"\n+\t\t\t\"\\n\"\n+\t\t\t\"\\n\"\n+\t\t\t\"PT1H\\n\"\n+\t\t\t\"PT23H\\n\"\n+\t\t\t\"false\\n\"\n+\t\t\t\"\\n\");\n+\t\tbreak;\n+\n+\tcase SCHEDULE_DAILY:\n+\t\tfprintf(tfile->fp,\n+\t\t\t\"2020-01-01T00:00:00\\n\"\n+\t\t\t\"true\\n\"\n+\t\t\t\"\\n\"\n+\t\t\t\"\\n\"\n+\t\t\t\"\\n\"\n+\t\t\t\"\\n\"\n+\t\t\t\"\\n\"\n+\t\t\t\"\\n\"\n+\t\t\t\"\\n\"\n+\t\t\t\"\\n\"\n+\t\t\t\"\\n\"\n+\t\t\t\"1\\n\"\n+\t\t\t\"\\n\");\n+\t\tbreak;\n+\n+\tcase SCHEDULE_WEEKLY:\n+\t\tfprintf(tfile->fp,\n+\t\t\t\"2020-01-01T00:00:00\\n\"\n+\t\t\t\"true\\n\"\n+\t\t\t\"\\n\"\n+\t\t\t\"\\n\"\n+\t\t\t\"\\n\"\n+\t\t\t\"\\n\"\n+\t\t\t\"1\\n\"\n+\t\t\t\"\\n\");\n+\t\tbreak;\n+\n+\tdefault:\n+\t\tbreak;\n+\t}\n+\n+\txml = \"\\n\"\n+\t \"\\n\"\n+\t \"\\n\"\n+\t \"\\n\"\n+\t \"InteractiveToken\\n\"\n+\t \"LeastPrivilege\\n\"\n+\t \"\\n\"\n+\t \"\\n\"\n+\t \"\\n\"\n+\t \"IgnoreNew\\n\"\n+\t \"true\\n\"\n+\t \"true\\n\"\n+\t \"true\\n\"\n+\t \"false\\n\"\n+\t \"PT72H\\n\"\n+\t \"7\\n\"\n+\t \"\\n\"\n+\t \"\\n\"\n+\t \"\\n\"\n+\t \"\\\"%s\\\\git.exe\\\"\\n\"\n+\t \"--exec-path=\\\"%s\\\" for-each-repo --config=maintenance.repo maintenance run --schedule=%s\\n\"\n+\t \"\\n\"\n+\t \"\\n\"\n+\t \"\\n\";\n+\tfprintf(tfile->fp, xml, exec_path, exec_path, frequency);\n+\tstrvec_split(&child.args, cmd);\n+\tstrvec_pushl(&child.args, \"/create\", \"/tn\", name, \"/f\", \"/xml\", tfile->filename.buf, NULL);\n+\tclose_tempfile_gently(tfile);\n+\n+\tchild.no_stdout = 1;\n+\tchild.no_stderr = 1;\n+\n+\tif (start_command(&child))\n+\t\tdie(_(\"failed to start schtasks\"));\n+\tresult = finish_command(&child);\n+\n+\tdelete_tempfile(&tfile);\n+\tfree(name);\n+\treturn result;\n+}\n+\n+static int schtasks_schedule_tasks(const char *cmd)\n+{\n+\tconst char *exec_path = git_exec_path();\n+\n+\treturn schtasks_schedule_task(exec_path, SCHEDULE_HOURLY, cmd) ||\n+\t\tschtasks_schedule_task(exec_path, SCHEDULE_DAILY, cmd) ||\n+\t\tschtasks_schedule_task(exec_path, SCHEDULE_WEEKLY, cmd);\n+}\n+\n+static int schtasks_update_schedule(int run_maintenance, int fd, const char *cmd)\n+{\n+\tif (run_maintenance)\n+\t\treturn schtasks_schedule_tasks(cmd);\n+\telse\n+\t\treturn schtasks_remove_tasks(cmd);\n+}\n+\n #define BEGIN_LINE \"# BEGIN GIT MAINTENANCE SCHEDULE\"\n #define END_LINE \"# END GIT MAINTENANCE SCHEDULE\"\n \n@@ -1761,6 +1917,8 @@ static int crontab_update_schedule(int run_maintenance, int fd, const char *cmd)\n \n #if defined(__APPLE__)\n static const char platform_scheduler[] = \"launchctl\";\n+#elif defined(GIT_WINDOWS_NATIVE)\n+static const char platform_scheduler[] = \"schtasks\";\n #else\n static const char platform_scheduler[] = \"crontab\";\n #endif\n@@ -1789,6 +1947,8 @@ static int update_background_schedule(int enable)\n \n \tif (!strcmp(scheduler, \"launchctl\"))\n \t\tresult = launchctl_update_schedule(enable, lk.tempfile->fd, cmd);\n+\telse if (!strcmp(scheduler, \"schtasks\"))\n+\t\tresult = schtasks_update_schedule(enable, lk.tempfile->fd, cmd);\n \telse if (!strcmp(scheduler, \"crontab\"))\n \t\tresult = crontab_update_schedule(enable, lk.tempfile->fd, cmd);\n \telse\ndiff --git a/t/t7900-maintenance.sh b/t/t7900-maintenance.sh\nindex 6d37312901..0246e4ce30 100755\n--- a/t/t7900-maintenance.sh\n+++ b/t/t7900-maintenance.sh\n@@ -453,6 +453,50 @@ test_expect_success !MINGW 'start and stop macOS maintenance' '\n \ttest_line_count = 0 actual\n '\n \n+test_expect_success 'start and stop Windows maintenance' '\n+\twrite_script print-args <<-\\EOF &&\n+\techo $* >>args\n+\twhile test $# -gt 0\n+\tdo\n+\t\tcase \"$1\" in\n+\t\t/xml) shift; xmlfile=$1; break ;;\n+\t\t*) shift ;;\n+\t\tesac\n+\tdone\n+\tlines=$(wc -l args | awk \"{print \\$1;}\")\n+\ttest -z \"$xmlfile\" || cp \"$xmlfile\" \"schedule-$lines.xml\"\n+\tEOF\n+\n+\trm -f args &&\n+\tGIT_TEST_MAINT_SCHEDULER=\"schtasks:./print-args\" git maintenance start &&\n+\n+\t# start registers the repo\n+\tgit config --get --global maintenance.repo \"$(pwd)\" &&\n+\n+\tfor frequency in hourly daily weekly\n+\tdo\n+\t\tgrep \"/create /tn Git Maintenance ($frequency) /f /xml\" args \\\n+\t\t\t|| return 1\n+\tdone &&\n+\n+\tfor i in 1 2 3\n+\tdo\n+\t\ttest_xmllint \"schedule-$i.xml\" &&\n+\t\tgrep \"encoding=.US-ASCII.\" \"schedule-$i.xml\" || return 1\n+\tdone &&\n+\n+\trm -f args &&\n+\tGIT_TEST_MAINT_SCHEDULER=\"schtasks:./print-args\" git maintenance stop &&\n+\n+\t# stop does not unregister the repo\n+\tgit config --get --global maintenance.repo \"$(pwd)\" &&\n+\n+\trm expect &&\n+\tprintf \"/delete /tn Git Maintenance (%s) /f\\n\" \\\n+\t\thourly daily weekly >expect &&\n+\ttest_cmp expect args\n+'\n+\n test_expect_success 'register preserves existing strategy' '\n \tgit config maintenance.strategy none &&\n \tgit maintenance register &&\n-- \ngitgitgadget\n"},{"id":"410635","messageId":"cd015a5cbd7231deb01f5b783205df21bb180ed8.1606191405.git.gitgitgadget@gmail.com","threadId":"54562","inReplyTo":"pull.776.v5.git.1606191405.gitgitgadget@gmail.com","subject":"[PATCH v5 3/4] maintenance: use launchctl on macOS","fromName":"Derrick Stolee via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2020-11-24T04:16:44Z","receivedAt":"2020-11-24T04:16:56Z","isPatch":true,"sender":{"key":"stolee@gmail.com","avatar":"https://avatars.githubusercontent.com/u/570044?v=4"},"body":"From: Derrick Stolee \n\nThe existing mechanism for scheduling background maintenance is done\nthrough cron. The 'crontab -e' command allows updating the schedule\nwhile cron itself runs those commands. While this is technically\nsupported by macOS, it has some significant deficiencies:\n\n1. Every run of 'crontab -e' must request elevated privileges through\n the user interface. When running 'git maintenance start' from the\n Terminal app, it presents a dialog box saying \"Terminal.app would\n like to administer your computer. Administration can include\n modifying passwords, networking, and system settings.\" This is more\n alarming than what we are hoping to achieve. If this alert had some\n information about how \"git\" is trying to run \"crontab\" then we would\n have some reason to believe that this dialog might be fine. However,\n it also doesn't help that some scenarios just leave Git waiting for\n a response without presenting anything to the user. I experienced\n this when executing the command from a Bash terminal view inside\n Visual Studio Code.\n\n2. While cron initializes a user environment enough for \"git config\n --global --show-origin\" to show the correct config file information,\n it does not set up the environment enough for Git Credential Manager\n Core to load credentials during a 'prefetch' task. My prefetches\n against private repositories required re-authenticating through UI\n pop-ups in a way that should not be required.\n\nThe solution is to switch from cron to the Apple-recommended [1]\n'launchd' tool.\n\n[1] https://developer.apple.com/library/archive/documentation/MacOSX/Conceptual/BPSystemStartup/Chapters/ScheduledJobs.html\n\nThe basics of this tool is that we need to create XML-formatted\n\"plist\" files inside \"~/Library/LaunchAgents/\" and then use the\n'launchctl' tool to make launchd aware of them. The plist files\ninclude all of the scheduling information, along with the command-line\narguments split across an array of tags.\n\nFor example, here is my plist file for the weekly scheduled tasks:\n\n\n\n\nLabelorg.git-scm.git.weekly\nProgramArguments\n\n/usr/local/libexec/git-core/git\n--exec-path=/usr/local/libexec/git-core\nfor-each-repo\n--config=maintenance.repo\nmaintenance\nrun\n--schedule=weekly\n\nStartCalendarInterval\n\n\nDay0\nHour0\nMinute0\n\n\n\n\n\nThe schedules for the daily and hourly tasks are more complicated\nsince we need to use an array for the StartCalendarInterval with\nan entry for each of the six days other than the 0th day (to avoid\ncolliding with the weekly task), and each of the 23 hours other\nthan the 0th hour (to avoid colliding with the daily task).\n\nThe \"Label\" value is currently filled with \"org.git-scm.git.X\"\nwhere X is the frequency. We need a different plist file for each\nfrequency.\n\nThe launchctl command needs to be aligned with a user id in order\nto initialize the command environment. This must be done using\nthe 'launchctl bootstrap' subcommand. This subcommand is new as\nof macOS 10.11, which was released in September 2015. Before that\nrelease the 'launchctl load' subcommand was recommended. The best\nsource of information on this transition I have seen is available\nat [2]. The current design does not preclude a future version that\ndetects the available fatures of 'launchctl' to use the older\ncommands. However, it is best to rely on the newest version since\nApple might completely remove the deprecated version on short\nnotice.\n\n[2] https://babodee.wordpress.com/2016/04/09/launchctl-2-0-syntax/\n\nTo remove a schedule, we must run 'launchctl bootout' with a valid\nplist file. We also need to 'bootout' a task before the 'bootstrap'\nsubcommand will succeed, if such a task already exists.\n\nThe need for a user id requires us to run 'id -u' which works on\nPOSIX systems but not Windows. Further, the need for fully-qualitifed\npath names including $HOME behaves differently in the Git internals and\nthe external test suite. The $HOME variable starts with \"C:\\...\" instead\nof the \"/c/...\" that is provided by Git in these subcommands. The test\ntherefore has a prerequisite that we are not on Windows. The cross-\nplatform logic still allows us to test the macOS logic on a Linux\nmachine.\n\nWe can verify the commands that were run by 'git maintenance start'\nand 'git maintenance stop' by injecting a script that writes the\ncommand-line arguments into GIT_TEST_MAINT_SCHEDULER.\n\nAn earlier version of this patch accidentally had an opening\n\"\" tag when it should have had a closing \"\" tag. This\nwas caught during manual testing with actual 'launchctl' commands,\nbut we do not want to update developers' tasks when running tests.\nIt appears that macOS includes the \"xmllint\" tool which can verify\nthe XML format. This is useful for any system that might contain\nthe tool, so use it whenever it is available.\n\nCo-authored-by: Eric Sunshine \nSigned-off-by: Eric Sunshine \nSigned-off-by: Derrick Stolee \n---\n Documentation/git-maintenance.txt | 40 +++++++\n builtin/gc.c | 188 +++++++++++++++++++++++++++++-\n t/t7900-maintenance.sh | 58 +++++++++\n 3 files changed, 285 insertions(+), 1 deletion(-)\n\ndiff --git a/Documentation/git-maintenance.txt b/Documentation/git-maintenance.txt\nindex 1aa1112418..5f8f63f098 100644\n--- a/Documentation/git-maintenance.txt\n+++ b/Documentation/git-maintenance.txt\n@@ -273,6 +273,46 @@ schedule to ensure you are executing the correct binaries in your\n schedule.\n \n \n+BACKGROUND MAINTENANCE ON MACOS SYSTEMS\n+---------------------------------------\n+\n+While macOS technically supports `cron`, using `crontab -e` requires\n+elevated privileges and the executed process does not have a full user\n+context. Without a full user context, Git and its credential helpers\n+cannot access stored credentials, so some maintenance tasks are not\n+functional.\n+\n+Instead, `git maintenance start` interacts with the `launchctl` tool,\n+which is the recommended way to schedule timed jobs in macOS. Scheduling\n+maintenance through `git maintenance (start|stop)` requires some\n+`launchctl` features available only in macOS 10.11 or later.\n+\n+Your user-specific scheduled tasks are stored as XML-formatted `.plist`\n+files in `~/Library/LaunchAgents/`. You can see the currently-registered\n+tasks using the following command:\n+\n+-----------------------------------------------------------------------\n+$ ls ~/Library/LaunchAgents/org.git-scm.git*\n+org.git-scm.git.daily.plist\n+org.git-scm.git.hourly.plist\n+org.git-scm.git.weekly.plist\n+-----------------------------------------------------------------------\n+\n+One task is registered for each `--schedule=` option. To\n+inspect how the XML format describes each schedule, open one of these\n+`.plist` files in an editor and inspect the `` element following\n+the `StartCalendarInterval` element.\n+\n+`git maintenance start` will overwrite these files and register the\n+tasks again with `launchctl`, so any customizations should be done by\n+creating your own `.plist` files with distinct names. Similarly, the\n+`git maintenance stop` command will unregister the tasks with `launchctl`\n+and delete the `.plist` files.\n+\n+To create more advanced customizations to your background tasks, see\n+launchctl.plist(5) for more information.\n+\n+\n GIT\n ---\n Part of the linkgit:git[1] suite\ndiff --git a/builtin/gc.c b/builtin/gc.c\nindex 18ae7f7138..782769f243 100644\n--- a/builtin/gc.c\n+++ b/builtin/gc.c\n@@ -1491,6 +1491,186 @@ static int maintenance_unregister(void)\n \treturn run_command(&config_unset);\n }\n \n+static const char *get_frequency(enum schedule_priority schedule)\n+{\n+\tswitch (schedule) {\n+\tcase SCHEDULE_HOURLY:\n+\t\treturn \"hourly\";\n+\tcase SCHEDULE_DAILY:\n+\t\treturn \"daily\";\n+\tcase SCHEDULE_WEEKLY:\n+\t\treturn \"weekly\";\n+\tdefault:\n+\t\tBUG(\"invalid schedule %d\", schedule);\n+\t}\n+}\n+\n+static char *launchctl_service_name(const char *frequency)\n+{\n+\tstruct strbuf label = STRBUF_INIT;\n+\tstrbuf_addf(&label, \"org.git-scm.git.%s\", frequency);\n+\treturn strbuf_detach(&label, NULL);\n+}\n+\n+static char *launchctl_service_filename(const char *name)\n+{\n+\tchar *expanded;\n+\tstruct strbuf filename = STRBUF_INIT;\n+\tstrbuf_addf(&filename, \"~/Library/LaunchAgents/%s.plist\", name);\n+\n+\texpanded = expand_user_path(filename.buf, 1);\n+\tif (!expanded)\n+\t\tdie(_(\"failed to expand path '%s'\"), filename.buf);\n+\n+\tstrbuf_release(&filename);\n+\treturn expanded;\n+}\n+\n+static char *launchctl_get_uid(void)\n+{\n+\treturn xstrfmt(\"gui/%d\", getuid());\n+}\n+\n+static int launchctl_boot_plist(int enable, const char *filename, const char *cmd)\n+{\n+\tint result;\n+\tstruct child_process child = CHILD_PROCESS_INIT;\n+\tchar *uid = launchctl_get_uid();\n+\n+\tstrvec_split(&child.args, cmd);\n+\tif (enable)\n+\t\tstrvec_push(&child.args, \"bootstrap\");\n+\telse\n+\t\tstrvec_push(&child.args, \"bootout\");\n+\tstrvec_push(&child.args, uid);\n+\tstrvec_push(&child.args, filename);\n+\n+\tchild.no_stderr = 1;\n+\tchild.no_stdout = 1;\n+\n+\tif (start_command(&child))\n+\t\tdie(_(\"failed to start launchctl\"));\n+\n+\tresult = finish_command(&child);\n+\n+\tfree(uid);\n+\treturn result;\n+}\n+\n+static int launchctl_remove_plist(enum schedule_priority schedule, const char *cmd)\n+{\n+\tconst char *frequency = get_frequency(schedule);\n+\tchar *name = launchctl_service_name(frequency);\n+\tchar *filename = launchctl_service_filename(name);\n+\tint result = launchctl_boot_plist(0, filename, cmd);\n+\tunlink(filename);\n+\tfree(filename);\n+\tfree(name);\n+\treturn result;\n+}\n+\n+static int launchctl_remove_plists(const char *cmd)\n+{\n+\treturn launchctl_remove_plist(SCHEDULE_HOURLY, cmd) ||\n+\t\tlaunchctl_remove_plist(SCHEDULE_DAILY, cmd) ||\n+\t\tlaunchctl_remove_plist(SCHEDULE_WEEKLY, cmd);\n+}\n+\n+static int launchctl_schedule_plist(const char *exec_path, enum schedule_priority schedule, const char *cmd)\n+{\n+\tFILE *plist;\n+\tint i;\n+\tconst char *preamble, *repeat;\n+\tconst char *frequency = get_frequency(schedule);\n+\tchar *name = launchctl_service_name(frequency);\n+\tchar *filename = launchctl_service_filename(name);\n+\n+\tif (safe_create_leading_directories(filename))\n+\t\tdie(_(\"failed to create directories for '%s'\"), filename);\n+\tplist = xfopen(filename, \"w\");\n+\n+\tpreamble = \"\\n\"\n+\t\t \"\\n\"\n+\t\t \"\"\n+\t\t \"\\n\"\n+\t\t \"Label%s\\n\"\n+\t\t \"ProgramArguments\\n\"\n+\t\t \"\\n\"\n+\t\t \"%s/git\\n\"\n+\t\t \"--exec-path=%s\\n\"\n+\t\t \"for-each-repo\\n\"\n+\t\t \"--config=maintenance.repo\\n\"\n+\t\t \"maintenance\\n\"\n+\t\t \"run\\n\"\n+\t\t \"--schedule=%s\\n\"\n+\t\t \"\\n\"\n+\t\t \"StartCalendarInterval\\n\"\n+\t\t \"\\n\";\n+\tfprintf(plist, preamble, name, exec_path, exec_path, frequency);\n+\n+\tswitch (schedule) {\n+\tcase SCHEDULE_HOURLY:\n+\t\trepeat = \"\\n\"\n+\t\t\t \"Hour%d\\n\"\n+\t\t\t \"Minute0\\n\"\n+\t\t\t \"\\n\";\n+\t\tfor (i = 1; i <= 23; i++)\n+\t\t\tfprintf(plist, repeat, i);\n+\t\tbreak;\n+\n+\tcase SCHEDULE_DAILY:\n+\t\trepeat = \"\\n\"\n+\t\t\t \"Day%d\\n\"\n+\t\t\t \"Hour0\\n\"\n+\t\t\t \"Minute0\\n\"\n+\t\t\t \"\\n\";\n+\t\tfor (i = 1; i <= 6; i++)\n+\t\t\tfprintf(plist, repeat, i);\n+\t\tbreak;\n+\n+\tcase SCHEDULE_WEEKLY:\n+\t\tfprintf(plist,\n+\t\t\t\"\\n\"\n+\t\t\t\"Day0\\n\"\n+\t\t\t\"Hour0\\n\"\n+\t\t\t\"Minute0\\n\"\n+\t\t\t\"\\n\");\n+\t\tbreak;\n+\n+\tdefault:\n+\t\t/* unreachable */\n+\t\tbreak;\n+\t}\n+\tfprintf(plist, \"\\n\\n\\n\");\n+\tfclose(plist);\n+\n+\t/* bootout might fail if not already running, so ignore */\n+\tlaunchctl_boot_plist(0, filename, cmd);\n+\tif (launchctl_boot_plist(1, filename, cmd))\n+\t\tdie(_(\"failed to bootstrap service %s\"), filename);\n+\n+\tfree(filename);\n+\tfree(name);\n+\treturn 0;\n+}\n+\n+static int launchctl_add_plists(const char *cmd)\n+{\n+\tconst char *exec_path = git_exec_path();\n+\n+\treturn launchctl_schedule_plist(exec_path, SCHEDULE_HOURLY, cmd) ||\n+\t\tlaunchctl_schedule_plist(exec_path, SCHEDULE_DAILY, cmd) ||\n+\t\tlaunchctl_schedule_plist(exec_path, SCHEDULE_WEEKLY, cmd);\n+}\n+\n+static int launchctl_update_schedule(int run_maintenance, int fd, const char *cmd)\n+{\n+\tif (run_maintenance)\n+\t\treturn launchctl_add_plists(cmd);\n+\telse\n+\t\treturn launchctl_remove_plists(cmd);\n+}\n+\n #define BEGIN_LINE \"# BEGIN GIT MAINTENANCE SCHEDULE\"\n #define END_LINE \"# END GIT MAINTENANCE SCHEDULE\"\n \n@@ -1579,7 +1759,11 @@ static int crontab_update_schedule(int run_maintenance, int fd, const char *cmd)\n \treturn result;\n }\n \n+#if defined(__APPLE__)\n+static const char platform_scheduler[] = \"launchctl\";\n+#else\n static const char platform_scheduler[] = \"crontab\";\n+#endif\n \n static int update_background_schedule(int enable)\n {\n@@ -1603,7 +1787,9 @@ static int update_background_schedule(int enable)\n \tif (hold_lock_file_for_update(&lk, lock_path, LOCK_NO_DEREF) < 0)\n \t\treturn error(_(\"another process is scheduling background maintenance\"));\n \n-\tif (!strcmp(scheduler, \"crontab\"))\n+\tif (!strcmp(scheduler, \"launchctl\"))\n+\t\tresult = launchctl_update_schedule(enable, lk.tempfile->fd, cmd);\n+\telse if (!strcmp(scheduler, \"crontab\"))\n \t\tresult = crontab_update_schedule(enable, lk.tempfile->fd, cmd);\n \telse\n \t\tdie(\"unknown background scheduler: %s\", scheduler);\ndiff --git a/t/t7900-maintenance.sh b/t/t7900-maintenance.sh\nindex eeb939168d..6d37312901 100755\n--- a/t/t7900-maintenance.sh\n+++ b/t/t7900-maintenance.sh\n@@ -7,6 +7,19 @@ test_description='git maintenance builtin'\n GIT_TEST_COMMIT_GRAPH=0\n GIT_TEST_MULTI_PACK_INDEX=0\n \n+test_lazy_prereq XMLLINT '\n+\txmllint --version\n+'\n+\n+test_xmllint () {\n+\tif test_have_prereq XMLLINT\n+\tthen\n+\t\txmllint --noout \"$@\"\n+\telse\n+\t\ttrue\n+\tfi\n+}\n+\n test_expect_success 'help text' '\n \ttest_expect_code 129 git maintenance -h 2>err &&\n \ttest_i18ngrep \"usage: git maintenance \" err &&\n@@ -395,6 +408,51 @@ test_expect_success 'start preserves existing schedule' '\n \tgrep \"Important information!\" cron.txt\n '\n \n+test_expect_success !MINGW 'start and stop macOS maintenance' '\n+\tuid=$(id -u) &&\n+\n+\twrite_script print-args <<-\\EOF &&\n+\techo $* >>args\n+\tEOF\n+\n+\trm -f args &&\n+\tGIT_TEST_MAINT_SCHEDULER=launchctl:./print-args git maintenance start &&\n+\n+\t# start registers the repo\n+\tgit config --get --global maintenance.repo \"$(pwd)\" &&\n+\n+\tls \"$HOME/Library/LaunchAgents\" >actual &&\n+\tcat >expect <<-\\EOF &&\n+\torg.git-scm.git.daily.plist\n+\torg.git-scm.git.hourly.plist\n+\torg.git-scm.git.weekly.plist\n+\tEOF\n+\ttest_cmp expect actual &&\n+\n+\trm expect &&\n+\tfor frequency in hourly daily weekly\n+\tdo\n+\t\tPLIST=\"$HOME/Library/LaunchAgents/org.git-scm.git.$frequency.plist\" &&\n+\t\ttest_xmllint \"$PLIST\" &&\n+\t\tgrep schedule=$frequency \"$PLIST\" &&\n+\t\techo \"bootout gui/$uid $PLIST\" >>expect &&\n+\t\techo \"bootstrap gui/$uid $PLIST\" >>expect || return 1\n+\tdone &&\n+\ttest_cmp expect args &&\n+\n+\trm -f args &&\n+\tGIT_TEST_MAINT_SCHEDULER=launchctl:./print-args git maintenance stop &&\n+\n+\t# stop does not unregister the repo\n+\tgit config --get --global maintenance.repo \"$(pwd)\" &&\n+\n+\tprintf \"bootout gui/$uid $HOME/Library/LaunchAgents/org.git-scm.git.%s.plist\\n\" \\\n+\t\thourly daily weekly >expect &&\n+\ttest_cmp expect args &&\n+\tls \"$HOME/Library/LaunchAgents\" >actual &&\n+\ttest_line_count = 0 actual\n+'\n+\n test_expect_success 'register preserves existing strategy' '\n \tgit config maintenance.strategy none &&\n \tgit maintenance register &&\n-- \ngitgitgadget\n\n"},{"id":"410921","messageId":"CAPig+cQ6vZzbb36t5Kn=NM9wXC8i1MpcGXfK=QwnNRoTQsQP0A@mail.gmail.com","threadId":"54562","inReplyTo":"ac9a28bea39b4d1897ad6a41331e52770c8c7531.1606191405.git.gitgitgadget@gmail.com","subject":"Re: [PATCH v5 4/4] maintenance: use Windows scheduled tasks","fromName":"Eric Sunshine","fromEmail":"sunshine@sunshineco.com","sentAt":"2020-11-27T09:08:10Z","receivedAt":"2020-11-27T09:08:25Z","isPatch":true,"sender":{"key":"sunshine@sunshineco.com","avatar":"https://avatars.githubusercontent.com/u/163641?v=4"},"body":"On Mon, Nov 23, 2020 at 11:16 PM Derrick Stolee via GitGitGadget\n wrote:\n> Git's background maintenance uses cron by default, but this is not\n> available on Windows. Instead, integrate with Task Scheduler.\n> [...]\n> Signed-off-by: Derrick Stolee \n> ---\n> diff --git a/builtin/gc.c b/builtin/gc.c\n> @@ -1671,6 +1671,162 @@ static int launchctl_update_schedule(int run_maintenance, int fd, const char *cm\n> +static int schtasks_schedule_task(const char *exec_path, enum schedule_priority schedule, const char *cmd)\n> +{\n> + const char *frequency = get_frequency(schedule);\n> +\n> + tfile = xmks_tempfile(\"schedule_XXXXXX\");\n> + if (!tfile || !fdopen_tempfile(tfile, \"w\"))\n> + die(_(\"failed to create temp xml file\"));\n\nSeveral comments:\n\nThe \"x\" prefix on xmks_tempfile() means that it will die() if it can't\nopen the tempfile, so the `!tfile` condition is pointless, thus it\ncould be written:\n\n if (!fdopen_tempfile(tfile, \"w\"))\n\nThe mks_tempfile_t*() functions with a trailing \"t\" will place the\ntemporary file in TMPDIR, whereas xmks_tempfile() used here places it\nin the worktree directory, which is not as desirable. Ideally, this\nwould be using xmks_tempfile_t(), however, that function doesn't exist\nyet in tempfile.h, so the best you can do (without the extra work of\nalso adding the missing function) is to use mks_tempfile_t(). That\ndoesn't die(), so `!tfile` would still be needed in the conditional.\n\nIn earlier versions, you incorporated `frequency` into the temporary\nfilename which was nice because it made the test easier to understand.\nIt's not hard to do so here, as well, nor to extract a useful filename\nin the test (as I'll show below). For instance:\n\n struct strbuf tpath = STRBUF_INIT;\n strbuf_addf(&tpath, \"schedule-%s-XXXXXX\", frequency);\n tfile = mks_tempfile_t(tpath.buf);\n strbuf_release(&tpath);\n if (!tfile || !fdopen(tempfile(tfile, \"w\"))\n die(...);\n\n> + strvec_pushl(&child.args, \"/create\", \"/tn\", name, \"/f\", \"/xml\", tfile->filename.buf, NULL);\n\nAlternately, use the getter:\n\n strvec_pushl(..., get_tempfile_path(&tfile), ...);\n\n> diff --git a/t/t7900-maintenance.sh b/t/t7900-maintenance.sh\n> @@ -453,6 +453,50 @@ test_expect_success !MINGW 'start and stop macOS maintenance' '\n> +test_expect_success 'start and stop Windows maintenance' '\n> + write_script print-args <<-\\EOF &&\n> + echo $* >>args\n> + while test $# -gt 0\n> + do\n> + case \"$1\" in\n> + /xml) shift; xmlfile=$1; break ;;\n> + *) shift ;;\n> + esac\n> + done\n> + lines=$(wc -l args | awk \"{print \\$1;}\")\n\nYou're using `awk` to pluck out the line count and ignore the\nfilename, but the same can be accomplished by feeding the file as\nstdin to `wc` rather than naming it as an argument, thus this is\nequivalent:\n\n lines=$(wc -l + test -z \"$xmlfile\" || cp \"$xmlfile\" \"schedule-$lines.xml\"\n\nIf you take the suggestion earlier in this review of naming the file\n\"schedule-${frequency}-XXXXXX.xml\", then you can strip it down to just\n\"schedule-${frequency}.xml\" using the expression `${xmlfile%-*}.xml`.\nThere is no need for `$lines`. Thus, copying the file would become:\n\n test -z \"$xmlfile\" || cp \"$xmlfile\" \"${xmlfile%-*}.xml\"\n\n> + for i in 1 2 3\n> + do\n\nWhich means that you can use the more easily understood `hourly daily\nweekly` enumeration here rather than `1 2 3`.\n\nHaving said all this, I'm not sure it's worth a re-roll. These sort of\ntweaks can be done atop the current series if someone wants to tackle\nit.\n"},{"id":"411897","messageId":"7cc70a8fe7bad92179961b31520147af39d5353c.1607542142.git.gitgitgadget@gmail.com","threadId":"54562","inReplyTo":"pull.776.v6.git.1607542142.gitgitgadget@gmail.com","subject":"[PATCH v6 2/4] maintenance: include 'cron' details in docs","fromName":"Derrick Stolee via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2020-12-09T19:29:00Z","receivedAt":"2020-12-09T19:31:31Z","isPatch":true,"sender":{"key":"stolee@gmail.com","avatar":"https://avatars.githubusercontent.com/u/570044?v=4"},"body":"From: Derrick Stolee \n\nAdvanced and expert users may want to know how 'git maintenance start'\nschedules background maintenance in order to customize their own\nschedules beyond what the maintenance.* config values allow. Start a new\nset of sections in git-maintenance.txt that describe how 'cron' is used\nto run these tasks.\n\nThis is particularly valuable for users who want to inspect what Git is\ndoing or for users who want to customize the schedule further. Having a\nbaseline can provide a way forward for users who have never worked with\ncron schedules.\n\nHelped-by: Eric Sunshine \nSigned-off-by: Derrick Stolee \n---\n Documentation/git-maintenance.txt | 54 +++++++++++++++++++++++++++++++\n 1 file changed, 54 insertions(+)\n\ndiff --git a/Documentation/git-maintenance.txt b/Documentation/git-maintenance.txt\nindex 6fec1eb8dc2..1aa11124186 100644\n--- a/Documentation/git-maintenance.txt\n+++ b/Documentation/git-maintenance.txt\n@@ -218,6 +218,60 @@ Further, the `git gc` command should not be combined with\n but does not take the lock in the same way as `git maintenance run`. If\n possible, use `git maintenance run --task=gc` instead of `git gc`.\n \n+The following sections describe the mechanisms put in place to run\n+background maintenance by `git maintenance start` and how to customize\n+them.\n+\n+BACKGROUND MAINTENANCE ON POSIX SYSTEMS\n+---------------------------------------\n+\n+The standard mechanism for scheduling background tasks on POSIX systems\n+is cron(8). This tool executes commands based on a given schedule. The\n+current list of user-scheduled tasks can be found by running `crontab -l`.\n+The schedule written by `git maintenance start` is similar to this:\n+\n+-----------------------------------------------------------------------\n+# BEGIN GIT MAINTENANCE SCHEDULE\n+# The following schedule was created by Git\n+# Any edits made in this region might be\n+# replaced in the future by a Git command.\n+\n+0 1-23 * * * \"//git\" --exec-path=\"/\" for-each-repo --config=maintenance.repo maintenance run --schedule=hourly\n+0 0 * * 1-6 \"//git\" --exec-path=\"/\" for-each-repo --config=maintenance.repo maintenance run --schedule=daily\n+0 0 * * 0 \"//git\" --exec-path=\"/\" for-each-repo --config=maintenance.repo maintenance run --schedule=weekly\n+\n+# END GIT MAINTENANCE SCHEDULE\n+-----------------------------------------------------------------------\n+\n+The comments are used as a region to mark the schedule as written by Git.\n+Any modifications within this region will be completely deleted by\n+`git maintenance stop` or overwritten by `git maintenance start`.\n+\n+The `crontab` entry specifies the full path of the `git` executable to\n+ensure that the executed `git` command is the same one with which\n+`git maintenance start` was issued independent of `PATH`. If the same user\n+runs `git maintenance start` with multiple Git executables, then only the\n+latest executable is used.\n+\n+These commands use `git for-each-repo --config=maintenance.repo` to run\n+`git maintenance run --schedule=` on each repository listed in\n+the multi-valued `maintenance.repo` config option. These are typically\n+loaded from the user-specific global config. The `git maintenance` process\n+then determines which maintenance tasks are configured to run on each\n+repository with each `` using the `maintenance..schedule`\n+config options. These values are loaded from the global or repository\n+config values.\n+\n+If the config values are insufficient to achieve your desired background\n+maintenance schedule, then you can create your own schedule. If you run\n+`crontab -e`, then an editor will load with your user-specific `cron`\n+schedule. In that editor, you can add your own schedule lines. You could\n+start by adapting the default schedule listed earlier, or you could read\n+the crontab(5) documentation for advanced scheduling techniques. Please\n+do use the full path and `--exec-path` techniques from the default\n+schedule to ensure you are executing the correct binaries in your\n+schedule.\n+\n \n GIT\n ---\n-- \ngitgitgadget\n\n"},{"id":"411898","messageId":"pull.776.v6.git.1607542142.gitgitgadget@gmail.com","threadId":"54562","inReplyTo":"pull.776.v5.git.1606191405.gitgitgadget@gmail.com","subject":"[PATCH v6 0/4] Maintenance IV: Platform-specific background maintenance","fromName":"Derrick Stolee via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2020-12-09T19:28:58Z","receivedAt":"2020-12-09T19:31:31Z","isPatch":true,"sender":{"key":"stolee@gmail.com","avatar":"https://avatars.githubusercontent.com/u/570044?v=4"},"body":"This is based on ds/maintenance-part-3.\n\nAfter sitting with the background maintenance as it has been cooking, I\nwanted to come back around and implement the background maintenance for\nWindows. However, I noticed that there were some things bothering me with\nbackground maintenance on my macOS machine. These are detailed in PATCH 3,\nbut the tl;dr is that 'cron' is not recommended by Apple and instead\n'launchd' satisfies our needs.\n\nThis series implements the background scheduling so git maintenance\n(start|stop) works on those platforms. I've been operating with these\nschedules for a while now without the problems described in the patches.\n\nThere is a particularly annoying case about console windows popping up on\nWindows, but PATCH 4 describes a plan to get around that.\n\n\nUpdate in V6\n============\n\n * The Windows platform uses the tempfile API a bit better, including using\n the frequency in the filename to make the test simpler.\n\nThanks, -Stolee\n\ncc: jrnieder@gmail.com cc: jonathantanmy@google.com cc: sluongng@gmail.com\ncc: Đoàn Trần Công Danh congdanhqx@gmail.com cc: Martin Ågren\nmartin.agren@gmail.com cc: Eric Sunshine sunshine@sunshineco.com cc: Derrick\nStolee stolee@gmail.com\n\nDerrick Stolee (4):\n maintenance: extract platform-specific scheduling\n maintenance: include 'cron' details in docs\n maintenance: use launchctl on macOS\n maintenance: use Windows scheduled tasks\n\n Documentation/git-maintenance.txt | 116 ++++++++\n builtin/gc.c | 421 ++++++++++++++++++++++++++++--\n t/t7900-maintenance.sh | 105 +++++++-\n t/test-lib.sh | 7 +-\n 4 files changed, 615 insertions(+), 34 deletions(-)\n\n\nbase-commit: 0016b618182f642771dc589cf0090289f9fe1b4f\nPublished-As: https://github.com/gitgitgadget/git/releases/tag/pr-776%2Fderrickstolee%2Fmaintenance%2FmacOS-v6\nFetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-776/derrickstolee/maintenance/macOS-v6\nPull-Request: https://github.com/gitgitgadget/git/pull/776\n\nRange-diff vs v5:\n\n 1: 4807342b001 = 1: 4807342b001 maintenance: extract platform-specific scheduling\n 2: 7cc70a8fe7b = 2: 7cc70a8fe7b maintenance: include 'cron' details in docs\n 3: cd015a5cbd7 = 3: cd015a5cbd7 maintenance: use launchctl on macOS\n 4: ac9a28bea39 ! 4: 6ad4a6b98c6 maintenance: use Windows scheduled tasks\n @@ Commit message\n by Git is valid when xmllint exists on the system.\n \n Since we use a temporary file for the XML files sent to 'schtasks', we\n - must copy the file to a predictable filename. Use the number of lines in\n - the 'args' file to provide a filename for xmllint. Instead of an exact\n - match on the 'args' file, we 'grep' for the arguments other than the\n - filename.\n + prefix the random characters with the frequency so it is easier to\n + examine the proper file during tests. Instead of an exact match on the\n + 'args' file, we 'grep' for the arguments other than the filename.\n \n There is a deficiency in the current design. Windows has two kinds of\n applications: GUI applications that start by \"winmain()\" and console\n @@ builtin/gc.c: static int launchctl_update_schedule(int run_maintenance, int fd,\n +\tstruct tempfile *tfile;\n +\tconst char *frequency = get_frequency(schedule);\n +\tchar *name = schtasks_task_name(frequency);\n ++\tstruct strbuf tfilename = STRBUF_INIT;\n +\n -+\ttfile = xmks_tempfile(\"schedule_XXXXXX\");\n -+\tif (!tfile || !fdopen_tempfile(tfile, \"w\"))\n ++\tstrbuf_addf(&tfilename, \"schedule_%s_XXXXXX\", frequency);\n ++\ttfile = xmks_tempfile(tfilename.buf);\n ++\tstrbuf_release(&tfilename);\n ++\n ++\tif (!fdopen_tempfile(tfile, \"w\"))\n +\t\tdie(_(\"failed to create temp xml file\"));\n +\n +\txml = \"\\n\"\n @@ builtin/gc.c: static int launchctl_update_schedule(int run_maintenance, int fd,\n +\t \"\\n\";\n +\tfprintf(tfile->fp, xml, exec_path, exec_path, frequency);\n +\tstrvec_split(&child.args, cmd);\n -+\tstrvec_pushl(&child.args, \"/create\", \"/tn\", name, \"/f\", \"/xml\", tfile->filename.buf, NULL);\n ++\tstrvec_pushl(&child.args, \"/create\", \"/tn\", name, \"/f\", \"/xml\",\n ++\t\t\t\t get_tempfile_path(tfile), NULL);\n +\tclose_tempfile_gently(tfile);\n +\n +\tchild.no_stdout = 1;\n @@ t/t7900-maintenance.sh: test_expect_success !MINGW 'start and stop macOS mainten\n +\t\t*) shift ;;\n +\t\tesac\n +\tdone\n -+\tlines=$(wc -l args | awk \"{print \\$1;}\")\n -+\ttest -z \"$xmlfile\" || cp \"$xmlfile\" \"schedule-$lines.xml\"\n ++\ttest -z \"$xmlfile\" || cp \"$xmlfile\" \"$xmlfile.xml\"\n +\tEOF\n +\n +\trm -f args &&\n -+\tGIT_TEST_MAINT_SCHEDULER=\"schtasks:./print-args\" git maintenance start &&\n ++\tGIT_TEST_MAINT_SCHEDULER=\"schtasks:./print-args\" GIT_TRACE2_PERF=1 git maintenance start &&\n +\n +\t# start registers the repo\n +\tgit config --get --global maintenance.repo \"$(pwd)\" &&\n +\n +\tfor frequency in hourly daily weekly\n +\tdo\n -+\t\tgrep \"/create /tn Git Maintenance ($frequency) /f /xml\" args \\\n -+\t\t\t|| return 1\n -+\tdone &&\n -+\n -+\tfor i in 1 2 3\n -+\tdo\n -+\t\ttest_xmllint \"schedule-$i.xml\" &&\n -+\t\tgrep \"encoding=.US-ASCII.\" \"schedule-$i.xml\" || return 1\n ++\t\tgrep \"/create /tn Git Maintenance ($frequency) /f /xml\" args &&\n ++\t\tfile=$(ls schedule_$frequency*.xml) &&\n ++\t\ttest_xmllint \"$file\" &&\n ++\t\tgrep \"encoding=.US-ASCII.\" \"$file\" || return 1\n +\tdone &&\n +\n +\trm -f args &&\n\n-- \ngitgitgadget\n"},{"id":"411899","messageId":"4807342b0019be29bb369ed3403a485f0ce9c15d.1607542142.git.gitgitgadget@gmail.com","threadId":"54562","inReplyTo":"pull.776.v6.git.1607542142.gitgitgadget@gmail.com","subject":"[PATCH v6 1/4] maintenance: extract platform-specific scheduling","fromName":"Derrick Stolee via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2020-12-09T19:28:59Z","receivedAt":"2020-12-09T19:31:39Z","isPatch":true,"sender":{"key":"stolee@gmail.com","avatar":"https://avatars.githubusercontent.com/u/570044?v=4"},"body":"From: Derrick Stolee \n\nThe existing schedule mechanism using 'cron' is supported by POSIX\nplatforms, but not Windows. It also works slightly differently on\nmacOS to significant detriment of the user experience. To allow for\nnew implementations on these platforms, extract a method that\nperforms the platform-specific scheduling mechanism. This will be\nswapped at compile time with new implementations on specialized\nplatforms.\n\nAs we add this generality, rename GIT_TEST_CRONTAB to\nGIT_TEST_MAINT_SCHEDULER. Further, this variable is now parsed as\n\":\" so we can test platform-specific scheduling\nlogic even when not on the correct platform. By specifying the\n in this string, we will be able to test all three sets of\nGit logic from a Linux machine.\n\nCo-authored-by: Eric Sunshine \nSigned-off-by: Eric Sunshine \nSigned-off-by: Derrick Stolee \n---\n builtin/gc.c | 70 ++++++++++++++++++++++++++----------------\n t/t7900-maintenance.sh | 8 ++---\n t/test-lib.sh | 7 +++--\n 3 files changed, 51 insertions(+), 34 deletions(-)\n\ndiff --git a/builtin/gc.c b/builtin/gc.c\nindex e3098ef6a12..18ae7f7138a 100644\n--- a/builtin/gc.c\n+++ b/builtin/gc.c\n@@ -1494,35 +1494,23 @@ static int maintenance_unregister(void)\n #define BEGIN_LINE \"# BEGIN GIT MAINTENANCE SCHEDULE\"\n #define END_LINE \"# END GIT MAINTENANCE SCHEDULE\"\n \n-static int update_background_schedule(int run_maintenance)\n+static int crontab_update_schedule(int run_maintenance, int fd, const char *cmd)\n {\n \tint result = 0;\n \tint in_old_region = 0;\n \tstruct child_process crontab_list = CHILD_PROCESS_INIT;\n \tstruct child_process crontab_edit = CHILD_PROCESS_INIT;\n \tFILE *cron_list, *cron_in;\n-\tconst char *crontab_name;\n \tstruct strbuf line = STRBUF_INIT;\n-\tstruct lock_file lk;\n-\tchar *lock_path = xstrfmt(\"%s/schedule\", the_repository->objects->odb->path);\n \n-\tif (hold_lock_file_for_update(&lk, lock_path, LOCK_NO_DEREF) < 0)\n-\t\treturn error(_(\"another process is scheduling background maintenance\"));\n-\n-\tcrontab_name = getenv(\"GIT_TEST_CRONTAB\");\n-\tif (!crontab_name)\n-\t\tcrontab_name = \"crontab\";\n-\n-\tstrvec_split(&crontab_list.args, crontab_name);\n+\tstrvec_split(&crontab_list.args, cmd);\n \tstrvec_push(&crontab_list.args, \"-l\");\n \tcrontab_list.in = -1;\n-\tcrontab_list.out = dup(lk.tempfile->fd);\n+\tcrontab_list.out = dup(fd);\n \tcrontab_list.git_cmd = 0;\n \n-\tif (start_command(&crontab_list)) {\n-\t\tresult = error(_(\"failed to run 'crontab -l'; your system might not support 'cron'\"));\n-\t\tgoto cleanup;\n-\t}\n+\tif (start_command(&crontab_list))\n+\t\treturn error(_(\"failed to run 'crontab -l'; your system might not support 'cron'\"));\n \n \t/* Ignore exit code, as an empty crontab will return error. */\n \tfinish_command(&crontab_list);\n@@ -1531,17 +1519,15 @@ static int update_background_schedule(int run_maintenance)\n \t * Read from the .lock file, filtering out the old\n \t * schedule while appending the new schedule.\n \t */\n-\tcron_list = fdopen(lk.tempfile->fd, \"r\");\n+\tcron_list = fdopen(fd, \"r\");\n \trewind(cron_list);\n \n-\tstrvec_split(&crontab_edit.args, crontab_name);\n+\tstrvec_split(&crontab_edit.args, cmd);\n \tcrontab_edit.in = -1;\n \tcrontab_edit.git_cmd = 0;\n \n-\tif (start_command(&crontab_edit)) {\n-\t\tresult = error(_(\"failed to run 'crontab'; your system might not support 'cron'\"));\n-\t\tgoto cleanup;\n-\t}\n+\tif (start_command(&crontab_edit))\n+\t\treturn error(_(\"failed to run 'crontab'; your system might not support 'cron'\"));\n \n \tcron_in = fdopen(crontab_edit.in, \"w\");\n \tif (!cron_in) {\n@@ -1586,14 +1572,44 @@ static int update_background_schedule(int run_maintenance)\n \tclose(crontab_edit.in);\n \n done_editing:\n-\tif (finish_command(&crontab_edit)) {\n+\tif (finish_command(&crontab_edit))\n \t\tresult = error(_(\"'crontab' died\"));\n-\t\tgoto cleanup;\n+\telse\n+\t\tfclose(cron_list);\n+\treturn result;\n+}\n+\n+static const char platform_scheduler[] = \"crontab\";\n+\n+static int update_background_schedule(int enable)\n+{\n+\tint result;\n+\tconst char *scheduler = platform_scheduler;\n+\tconst char *cmd = scheduler;\n+\tchar *testing;\n+\tstruct lock_file lk;\n+\tchar *lock_path = xstrfmt(\"%s/schedule\", the_repository->objects->odb->path);\n+\n+\ttesting = xstrdup_or_null(getenv(\"GIT_TEST_MAINT_SCHEDULER\"));\n+\tif (testing) {\n+\t\tchar *sep = strchr(testing, ':');\n+\t\tif (!sep)\n+\t\t\tdie(\"GIT_TEST_MAINT_SCHEDULER unparseable: %s\", testing);\n+\t\t*sep = '\\0';\n+\t\tscheduler = testing;\n+\t\tcmd = sep + 1;\n \t}\n-\tfclose(cron_list);\n \n-cleanup:\n+\tif (hold_lock_file_for_update(&lk, lock_path, LOCK_NO_DEREF) < 0)\n+\t\treturn error(_(\"another process is scheduling background maintenance\"));\n+\n+\tif (!strcmp(scheduler, \"crontab\"))\n+\t\tresult = crontab_update_schedule(enable, lk.tempfile->fd, cmd);\n+\telse\n+\t\tdie(\"unknown background scheduler: %s\", scheduler);\n+\n \trollback_lock_file(&lk);\n+\tfree(testing);\n \treturn result;\n }\n \ndiff --git a/t/t7900-maintenance.sh b/t/t7900-maintenance.sh\nindex 20184e96e1a..eeb939168da 100755\n--- a/t/t7900-maintenance.sh\n+++ b/t/t7900-maintenance.sh\n@@ -368,7 +368,7 @@ test_expect_success 'register and unregister' '\n '\n \n test_expect_success 'start from empty cron table' '\n-\tGIT_TEST_CRONTAB=\"test-tool crontab cron.txt\" git maintenance start &&\n+\tGIT_TEST_MAINT_SCHEDULER=\"crontab:test-tool crontab cron.txt\" git maintenance start &&\n \n \t# start registers the repo\n \tgit config --get --global maintenance.repo \"$(pwd)\" &&\n@@ -379,19 +379,19 @@ test_expect_success 'start from empty cron table' '\n '\n \n test_expect_success 'stop from existing schedule' '\n-\tGIT_TEST_CRONTAB=\"test-tool crontab cron.txt\" git maintenance stop &&\n+\tGIT_TEST_MAINT_SCHEDULER=\"crontab:test-tool crontab cron.txt\" git maintenance stop &&\n \n \t# stop does not unregister the repo\n \tgit config --get --global maintenance.repo \"$(pwd)\" &&\n \n \t# Operation is idempotent\n-\tGIT_TEST_CRONTAB=\"test-tool crontab cron.txt\" git maintenance stop &&\n+\tGIT_TEST_MAINT_SCHEDULER=\"crontab:test-tool crontab cron.txt\" git maintenance stop &&\n \ttest_must_be_empty cron.txt\n '\n \n test_expect_success 'start preserves existing schedule' '\n \techo \"Important information!\" >cron.txt &&\n-\tGIT_TEST_CRONTAB=\"test-tool crontab cron.txt\" git maintenance start &&\n+\tGIT_TEST_MAINT_SCHEDULER=\"crontab:test-tool crontab cron.txt\" git maintenance start &&\n \tgrep \"Important information!\" cron.txt\n '\n \ndiff --git a/t/test-lib.sh b/t/test-lib.sh\nindex 4a60d1ed766..ddbeee1f5eb 100644\n--- a/t/test-lib.sh\n+++ b/t/test-lib.sh\n@@ -1704,7 +1704,8 @@ test_lazy_prereq REBASE_P '\n '\n \n # Ensure that no test accidentally triggers a Git command\n-# that runs 'crontab', affecting a user's cron schedule.\n-# Tests that verify the cron integration must set this locally\n+# that runs the actual maintenance scheduler, affecting a user's\n+# system permanently.\n+# Tests that verify the scheduler integration must set this locally\n # to avoid errors.\n-GIT_TEST_CRONTAB=\"exit 1\"\n+GIT_TEST_MAINT_SCHEDULER=\"none:exit 1\"\n-- \ngitgitgadget\n\n"},{"id":"411900","messageId":"6ad4a6b98c63abc243bb16cfd10fbd583e90a3ae.1607542142.git.gitgitgadget@gmail.com","threadId":"54562","inReplyTo":"pull.776.v6.git.1607542142.gitgitgadget@gmail.com","subject":"[PATCH v6 4/4] maintenance: use Windows scheduled tasks","fromName":"Derrick Stolee via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2020-12-09T19:29:02Z","receivedAt":"2020-12-09T19:32:32Z","isPatch":true,"sender":{"key":"stolee@gmail.com","avatar":"https://avatars.githubusercontent.com/u/570044?v=4"},"body":"From: Derrick Stolee \n\nGit's background maintenance uses cron by default, but this is not\navailable on Windows. Instead, integrate with Task Scheduler.\n\nTasks can be scheduled using the 'schtasks' command. There are several\ncommand-line options that can allow for some advanced scheduling, but\nunfortunately these seem to all require authenticating using a password.\n\nInstead, use the \"/xml\" option to pass an XML file that contains the\nconfiguration for the necessary schedule. These XML files are based on\nsome that I exported after constructing a schedule in the Task Scheduler\nGUI. These options only run background maintenance when the user is\nlogged in, and more fields are populated with the current username and\nSID at run-time by 'schtasks'.\n\nSince the GIT_TEST_MAINT_SCHEDULER environment variable allows us to\nspecify 'schtasks' as the scheduler, we can test the Windows-specific\nlogic on other platforms. Thus, add a check that the XML file written\nby Git is valid when xmllint exists on the system.\n\nSince we use a temporary file for the XML files sent to 'schtasks', we\nprefix the random characters with the frequency so it is easier to\nexamine the proper file during tests. Instead of an exact match on the\n'args' file, we 'grep' for the arguments other than the filename.\n\nThere is a deficiency in the current design. Windows has two kinds of\napplications: GUI applications that start by \"winmain()\" and console\napplications that start by \"main()\". Console applications are attached\nto a new Console window if they are not already associated with a GUI\napplication. This means that every hour the scheudled task launches a\ncommand window for the scheduled tasks. Not only is this visually\nobtrusive, but it also takes focus from whatever else the user is\ndoing!\n\nA simple fix would be to insert a GUI application that acts as a shim\nbetween the scheduled task and Git. This is currently possible in Git\nfor Windows by setting the tag equal to\n\n C:\\Program Files\\Git\\git-bash.exe\n\nwith options \"--hide --no-needs-console --command=cmd\\git.exe\"\nfollowed by the arguments currently used. Since git-bash.exe is not\nincluded in Windows builds of core Git, I chose to leave out this\nfeature. My plan is to submit a small patch to Git for Windows that\nconverts the use of git.exe with this use of git-bash.exe in the\nshort term. In the long term, we can consider creating this GUI\nshim application within core Git, perhaps in contrib/.\n\nCo-authored-by: Eric Sunshine \nSigned-off-by: Eric Sunshine \nSigned-off-by: Derrick Stolee \n---\n Documentation/git-maintenance.txt | 22 ++++\n builtin/gc.c | 165 ++++++++++++++++++++++++++++++\n t/t7900-maintenance.sh | 39 +++++++\n 3 files changed, 226 insertions(+)\n\ndiff --git a/Documentation/git-maintenance.txt b/Documentation/git-maintenance.txt\nindex 5f8f63f0988..6970f2b8983 100644\n--- a/Documentation/git-maintenance.txt\n+++ b/Documentation/git-maintenance.txt\n@@ -313,6 +313,28 @@ To create more advanced customizations to your background tasks, see\n launchctl.plist(5) for more information.\n \n \n+BACKGROUND MAINTENANCE ON WINDOWS SYSTEMS\n+-----------------------------------------\n+\n+Windows does not support `cron` and instead has its own system for\n+scheduling background tasks. The `git maintenance start` command uses\n+the `schtasks` command to submit tasks to this system. You can inspect\n+all background tasks using the Task Scheduler application. The tasks\n+added by Git have names of the form `Git Maintenance ()`.\n+The Task Scheduler GUI has ways to inspect these tasks, but you can also\n+export the tasks to XML files and view the details there.\n+\n+Note that since Git is a console application, these background tasks\n+create a console window visible to the current user. This can be changed\n+manually by selecting the \"Run whether user is logged in or not\" option\n+in Task Scheduler. This change requires a password input, which is why\n+`git maintenance start` does not select it by default.\n+\n+If you want to customize the background tasks, please rename the tasks\n+so future calls to `git maintenance (start|stop)` do not overwrite your\n+custom tasks.\n+\n+\n GIT\n ---\n Part of the linkgit:git[1] suite\ndiff --git a/builtin/gc.c b/builtin/gc.c\nindex 782769f2438..7c989904671 100644\n--- a/builtin/gc.c\n+++ b/builtin/gc.c\n@@ -1671,6 +1671,167 @@ static int launchctl_update_schedule(int run_maintenance, int fd, const char *cm\n \t\treturn launchctl_remove_plists(cmd);\n }\n \n+static char *schtasks_task_name(const char *frequency)\n+{\n+\tstruct strbuf label = STRBUF_INIT;\n+\tstrbuf_addf(&label, \"Git Maintenance (%s)\", frequency);\n+\treturn strbuf_detach(&label, NULL);\n+}\n+\n+static int schtasks_remove_task(enum schedule_priority schedule, const char *cmd)\n+{\n+\tint result;\n+\tstruct strvec args = STRVEC_INIT;\n+\tconst char *frequency = get_frequency(schedule);\n+\tchar *name = schtasks_task_name(frequency);\n+\n+\tstrvec_split(&args, cmd);\n+\tstrvec_pushl(&args, \"/delete\", \"/tn\", name, \"/f\", NULL);\n+\n+\tresult = run_command_v_opt(args.v, 0);\n+\n+\tstrvec_clear(&args);\n+\tfree(name);\n+\treturn result;\n+}\n+\n+static int schtasks_remove_tasks(const char *cmd)\n+{\n+\treturn schtasks_remove_task(SCHEDULE_HOURLY, cmd) ||\n+\t\tschtasks_remove_task(SCHEDULE_DAILY, cmd) ||\n+\t\tschtasks_remove_task(SCHEDULE_WEEKLY, cmd);\n+}\n+\n+static int schtasks_schedule_task(const char *exec_path, enum schedule_priority schedule, const char *cmd)\n+{\n+\tint result;\n+\tstruct child_process child = CHILD_PROCESS_INIT;\n+\tconst char *xml;\n+\tstruct tempfile *tfile;\n+\tconst char *frequency = get_frequency(schedule);\n+\tchar *name = schtasks_task_name(frequency);\n+\tstruct strbuf tfilename = STRBUF_INIT;\n+\n+\tstrbuf_addf(&tfilename, \"schedule_%s_XXXXXX\", frequency);\n+\ttfile = xmks_tempfile(tfilename.buf);\n+\tstrbuf_release(&tfilename);\n+\n+\tif (!fdopen_tempfile(tfile, \"w\"))\n+\t\tdie(_(\"failed to create temp xml file\"));\n+\n+\txml = \"\\n\"\n+\t \"\\n\"\n+\t \"\\n\"\n+\t \"\\n\";\n+\tfputs(xml, tfile->fp);\n+\n+\tswitch (schedule) {\n+\tcase SCHEDULE_HOURLY:\n+\t\tfprintf(tfile->fp,\n+\t\t\t\"2020-01-01T01:00:00\\n\"\n+\t\t\t\"true\\n\"\n+\t\t\t\"\\n\"\n+\t\t\t\"1\\n\"\n+\t\t\t\"\\n\"\n+\t\t\t\"\\n\"\n+\t\t\t\"PT1H\\n\"\n+\t\t\t\"PT23H\\n\"\n+\t\t\t\"false\\n\"\n+\t\t\t\"\\n\");\n+\t\tbreak;\n+\n+\tcase SCHEDULE_DAILY:\n+\t\tfprintf(tfile->fp,\n+\t\t\t\"2020-01-01T00:00:00\\n\"\n+\t\t\t\"true\\n\"\n+\t\t\t\"\\n\"\n+\t\t\t\"\\n\"\n+\t\t\t\"\\n\"\n+\t\t\t\"\\n\"\n+\t\t\t\"\\n\"\n+\t\t\t\"\\n\"\n+\t\t\t\"\\n\"\n+\t\t\t\"\\n\"\n+\t\t\t\"\\n\"\n+\t\t\t\"1\\n\"\n+\t\t\t\"\\n\");\n+\t\tbreak;\n+\n+\tcase SCHEDULE_WEEKLY:\n+\t\tfprintf(tfile->fp,\n+\t\t\t\"2020-01-01T00:00:00\\n\"\n+\t\t\t\"true\\n\"\n+\t\t\t\"\\n\"\n+\t\t\t\"\\n\"\n+\t\t\t\"\\n\"\n+\t\t\t\"\\n\"\n+\t\t\t\"1\\n\"\n+\t\t\t\"\\n\");\n+\t\tbreak;\n+\n+\tdefault:\n+\t\tbreak;\n+\t}\n+\n+\txml = \"\\n\"\n+\t \"\\n\"\n+\t \"\\n\"\n+\t \"\\n\"\n+\t \"InteractiveToken\\n\"\n+\t \"LeastPrivilege\\n\"\n+\t \"\\n\"\n+\t \"\\n\"\n+\t \"\\n\"\n+\t \"IgnoreNew\\n\"\n+\t \"true\\n\"\n+\t \"true\\n\"\n+\t \"true\\n\"\n+\t \"false\\n\"\n+\t \"PT72H\\n\"\n+\t \"7\\n\"\n+\t \"\\n\"\n+\t \"\\n\"\n+\t \"\\n\"\n+\t \"\\\"%s\\\\git.exe\\\"\\n\"\n+\t \"--exec-path=\\\"%s\\\" for-each-repo --config=maintenance.repo maintenance run --schedule=%s\\n\"\n+\t \"\\n\"\n+\t \"\\n\"\n+\t \"\\n\";\n+\tfprintf(tfile->fp, xml, exec_path, exec_path, frequency);\n+\tstrvec_split(&child.args, cmd);\n+\tstrvec_pushl(&child.args, \"/create\", \"/tn\", name, \"/f\", \"/xml\",\n+\t\t\t\t get_tempfile_path(tfile), NULL);\n+\tclose_tempfile_gently(tfile);\n+\n+\tchild.no_stdout = 1;\n+\tchild.no_stderr = 1;\n+\n+\tif (start_command(&child))\n+\t\tdie(_(\"failed to start schtasks\"));\n+\tresult = finish_command(&child);\n+\n+\tdelete_tempfile(&tfile);\n+\tfree(name);\n+\treturn result;\n+}\n+\n+static int schtasks_schedule_tasks(const char *cmd)\n+{\n+\tconst char *exec_path = git_exec_path();\n+\n+\treturn schtasks_schedule_task(exec_path, SCHEDULE_HOURLY, cmd) ||\n+\t\tschtasks_schedule_task(exec_path, SCHEDULE_DAILY, cmd) ||\n+\t\tschtasks_schedule_task(exec_path, SCHEDULE_WEEKLY, cmd);\n+}\n+\n+static int schtasks_update_schedule(int run_maintenance, int fd, const char *cmd)\n+{\n+\tif (run_maintenance)\n+\t\treturn schtasks_schedule_tasks(cmd);\n+\telse\n+\t\treturn schtasks_remove_tasks(cmd);\n+}\n+\n #define BEGIN_LINE \"# BEGIN GIT MAINTENANCE SCHEDULE\"\n #define END_LINE \"# END GIT MAINTENANCE SCHEDULE\"\n \n@@ -1761,6 +1922,8 @@ static int crontab_update_schedule(int run_maintenance, int fd, const char *cmd)\n \n #if defined(__APPLE__)\n static const char platform_scheduler[] = \"launchctl\";\n+#elif defined(GIT_WINDOWS_NATIVE)\n+static const char platform_scheduler[] = \"schtasks\";\n #else\n static const char platform_scheduler[] = \"crontab\";\n #endif\n@@ -1789,6 +1952,8 @@ static int update_background_schedule(int enable)\n \n \tif (!strcmp(scheduler, \"launchctl\"))\n \t\tresult = launchctl_update_schedule(enable, lk.tempfile->fd, cmd);\n+\telse if (!strcmp(scheduler, \"schtasks\"))\n+\t\tresult = schtasks_update_schedule(enable, lk.tempfile->fd, cmd);\n \telse if (!strcmp(scheduler, \"crontab\"))\n \t\tresult = crontab_update_schedule(enable, lk.tempfile->fd, cmd);\n \telse\ndiff --git a/t/t7900-maintenance.sh b/t/t7900-maintenance.sh\nindex 6d373129016..f080c29a61d 100755\n--- a/t/t7900-maintenance.sh\n+++ b/t/t7900-maintenance.sh\n@@ -453,6 +453,45 @@ test_expect_success !MINGW 'start and stop macOS maintenance' '\n \ttest_line_count = 0 actual\n '\n \n+test_expect_success 'start and stop Windows maintenance' '\n+\twrite_script print-args <<-\\EOF &&\n+\techo $* >>args\n+\twhile test $# -gt 0\n+\tdo\n+\t\tcase \"$1\" in\n+\t\t/xml) shift; xmlfile=$1; break ;;\n+\t\t*) shift ;;\n+\t\tesac\n+\tdone\n+\ttest -z \"$xmlfile\" || cp \"$xmlfile\" \"$xmlfile.xml\"\n+\tEOF\n+\n+\trm -f args &&\n+\tGIT_TEST_MAINT_SCHEDULER=\"schtasks:./print-args\" GIT_TRACE2_PERF=1 git maintenance start &&\n+\n+\t# start registers the repo\n+\tgit config --get --global maintenance.repo \"$(pwd)\" &&\n+\n+\tfor frequency in hourly daily weekly\n+\tdo\n+\t\tgrep \"/create /tn Git Maintenance ($frequency) /f /xml\" args &&\n+\t\tfile=$(ls schedule_$frequency*.xml) &&\n+\t\ttest_xmllint \"$file\" &&\n+\t\tgrep \"encoding=.US-ASCII.\" \"$file\" || return 1\n+\tdone &&\n+\n+\trm -f args &&\n+\tGIT_TEST_MAINT_SCHEDULER=\"schtasks:./print-args\" git maintenance stop &&\n+\n+\t# stop does not unregister the repo\n+\tgit config --get --global maintenance.repo \"$(pwd)\" &&\n+\n+\trm expect &&\n+\tprintf \"/delete /tn Git Maintenance (%s) /f\\n\" \\\n+\t\thourly daily weekly >expect &&\n+\ttest_cmp expect args\n+'\n+\n test_expect_success 'register preserves existing strategy' '\n \tgit config maintenance.strategy none &&\n \tgit maintenance register &&\n-- \ngitgitgadget\n"},{"id":"411901","messageId":"cd015a5cbd7231deb01f5b783205df21bb180ed8.1607542142.git.gitgitgadget@gmail.com","threadId":"54562","inReplyTo":"pull.776.v6.git.1607542142.gitgitgadget@gmail.com","subject":"[PATCH v6 3/4] maintenance: use launchctl on macOS","fromName":"Derrick Stolee via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2020-12-09T19:29:01Z","receivedAt":"2020-12-09T19:32:56Z","isPatch":true,"sender":{"key":"stolee@gmail.com","avatar":"https://avatars.githubusercontent.com/u/570044?v=4"},"body":"From: Derrick Stolee \n\nThe existing mechanism for scheduling background maintenance is done\nthrough cron. The 'crontab -e' command allows updating the schedule\nwhile cron itself runs those commands. While this is technically\nsupported by macOS, it has some significant deficiencies:\n\n1. Every run of 'crontab -e' must request elevated privileges through\n the user interface. When running 'git maintenance start' from the\n Terminal app, it presents a dialog box saying \"Terminal.app would\n like to administer your computer. Administration can include\n modifying passwords, networking, and system settings.\" This is more\n alarming than what we are hoping to achieve. If this alert had some\n information about how \"git\" is trying to run \"crontab\" then we would\n have some reason to believe that this dialog might be fine. However,\n it also doesn't help that some scenarios just leave Git waiting for\n a response without presenting anything to the user. I experienced\n this when executing the command from a Bash terminal view inside\n Visual Studio Code.\n\n2. While cron initializes a user environment enough for \"git config\n --global --show-origin\" to show the correct config file information,\n it does not set up the environment enough for Git Credential Manager\n Core to load credentials during a 'prefetch' task. My prefetches\n against private repositories required re-authenticating through UI\n pop-ups in a way that should not be required.\n\nThe solution is to switch from cron to the Apple-recommended [1]\n'launchd' tool.\n\n[1] https://developer.apple.com/library/archive/documentation/MacOSX/Conceptual/BPSystemStartup/Chapters/ScheduledJobs.html\n\nThe basics of this tool is that we need to create XML-formatted\n\"plist\" files inside \"~/Library/LaunchAgents/\" and then use the\n'launchctl' tool to make launchd aware of them. The plist files\ninclude all of the scheduling information, along with the command-line\narguments split across an array of tags.\n\nFor example, here is my plist file for the weekly scheduled tasks:\n\n\n\n\nLabelorg.git-scm.git.weekly\nProgramArguments\n\n/usr/local/libexec/git-core/git\n--exec-path=/usr/local/libexec/git-core\nfor-each-repo\n--config=maintenance.repo\nmaintenance\nrun\n--schedule=weekly\n\nStartCalendarInterval\n\n\nDay0\nHour0\nMinute0\n\n\n\n\n\nThe schedules for the daily and hourly tasks are more complicated\nsince we need to use an array for the StartCalendarInterval with\nan entry for each of the six days other than the 0th day (to avoid\ncolliding with the weekly task), and each of the 23 hours other\nthan the 0th hour (to avoid colliding with the daily task).\n\nThe \"Label\" value is currently filled with \"org.git-scm.git.X\"\nwhere X is the frequency. We need a different plist file for each\nfrequency.\n\nThe launchctl command needs to be aligned with a user id in order\nto initialize the command environment. This must be done using\nthe 'launchctl bootstrap' subcommand. This subcommand is new as\nof macOS 10.11, which was released in September 2015. Before that\nrelease the 'launchctl load' subcommand was recommended. The best\nsource of information on this transition I have seen is available\nat [2]. The current design does not preclude a future version that\ndetects the available fatures of 'launchctl' to use the older\ncommands. However, it is best to rely on the newest version since\nApple might completely remove the deprecated version on short\nnotice.\n\n[2] https://babodee.wordpress.com/2016/04/09/launchctl-2-0-syntax/\n\nTo remove a schedule, we must run 'launchctl bootout' with a valid\nplist file. We also need to 'bootout' a task before the 'bootstrap'\nsubcommand will succeed, if such a task already exists.\n\nThe need for a user id requires us to run 'id -u' which works on\nPOSIX systems but not Windows. Further, the need for fully-qualitifed\npath names including $HOME behaves differently in the Git internals and\nthe external test suite. The $HOME variable starts with \"C:\\...\" instead\nof the \"/c/...\" that is provided by Git in these subcommands. The test\ntherefore has a prerequisite that we are not on Windows. The cross-\nplatform logic still allows us to test the macOS logic on a Linux\nmachine.\n\nWe can verify the commands that were run by 'git maintenance start'\nand 'git maintenance stop' by injecting a script that writes the\ncommand-line arguments into GIT_TEST_MAINT_SCHEDULER.\n\nAn earlier version of this patch accidentally had an opening\n\"\" tag when it should have had a closing \"\" tag. This\nwas caught during manual testing with actual 'launchctl' commands,\nbut we do not want to update developers' tasks when running tests.\nIt appears that macOS includes the \"xmllint\" tool which can verify\nthe XML format. This is useful for any system that might contain\nthe tool, so use it whenever it is available.\n\nCo-authored-by: Eric Sunshine \nSigned-off-by: Eric Sunshine \nSigned-off-by: Derrick Stolee \n---\n Documentation/git-maintenance.txt | 40 +++++++\n builtin/gc.c | 188 +++++++++++++++++++++++++++++-\n t/t7900-maintenance.sh | 58 +++++++++\n 3 files changed, 285 insertions(+), 1 deletion(-)\n\ndiff --git a/Documentation/git-maintenance.txt b/Documentation/git-maintenance.txt\nindex 1aa11124186..5f8f63f0988 100644\n--- a/Documentation/git-maintenance.txt\n+++ b/Documentation/git-maintenance.txt\n@@ -273,6 +273,46 @@ schedule to ensure you are executing the correct binaries in your\n schedule.\n \n \n+BACKGROUND MAINTENANCE ON MACOS SYSTEMS\n+---------------------------------------\n+\n+While macOS technically supports `cron`, using `crontab -e` requires\n+elevated privileges and the executed process does not have a full user\n+context. Without a full user context, Git and its credential helpers\n+cannot access stored credentials, so some maintenance tasks are not\n+functional.\n+\n+Instead, `git maintenance start` interacts with the `launchctl` tool,\n+which is the recommended way to schedule timed jobs in macOS. Scheduling\n+maintenance through `git maintenance (start|stop)` requires some\n+`launchctl` features available only in macOS 10.11 or later.\n+\n+Your user-specific scheduled tasks are stored as XML-formatted `.plist`\n+files in `~/Library/LaunchAgents/`. You can see the currently-registered\n+tasks using the following command:\n+\n+-----------------------------------------------------------------------\n+$ ls ~/Library/LaunchAgents/org.git-scm.git*\n+org.git-scm.git.daily.plist\n+org.git-scm.git.hourly.plist\n+org.git-scm.git.weekly.plist\n+-----------------------------------------------------------------------\n+\n+One task is registered for each `--schedule=` option. To\n+inspect how the XML format describes each schedule, open one of these\n+`.plist` files in an editor and inspect the `` element following\n+the `StartCalendarInterval` element.\n+\n+`git maintenance start` will overwrite these files and register the\n+tasks again with `launchctl`, so any customizations should be done by\n+creating your own `.plist` files with distinct names. Similarly, the\n+`git maintenance stop` command will unregister the tasks with `launchctl`\n+and delete the `.plist` files.\n+\n+To create more advanced customizations to your background tasks, see\n+launchctl.plist(5) for more information.\n+\n+\n GIT\n ---\n Part of the linkgit:git[1] suite\ndiff --git a/builtin/gc.c b/builtin/gc.c\nindex 18ae7f7138a..782769f2438 100644\n--- a/builtin/gc.c\n+++ b/builtin/gc.c\n@@ -1491,6 +1491,186 @@ static int maintenance_unregister(void)\n \treturn run_command(&config_unset);\n }\n \n+static const char *get_frequency(enum schedule_priority schedule)\n+{\n+\tswitch (schedule) {\n+\tcase SCHEDULE_HOURLY:\n+\t\treturn \"hourly\";\n+\tcase SCHEDULE_DAILY:\n+\t\treturn \"daily\";\n+\tcase SCHEDULE_WEEKLY:\n+\t\treturn \"weekly\";\n+\tdefault:\n+\t\tBUG(\"invalid schedule %d\", schedule);\n+\t}\n+}\n+\n+static char *launchctl_service_name(const char *frequency)\n+{\n+\tstruct strbuf label = STRBUF_INIT;\n+\tstrbuf_addf(&label, \"org.git-scm.git.%s\", frequency);\n+\treturn strbuf_detach(&label, NULL);\n+}\n+\n+static char *launchctl_service_filename(const char *name)\n+{\n+\tchar *expanded;\n+\tstruct strbuf filename = STRBUF_INIT;\n+\tstrbuf_addf(&filename, \"~/Library/LaunchAgents/%s.plist\", name);\n+\n+\texpanded = expand_user_path(filename.buf, 1);\n+\tif (!expanded)\n+\t\tdie(_(\"failed to expand path '%s'\"), filename.buf);\n+\n+\tstrbuf_release(&filename);\n+\treturn expanded;\n+}\n+\n+static char *launchctl_get_uid(void)\n+{\n+\treturn xstrfmt(\"gui/%d\", getuid());\n+}\n+\n+static int launchctl_boot_plist(int enable, const char *filename, const char *cmd)\n+{\n+\tint result;\n+\tstruct child_process child = CHILD_PROCESS_INIT;\n+\tchar *uid = launchctl_get_uid();\n+\n+\tstrvec_split(&child.args, cmd);\n+\tif (enable)\n+\t\tstrvec_push(&child.args, \"bootstrap\");\n+\telse\n+\t\tstrvec_push(&child.args, \"bootout\");\n+\tstrvec_push(&child.args, uid);\n+\tstrvec_push(&child.args, filename);\n+\n+\tchild.no_stderr = 1;\n+\tchild.no_stdout = 1;\n+\n+\tif (start_command(&child))\n+\t\tdie(_(\"failed to start launchctl\"));\n+\n+\tresult = finish_command(&child);\n+\n+\tfree(uid);\n+\treturn result;\n+}\n+\n+static int launchctl_remove_plist(enum schedule_priority schedule, const char *cmd)\n+{\n+\tconst char *frequency = get_frequency(schedule);\n+\tchar *name = launchctl_service_name(frequency);\n+\tchar *filename = launchctl_service_filename(name);\n+\tint result = launchctl_boot_plist(0, filename, cmd);\n+\tunlink(filename);\n+\tfree(filename);\n+\tfree(name);\n+\treturn result;\n+}\n+\n+static int launchctl_remove_plists(const char *cmd)\n+{\n+\treturn launchctl_remove_plist(SCHEDULE_HOURLY, cmd) ||\n+\t\tlaunchctl_remove_plist(SCHEDULE_DAILY, cmd) ||\n+\t\tlaunchctl_remove_plist(SCHEDULE_WEEKLY, cmd);\n+}\n+\n+static int launchctl_schedule_plist(const char *exec_path, enum schedule_priority schedule, const char *cmd)\n+{\n+\tFILE *plist;\n+\tint i;\n+\tconst char *preamble, *repeat;\n+\tconst char *frequency = get_frequency(schedule);\n+\tchar *name = launchctl_service_name(frequency);\n+\tchar *filename = launchctl_service_filename(name);\n+\n+\tif (safe_create_leading_directories(filename))\n+\t\tdie(_(\"failed to create directories for '%s'\"), filename);\n+\tplist = xfopen(filename, \"w\");\n+\n+\tpreamble = \"\\n\"\n+\t\t \"\\n\"\n+\t\t \"\"\n+\t\t \"\\n\"\n+\t\t \"Label%s\\n\"\n+\t\t \"ProgramArguments\\n\"\n+\t\t \"\\n\"\n+\t\t \"%s/git\\n\"\n+\t\t \"--exec-path=%s\\n\"\n+\t\t \"for-each-repo\\n\"\n+\t\t \"--config=maintenance.repo\\n\"\n+\t\t \"maintenance\\n\"\n+\t\t \"run\\n\"\n+\t\t \"--schedule=%s\\n\"\n+\t\t \"\\n\"\n+\t\t \"StartCalendarInterval\\n\"\n+\t\t \"\\n\";\n+\tfprintf(plist, preamble, name, exec_path, exec_path, frequency);\n+\n+\tswitch (schedule) {\n+\tcase SCHEDULE_HOURLY:\n+\t\trepeat = \"\\n\"\n+\t\t\t \"Hour%d\\n\"\n+\t\t\t \"Minute0\\n\"\n+\t\t\t \"\\n\";\n+\t\tfor (i = 1; i <= 23; i++)\n+\t\t\tfprintf(plist, repeat, i);\n+\t\tbreak;\n+\n+\tcase SCHEDULE_DAILY:\n+\t\trepeat = \"\\n\"\n+\t\t\t \"Day%d\\n\"\n+\t\t\t \"Hour0\\n\"\n+\t\t\t \"Minute0\\n\"\n+\t\t\t \"\\n\";\n+\t\tfor (i = 1; i <= 6; i++)\n+\t\t\tfprintf(plist, repeat, i);\n+\t\tbreak;\n+\n+\tcase SCHEDULE_WEEKLY:\n+\t\tfprintf(plist,\n+\t\t\t\"\\n\"\n+\t\t\t\"Day0\\n\"\n+\t\t\t\"Hour0\\n\"\n+\t\t\t\"Minute0\\n\"\n+\t\t\t\"\\n\");\n+\t\tbreak;\n+\n+\tdefault:\n+\t\t/* unreachable */\n+\t\tbreak;\n+\t}\n+\tfprintf(plist, \"\\n\\n\\n\");\n+\tfclose(plist);\n+\n+\t/* bootout might fail if not already running, so ignore */\n+\tlaunchctl_boot_plist(0, filename, cmd);\n+\tif (launchctl_boot_plist(1, filename, cmd))\n+\t\tdie(_(\"failed to bootstrap service %s\"), filename);\n+\n+\tfree(filename);\n+\tfree(name);\n+\treturn 0;\n+}\n+\n+static int launchctl_add_plists(const char *cmd)\n+{\n+\tconst char *exec_path = git_exec_path();\n+\n+\treturn launchctl_schedule_plist(exec_path, SCHEDULE_HOURLY, cmd) ||\n+\t\tlaunchctl_schedule_plist(exec_path, SCHEDULE_DAILY, cmd) ||\n+\t\tlaunchctl_schedule_plist(exec_path, SCHEDULE_WEEKLY, cmd);\n+}\n+\n+static int launchctl_update_schedule(int run_maintenance, int fd, const char *cmd)\n+{\n+\tif (run_maintenance)\n+\t\treturn launchctl_add_plists(cmd);\n+\telse\n+\t\treturn launchctl_remove_plists(cmd);\n+}\n+\n #define BEGIN_LINE \"# BEGIN GIT MAINTENANCE SCHEDULE\"\n #define END_LINE \"# END GIT MAINTENANCE SCHEDULE\"\n \n@@ -1579,7 +1759,11 @@ static int crontab_update_schedule(int run_maintenance, int fd, const char *cmd)\n \treturn result;\n }\n \n+#if defined(__APPLE__)\n+static const char platform_scheduler[] = \"launchctl\";\n+#else\n static const char platform_scheduler[] = \"crontab\";\n+#endif\n \n static int update_background_schedule(int enable)\n {\n@@ -1603,7 +1787,9 @@ static int update_background_schedule(int enable)\n \tif (hold_lock_file_for_update(&lk, lock_path, LOCK_NO_DEREF) < 0)\n \t\treturn error(_(\"another process is scheduling background maintenance\"));\n \n-\tif (!strcmp(scheduler, \"crontab\"))\n+\tif (!strcmp(scheduler, \"launchctl\"))\n+\t\tresult = launchctl_update_schedule(enable, lk.tempfile->fd, cmd);\n+\telse if (!strcmp(scheduler, \"crontab\"))\n \t\tresult = crontab_update_schedule(enable, lk.tempfile->fd, cmd);\n \telse\n \t\tdie(\"unknown background scheduler: %s\", scheduler);\ndiff --git a/t/t7900-maintenance.sh b/t/t7900-maintenance.sh\nindex eeb939168da..6d373129016 100755\n--- a/t/t7900-maintenance.sh\n+++ b/t/t7900-maintenance.sh\n@@ -7,6 +7,19 @@ test_description='git maintenance builtin'\n GIT_TEST_COMMIT_GRAPH=0\n GIT_TEST_MULTI_PACK_INDEX=0\n \n+test_lazy_prereq XMLLINT '\n+\txmllint --version\n+'\n+\n+test_xmllint () {\n+\tif test_have_prereq XMLLINT\n+\tthen\n+\t\txmllint --noout \"$@\"\n+\telse\n+\t\ttrue\n+\tfi\n+}\n+\n test_expect_success 'help text' '\n \ttest_expect_code 129 git maintenance -h 2>err &&\n \ttest_i18ngrep \"usage: git maintenance \" err &&\n@@ -395,6 +408,51 @@ test_expect_success 'start preserves existing schedule' '\n \tgrep \"Important information!\" cron.txt\n '\n \n+test_expect_success !MINGW 'start and stop macOS maintenance' '\n+\tuid=$(id -u) &&\n+\n+\twrite_script print-args <<-\\EOF &&\n+\techo $* >>args\n+\tEOF\n+\n+\trm -f args &&\n+\tGIT_TEST_MAINT_SCHEDULER=launchctl:./print-args git maintenance start &&\n+\n+\t# start registers the repo\n+\tgit config --get --global maintenance.repo \"$(pwd)\" &&\n+\n+\tls \"$HOME/Library/LaunchAgents\" >actual &&\n+\tcat >expect <<-\\EOF &&\n+\torg.git-scm.git.daily.plist\n+\torg.git-scm.git.hourly.plist\n+\torg.git-scm.git.weekly.plist\n+\tEOF\n+\ttest_cmp expect actual &&\n+\n+\trm expect &&\n+\tfor frequency in hourly daily weekly\n+\tdo\n+\t\tPLIST=\"$HOME/Library/LaunchAgents/org.git-scm.git.$frequency.plist\" &&\n+\t\ttest_xmllint \"$PLIST\" &&\n+\t\tgrep schedule=$frequency \"$PLIST\" &&\n+\t\techo \"bootout gui/$uid $PLIST\" >>expect &&\n+\t\techo \"bootstrap gui/$uid $PLIST\" >>expect || return 1\n+\tdone &&\n+\ttest_cmp expect args &&\n+\n+\trm -f args &&\n+\tGIT_TEST_MAINT_SCHEDULER=launchctl:./print-args git maintenance stop &&\n+\n+\t# stop does not unregister the repo\n+\tgit config --get --global maintenance.repo \"$(pwd)\" &&\n+\n+\tprintf \"bootout gui/$uid $HOME/Library/LaunchAgents/org.git-scm.git.%s.plist\\n\" \\\n+\t\thourly daily weekly >expect &&\n+\ttest_cmp expect args &&\n+\tls \"$HOME/Library/LaunchAgents\" >actual &&\n+\ttest_line_count = 0 actual\n+'\n+\n test_expect_success 'register preserves existing strategy' '\n \tgit config maintenance.strategy none &&\n \tgit maintenance register &&\n-- \ngitgitgadget\n\n"},{"id":"411957","messageId":"CAPig+cSciupmmupdgynfBO98EFke5fneC5W+k2vCo9Zkh3KF5w@mail.gmail.com","threadId":"54562","inReplyTo":"xmqqpn3izeit.fsf@gitster.c.googlers.com","subject":"Re: [PATCH v6 0/4] Maintenance IV: Platform-specific background maintenance","fromName":"Eric Sunshine","fromEmail":"sunshine@sunshineco.com","sentAt":"2020-12-10T00:49:27Z","receivedAt":"2020-12-10T00:51:05Z","isPatch":true,"sender":{"key":"sunshine@sunshineco.com","avatar":"https://avatars.githubusercontent.com/u/163641?v=4"},"body":"On Wed, Dec 9, 2020 at 7:33 PM Junio C Hamano wrote:\n> \"Derrick Stolee via GitGitGadget\" writes:\n> > Update in V6\n> > ============\n> >\n> > * The Windows platform uses the tempfile API a bit better, including using\n> > the frequency in the filename to make the test simpler.\n>\n> Are two fix-up patches from Eric that have been queued near the top\n> of ds/maintenance-part-4 still relevant?\n\nBoth of the patches from Sunshine are still relevant atop Stolee's\nlatest (v6), and they should apply cleanly (I would think) since v6\ndidn't change anything related to those patches.\n"},{"id":"411958","messageId":"xmqqpn3izeit.fsf@gitster.c.googlers.com","threadId":"54562","inReplyTo":"pull.776.v6.git.1607542142.gitgitgadget@gmail.com","subject":"Re: [PATCH v6 0/4] Maintenance IV: Platform-specific background maintenance","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2020-12-10T00:32:58Z","receivedAt":"2020-12-10T00:51:32Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"\"Derrick Stolee via GitGitGadget\" writes:\n\n> This is based on ds/maintenance-part-3.\n>\n> After sitting with the background maintenance as it has been cooking, I\n> wanted to come back around and implement the background maintenance for\n> Windows. However, I noticed that there were some things bothering me with\n> background maintenance on my macOS machine. These are detailed in PATCH 3,\n> but the tl;dr is that 'cron' is not recommended by Apple and instead\n> 'launchd' satisfies our needs.\n>\n> This series implements the background scheduling so git maintenance\n> (start|stop) works on those platforms. I've been operating with these\n> schedules for a while now without the problems described in the patches.\n>\n> There is a particularly annoying case about console windows popping up on\n> Windows, but PATCH 4 describes a plan to get around that.\n>\n>\n> Update in V6\n> ============\n>\n> * The Windows platform uses the tempfile API a bit better, including using\n> the frequency in the filename to make the test simpler.\n\nAre two fix-up patches from Eric that have been queued near the top\nof ds/maintenance-part-4 still relevant? \n\nAt least, the \"when invoked individually\" patch that added an \"-f\"\noption to two invocations of \"rm\" is still applicable, I would\nthink (I didn't look at the other one).\n\ncommit e3801c41e4d4cb1dd899942e04ab78310e781d07\nAuthor: Eric Sunshine \n\n t7900: make macOS-specific test work on Windows\n\nNotes (amlog):\n Message-Id: <20201130044224.12298-3-sunshine@sunshineco.com>\n\ncommit 1e5ddd79e2da18ee19b665a045d4187c5dc6234e\nAuthor: Eric Sunshine \n\n t7900: fix test failures when invoked individually via --run\n\nNotes (amlog):\n Message-Id: <20201130044224.12298-2-sunshine@sunshineco.com>\n"},{"id":"411963","messageId":"xmqq5z5azd2r.fsf@gitster.c.googlers.com","threadId":"54562","inReplyTo":"CAPig+cSciupmmupdgynfBO98EFke5fneC5W+k2vCo9Zkh3KF5w@mail.gmail.com","subject":"Re: [PATCH v6 0/4] Maintenance IV: Platform-specific background maintenance","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2020-12-10T01:04:12Z","receivedAt":"2020-12-10T01:05:11Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Eric Sunshine writes:\n\n> On Wed, Dec 9, 2020 at 7:33 PM Junio C Hamano wrote:\n>> \"Derrick Stolee via GitGitGadget\" writes:\n>> > Update in V6\n>> > ============\n>> >\n>> > * The Windows platform uses the tempfile API a bit better, including using\n>> > the frequency in the filename to make the test simpler.\n>>\n>> Are two fix-up patches from Eric that have been queued near the top\n>> of ds/maintenance-part-4 still relevant?\n>\n> Both of the patches from Sunshine are still relevant atop Stolee's\n> latest (v6), and they should apply cleanly (I would think) since v6\n> didn't change anything related to those patches.\n\nYup, I tried rebasing these two and they applied cleanly, so I'll\ninclude them in today's pushout (which I haven't finished yet).\n\nI probably would not notice if the updated 4-patch series already\nsolved the issue in another way without causing the textual conflict\nwith your two fix-up patches, though ;-)\n\nThanks.\n\n\n"},{"id":"413447","messageId":"73383c58-6b78-bded-da7b-fc44c8b9c02b@gmail.com","threadId":"54562","inReplyTo":"xmqq5z5azd2r.fsf@gitster.c.googlers.com","subject":"Re: [PATCH v6 0/4] Maintenance IV: Platform-specific background maintenance","fromName":"Derrick Stolee","fromEmail":"stolee@gmail.com","sentAt":"2021-01-05T12:17:57Z","receivedAt":"2021-01-05T12:18:42Z","isPatch":true,"sender":{"key":"stolee@gmail.com","avatar":"https://avatars.githubusercontent.com/u/570044?v=4"},"body":"On 12/9/2020 8:04 PM, Junio C Hamano wrote:\n> Eric Sunshine writes:\n> \n>> On Wed, Dec 9, 2020 at 7:33 PM Junio C Hamano wrote:\n>>> \"Derrick Stolee via GitGitGadget\" writes:\n>>>> Update in V6\n>>>> ============\n>>>>\n>>>> * The Windows platform uses the tempfile API a bit better, including using\n>>>> the frequency in the filename to make the test simpler.\n>>>\n>>> Are two fix-up patches from Eric that have been queued near the top\n>>> of ds/maintenance-part-4 still relevant?\n>>\n>> Both of the patches from Sunshine are still relevant atop Stolee's\n>> latest (v6), and they should apply cleanly (I would think) since v6\n>> didn't change anything related to those patches.\n> \n> Yup, I tried rebasing these two and they applied cleanly, so I'll\n> include them in today's pushout (which I haven't finished yet).\n> \n> I probably would not notice if the updated 4-patch series already\n> solved the issue in another way without causing the textual conflict\n> with your two fix-up patches, though ;-)\n\nI noticed a subtle issue with the v6 series, so I _will_ reroll the\nseries squashing in Eric's patches. He will remain a co-author and\nI'll add the Helped-by: Ævar along with the details for that patch.\n\nThanks,\n-Stolee\n\n"},{"id":"413450","messageId":"pull.776.v7.git.1609852108.gitgitgadget@gmail.com","threadId":"54562","inReplyTo":"pull.776.v6.git.1607542142.gitgitgadget@gmail.com","subject":"[PATCH v7 0/4] Maintenance IV: Platform-specific background maintenance","fromName":"Derrick Stolee via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2021-01-05T13:08:24Z","receivedAt":"2021-01-05T13:09:14Z","isPatch":true,"sender":{"key":"stolee@gmail.com","avatar":"https://avatars.githubusercontent.com/u/570044?v=4"},"body":"This is based on ds/maintenance-part-3.\n\nAfter sitting with the background maintenance as it has been cooking, I\nwanted to come back around and implement the background maintenance for\nWindows. However, I noticed that there were some things bothering me with\nbackground maintenance on my macOS machine. These are detailed in PATCH 3,\nbut the tl;dr is that 'cron' is not recommended by Apple and instead\n'launchd' satisfies our needs.\n\nThis series implements the background scheduling so git maintenance\n(start|stop) works on those platforms. I've been operating with these\nschedules for a while now without the problems described in the patches.\n\nThere is a particularly annoying case about console windows popping up on\nWindows, but PATCH 4 describes a plan to get around that.\n\n\nUpdate in V7\n============\n\n * I had included an \"encoding\" string in the XML file for schtasks based on\n an example using UTF-8. The cross-platform tests then complained (in\n xmllint) because they wrote in ASCII instead. However, actually testing\n the situation on Windows (see [1]) against the real schtasks finds that\n it doesn't like that encoding string. I removed it entirely, and\n everything seems happier.\n\n * I squashed Eric's two commits making the tests better. He remains a\n co-author and I kept his Helped-by. I had to rearrange the commit message\n a bit to point out the care he took for the cross-platform tests without\n referring to the test doing the wrong thing.\n\n[1] https://github.com/microsoft/git/pull/304\n\nThanks, -Stolee\n\ncc: jrnieder@gmail.com cc: jonathantanmy@google.com cc: sluongng@gmail.com\ncc: Đoàn Trần Công Danh congdanhqx@gmail.com cc: Martin Ågren\nmartin.agren@gmail.com cc: Eric Sunshine sunshine@sunshineco.com cc: Derrick\nStolee stolee@gmail.com\n\nDerrick Stolee (4):\n maintenance: extract platform-specific scheduling\n maintenance: include 'cron' details in docs\n maintenance: use launchctl on macOS\n maintenance: use Windows scheduled tasks\n\n Documentation/git-maintenance.txt | 116 ++++++++\n builtin/gc.c | 422 ++++++++++++++++++++++++++++--\n t/t7900-maintenance.sh | 104 +++++++-\n t/test-lib.sh | 7 +-\n 4 files changed, 615 insertions(+), 34 deletions(-)\n\n\nbase-commit: 0016b618182f642771dc589cf0090289f9fe1b4f\nPublished-As: https://github.com/gitgitgadget/git/releases/tag/pr-776%2Fderrickstolee%2Fmaintenance%2FmacOS-v7\nFetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-776/derrickstolee/maintenance/macOS-v7\nPull-Request: https://github.com/gitgitgadget/git/pull/776\n\nRange-diff vs v6:\n\n 1: 4807342b001 = 1: 4807342b001 maintenance: extract platform-specific scheduling\n 2: 7cc70a8fe7b = 2: 7cc70a8fe7b maintenance: include 'cron' details in docs\n 3: cd015a5cbd7 ! 3: 3576c7aa54e maintenance: use launchctl on macOS\n @@ Commit message\n the XML format. This is useful for any system that might contain\n the tool, so use it whenever it is available.\n \n + We strive to make these tests work on all platforms, but Windows caused\n + some headaches. In particular, the value of getuid() called by the C\n + code is not guaranteed to be the same as `$(id -u)` invoked by a test.\n + This is because `git.exe` is a native Windows program, whereas the\n + utility programs run by the test script mostly utilize the MSYS2 runtime,\n + which emulates a POSIX-like environment. Since the purpose of the test\n + is to check that the input to the hook is well-formed, the actual user\n + ID is immaterial, thus we can work around the problem by making the the\n + test UID-agnostic. Another subtle issue is the $HOME environment\n + variable being a Windows-style path instead of a Unix-style path. We can\n + be more flexible here instead of expecting exact path matches.\n +\n + Helped-by: Ævar Arnfjörð Bjarmason \n Co-authored-by: Eric Sunshine \n Signed-off-by: Eric Sunshine \n Signed-off-by: Derrick Stolee \n @@ t/t7900-maintenance.sh: test_expect_success 'start preserves existing schedule'\n \tgrep \"Important information!\" cron.txt\n '\n \n -+test_expect_success !MINGW 'start and stop macOS maintenance' '\n -+\tuid=$(id -u) &&\n ++test_expect_success 'start and stop macOS maintenance' '\n ++\t# ensure $HOME can be compared against hook arguments on all platforms\n ++\tpfx=$(cd \"$HOME\" && pwd) &&\n +\n +\twrite_script print-args <<-\\EOF &&\n -+\techo $* >>args\n ++\techo $* | sed \"s:gui/[0-9][0-9]*:gui/[UID]:\" >>args\n +\tEOF\n +\n +\trm -f args &&\n @@ t/t7900-maintenance.sh: test_expect_success 'start preserves existing schedule'\n +\tEOF\n +\ttest_cmp expect actual &&\n +\n -+\trm expect &&\n ++\trm -f expect &&\n +\tfor frequency in hourly daily weekly\n +\tdo\n -+\t\tPLIST=\"$HOME/Library/LaunchAgents/org.git-scm.git.$frequency.plist\" &&\n ++\t\tPLIST=\"$pfx/Library/LaunchAgents/org.git-scm.git.$frequency.plist\" &&\n +\t\ttest_xmllint \"$PLIST\" &&\n +\t\tgrep schedule=$frequency \"$PLIST\" &&\n -+\t\techo \"bootout gui/$uid $PLIST\" >>expect &&\n -+\t\techo \"bootstrap gui/$uid $PLIST\" >>expect || return 1\n ++\t\techo \"bootout gui/[UID] $PLIST\" >>expect &&\n ++\t\techo \"bootstrap gui/[UID] $PLIST\" >>expect || return 1\n +\tdone &&\n +\ttest_cmp expect args &&\n +\n @@ t/t7900-maintenance.sh: test_expect_success 'start preserves existing schedule'\n +\t# stop does not unregister the repo\n +\tgit config --get --global maintenance.repo \"$(pwd)\" &&\n +\n -+\tprintf \"bootout gui/$uid $HOME/Library/LaunchAgents/org.git-scm.git.%s.plist\\n\" \\\n ++\tprintf \"bootout gui/[UID] $pfx/Library/LaunchAgents/org.git-scm.git.%s.plist\\n\" \\\n +\t\thourly daily weekly >expect &&\n +\ttest_cmp expect args &&\n +\tls \"$HOME/Library/LaunchAgents\" >actual &&\n 4: 6ad4a6b98c6 ! 4: 68f5013dee3 maintenance: use Windows scheduled tasks\n @@ Documentation/git-maintenance.txt: To create more advanced customizations to you\n Part of the linkgit:git[1] suite\n \n ## builtin/gc.c ##\n +@@ builtin/gc.c: static int launchctl_schedule_plist(const char *exec_path, enum schedule_priorit\n + \t\tdie(_(\"failed to create directories for '%s'\"), filename);\n + \tplist = xfopen(filename, \"w\");\n + \n +-\tpreamble = \"\\n\"\n ++\tpreamble = \"\\n\"\n + \t\t \"\\n\"\n + \t\t \"\"\n + \t\t \"\\n\"\n @@ builtin/gc.c: static int launchctl_update_schedule(int run_maintenance, int fd, const char *cm\n \t\treturn launchctl_remove_plists(cmd);\n }\n @@ builtin/gc.c: static int launchctl_update_schedule(int run_maintenance, int fd,\n +\tchar *name = schtasks_task_name(frequency);\n +\tstruct strbuf tfilename = STRBUF_INIT;\n +\n -+\tstrbuf_addf(&tfilename, \"schedule_%s_XXXXXX\", frequency);\n ++\tstrbuf_addf(&tfilename, \"%s/schedule_%s_XXXXXX\",\n ++\t\t get_git_common_dir(), frequency);\n +\ttfile = xmks_tempfile(tfilename.buf);\n +\tstrbuf_release(&tfilename);\n +\n +\tif (!fdopen_tempfile(tfile, \"w\"))\n +\t\tdie(_(\"failed to create temp xml file\"));\n +\n -+\txml = \"\\n\"\n ++\txml = \"\\n\"\n +\t \"\\n\"\n +\t \"\\n\"\n +\t \"\\n\";\n @@ builtin/gc.c: static int update_background_schedule(int enable)\n \telse\n \n ## t/t7900-maintenance.sh ##\n -@@ t/t7900-maintenance.sh: test_expect_success !MINGW 'start and stop macOS maintenance' '\n +@@ t/t7900-maintenance.sh: test_expect_success 'start and stop macOS maintenance' '\n \ttest_line_count = 0 actual\n '\n \n @@ t/t7900-maintenance.sh: test_expect_success !MINGW 'start and stop macOS mainten\n +\tEOF\n +\n +\trm -f args &&\n -+\tGIT_TEST_MAINT_SCHEDULER=\"schtasks:./print-args\" GIT_TRACE2_PERF=1 git maintenance start &&\n ++\tGIT_TEST_MAINT_SCHEDULER=\"schtasks:./print-args\" git maintenance start &&\n +\n +\t# start registers the repo\n +\tgit config --get --global maintenance.repo \"$(pwd)\" &&\n @@ t/t7900-maintenance.sh: test_expect_success !MINGW 'start and stop macOS mainten\n +\tfor frequency in hourly daily weekly\n +\tdo\n +\t\tgrep \"/create /tn Git Maintenance ($frequency) /f /xml\" args &&\n -+\t\tfile=$(ls schedule_$frequency*.xml) &&\n -+\t\ttest_xmllint \"$file\" &&\n -+\t\tgrep \"encoding=.US-ASCII.\" \"$file\" || return 1\n ++\t\tfile=$(ls .git/schedule_${frequency}*.xml) &&\n ++\t\ttest_xmllint \"$file\" || return 1\n +\tdone &&\n +\n +\trm -f args &&\n @@ t/t7900-maintenance.sh: test_expect_success !MINGW 'start and stop macOS mainten\n +\t# stop does not unregister the repo\n +\tgit config --get --global maintenance.repo \"$(pwd)\" &&\n +\n -+\trm expect &&\n +\tprintf \"/delete /tn Git Maintenance (%s) /f\\n\" \\\n +\t\thourly daily weekly >expect &&\n +\ttest_cmp expect args\n\n-- \ngitgitgadget\n"},{"id":"413451","messageId":"7cc70a8fe7bad92179961b31520147af39d5353c.1609852109.git.gitgitgadget@gmail.com","threadId":"54562","inReplyTo":"pull.776.v7.git.1609852108.gitgitgadget@gmail.com","subject":"[PATCH v7 2/4] maintenance: include 'cron' details in docs","fromName":"Derrick Stolee via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2021-01-05T13:08:26Z","receivedAt":"2021-01-05T13:09:31Z","isPatch":true,"sender":{"key":"stolee@gmail.com","avatar":"https://avatars.githubusercontent.com/u/570044?v=4"},"body":"From: Derrick Stolee \n\nAdvanced and expert users may want to know how 'git maintenance start'\nschedules background maintenance in order to customize their own\nschedules beyond what the maintenance.* config values allow. Start a new\nset of sections in git-maintenance.txt that describe how 'cron' is used\nto run these tasks.\n\nThis is particularly valuable for users who want to inspect what Git is\ndoing or for users who want to customize the schedule further. Having a\nbaseline can provide a way forward for users who have never worked with\ncron schedules.\n\nHelped-by: Eric Sunshine \nSigned-off-by: Derrick Stolee \n---\n Documentation/git-maintenance.txt | 54 +++++++++++++++++++++++++++++++\n 1 file changed, 54 insertions(+)\n\ndiff --git a/Documentation/git-maintenance.txt b/Documentation/git-maintenance.txt\nindex 6fec1eb8dc2..1aa11124186 100644\n--- a/Documentation/git-maintenance.txt\n+++ b/Documentation/git-maintenance.txt\n@@ -218,6 +218,60 @@ Further, the `git gc` command should not be combined with\n but does not take the lock in the same way as `git maintenance run`. If\n possible, use `git maintenance run --task=gc` instead of `git gc`.\n \n+The following sections describe the mechanisms put in place to run\n+background maintenance by `git maintenance start` and how to customize\n+them.\n+\n+BACKGROUND MAINTENANCE ON POSIX SYSTEMS\n+---------------------------------------\n+\n+The standard mechanism for scheduling background tasks on POSIX systems\n+is cron(8). This tool executes commands based on a given schedule. The\n+current list of user-scheduled tasks can be found by running `crontab -l`.\n+The schedule written by `git maintenance start` is similar to this:\n+\n+-----------------------------------------------------------------------\n+# BEGIN GIT MAINTENANCE SCHEDULE\n+# The following schedule was created by Git\n+# Any edits made in this region might be\n+# replaced in the future by a Git command.\n+\n+0 1-23 * * * \"//git\" --exec-path=\"/\" for-each-repo --config=maintenance.repo maintenance run --schedule=hourly\n+0 0 * * 1-6 \"//git\" --exec-path=\"/\" for-each-repo --config=maintenance.repo maintenance run --schedule=daily\n+0 0 * * 0 \"//git\" --exec-path=\"/\" for-each-repo --config=maintenance.repo maintenance run --schedule=weekly\n+\n+# END GIT MAINTENANCE SCHEDULE\n+-----------------------------------------------------------------------\n+\n+The comments are used as a region to mark the schedule as written by Git.\n+Any modifications within this region will be completely deleted by\n+`git maintenance stop` or overwritten by `git maintenance start`.\n+\n+The `crontab` entry specifies the full path of the `git` executable to\n+ensure that the executed `git` command is the same one with which\n+`git maintenance start` was issued independent of `PATH`. If the same user\n+runs `git maintenance start` with multiple Git executables, then only the\n+latest executable is used.\n+\n+These commands use `git for-each-repo --config=maintenance.repo` to run\n+`git maintenance run --schedule=` on each repository listed in\n+the multi-valued `maintenance.repo` config option. These are typically\n+loaded from the user-specific global config. The `git maintenance` process\n+then determines which maintenance tasks are configured to run on each\n+repository with each `` using the `maintenance..schedule`\n+config options. These values are loaded from the global or repository\n+config values.\n+\n+If the config values are insufficient to achieve your desired background\n+maintenance schedule, then you can create your own schedule. If you run\n+`crontab -e`, then an editor will load with your user-specific `cron`\n+schedule. In that editor, you can add your own schedule lines. You could\n+start by adapting the default schedule listed earlier, or you could read\n+the crontab(5) documentation for advanced scheduling techniques. Please\n+do use the full path and `--exec-path` techniques from the default\n+schedule to ensure you are executing the correct binaries in your\n+schedule.\n+\n \n GIT\n ---\n-- \ngitgitgadget\n\n"},{"id":"413452","messageId":"4807342b0019be29bb369ed3403a485f0ce9c15d.1609852108.git.gitgitgadget@gmail.com","threadId":"54562","inReplyTo":"pull.776.v7.git.1609852108.gitgitgadget@gmail.com","subject":"[PATCH v7 1/4] maintenance: extract platform-specific scheduling","fromName":"Derrick Stolee via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2021-01-05T13:08:25Z","receivedAt":"2021-01-05T13:09:31Z","isPatch":true,"sender":{"key":"stolee@gmail.com","avatar":"https://avatars.githubusercontent.com/u/570044?v=4"},"body":"From: Derrick Stolee \n\nThe existing schedule mechanism using 'cron' is supported by POSIX\nplatforms, but not Windows. It also works slightly differently on\nmacOS to significant detriment of the user experience. To allow for\nnew implementations on these platforms, extract a method that\nperforms the platform-specific scheduling mechanism. This will be\nswapped at compile time with new implementations on specialized\nplatforms.\n\nAs we add this generality, rename GIT_TEST_CRONTAB to\nGIT_TEST_MAINT_SCHEDULER. Further, this variable is now parsed as\n\":\" so we can test platform-specific scheduling\nlogic even when not on the correct platform. By specifying the\n in this string, we will be able to test all three sets of\nGit logic from a Linux machine.\n\nCo-authored-by: Eric Sunshine \nSigned-off-by: Eric Sunshine \nSigned-off-by: Derrick Stolee \n---\n builtin/gc.c | 70 ++++++++++++++++++++++++++----------------\n t/t7900-maintenance.sh | 8 ++---\n t/test-lib.sh | 7 +++--\n 3 files changed, 51 insertions(+), 34 deletions(-)\n\ndiff --git a/builtin/gc.c b/builtin/gc.c\nindex e3098ef6a12..18ae7f7138a 100644\n--- a/builtin/gc.c\n+++ b/builtin/gc.c\n@@ -1494,35 +1494,23 @@ static int maintenance_unregister(void)\n #define BEGIN_LINE \"# BEGIN GIT MAINTENANCE SCHEDULE\"\n #define END_LINE \"# END GIT MAINTENANCE SCHEDULE\"\n \n-static int update_background_schedule(int run_maintenance)\n+static int crontab_update_schedule(int run_maintenance, int fd, const char *cmd)\n {\n \tint result = 0;\n \tint in_old_region = 0;\n \tstruct child_process crontab_list = CHILD_PROCESS_INIT;\n \tstruct child_process crontab_edit = CHILD_PROCESS_INIT;\n \tFILE *cron_list, *cron_in;\n-\tconst char *crontab_name;\n \tstruct strbuf line = STRBUF_INIT;\n-\tstruct lock_file lk;\n-\tchar *lock_path = xstrfmt(\"%s/schedule\", the_repository->objects->odb->path);\n \n-\tif (hold_lock_file_for_update(&lk, lock_path, LOCK_NO_DEREF) < 0)\n-\t\treturn error(_(\"another process is scheduling background maintenance\"));\n-\n-\tcrontab_name = getenv(\"GIT_TEST_CRONTAB\");\n-\tif (!crontab_name)\n-\t\tcrontab_name = \"crontab\";\n-\n-\tstrvec_split(&crontab_list.args, crontab_name);\n+\tstrvec_split(&crontab_list.args, cmd);\n \tstrvec_push(&crontab_list.args, \"-l\");\n \tcrontab_list.in = -1;\n-\tcrontab_list.out = dup(lk.tempfile->fd);\n+\tcrontab_list.out = dup(fd);\n \tcrontab_list.git_cmd = 0;\n \n-\tif (start_command(&crontab_list)) {\n-\t\tresult = error(_(\"failed to run 'crontab -l'; your system might not support 'cron'\"));\n-\t\tgoto cleanup;\n-\t}\n+\tif (start_command(&crontab_list))\n+\t\treturn error(_(\"failed to run 'crontab -l'; your system might not support 'cron'\"));\n \n \t/* Ignore exit code, as an empty crontab will return error. */\n \tfinish_command(&crontab_list);\n@@ -1531,17 +1519,15 @@ static int update_background_schedule(int run_maintenance)\n \t * Read from the .lock file, filtering out the old\n \t * schedule while appending the new schedule.\n \t */\n-\tcron_list = fdopen(lk.tempfile->fd, \"r\");\n+\tcron_list = fdopen(fd, \"r\");\n \trewind(cron_list);\n \n-\tstrvec_split(&crontab_edit.args, crontab_name);\n+\tstrvec_split(&crontab_edit.args, cmd);\n \tcrontab_edit.in = -1;\n \tcrontab_edit.git_cmd = 0;\n \n-\tif (start_command(&crontab_edit)) {\n-\t\tresult = error(_(\"failed to run 'crontab'; your system might not support 'cron'\"));\n-\t\tgoto cleanup;\n-\t}\n+\tif (start_command(&crontab_edit))\n+\t\treturn error(_(\"failed to run 'crontab'; your system might not support 'cron'\"));\n \n \tcron_in = fdopen(crontab_edit.in, \"w\");\n \tif (!cron_in) {\n@@ -1586,14 +1572,44 @@ static int update_background_schedule(int run_maintenance)\n \tclose(crontab_edit.in);\n \n done_editing:\n-\tif (finish_command(&crontab_edit)) {\n+\tif (finish_command(&crontab_edit))\n \t\tresult = error(_(\"'crontab' died\"));\n-\t\tgoto cleanup;\n+\telse\n+\t\tfclose(cron_list);\n+\treturn result;\n+}\n+\n+static const char platform_scheduler[] = \"crontab\";\n+\n+static int update_background_schedule(int enable)\n+{\n+\tint result;\n+\tconst char *scheduler = platform_scheduler;\n+\tconst char *cmd = scheduler;\n+\tchar *testing;\n+\tstruct lock_file lk;\n+\tchar *lock_path = xstrfmt(\"%s/schedule\", the_repository->objects->odb->path);\n+\n+\ttesting = xstrdup_or_null(getenv(\"GIT_TEST_MAINT_SCHEDULER\"));\n+\tif (testing) {\n+\t\tchar *sep = strchr(testing, ':');\n+\t\tif (!sep)\n+\t\t\tdie(\"GIT_TEST_MAINT_SCHEDULER unparseable: %s\", testing);\n+\t\t*sep = '\\0';\n+\t\tscheduler = testing;\n+\t\tcmd = sep + 1;\n \t}\n-\tfclose(cron_list);\n \n-cleanup:\n+\tif (hold_lock_file_for_update(&lk, lock_path, LOCK_NO_DEREF) < 0)\n+\t\treturn error(_(\"another process is scheduling background maintenance\"));\n+\n+\tif (!strcmp(scheduler, \"crontab\"))\n+\t\tresult = crontab_update_schedule(enable, lk.tempfile->fd, cmd);\n+\telse\n+\t\tdie(\"unknown background scheduler: %s\", scheduler);\n+\n \trollback_lock_file(&lk);\n+\tfree(testing);\n \treturn result;\n }\n \ndiff --git a/t/t7900-maintenance.sh b/t/t7900-maintenance.sh\nindex 20184e96e1a..eeb939168da 100755\n--- a/t/t7900-maintenance.sh\n+++ b/t/t7900-maintenance.sh\n@@ -368,7 +368,7 @@ test_expect_success 'register and unregister' '\n '\n \n test_expect_success 'start from empty cron table' '\n-\tGIT_TEST_CRONTAB=\"test-tool crontab cron.txt\" git maintenance start &&\n+\tGIT_TEST_MAINT_SCHEDULER=\"crontab:test-tool crontab cron.txt\" git maintenance start &&\n \n \t# start registers the repo\n \tgit config --get --global maintenance.repo \"$(pwd)\" &&\n@@ -379,19 +379,19 @@ test_expect_success 'start from empty cron table' '\n '\n \n test_expect_success 'stop from existing schedule' '\n-\tGIT_TEST_CRONTAB=\"test-tool crontab cron.txt\" git maintenance stop &&\n+\tGIT_TEST_MAINT_SCHEDULER=\"crontab:test-tool crontab cron.txt\" git maintenance stop &&\n \n \t# stop does not unregister the repo\n \tgit config --get --global maintenance.repo \"$(pwd)\" &&\n \n \t# Operation is idempotent\n-\tGIT_TEST_CRONTAB=\"test-tool crontab cron.txt\" git maintenance stop &&\n+\tGIT_TEST_MAINT_SCHEDULER=\"crontab:test-tool crontab cron.txt\" git maintenance stop &&\n \ttest_must_be_empty cron.txt\n '\n \n test_expect_success 'start preserves existing schedule' '\n \techo \"Important information!\" >cron.txt &&\n-\tGIT_TEST_CRONTAB=\"test-tool crontab cron.txt\" git maintenance start &&\n+\tGIT_TEST_MAINT_SCHEDULER=\"crontab:test-tool crontab cron.txt\" git maintenance start &&\n \tgrep \"Important information!\" cron.txt\n '\n \ndiff --git a/t/test-lib.sh b/t/test-lib.sh\nindex 4a60d1ed766..ddbeee1f5eb 100644\n--- a/t/test-lib.sh\n+++ b/t/test-lib.sh\n@@ -1704,7 +1704,8 @@ test_lazy_prereq REBASE_P '\n '\n \n # Ensure that no test accidentally triggers a Git command\n-# that runs 'crontab', affecting a user's cron schedule.\n-# Tests that verify the cron integration must set this locally\n+# that runs the actual maintenance scheduler, affecting a user's\n+# system permanently.\n+# Tests that verify the scheduler integration must set this locally\n # to avoid errors.\n-GIT_TEST_CRONTAB=\"exit 1\"\n+GIT_TEST_MAINT_SCHEDULER=\"none:exit 1\"\n-- \ngitgitgadget\n\n"},{"id":"413453","messageId":"68f5013dee3762d5ad2fdc4bb229b522881efdf7.1609852109.git.gitgitgadget@gmail.com","threadId":"54562","inReplyTo":"pull.776.v7.git.1609852108.gitgitgadget@gmail.com","subject":"[PATCH v7 4/4] maintenance: use Windows scheduled tasks","fromName":"Derrick Stolee via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2021-01-05T13:08:28Z","receivedAt":"2021-01-05T13:09:31Z","isPatch":true,"sender":{"key":"stolee@gmail.com","avatar":"https://avatars.githubusercontent.com/u/570044?v=4"},"body":"From: Derrick Stolee \n\nGit's background maintenance uses cron by default, but this is not\navailable on Windows. Instead, integrate with Task Scheduler.\n\nTasks can be scheduled using the 'schtasks' command. There are several\ncommand-line options that can allow for some advanced scheduling, but\nunfortunately these seem to all require authenticating using a password.\n\nInstead, use the \"/xml\" option to pass an XML file that contains the\nconfiguration for the necessary schedule. These XML files are based on\nsome that I exported after constructing a schedule in the Task Scheduler\nGUI. These options only run background maintenance when the user is\nlogged in, and more fields are populated with the current username and\nSID at run-time by 'schtasks'.\n\nSince the GIT_TEST_MAINT_SCHEDULER environment variable allows us to\nspecify 'schtasks' as the scheduler, we can test the Windows-specific\nlogic on other platforms. Thus, add a check that the XML file written\nby Git is valid when xmllint exists on the system.\n\nSince we use a temporary file for the XML files sent to 'schtasks', we\nprefix the random characters with the frequency so it is easier to\nexamine the proper file during tests. Instead of an exact match on the\n'args' file, we 'grep' for the arguments other than the filename.\n\nThere is a deficiency in the current design. Windows has two kinds of\napplications: GUI applications that start by \"winmain()\" and console\napplications that start by \"main()\". Console applications are attached\nto a new Console window if they are not already associated with a GUI\napplication. This means that every hour the scheudled task launches a\ncommand window for the scheduled tasks. Not only is this visually\nobtrusive, but it also takes focus from whatever else the user is\ndoing!\n\nA simple fix would be to insert a GUI application that acts as a shim\nbetween the scheduled task and Git. This is currently possible in Git\nfor Windows by setting the tag equal to\n\n C:\\Program Files\\Git\\git-bash.exe\n\nwith options \"--hide --no-needs-console --command=cmd\\git.exe\"\nfollowed by the arguments currently used. Since git-bash.exe is not\nincluded in Windows builds of core Git, I chose to leave out this\nfeature. My plan is to submit a small patch to Git for Windows that\nconverts the use of git.exe with this use of git-bash.exe in the\nshort term. In the long term, we can consider creating this GUI\nshim application within core Git, perhaps in contrib/.\n\nCo-authored-by: Eric Sunshine \nSigned-off-by: Eric Sunshine \nSigned-off-by: Derrick Stolee \n---\n Documentation/git-maintenance.txt | 22 ++++\n builtin/gc.c | 168 +++++++++++++++++++++++++++++-\n t/t7900-maintenance.sh | 37 +++++++\n 3 files changed, 226 insertions(+), 1 deletion(-)\n\ndiff --git a/Documentation/git-maintenance.txt b/Documentation/git-maintenance.txt\nindex 5f8f63f0988..6970f2b8983 100644\n--- a/Documentation/git-maintenance.txt\n+++ b/Documentation/git-maintenance.txt\n@@ -313,6 +313,28 @@ To create more advanced customizations to your background tasks, see\n launchctl.plist(5) for more information.\n \n \n+BACKGROUND MAINTENANCE ON WINDOWS SYSTEMS\n+-----------------------------------------\n+\n+Windows does not support `cron` and instead has its own system for\n+scheduling background tasks. The `git maintenance start` command uses\n+the `schtasks` command to submit tasks to this system. You can inspect\n+all background tasks using the Task Scheduler application. The tasks\n+added by Git have names of the form `Git Maintenance ()`.\n+The Task Scheduler GUI has ways to inspect these tasks, but you can also\n+export the tasks to XML files and view the details there.\n+\n+Note that since Git is a console application, these background tasks\n+create a console window visible to the current user. This can be changed\n+manually by selecting the \"Run whether user is logged in or not\" option\n+in Task Scheduler. This change requires a password input, which is why\n+`git maintenance start` does not select it by default.\n+\n+If you want to customize the background tasks, please rename the tasks\n+so future calls to `git maintenance (start|stop)` do not overwrite your\n+custom tasks.\n+\n+\n GIT\n ---\n Part of the linkgit:git[1] suite\ndiff --git a/builtin/gc.c b/builtin/gc.c\nindex 782769f2438..fdc95d9e99f 100644\n--- a/builtin/gc.c\n+++ b/builtin/gc.c\n@@ -1589,7 +1589,7 @@ static int launchctl_schedule_plist(const char *exec_path, enum schedule_priorit\n \t\tdie(_(\"failed to create directories for '%s'\"), filename);\n \tplist = xfopen(filename, \"w\");\n \n-\tpreamble = \"\\n\"\n+\tpreamble = \"\\n\"\n \t\t \"\\n\"\n \t\t \"\"\n \t\t \"\\n\"\n@@ -1671,6 +1671,168 @@ static int launchctl_update_schedule(int run_maintenance, int fd, const char *cm\n \t\treturn launchctl_remove_plists(cmd);\n }\n \n+static char *schtasks_task_name(const char *frequency)\n+{\n+\tstruct strbuf label = STRBUF_INIT;\n+\tstrbuf_addf(&label, \"Git Maintenance (%s)\", frequency);\n+\treturn strbuf_detach(&label, NULL);\n+}\n+\n+static int schtasks_remove_task(enum schedule_priority schedule, const char *cmd)\n+{\n+\tint result;\n+\tstruct strvec args = STRVEC_INIT;\n+\tconst char *frequency = get_frequency(schedule);\n+\tchar *name = schtasks_task_name(frequency);\n+\n+\tstrvec_split(&args, cmd);\n+\tstrvec_pushl(&args, \"/delete\", \"/tn\", name, \"/f\", NULL);\n+\n+\tresult = run_command_v_opt(args.v, 0);\n+\n+\tstrvec_clear(&args);\n+\tfree(name);\n+\treturn result;\n+}\n+\n+static int schtasks_remove_tasks(const char *cmd)\n+{\n+\treturn schtasks_remove_task(SCHEDULE_HOURLY, cmd) ||\n+\t\tschtasks_remove_task(SCHEDULE_DAILY, cmd) ||\n+\t\tschtasks_remove_task(SCHEDULE_WEEKLY, cmd);\n+}\n+\n+static int schtasks_schedule_task(const char *exec_path, enum schedule_priority schedule, const char *cmd)\n+{\n+\tint result;\n+\tstruct child_process child = CHILD_PROCESS_INIT;\n+\tconst char *xml;\n+\tstruct tempfile *tfile;\n+\tconst char *frequency = get_frequency(schedule);\n+\tchar *name = schtasks_task_name(frequency);\n+\tstruct strbuf tfilename = STRBUF_INIT;\n+\n+\tstrbuf_addf(&tfilename, \"%s/schedule_%s_XXXXXX\",\n+\t\t get_git_common_dir(), frequency);\n+\ttfile = xmks_tempfile(tfilename.buf);\n+\tstrbuf_release(&tfilename);\n+\n+\tif (!fdopen_tempfile(tfile, \"w\"))\n+\t\tdie(_(\"failed to create temp xml file\"));\n+\n+\txml = \"\\n\"\n+\t \"\\n\"\n+\t \"\\n\"\n+\t \"\\n\";\n+\tfputs(xml, tfile->fp);\n+\n+\tswitch (schedule) {\n+\tcase SCHEDULE_HOURLY:\n+\t\tfprintf(tfile->fp,\n+\t\t\t\"2020-01-01T01:00:00\\n\"\n+\t\t\t\"true\\n\"\n+\t\t\t\"\\n\"\n+\t\t\t\"1\\n\"\n+\t\t\t\"\\n\"\n+\t\t\t\"\\n\"\n+\t\t\t\"PT1H\\n\"\n+\t\t\t\"PT23H\\n\"\n+\t\t\t\"false\\n\"\n+\t\t\t\"\\n\");\n+\t\tbreak;\n+\n+\tcase SCHEDULE_DAILY:\n+\t\tfprintf(tfile->fp,\n+\t\t\t\"2020-01-01T00:00:00\\n\"\n+\t\t\t\"true\\n\"\n+\t\t\t\"\\n\"\n+\t\t\t\"\\n\"\n+\t\t\t\"\\n\"\n+\t\t\t\"\\n\"\n+\t\t\t\"\\n\"\n+\t\t\t\"\\n\"\n+\t\t\t\"\\n\"\n+\t\t\t\"\\n\"\n+\t\t\t\"\\n\"\n+\t\t\t\"1\\n\"\n+\t\t\t\"\\n\");\n+\t\tbreak;\n+\n+\tcase SCHEDULE_WEEKLY:\n+\t\tfprintf(tfile->fp,\n+\t\t\t\"2020-01-01T00:00:00\\n\"\n+\t\t\t\"true\\n\"\n+\t\t\t\"\\n\"\n+\t\t\t\"\\n\"\n+\t\t\t\"\\n\"\n+\t\t\t\"\\n\"\n+\t\t\t\"1\\n\"\n+\t\t\t\"\\n\");\n+\t\tbreak;\n+\n+\tdefault:\n+\t\tbreak;\n+\t}\n+\n+\txml = \"\\n\"\n+\t \"\\n\"\n+\t \"\\n\"\n+\t \"\\n\"\n+\t \"InteractiveToken\\n\"\n+\t \"LeastPrivilege\\n\"\n+\t \"\\n\"\n+\t \"\\n\"\n+\t \"\\n\"\n+\t \"IgnoreNew\\n\"\n+\t \"true\\n\"\n+\t \"true\\n\"\n+\t \"true\\n\"\n+\t \"false\\n\"\n+\t \"PT72H\\n\"\n+\t \"7\\n\"\n+\t \"\\n\"\n+\t \"\\n\"\n+\t \"\\n\"\n+\t \"\\\"%s\\\\git.exe\\\"\\n\"\n+\t \"--exec-path=\\\"%s\\\" for-each-repo --config=maintenance.repo maintenance run --schedule=%s\\n\"\n+\t \"\\n\"\n+\t \"\\n\"\n+\t \"\\n\";\n+\tfprintf(tfile->fp, xml, exec_path, exec_path, frequency);\n+\tstrvec_split(&child.args, cmd);\n+\tstrvec_pushl(&child.args, \"/create\", \"/tn\", name, \"/f\", \"/xml\",\n+\t\t\t\t get_tempfile_path(tfile), NULL);\n+\tclose_tempfile_gently(tfile);\n+\n+\tchild.no_stdout = 1;\n+\tchild.no_stderr = 1;\n+\n+\tif (start_command(&child))\n+\t\tdie(_(\"failed to start schtasks\"));\n+\tresult = finish_command(&child);\n+\n+\tdelete_tempfile(&tfile);\n+\tfree(name);\n+\treturn result;\n+}\n+\n+static int schtasks_schedule_tasks(const char *cmd)\n+{\n+\tconst char *exec_path = git_exec_path();\n+\n+\treturn schtasks_schedule_task(exec_path, SCHEDULE_HOURLY, cmd) ||\n+\t\tschtasks_schedule_task(exec_path, SCHEDULE_DAILY, cmd) ||\n+\t\tschtasks_schedule_task(exec_path, SCHEDULE_WEEKLY, cmd);\n+}\n+\n+static int schtasks_update_schedule(int run_maintenance, int fd, const char *cmd)\n+{\n+\tif (run_maintenance)\n+\t\treturn schtasks_schedule_tasks(cmd);\n+\telse\n+\t\treturn schtasks_remove_tasks(cmd);\n+}\n+\n #define BEGIN_LINE \"# BEGIN GIT MAINTENANCE SCHEDULE\"\n #define END_LINE \"# END GIT MAINTENANCE SCHEDULE\"\n \n@@ -1761,6 +1923,8 @@ static int crontab_update_schedule(int run_maintenance, int fd, const char *cmd)\n \n #if defined(__APPLE__)\n static const char platform_scheduler[] = \"launchctl\";\n+#elif defined(GIT_WINDOWS_NATIVE)\n+static const char platform_scheduler[] = \"schtasks\";\n #else\n static const char platform_scheduler[] = \"crontab\";\n #endif\n@@ -1789,6 +1953,8 @@ static int update_background_schedule(int enable)\n \n \tif (!strcmp(scheduler, \"launchctl\"))\n \t\tresult = launchctl_update_schedule(enable, lk.tempfile->fd, cmd);\n+\telse if (!strcmp(scheduler, \"schtasks\"))\n+\t\tresult = schtasks_update_schedule(enable, lk.tempfile->fd, cmd);\n \telse if (!strcmp(scheduler, \"crontab\"))\n \t\tresult = crontab_update_schedule(enable, lk.tempfile->fd, cmd);\n \telse\ndiff --git a/t/t7900-maintenance.sh b/t/t7900-maintenance.sh\nindex adf24dee72d..135505f6195 100755\n--- a/t/t7900-maintenance.sh\n+++ b/t/t7900-maintenance.sh\n@@ -454,6 +454,43 @@ test_expect_success 'start and stop macOS maintenance' '\n \ttest_line_count = 0 actual\n '\n \n+test_expect_success 'start and stop Windows maintenance' '\n+\twrite_script print-args <<-\\EOF &&\n+\techo $* >>args\n+\twhile test $# -gt 0\n+\tdo\n+\t\tcase \"$1\" in\n+\t\t/xml) shift; xmlfile=$1; break ;;\n+\t\t*) shift ;;\n+\t\tesac\n+\tdone\n+\ttest -z \"$xmlfile\" || cp \"$xmlfile\" \"$xmlfile.xml\"\n+\tEOF\n+\n+\trm -f args &&\n+\tGIT_TEST_MAINT_SCHEDULER=\"schtasks:./print-args\" git maintenance start &&\n+\n+\t# start registers the repo\n+\tgit config --get --global maintenance.repo \"$(pwd)\" &&\n+\n+\tfor frequency in hourly daily weekly\n+\tdo\n+\t\tgrep \"/create /tn Git Maintenance ($frequency) /f /xml\" args &&\n+\t\tfile=$(ls .git/schedule_${frequency}*.xml) &&\n+\t\ttest_xmllint \"$file\" || return 1\n+\tdone &&\n+\n+\trm -f args &&\n+\tGIT_TEST_MAINT_SCHEDULER=\"schtasks:./print-args\" git maintenance stop &&\n+\n+\t# stop does not unregister the repo\n+\tgit config --get --global maintenance.repo \"$(pwd)\" &&\n+\n+\tprintf \"/delete /tn Git Maintenance (%s) /f\\n\" \\\n+\t\thourly daily weekly >expect &&\n+\ttest_cmp expect args\n+'\n+\n test_expect_success 'register preserves existing strategy' '\n \tgit config maintenance.strategy none &&\n \tgit maintenance register &&\n-- \ngitgitgadget\n"},{"id":"413454","messageId":"3576c7aa54e08c11ac86675ca9bc11b80366d390.1609852109.git.gitgitgadget@gmail.com","threadId":"54562","inReplyTo":"pull.776.v7.git.1609852108.gitgitgadget@gmail.com","subject":"[PATCH v7 3/4] maintenance: use launchctl on macOS","fromName":"Derrick Stolee via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2021-01-05T13:08:27Z","receivedAt":"2021-01-05T13:09:31Z","isPatch":true,"sender":{"key":"stolee@gmail.com","avatar":"https://avatars.githubusercontent.com/u/570044?v=4"},"body":"From: Derrick Stolee \n\nThe existing mechanism for scheduling background maintenance is done\nthrough cron. The 'crontab -e' command allows updating the schedule\nwhile cron itself runs those commands. While this is technically\nsupported by macOS, it has some significant deficiencies:\n\n1. Every run of 'crontab -e' must request elevated privileges through\n the user interface. When running 'git maintenance start' from the\n Terminal app, it presents a dialog box saying \"Terminal.app would\n like to administer your computer. Administration can include\n modifying passwords, networking, and system settings.\" This is more\n alarming than what we are hoping to achieve. If this alert had some\n information about how \"git\" is trying to run \"crontab\" then we would\n have some reason to believe that this dialog might be fine. However,\n it also doesn't help that some scenarios just leave Git waiting for\n a response without presenting anything to the user. I experienced\n this when executing the command from a Bash terminal view inside\n Visual Studio Code.\n\n2. While cron initializes a user environment enough for \"git config\n --global --show-origin\" to show the correct config file information,\n it does not set up the environment enough for Git Credential Manager\n Core to load credentials during a 'prefetch' task. My prefetches\n against private repositories required re-authenticating through UI\n pop-ups in a way that should not be required.\n\nThe solution is to switch from cron to the Apple-recommended [1]\n'launchd' tool.\n\n[1] https://developer.apple.com/library/archive/documentation/MacOSX/Conceptual/BPSystemStartup/Chapters/ScheduledJobs.html\n\nThe basics of this tool is that we need to create XML-formatted\n\"plist\" files inside \"~/Library/LaunchAgents/\" and then use the\n'launchctl' tool to make launchd aware of them. The plist files\ninclude all of the scheduling information, along with the command-line\narguments split across an array of tags.\n\nFor example, here is my plist file for the weekly scheduled tasks:\n\n\n\n\nLabelorg.git-scm.git.weekly\nProgramArguments\n\n/usr/local/libexec/git-core/git\n--exec-path=/usr/local/libexec/git-core\nfor-each-repo\n--config=maintenance.repo\nmaintenance\nrun\n--schedule=weekly\n\nStartCalendarInterval\n\n\nDay0\nHour0\nMinute0\n\n\n\n\n\nThe schedules for the daily and hourly tasks are more complicated\nsince we need to use an array for the StartCalendarInterval with\nan entry for each of the six days other than the 0th day (to avoid\ncolliding with the weekly task), and each of the 23 hours other\nthan the 0th hour (to avoid colliding with the daily task).\n\nThe \"Label\" value is currently filled with \"org.git-scm.git.X\"\nwhere X is the frequency. We need a different plist file for each\nfrequency.\n\nThe launchctl command needs to be aligned with a user id in order\nto initialize the command environment. This must be done using\nthe 'launchctl bootstrap' subcommand. This subcommand is new as\nof macOS 10.11, which was released in September 2015. Before that\nrelease the 'launchctl load' subcommand was recommended. The best\nsource of information on this transition I have seen is available\nat [2]. The current design does not preclude a future version that\ndetects the available fatures of 'launchctl' to use the older\ncommands. However, it is best to rely on the newest version since\nApple might completely remove the deprecated version on short\nnotice.\n\n[2] https://babodee.wordpress.com/2016/04/09/launchctl-2-0-syntax/\n\nTo remove a schedule, we must run 'launchctl bootout' with a valid\nplist file. We also need to 'bootout' a task before the 'bootstrap'\nsubcommand will succeed, if such a task already exists.\n\nThe need for a user id requires us to run 'id -u' which works on\nPOSIX systems but not Windows. Further, the need for fully-qualitifed\npath names including $HOME behaves differently in the Git internals and\nthe external test suite. The $HOME variable starts with \"C:\\...\" instead\nof the \"/c/...\" that is provided by Git in these subcommands. The test\ntherefore has a prerequisite that we are not on Windows. The cross-\nplatform logic still allows us to test the macOS logic on a Linux\nmachine.\n\nWe can verify the commands that were run by 'git maintenance start'\nand 'git maintenance stop' by injecting a script that writes the\ncommand-line arguments into GIT_TEST_MAINT_SCHEDULER.\n\nAn earlier version of this patch accidentally had an opening\n\"\" tag when it should have had a closing \"\" tag. This\nwas caught during manual testing with actual 'launchctl' commands,\nbut we do not want to update developers' tasks when running tests.\nIt appears that macOS includes the \"xmllint\" tool which can verify\nthe XML format. This is useful for any system that might contain\nthe tool, so use it whenever it is available.\n\nWe strive to make these tests work on all platforms, but Windows caused\nsome headaches. In particular, the value of getuid() called by the C\ncode is not guaranteed to be the same as `$(id -u)` invoked by a test.\nThis is because `git.exe` is a native Windows program, whereas the\nutility programs run by the test script mostly utilize the MSYS2 runtime,\nwhich emulates a POSIX-like environment. Since the purpose of the test\nis to check that the input to the hook is well-formed, the actual user\nID is immaterial, thus we can work around the problem by making the the\ntest UID-agnostic. Another subtle issue is the $HOME environment\nvariable being a Windows-style path instead of a Unix-style path. We can\nbe more flexible here instead of expecting exact path matches.\n\nHelped-by: Ævar Arnfjörð Bjarmason \nCo-authored-by: Eric Sunshine \nSigned-off-by: Eric Sunshine \nSigned-off-by: Derrick Stolee \n---\n Documentation/git-maintenance.txt | 40 +++++++\n builtin/gc.c | 188 +++++++++++++++++++++++++++++-\n t/t7900-maintenance.sh | 59 ++++++++++\n 3 files changed, 286 insertions(+), 1 deletion(-)\n\ndiff --git a/Documentation/git-maintenance.txt b/Documentation/git-maintenance.txt\nindex 1aa11124186..5f8f63f0988 100644\n--- a/Documentation/git-maintenance.txt\n+++ b/Documentation/git-maintenance.txt\n@@ -273,6 +273,46 @@ schedule to ensure you are executing the correct binaries in your\n schedule.\n \n \n+BACKGROUND MAINTENANCE ON MACOS SYSTEMS\n+---------------------------------------\n+\n+While macOS technically supports `cron`, using `crontab -e` requires\n+elevated privileges and the executed process does not have a full user\n+context. Without a full user context, Git and its credential helpers\n+cannot access stored credentials, so some maintenance tasks are not\n+functional.\n+\n+Instead, `git maintenance start` interacts with the `launchctl` tool,\n+which is the recommended way to schedule timed jobs in macOS. Scheduling\n+maintenance through `git maintenance (start|stop)` requires some\n+`launchctl` features available only in macOS 10.11 or later.\n+\n+Your user-specific scheduled tasks are stored as XML-formatted `.plist`\n+files in `~/Library/LaunchAgents/`. You can see the currently-registered\n+tasks using the following command:\n+\n+-----------------------------------------------------------------------\n+$ ls ~/Library/LaunchAgents/org.git-scm.git*\n+org.git-scm.git.daily.plist\n+org.git-scm.git.hourly.plist\n+org.git-scm.git.weekly.plist\n+-----------------------------------------------------------------------\n+\n+One task is registered for each `--schedule=` option. To\n+inspect how the XML format describes each schedule, open one of these\n+`.plist` files in an editor and inspect the `` element following\n+the `StartCalendarInterval` element.\n+\n+`git maintenance start` will overwrite these files and register the\n+tasks again with `launchctl`, so any customizations should be done by\n+creating your own `.plist` files with distinct names. Similarly, the\n+`git maintenance stop` command will unregister the tasks with `launchctl`\n+and delete the `.plist` files.\n+\n+To create more advanced customizations to your background tasks, see\n+launchctl.plist(5) for more information.\n+\n+\n GIT\n ---\n Part of the linkgit:git[1] suite\ndiff --git a/builtin/gc.c b/builtin/gc.c\nindex 18ae7f7138a..782769f2438 100644\n--- a/builtin/gc.c\n+++ b/builtin/gc.c\n@@ -1491,6 +1491,186 @@ static int maintenance_unregister(void)\n \treturn run_command(&config_unset);\n }\n \n+static const char *get_frequency(enum schedule_priority schedule)\n+{\n+\tswitch (schedule) {\n+\tcase SCHEDULE_HOURLY:\n+\t\treturn \"hourly\";\n+\tcase SCHEDULE_DAILY:\n+\t\treturn \"daily\";\n+\tcase SCHEDULE_WEEKLY:\n+\t\treturn \"weekly\";\n+\tdefault:\n+\t\tBUG(\"invalid schedule %d\", schedule);\n+\t}\n+}\n+\n+static char *launchctl_service_name(const char *frequency)\n+{\n+\tstruct strbuf label = STRBUF_INIT;\n+\tstrbuf_addf(&label, \"org.git-scm.git.%s\", frequency);\n+\treturn strbuf_detach(&label, NULL);\n+}\n+\n+static char *launchctl_service_filename(const char *name)\n+{\n+\tchar *expanded;\n+\tstruct strbuf filename = STRBUF_INIT;\n+\tstrbuf_addf(&filename, \"~/Library/LaunchAgents/%s.plist\", name);\n+\n+\texpanded = expand_user_path(filename.buf, 1);\n+\tif (!expanded)\n+\t\tdie(_(\"failed to expand path '%s'\"), filename.buf);\n+\n+\tstrbuf_release(&filename);\n+\treturn expanded;\n+}\n+\n+static char *launchctl_get_uid(void)\n+{\n+\treturn xstrfmt(\"gui/%d\", getuid());\n+}\n+\n+static int launchctl_boot_plist(int enable, const char *filename, const char *cmd)\n+{\n+\tint result;\n+\tstruct child_process child = CHILD_PROCESS_INIT;\n+\tchar *uid = launchctl_get_uid();\n+\n+\tstrvec_split(&child.args, cmd);\n+\tif (enable)\n+\t\tstrvec_push(&child.args, \"bootstrap\");\n+\telse\n+\t\tstrvec_push(&child.args, \"bootout\");\n+\tstrvec_push(&child.args, uid);\n+\tstrvec_push(&child.args, filename);\n+\n+\tchild.no_stderr = 1;\n+\tchild.no_stdout = 1;\n+\n+\tif (start_command(&child))\n+\t\tdie(_(\"failed to start launchctl\"));\n+\n+\tresult = finish_command(&child);\n+\n+\tfree(uid);\n+\treturn result;\n+}\n+\n+static int launchctl_remove_plist(enum schedule_priority schedule, const char *cmd)\n+{\n+\tconst char *frequency = get_frequency(schedule);\n+\tchar *name = launchctl_service_name(frequency);\n+\tchar *filename = launchctl_service_filename(name);\n+\tint result = launchctl_boot_plist(0, filename, cmd);\n+\tunlink(filename);\n+\tfree(filename);\n+\tfree(name);\n+\treturn result;\n+}\n+\n+static int launchctl_remove_plists(const char *cmd)\n+{\n+\treturn launchctl_remove_plist(SCHEDULE_HOURLY, cmd) ||\n+\t\tlaunchctl_remove_plist(SCHEDULE_DAILY, cmd) ||\n+\t\tlaunchctl_remove_plist(SCHEDULE_WEEKLY, cmd);\n+}\n+\n+static int launchctl_schedule_plist(const char *exec_path, enum schedule_priority schedule, const char *cmd)\n+{\n+\tFILE *plist;\n+\tint i;\n+\tconst char *preamble, *repeat;\n+\tconst char *frequency = get_frequency(schedule);\n+\tchar *name = launchctl_service_name(frequency);\n+\tchar *filename = launchctl_service_filename(name);\n+\n+\tif (safe_create_leading_directories(filename))\n+\t\tdie(_(\"failed to create directories for '%s'\"), filename);\n+\tplist = xfopen(filename, \"w\");\n+\n+\tpreamble = \"\\n\"\n+\t\t \"\\n\"\n+\t\t \"\"\n+\t\t \"\\n\"\n+\t\t \"Label%s\\n\"\n+\t\t \"ProgramArguments\\n\"\n+\t\t \"\\n\"\n+\t\t \"%s/git\\n\"\n+\t\t \"--exec-path=%s\\n\"\n+\t\t \"for-each-repo\\n\"\n+\t\t \"--config=maintenance.repo\\n\"\n+\t\t \"maintenance\\n\"\n+\t\t \"run\\n\"\n+\t\t \"--schedule=%s\\n\"\n+\t\t \"\\n\"\n+\t\t \"StartCalendarInterval\\n\"\n+\t\t \"\\n\";\n+\tfprintf(plist, preamble, name, exec_path, exec_path, frequency);\n+\n+\tswitch (schedule) {\n+\tcase SCHEDULE_HOURLY:\n+\t\trepeat = \"\\n\"\n+\t\t\t \"Hour%d\\n\"\n+\t\t\t \"Minute0\\n\"\n+\t\t\t \"\\n\";\n+\t\tfor (i = 1; i <= 23; i++)\n+\t\t\tfprintf(plist, repeat, i);\n+\t\tbreak;\n+\n+\tcase SCHEDULE_DAILY:\n+\t\trepeat = \"\\n\"\n+\t\t\t \"Day%d\\n\"\n+\t\t\t \"Hour0\\n\"\n+\t\t\t \"Minute0\\n\"\n+\t\t\t \"\\n\";\n+\t\tfor (i = 1; i <= 6; i++)\n+\t\t\tfprintf(plist, repeat, i);\n+\t\tbreak;\n+\n+\tcase SCHEDULE_WEEKLY:\n+\t\tfprintf(plist,\n+\t\t\t\"\\n\"\n+\t\t\t\"Day0\\n\"\n+\t\t\t\"Hour0\\n\"\n+\t\t\t\"Minute0\\n\"\n+\t\t\t\"\\n\");\n+\t\tbreak;\n+\n+\tdefault:\n+\t\t/* unreachable */\n+\t\tbreak;\n+\t}\n+\tfprintf(plist, \"\\n\\n\\n\");\n+\tfclose(plist);\n+\n+\t/* bootout might fail if not already running, so ignore */\n+\tlaunchctl_boot_plist(0, filename, cmd);\n+\tif (launchctl_boot_plist(1, filename, cmd))\n+\t\tdie(_(\"failed to bootstrap service %s\"), filename);\n+\n+\tfree(filename);\n+\tfree(name);\n+\treturn 0;\n+}\n+\n+static int launchctl_add_plists(const char *cmd)\n+{\n+\tconst char *exec_path = git_exec_path();\n+\n+\treturn launchctl_schedule_plist(exec_path, SCHEDULE_HOURLY, cmd) ||\n+\t\tlaunchctl_schedule_plist(exec_path, SCHEDULE_DAILY, cmd) ||\n+\t\tlaunchctl_schedule_plist(exec_path, SCHEDULE_WEEKLY, cmd);\n+}\n+\n+static int launchctl_update_schedule(int run_maintenance, int fd, const char *cmd)\n+{\n+\tif (run_maintenance)\n+\t\treturn launchctl_add_plists(cmd);\n+\telse\n+\t\treturn launchctl_remove_plists(cmd);\n+}\n+\n #define BEGIN_LINE \"# BEGIN GIT MAINTENANCE SCHEDULE\"\n #define END_LINE \"# END GIT MAINTENANCE SCHEDULE\"\n \n@@ -1579,7 +1759,11 @@ static int crontab_update_schedule(int run_maintenance, int fd, const char *cmd)\n \treturn result;\n }\n \n+#if defined(__APPLE__)\n+static const char platform_scheduler[] = \"launchctl\";\n+#else\n static const char platform_scheduler[] = \"crontab\";\n+#endif\n \n static int update_background_schedule(int enable)\n {\n@@ -1603,7 +1787,9 @@ static int update_background_schedule(int enable)\n \tif (hold_lock_file_for_update(&lk, lock_path, LOCK_NO_DEREF) < 0)\n \t\treturn error(_(\"another process is scheduling background maintenance\"));\n \n-\tif (!strcmp(scheduler, \"crontab\"))\n+\tif (!strcmp(scheduler, \"launchctl\"))\n+\t\tresult = launchctl_update_schedule(enable, lk.tempfile->fd, cmd);\n+\telse if (!strcmp(scheduler, \"crontab\"))\n \t\tresult = crontab_update_schedule(enable, lk.tempfile->fd, cmd);\n \telse\n \t\tdie(\"unknown background scheduler: %s\", scheduler);\ndiff --git a/t/t7900-maintenance.sh b/t/t7900-maintenance.sh\nindex eeb939168da..adf24dee72d 100755\n--- a/t/t7900-maintenance.sh\n+++ b/t/t7900-maintenance.sh\n@@ -7,6 +7,19 @@ test_description='git maintenance builtin'\n GIT_TEST_COMMIT_GRAPH=0\n GIT_TEST_MULTI_PACK_INDEX=0\n \n+test_lazy_prereq XMLLINT '\n+\txmllint --version\n+'\n+\n+test_xmllint () {\n+\tif test_have_prereq XMLLINT\n+\tthen\n+\t\txmllint --noout \"$@\"\n+\telse\n+\t\ttrue\n+\tfi\n+}\n+\n test_expect_success 'help text' '\n \ttest_expect_code 129 git maintenance -h 2>err &&\n \ttest_i18ngrep \"usage: git maintenance \" err &&\n@@ -395,6 +408,52 @@ test_expect_success 'start preserves existing schedule' '\n \tgrep \"Important information!\" cron.txt\n '\n \n+test_expect_success 'start and stop macOS maintenance' '\n+\t# ensure $HOME can be compared against hook arguments on all platforms\n+\tpfx=$(cd \"$HOME\" && pwd) &&\n+\n+\twrite_script print-args <<-\\EOF &&\n+\techo $* | sed \"s:gui/[0-9][0-9]*:gui/[UID]:\" >>args\n+\tEOF\n+\n+\trm -f args &&\n+\tGIT_TEST_MAINT_SCHEDULER=launchctl:./print-args git maintenance start &&\n+\n+\t# start registers the repo\n+\tgit config --get --global maintenance.repo \"$(pwd)\" &&\n+\n+\tls \"$HOME/Library/LaunchAgents\" >actual &&\n+\tcat >expect <<-\\EOF &&\n+\torg.git-scm.git.daily.plist\n+\torg.git-scm.git.hourly.plist\n+\torg.git-scm.git.weekly.plist\n+\tEOF\n+\ttest_cmp expect actual &&\n+\n+\trm -f expect &&\n+\tfor frequency in hourly daily weekly\n+\tdo\n+\t\tPLIST=\"$pfx/Library/LaunchAgents/org.git-scm.git.$frequency.plist\" &&\n+\t\ttest_xmllint \"$PLIST\" &&\n+\t\tgrep schedule=$frequency \"$PLIST\" &&\n+\t\techo \"bootout gui/[UID] $PLIST\" >>expect &&\n+\t\techo \"bootstrap gui/[UID] $PLIST\" >>expect || return 1\n+\tdone &&\n+\ttest_cmp expect args &&\n+\n+\trm -f args &&\n+\tGIT_TEST_MAINT_SCHEDULER=launchctl:./print-args git maintenance stop &&\n+\n+\t# stop does not unregister the repo\n+\tgit config --get --global maintenance.repo \"$(pwd)\" &&\n+\n+\tprintf \"bootout gui/[UID] $pfx/Library/LaunchAgents/org.git-scm.git.%s.plist\\n\" \\\n+\t\thourly daily weekly >expect &&\n+\ttest_cmp expect args &&\n+\tls \"$HOME/Library/LaunchAgents\" >actual &&\n+\ttest_line_count = 0 actual\n+'\n+\n test_expect_success 'register preserves existing strategy' '\n \tgit config maintenance.strategy none &&\n \tgit maintenance register &&\n-- \ngitgitgadget\n\n"},{"id":"413952","messageId":"CAPig+cRBK778CTj1mbg03HOcCCZXn9v+FLDkhA0ho+wd=GTCNQ@mail.gmail.com","threadId":"54562","inReplyTo":"3576c7aa54e08c11ac86675ca9bc11b80366d390.1609852109.git.gitgitgadget@gmail.com","subject":"Re: [PATCH v7 3/4] maintenance: use launchctl on macOS","fromName":"Eric Sunshine","fromEmail":"sunshine@sunshineco.com","sentAt":"2021-01-10T06:34:11Z","receivedAt":"2021-01-10T06:35:14Z","isPatch":true,"sender":{"key":"sunshine@sunshineco.com","avatar":"https://avatars.githubusercontent.com/u/163641?v=4"},"body":"On Tue, Jan 5, 2021 at 8:08 AM Derrick Stolee via GitGitGadget\n wrote:\n> [...]\n> The need for a user id requires us to run 'id -u' which works on\n> POSIX systems but not Windows. Further, the need for fully-qualitifed\n> path names including $HOME behaves differently in the Git internals and\n> the external test suite. The $HOME variable starts with \"C:\\...\" instead\n> of the \"/c/...\" that is provided by Git in these subcommands. The test\n> therefore has a prerequisite that we are not on Windows. The cross-\n> platform logic still allows us to test the macOS logic on a Linux\n> machine.\n\nYou forgot to remove the above paragraph...\n\n> We strive to make these tests work on all platforms, but Windows caused\n> some headaches. In particular, the value of getuid() called by the C\n> code is not guaranteed to be the same as `$(id -u)` invoked by a test.\n> This is because `git.exe` is a native Windows program, whereas the\n> utility programs run by the test script mostly utilize the MSYS2 runtime,\n> which emulates a POSIX-like environment. Since the purpose of the test\n> is to check that the input to the hook is well-formed, the actual user\n> ID is immaterial, thus we can work around the problem by making the the\n> test UID-agnostic. Another subtle issue is the $HOME environment\n> variable being a Windows-style path instead of a Unix-style path. We can\n> be more flexible here instead of expecting exact path matches.\n\n...when you added this paragraph from my separate patch which you\nfolded into this patch for v7. The two paragraphs are at odds with one\nanother.\n"}]}