[PATCH v2 0/4] Maintenance IV: Platform-specific background maintenance
- From
Derrick Stolee via GitGitGadget <gitgitgadget@gmail.com>
- Date
- Nov 4, 2020, 20:06 UTC
- Message-ID
- <pull.776.v2.git.1604520368.gitgitgadget@gmail.com>
- In-Reply-To
- <pull.776.git.1604412196.gitgitgadget@gmail.com>
This is based on ds/maintenance-part-3.
After sitting with the background maintenance as it has been cooking, I wanted to come back around and implement the background maintenance for Windows. However, I noticed that there were some things bothering me with background maintenance on my macOS machine. These are detailed in PATCH 3, but the tl;dr is that 'cron' is not recommended by Apple and instead 'launchd' satisfies our needs.
This series implements the background scheduling so git maintenance (start|stop) works on those platforms. I've been operating with these schedules for a while now without the problems described in the patches.
There is a particularly annoying case about console windows popping up on Windows, but PATCH 4 describes a plan to get around that.
Updates in V2 =============
* This is a faster turnaround for a v2 than I would normally like, but Eric inspired extra documentation about how to customize background schedules. * New extensions to git-maintenance.txt include guidelines for inspecting what git maintenance start does and how to customize beyond that. This includes a new PATCH 2 that includes documentation for 'cron' on non-macOS non-Windows systems. * Several improvements, especially in the tests, are included. * While testing manually, I noticed that somehow I had incorrectly had an opening <dict> tag instead of a closing </dict> tag in the hourly format on macOS. I found that the xmllint tool can verify the XML format of a file, which catches the bug. This seems like a good approach since the test is macOS-only. Does anyone have concerns about adding this dependency?
Thanks, -Stolee
Derrick Stolee (4): maintenance: extract platform-specific scheduling maintenance: include 'cron' details in docs maintenance: use launchctl on macOS maintenance: use Windows scheduled tasks
Documentation/git-maintenance.txt | 119 +++++++++ builtin/gc.c | 428 ++++++++++++++++++++++++++++-- t/t7900-maintenance.sh | 83 +++++- t/test-lib.sh | 4 + 4 files changed, 614 insertions(+), 20 deletions(-)
base-commit: 0016b618182f642771dc589cf0090289f9fe1b4f Published-As: https://github.com/gitgitgadget/git/releases/tag/pr-776%2Fderrickstolee%2Fmaintenance%2FmacOS-v2 Fetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-776/derrickstolee/maintenance/macOS-v2 Pull-Request: https://github.com/gitgitgadget/git/pull/776
Range-diff vs v1:
1: d35f1aa162 = 1: d35f1aa162 maintenance: extract platform-specific scheduling
-: ---------- > 2: 709a173720 maintenance: include 'cron' details in docs
2: 832fdf1687 ! 3: 0fafd75d10 maintenance: use launchctl on macOS
@@ Commit message
and 'git maintenance stop' by injecting a script that writes the
command-line arguments into GIT_TEST_CRONTAB.
+ An earlier version of this patch accidentally had an opening
+ "<dict>" tag when it should have had a closing "</dict>" tag. This
+ was caught during manual testing with actual 'launchctl' commands,
+ but we do not want to update developers' tasks when running tests.
+ It appears that macOS includes the "xmllint" tool which can verify
+ the XML format, so call it from the macOS-specific tests to ensure
+ the .plist files are well-formatted.
+
+ Helped-by: Eric Sunshine <sunshine@sunshineco.com>
Signed-off-by: Derrick Stolee <dstolee@microsoft.com>
+ ## Documentation/git-maintenance.txt ##
+@@ Documentation/git-maintenance.txt: for advanced scheduling techniques. Please do use the full path and
+ executing the correct binaries in your schedule.
+
+
++BACKGROUND MAINTENANCE ON MACOS SYSTEMS
++---------------------------------------
++
++While macOS technically supports `cron`, using `crontab -e` requires
++elevated privileges and the executed process do not have a full user
++context. Without a full user context, Git and its credential helpers
++cannot access stored credentials, so some maintenance tasks are not
++functional.
++
++Instead, `git maintenance start` interacts with the `launchctl` tool,
++which is the recommended way to
++https://developer.apple.com/library/archive/documentation/MacOSX/Conceptual/BPSystemStartup/Chapters/ScheduledJobs.html[schedule timed jobs in macOS].
++
++Scheduling maintenance through `git maintenance (start|stop)` requires
++some `launchctl` features available only in macOS 10.11 or later.
++
++Your user-specific scheduled tasks are stored as XML-formatted `.plist`
++files in `~/Library/LaunchAgents/`. You can see the currently-registered
++tasks using the following command:
++
++-----------------------------------------------------------------------
++$ ls ~/Library/LaunchAgents/ | grep org.git-scm.git
++org.git-scm.git.daily.plist
++org.git-scm.git.hourly.plist
++org.git-scm.git.weekly.plist
++-----------------------------------------------------------------------
++
++One task is registered for each `--schedule=<frequency>` option. To
++inspect how the XML format describes each schedule, open one of these
++`.plist` files in an editor and inspect the `<array>` element following
++the `<key>StartCalendarInterval</key>` element.
++
++`git maintenance start` will overwrite these files and register the
++tasks again with `launchctl`, so any customizations should be done by
++creating your own `.plist` files with distinct names. Similarly, the
++`git maintenance stop` command will unregister the tasks with `launchctl`
++and delete the `.plist` files.
++
++To create more advanced customizations to your background tasks, see
++https://developer.apple.com/library/archive/documentation/MacOSX/Conceptual/BPSystemStartup/Chapters/CreatingLaunchdJobs.html#//apple_ref/doc/uid/TP40001762-104142[the `launchctl` documentation]
++for more information.
++
++
+ GIT
+ ---
+ Part of the linkgit:git[1] suite
+
## builtin/gc.c ##
@@ builtin/gc.c: static int maintenance_unregister(void)
return run_command(&config_unset);
@@ builtin/gc.c: static int maintenance_unregister(void)
+ repeat = "<dict>\n"
+ "<key>Hour</key><integer>%d</integer>\n"
+ "<key>Minute</key><integer>0</integer>\n"
-+ "<dict>\n";
++ "</dict>\n";
+ for (i = 1; i <= 23; i++)
+ fprintf(plist, repeat, i);
+ break;
@@ t/t7900-maintenance.sh: test_expect_success 'stop from existing schedule' '
+ for frequency in hourly daily weekly
+ do
+ PLIST="$HOME/Library/LaunchAgents/org.git-scm.git.$frequency.plist" &&
++ xmllint "$PLIST" >/dev/null &&
+ grep schedule=$frequency "$PLIST" &&
+ echo "bootout gui/$UID $PLIST" >>expect &&
+ echo "bootstrap gui/$UID $PLIST" >>expect || return 1
3: a9221cc4aa ! 4: 84eb44de31 maintenance: use Windows scheduled tasks
@@ Commit message
short term. In the long term, we can consider creating this GUI
shim application within core Git, perhaps in contrib/.
+ Helped-by: Eric Sunshine <sunshine@sunshineco.com>
Signed-off-by: Derrick Stolee <dstolee@microsoft.com>
+ ## Documentation/git-maintenance.txt ##
+@@ Documentation/git-maintenance.txt: https://developer.apple.com/library/archive/documentation/MacOSX/Conceptual/BPSy
+ for more information.
+
+
++BACKGROUND MAINTENANCE ON WINDOWS SYSTEMS
++-----------------------------------------
++
++Windows does not support `cron` and instead has its own system for
++scheduling background tasks. The `git maintenance start` command uses
++the `schtasks` command to submit tasks to this system. You can inspect
++all background tasks using the Task Scheduler application. The tasks
++added by Git have names of the form `Git Maintenance (<frequency>)`.
++The Task Scheduler GUI has ways to inspect these tasks, but you can also
++export the tasks to XML files and view the details there.
++
++Note that since Git is a console application, these background tasks
++create a console window visible to the current user. This can be changed
++manually by selecting the "Run whether user is logged in or not" option
++in Task Scheduler. This change requires a password input, which is why
++`git maintenance start` does not select it by default.
++
++If you want to customize the background tasks, please rename the tasks
++so future calls to `git maintenance (start|stop)` do not overwrite your
++custom tasks.
++
++
+ GIT
+ ---
+ Part of the linkgit:git[1] suite
+
## builtin/gc.c ##
@@ builtin/gc.c: static int platform_update_schedule(int run_maintenance, int fd)
else
@@ t/t7900-maintenance.sh: test_expect_success MACOS_MAINTENANCE 'start and stop ma
'
+test_expect_success MINGW 'start and stop Windows maintenance' '
-+ echo "echo \$@ >>args" >print-args &&
-+ chmod a+x print-args &&
++ write_script print-args <<-\EOF &&
++ echo $* >>args
++ EOF
+
+ rm -f args &&
+ GIT_TEST_CRONTAB="/bin/sh print-args" git maintenance start &&
-+ cat args &&
+
+ # start registers the repo
+ git config --get --global maintenance.repo "$(pwd)" &&
+
-+ rm expect &&
+ for frequency in hourly daily weekly
+ do
-+ echo "/create /tn Git Maintenance ($frequency) /f /xml .git/objects/schedule-$frequency.xml" >>expect \
-+ || return 1
-+ done &&
++ printf "/create /tn Git Maintenance (%s) /f /xml .git/objects/schedule-%s.xml\n" \
++ $frequency $frequency
++ done >expect &&
+ test_cmp expect args &&
+
+ rm -f args &&
-+ GIT_TEST_CRONTAB="/bin/sh print-args" git maintenance stop &&
++ GIT_TEST_CRONTAB="/bin/sh print-args" git maintenance stop &&
+
+ # stop does not unregister the repo
+ git config --get --global maintenance.repo "$(pwd)" &&
+
+ rm expect &&
-+ for frequency in hourly daily weekly
-+ do
-+ echo "/delete /tn Git Maintenance ($frequency) /f" >>expect \
-+ || return 1
-+ done &&
++ printf "/delete /tn Git Maintenance (%s) /f\n" \
++ hourly daily weekly >expect &&
+ test_cmp expect args
+'
+-- gitgitgadget