{"thread":{"id":"50772","subject":"[PATCH 0/4] gc docs: modernize and fix the documentation","startedAt":"2019-03-18T16:15:17Z","lastAt":"2019-07-31T10:12:20Z","messageCount":64,"participants":["Ævar Arnfjörð Bjarmason","Jonathan Nieder","Jeff King","Duy Nguyen","Johannes Sixt","Andreas Heiduk","Junio C Hamano","Todd Zullinger"],"isPatch":true,"patchVersion":1,"patchTotal":4},"messages":[{"id":"371820","messageId":"20190318161502.7979-1-avarab@gmail.com","threadId":"50772","inReplyTo":null,"subject":"[PATCH 0/4] gc docs: modernize and fix the documentation","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2019-03-18T16:14:58Z","receivedAt":"2019-03-18T16:15:17Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"I've been annoyed by the \"gc\" docs for a while. This fixes most of the\nissues I've noticed, and also removes the duplication between the \"gc\"\nvariables in git-config(1) and in git-gc(1), which was made possible\nby Duy's Documentation/config/* series.\n\nThis should make the \"gc\" docs more awesome, and due to removing the\nde-duplication results in a net deletion of lines. Yay.\n\nThe only thing I was on the fence about was removing the 'gc\n\"refs/remotes/*' config example, but I think the remaining docs\nexplain it well enough. It can be added back if someone insists...\n\nNow, I was originally going to have 5 patches in this series by\nmodernizing the \"NOTES\" section, but that didn't make it in.\n\nThis series is unrelated (and does not conflict with) my in-flight gc\ncontention series\n(https://public-inbox.org/git/20190315155959.12390-1-avarab@gmail.com/),\nbut the \"git-gc\" docs should be updated to discuss the\ncore.filesRefLockTimeout option and how it impacts contention, see 8/8\nin that series for context. I.e. \"you may have contention, but\ncore.filesRefLockTimeout can mitigate blah blah\".\n\nI was going to do that, but then thought that we should also mention\nthat on the server-side we mitigate most/all of the contention via the\nquarantine, see \"QUARANTINE ENVIRONMENT\" in\ngit-receive-pack(1). I.e. we:\n\n 1. Get the temp pack\n 2. OK it (fsck, hooks etc.)\n 3. Move *complete* previously temp packs over\n 4. Update the refs\n\nI.e. we are immune from the \"concurrently with another process\" race,\nbut of course something concurrently updating the \"server\" repo\nwithout a quarantine environment may be subject to that race.\n\nThe only problem is that the last couple of paragraphs may be\nwrong. That's just my understanding from a brief reading of\n722ff7f876c (\"receive-pack: quarantine objects until pre-receive\naccepts\", 2016-10-03) so I didn't want to include that in this\nseries. Peff (or others), any comments?\n\nÆvar Arnfjörð Bjarmason (4):\n  gc docs: modernize the advice for manually running \"gc\"\n  gc docs: include the \"gc.*\" section from \"config\" in \"gc\"\n  gc docs: de-duplicate \"OPTIONS\" and \"CONFIGURATION\"\n  gc docs: downplay the usefulness of --aggressive\n\n Documentation/config/gc.txt |  39 ++++++++++--\n Documentation/git-gc.txt    | 120 +++++++++---------------------------\n 2 files changed, 65 insertions(+), 94 deletions(-)\n\n-- \n2.21.0.360.g471c308f928\n\n"},{"id":"371821","messageId":"20190318161502.7979-2-avarab@gmail.com","threadId":"50772","inReplyTo":"20190318161502.7979-1-avarab@gmail.com","subject":"[PATCH 1/4] gc docs: modernize the advice for manually running \"gc\"","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2019-03-18T16:14:59Z","receivedAt":"2019-03-18T16:15:20Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"The docs have been recommending that users need to run this manually,\nbut that hasn't been needed in practice for a long time.\n\nLet's instead have this reflect reality and say that most users don't\nneed to run this manually at all.\n\nSigned-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>\n---\n Documentation/git-gc.txt | 16 ++++++++++------\n 1 file changed, 10 insertions(+), 6 deletions(-)\n\ndiff --git a/Documentation/git-gc.txt b/Documentation/git-gc.txt\nindex a7c1b0f60ed..cc82971022e 100644\n--- a/Documentation/git-gc.txt\n+++ b/Documentation/git-gc.txt\n@@ -20,13 +20,17 @@ created from prior invocations of 'git add', packing refs, pruning\n reflog, rerere metadata or stale working trees. May also update ancillary\n indexes such as the commit-graph.\n \n-Users are encouraged to run this task on a regular basis within\n-each repository to maintain good disk space utilization and good\n-operating performance.\n+Most users should not have to run this command manually. When common\n+porcelain operations that create objects are run, such as\n+linkgit:git-commit[1] and linkgit:git-fetch[1], `git gc --auto` will\n+be run automatically.\n \n-Some git commands may automatically run 'git gc'; see the `--auto` flag\n-below for details. If you know what you're doing and all you want is to\n-disable this behavior permanently without further considerations, just do:\n+You should only need to run `git gc` manually when adding objects to a\n+repository without regularly running such porcelain commands. Another\n+use-case is wanting to do a one-off repository optimization.\n+\n+If you know what you're doing and all you want is to disable automatic\n+runs, do:\n \n ----------------------\n $ git config --global gc.auto 0\n-- \n2.21.0.360.g471c308f928\n\n"},{"id":"371822","messageId":"20190318161502.7979-3-avarab@gmail.com","threadId":"50772","inReplyTo":"20190318161502.7979-1-avarab@gmail.com","subject":"[PATCH 2/4] gc docs: include the \"gc.*\" section from \"config\" in \"gc\"","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2019-03-18T16:15:00Z","receivedAt":"2019-03-18T16:15:23Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"Rather than duplicating the documentation for the various \"gc\" options\nlet's include the \"gc\" docs from git-config. They were mostly better\nalready, and now we don't have the same docs in two places with subtly\ndifferent wording.\n\nIn the cases where the git-gc(1) docs were saying something the \"gc\"\ndocs in git-config(1) didn't cover move the relevant section over to\nthe git-config(1) docs.\n\nSigned-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>\n---\n Documentation/config/gc.txt | 12 +++++++\n Documentation/git-gc.txt    | 65 +++----------------------------------\n 2 files changed, 16 insertions(+), 61 deletions(-)\n\ndiff --git a/Documentation/config/gc.txt b/Documentation/config/gc.txt\nindex c6fbb8a96f9..a834a801cd6 100644\n--- a/Documentation/config/gc.txt\n+++ b/Documentation/config/gc.txt\n@@ -2,11 +2,17 @@ gc.aggressiveDepth::\n \tThe depth parameter used in the delta compression\n \talgorithm used by 'git gc --aggressive'.  This defaults\n \tto 50.\n++\n+See the documentation for the `--depth` option in\n+linkgit:git-repack[1] for more details.\n \n gc.aggressiveWindow::\n \tThe window size parameter used in the delta compression\n \talgorithm used by 'git gc --aggressive'.  This defaults\n \tto 250.\n++\n+See the documentation for the `--window` option in\n+linkgit:git-repack[1] for more details.\n \n gc.auto::\n \tWhen there are approximately more than this many loose\n@@ -94,6 +100,12 @@ gc.<pattern>.reflogExpireUnreachable::\n \tWith \"<pattern>\" (e.g. \"refs/stash\")\n \tin the middle, the setting applies only to the refs that\n \tmatch the <pattern>.\n++\n+These types of entries are generally created as a result of using `git\n+commit --amend` or `git rebase` and are the commits prior to the amend\n+or rebase occurring. Since these changes are not part of the current\n+project history most users will want to expire them sooner, which is\n+why the default is more aggressive than `gc.reflogExpire`.\n \n gc.rerereResolved::\n \tRecords of conflicted merge you resolved earlier are\ndiff --git a/Documentation/git-gc.txt b/Documentation/git-gc.txt\nindex cc82971022e..9edf4e465b4 100644\n--- a/Documentation/git-gc.txt\n+++ b/Documentation/git-gc.txt\n@@ -29,8 +29,7 @@ You should only need to run `git gc` manually when adding objects to a\n repository without regularly running such porcelain commands. Another\n use-case is wanting to do a one-off repository optimization.\n \n-If you know what you're doing and all you want is to disable automatic\n-runs, do:\n+If you know what you're doing and want to disable automatic runs, do:\n \n ----------------------\n $ git config --global gc.auto 0\n@@ -103,66 +102,10 @@ be performed as well.\n CONFIGURATION\n -------------\n \n-The optional configuration variable `gc.reflogExpire` can be\n-set to indicate how long historical entries within each branch's\n-reflog should remain available in this repository.  The setting is\n-expressed as a length of time, for example '90 days' or '3 months'.\n-It defaults to '90 days'.\n-\n-The optional configuration variable `gc.reflogExpireUnreachable`\n-can be set to indicate how long historical reflog entries which\n-are not part of the current branch should remain available in\n-this repository.  These types of entries are generally created as\n-a result of using `git commit --amend` or `git rebase` and are the\n-commits prior to the amend or rebase occurring.  Since these changes\n-are not part of the current project most users will want to expire\n-them sooner.  This option defaults to '30 days'.\n-\n-The above two configuration variables can be given to a pattern.  For\n-example, this sets non-default expiry values only to remote-tracking\n-branches:\n-\n-------------\n-[gc \"refs/remotes/*\"]\n-\treflogExpire = never\n-\treflogExpireUnreachable = 3 days\n-------------\n-\n-The optional configuration variable `gc.rerereResolved` indicates\n-how long records of conflicted merge you resolved earlier are\n-kept.  This defaults to 60 days.\n-\n-The optional configuration variable `gc.rerereUnresolved` indicates\n-how long records of conflicted merge you have not resolved are\n-kept.  This defaults to 15 days.\n-\n-The optional configuration variable `gc.packRefs` determines if\n-'git gc' runs 'git pack-refs'. This can be set to \"notbare\" to enable\n-it within all non-bare repos or it can be set to a boolean value.\n-This defaults to true.\n-\n-The optional configuration variable `gc.writeCommitGraph` determines if\n-'git gc' should run 'git commit-graph write'. This can be set to a\n-boolean value. This defaults to false.\n-\n-The optional configuration variable `gc.aggressiveWindow` controls how\n-much time is spent optimizing the delta compression of the objects in\n-the repository when the --aggressive option is specified.  The larger\n-the value, the more time is spent optimizing the delta compression.  See\n-the documentation for the --window option in linkgit:git-repack[1] for\n-more details.  This defaults to 250.\n-\n-Similarly, the optional configuration variable `gc.aggressiveDepth`\n-controls --depth option in linkgit:git-repack[1]. This defaults to 50.\n-\n-The optional configuration variable `gc.pruneExpire` controls how old\n-the unreferenced loose objects have to be before they are pruned.  The\n-default is \"2 weeks ago\".\n-\n-Optional configuration variable `gc.worktreePruneExpire` controls how\n-old a stale working tree should be before `git worktree prune` deletes\n-it. Default is \"3 months ago\".\n+The below documentation is the same as what's found in\n+linkgit:git-config[1]:\n \n+include::config/gc.txt[]\n \n NOTES\n -----\n-- \n2.21.0.360.g471c308f928\n\n"},{"id":"371823","messageId":"20190318161502.7979-4-avarab@gmail.com","threadId":"50772","inReplyTo":"20190318161502.7979-1-avarab@gmail.com","subject":"[PATCH 3/4] gc docs: de-duplicate \"OPTIONS\" and \"CONFIGURATION\"","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2019-03-18T16:15:01Z","receivedAt":"2019-03-18T16:15:24Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"In an earlier commit I started including the \"gc.*\" documentation from\ngit-config(1) in the git-gc(1) documentation. That still left us in a\nstate where the \"--auto\" option and \"gc.auto\" were redundantly\ndiscussing the same thing.\n\nFix that by briefly discussing how the option itself works for\n\"--auto\", and for the rest referring to the configuration\ndocumentation.\n\nThis revealed existing blind spots in the configuration documentation,\nmove over the documentation and reword as appropriate.\n\nSigned-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>\n---\n Documentation/config/gc.txt | 27 +++++++++++++++++++++++----\n Documentation/git-gc.txt    | 25 ++++---------------------\n 2 files changed, 27 insertions(+), 25 deletions(-)\n\ndiff --git a/Documentation/config/gc.txt b/Documentation/config/gc.txt\nindex a834a801cd6..605e14bc80b 100644\n--- a/Documentation/config/gc.txt\n+++ b/Documentation/config/gc.txt\n@@ -19,13 +19,27 @@ gc.auto::\n \tobjects in the repository, `git gc --auto` will pack them.\n \tSome Porcelain commands use this command to perform a\n \tlight-weight garbage collection from time to time.  The\n-\tdefault value is 6700.  Setting this to 0 disables it.\n+\tdefault value is 6700.\n++\n+Setting this to 0 disables not only automatic packing based on the\n+number of loose objects, but any other heuristic `git gc --auto` will\n+otherwise use to determine if there's work to do, such as\n+`gc.autoPackLimit`.\n++\n+The repacking of loose objects will be performed with `git repack -d\n+-l`.\n \n gc.autoPackLimit::\n+\n \tWhen there are more than this many packs that are not\n \tmarked with `*.keep` file in the repository, `git gc\n \t--auto` consolidates them into one larger pack.  The\n-\tdefault\tvalue is 50.  Setting this to 0 disables it.\n+\tdefault value is 50.  Setting this (or `gc.auto`) to 0\n+\tdisables it. Packs will be consolidated using the `-A` option\n+\tof `git repack`.\n++\n+See the `gc.bigPackThreshold` configuration variable below. When in\n+use it'll effect how the auto pack limit works.\n \n gc.autoDetach::\n \tMake `git gc --auto` return immediately and run in background\n@@ -35,13 +49,18 @@ gc.bigPackThreshold::\n \tIf non-zero, all packs larger than this limit are kept when\n \t`git gc` is run. This is very similar to `--keep-base-pack`\n \texcept that all packs that meet the threshold are kept, not\n-\tjust the base pack. Defaults to zero. Common unit suffixes of\n-\t'k', 'm', or 'g' are supported.\n+\tjust the base pack. Defaults to zero or a memory heuristic.\n+\tCommon unit suffixes of 'k', 'm', or 'g' are supported.\n +\n Note that if the number of kept packs is more than gc.autoPackLimit,\n this configuration variable is ignored, all packs except the base pack\n will be repacked. After this the number of packs should go below\n gc.autoPackLimit and gc.bigPackThreshold should be respected again.\n++\n+If the amount of memory is estimated not enough for `git repack` to\n+run smoothly and `gc.bigPackThreshold` is not set, the largest pack\n+will also be excluded (which is the equivalent of running `git gc`\n+with `--keep-base-pack`).\n \n gc.writeCommitGraph::\n \tIf true, then gc will rewrite the commit-graph file when\ndiff --git a/Documentation/git-gc.txt b/Documentation/git-gc.txt\nindex 9edf4e465b4..154c7c5e652 100644\n--- a/Documentation/git-gc.txt\n+++ b/Documentation/git-gc.txt\n@@ -49,29 +49,12 @@ OPTIONS\n --auto::\n \tWith this option, 'git gc' checks whether any housekeeping is\n \trequired; if not, it exits without performing any work.\n-\tSome git commands run `git gc --auto` after performing\n-\toperations that could create many loose objects. Housekeeping\n-\tis required if there are too many loose objects or too many\n-\tpacks in the repository.\n +\n-If the number of loose objects exceeds the value of the `gc.auto`\n-configuration variable, then all loose objects are combined into a\n-single pack using `git repack -d -l`.  Setting the value of `gc.auto`\n-to 0 disables automatic packing of loose objects.\n+See the `gc.auto' option in the \"CONFIGURATION\" below for how this\n+heuristic works.\n +\n-If the number of packs exceeds the value of `gc.autoPackLimit`,\n-then existing packs (except those marked with a `.keep` file\n-or over `gc.bigPackThreshold` limit)\n-are consolidated into a single pack by using the `-A` option of\n-'git repack'.\n-If the amount of memory is estimated not enough for `git repack` to\n-run smoothly and `gc.bigPackThreshold` is not set, the largest\n-pack will also be excluded (this is the equivalent of running `git gc`\n-with `--keep-base-pack`).\n-Setting `gc.autoPackLimit` to 0 disables automatic consolidation of\n-packs.\n-+\n-If houskeeping is required due to many loose objects or packs, all\n+Once housekeeping is triggered by exceeding the limits of\n+configurations options such as `gc.auto` and `gc.autoPackLimit`, all\n other housekeeping tasks (e.g. rerere, working trees, reflog...) will\n be performed as well.\n \n-- \n2.21.0.360.g471c308f928\n\n"},{"id":"371824","messageId":"20190318161502.7979-5-avarab@gmail.com","threadId":"50772","inReplyTo":"20190318161502.7979-1-avarab@gmail.com","subject":"[PATCH 4/4] gc docs: downplay the usefulness of --aggressive","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2019-03-18T16:15:02Z","receivedAt":"2019-03-18T16:15:26Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"The existing \"gc --aggressive\" docs come just short of recommending to\nusers that they run it regularly. In reality it's a waste of CPU for\nmost users, and may even make things actively worse. I've personally\ntalked to many users who've taken these docs as an advice to use this\noption, and have.\n\nLet's change this documentation to better reflect reality, i.e. for\nmost users using --aggressive is a waste of time, and may even be\nactively making things worse.\n\nLet's also clarify the \"The effects [...] are persistent\" to clearly\nnote that that's true to the extent that subsequent gc's aren't going\nto re-roll existing packs generated with --aggressive into a new set\nof packs.\n\nSigned-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>\n---\n Documentation/git-gc.txt | 18 ++++++++++++++----\n 1 file changed, 14 insertions(+), 4 deletions(-)\n\ndiff --git a/Documentation/git-gc.txt b/Documentation/git-gc.txt\nindex 154c7c5e652..d0eaba98db5 100644\n--- a/Documentation/git-gc.txt\n+++ b/Documentation/git-gc.txt\n@@ -41,10 +41,20 @@ OPTIONS\n --aggressive::\n \tUsually 'git gc' runs very quickly while providing good disk\n \tspace utilization and performance.  This option will cause\n-\t'git gc' to more aggressively optimize the repository at the expense\n-\tof taking much more time.  The effects of this optimization are\n-\tpersistent, so this option only needs to be used occasionally; every\n-\tfew hundred changesets or so.\n+\t'git gc' to more aggressively optimize the repository to save storage space\n+\tat the expense of taking much more time.\n++\n+Using this option may optimize for disk space at the expense of\n+runtime performance. See the `--depth` and `--window` documentation in\n+linkgit:git-repack[1]. It is not recommended that this option be used\n+to improve performance for a given repository without running tailored\n+performance benchmarks on it. It may make things better, or worse. Not\n+using this at all is the right trade-off for most users and their\n+repositories.\n++\n+The effects of this option are persistent to the extent that\n+`gc.autoPackLimit` and friends don't cause a consolidation of existing\n+pack(s) generated with this option.\n \n --auto::\n \tWith this option, 'git gc' checks whether any housekeeping is\n-- \n2.21.0.360.g471c308f928\n\n"},{"id":"371839","messageId":"20190318202824.GA24222@gmail.com","threadId":"50772","inReplyTo":"20190318161502.7979-5-avarab@gmail.com","subject":"Re: [PATCH 4/4] gc docs: downplay the usefulness of --aggressive","fromName":"Jonathan Nieder","fromEmail":"jrnieder@gmail.com","sentAt":"2019-03-18T20:28:24Z","receivedAt":"2019-03-18T20:28:29Z","isPatch":true,"sender":{"key":"jrnieder@gmail.com","avatar":"https://avatars.githubusercontent.com/u/281595?v=4"},"body":"Hi,\n\nÆvar Arnfjörð Bjarmason wrote:\n\n> --- a/Documentation/git-gc.txt\n> +++ b/Documentation/git-gc.txt\n> @@ -41,10 +41,20 @@ OPTIONS\n>  --aggressive::\n>  \tUsually 'git gc' runs very quickly while providing good disk\n>  \tspace utilization and performance.  This option will cause\n> -\t'git gc' to more aggressively optimize the repository at the expense\n> -\tof taking much more time.  The effects of this optimization are\n> -\tpersistent, so this option only needs to be used occasionally; every\n> -\tfew hundred changesets or so.\n> +\t'git gc' to more aggressively optimize the repository to save storage space\n> +\tat the expense of taking much more time.\n\nThis part looks good.\n\n> ++\n> +Using this option may optimize for disk space at the expense of\n> +runtime performance. See the `--depth` and `--window` documentation in\n> +linkgit:git-repack[1]. It is not recommended that this option be used\n> +to improve performance for a given repository without running tailored\n> +performance benchmarks on it. It may make things better, or worse. Not\n> +using this at all is the right trade-off for most users and their\n> +repositories.\n\nThis part kind of feels like giving up.  Can we make --aggressive have\ngood runtime read performance so we don't have to hedge this way?\nE.g. is this patch papering over a poor choice of --depth setting in\n--aggressive?\n\n> ++\n> +The effects of this option are persistent to the extent that\n> +`gc.autoPackLimit` and friends don't cause a consolidation of existing\n> +pack(s) generated with this option.\n\nThanks and hope that helps,\nJonathan\n"},{"id":"371845","messageId":"20190318212227.GD29661@sigill.intra.peff.net","threadId":"50772","inReplyTo":"20190318202824.GA24222@gmail.com","subject":"Re: [PATCH 4/4] gc docs: downplay the usefulness of --aggressive","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2019-03-18T21:22:27Z","receivedAt":"2019-03-18T21:22:31Z","isPatch":true,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Mon, Mar 18, 2019 at 01:28:24PM -0700, Jonathan Nieder wrote:\n\n> > +Using this option may optimize for disk space at the expense of\n> > +runtime performance. See the `--depth` and `--window` documentation in\n> > +linkgit:git-repack[1]. It is not recommended that this option be used\n> > +to improve performance for a given repository without running tailored\n> > +performance benchmarks on it. It may make things better, or worse. Not\n> > +using this at all is the right trade-off for most users and their\n> > +repositories.\n> \n> This part kind of feels like giving up.  Can we make --aggressive have\n> good runtime read performance so we don't have to hedge this way?\n> E.g. is this patch papering over a poor choice of --depth setting in\n> --aggressive?\n\nI thought we already did that, in 07e7dbf0db (gc: default aggressive\ndepth to 50, 2016-08-11). As far as I know, \"--aggressive\" produces\npacks with similar runtime performance.\n\nIt is possible, if it finds more deltas due to the larger window, that\nwe'd spend more time accessing those deltas. But if the chains aren't\nlong, the base cache tends to perform well, and delta reconstruction is\nabout the same cost as zlib inflating. And we have a smaller disk cache\nfootprint.\n\n-Peff\n"},{"id":"371846","messageId":"20190318212719.GE29661@sigill.intra.peff.net","threadId":"50772","inReplyTo":"20190318161502.7979-2-avarab@gmail.com","subject":"Re: [PATCH 1/4] gc docs: modernize the advice for manually running \"gc\"","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2019-03-18T21:27:20Z","receivedAt":"2019-03-18T21:27:23Z","isPatch":true,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Mon, Mar 18, 2019 at 05:14:59PM +0100, Ævar Arnfjörð Bjarmason wrote:\n\n> The docs have been recommending that users need to run this manually,\n> but that hasn't been needed in practice for a long time.\n> \n> Let's instead have this reflect reality and say that most users don't\n> need to run this manually at all.\n\nYeah, I think this makes sense.\n\n> -Users are encouraged to run this task on a regular basis within\n> -each repository to maintain good disk space utilization and good\n> -operating performance.\n> +Most users should not have to run this command manually. When common\n> +porcelain operations that create objects are run, such as\n> +linkgit:git-commit[1] and linkgit:git-fetch[1], `git gc --auto` will\n> +be run automatically.\n\nThis is in the description, before \"--auto\" is introduced. I wonder if\nit is worth just describing it briefly, like:\n\n  When common porcelain operations that creates objects are run, they\n  will check whether the repository has grown substantially since the\n  last maintenance, and if so run `git gc` automatically.\n\nThat gives a first-time reader an idea of whether they need to care\nabout this command without having to dig into what \"--auto\" is.\n\n-Peff\n"},{"id":"371847","messageId":"20190318213139.GF29661@sigill.intra.peff.net","threadId":"50772","inReplyTo":"20190318161502.7979-3-avarab@gmail.com","subject":"Re: [PATCH 2/4] gc docs: include the \"gc.*\" section from \"config\" in \"gc\"","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2019-03-18T21:31:39Z","receivedAt":"2019-03-18T21:31:42Z","isPatch":true,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Mon, Mar 18, 2019 at 05:15:00PM +0100, Ævar Arnfjörð Bjarmason wrote:\n\n> Rather than duplicating the documentation for the various \"gc\" options\n> let's include the \"gc\" docs from git-config. They were mostly better\n> already, and now we don't have the same docs in two places with subtly\n> different wording.\n> \n> In the cases where the git-gc(1) docs were saying something the \"gc\"\n> docs in git-config(1) didn't cover move the relevant section over to\n> the git-config(1) docs.\n\nMakes sense.\n\nI think we lose the actual example for gc.*.reflogExpire:\n\n> -The above two configuration variables can be given to a pattern.  For\n> -example, this sets non-default expiry values only to remote-tracking\n> -branches:\n> -\n> -------------\n> -[gc \"refs/remotes/*\"]\n> -\treflogExpire = never\n> -\treflogExpireUnreachable = 3 days\n> -------------\n\nI don't actually think it's that big a loss. If we wanted to retain it,\nthough, it might make sense in the \"EXAMPLES\" section.\n\n-Peff\n"},{"id":"371850","messageId":"20190318214905.GG29661@sigill.intra.peff.net","threadId":"50772","inReplyTo":"20190318161502.7979-4-avarab@gmail.com","subject":"Re: [PATCH 3/4] gc docs: de-duplicate \"OPTIONS\" and \"CONFIGURATION\"","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2019-03-18T21:49:05Z","receivedAt":"2019-03-18T21:49:09Z","isPatch":true,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Mon, Mar 18, 2019 at 05:15:01PM +0100, Ævar Arnfjörð Bjarmason wrote:\n\n> In an earlier commit I started including the \"gc.*\" documentation from\n> git-config(1) in the git-gc(1) documentation. That still left us in a\n> state where the \"--auto\" option and \"gc.auto\" were redundantly\n> discussing the same thing.\n> \n> Fix that by briefly discussing how the option itself works for\n> \"--auto\", and for the rest referring to the configuration\n> documentation.\n> \n> This revealed existing blind spots in the configuration documentation,\n> move over the documentation and reword as appropriate.\n\nNice improvement. A few comments:\n\n> diff --git a/Documentation/config/gc.txt b/Documentation/config/gc.txt\n> index a834a801cd6..605e14bc80b 100644\n> --- a/Documentation/config/gc.txt\n> +++ b/Documentation/config/gc.txt\n> @@ -19,13 +19,27 @@ gc.auto::\n>  \tobjects in the repository, `git gc --auto` will pack them.\n>  \tSome Porcelain commands use this command to perform a\n>  \tlight-weight garbage collection from time to time.  The\n> -\tdefault value is 6700.  Setting this to 0 disables it.\n> +\tdefault value is 6700.\n> ++\n> +Setting this to 0 disables not only automatic packing based on the\n> +number of loose objects, but any other heuristic `git gc --auto` will\n> +otherwise use to determine if there's work to do, such as\n> +`gc.autoPackLimit`.\n> ++\n> +The repacking of loose objects will be performed with `git repack -d\n> +-l`.\n\nI know this last sentence came from the existing documentation, but I\nwonder if we should be more vague here. We'd pack with \"repack -dl\" when\nwe have just loose objects, and \"repack -Adl\" when we have too many\npacks. Or \"repack -adl\" if we're pruning now, and \"--unpack-unreachable\"\notherwise.\n\nI think the point of git-gc is that you don't have to care about that\nstuff. It works magically, and if you are implementing your own custom\ngc scheme, then you are probably better off reading the output of\nGIT_TRACE or looking at the source, rather than this documentation.\n\n>  gc.autoPackLimit::\n> +\n>  \tWhen there are more than this many packs that are not\n\nWhat's this newline for? I'm not completely opposed if that's the style\nwe want, but it seems odd that just this one has a blank between the\nvariable name and the text.\n\n>  \tmarked with `*.keep` file in the repository, `git gc\n>  \t--auto` consolidates them into one larger pack.  The\n> -\tdefault\tvalue is 50.  Setting this to 0 disables it.\n> +\tdefault value is 50.  Setting this (or `gc.auto`) to 0\n> +\tdisables it. Packs will be consolidated using the `-A` option\n> +\tof `git repack`.\n\nIf we do revise the \"-d -l\" bit for the loose limit, we'd probably want\nto adjust this to match.\n\n> @@ -35,13 +49,18 @@ gc.bigPackThreshold::\n>  \tIf non-zero, all packs larger than this limit are kept when\n>  \t`git gc` is run. This is very similar to `--keep-base-pack`\n>  \texcept that all packs that meet the threshold are kept, not\n> -\tjust the base pack. Defaults to zero. Common unit suffixes of\n> -\t'k', 'm', or 'g' are supported.\n> +\tjust the base pack. Defaults to zero or a memory heuristic.\n> +\tCommon unit suffixes of 'k', 'm', or 'g' are supported.\n\nI'm not sure how to read this \"or\". What's the difference between \"0\" or\nthe memory heuristic, and when is one used? Or is that what the \"if the\nnumber of kept packs is more than...\" below is trying to say?\n\nIf so, I wonder if it would be simpler to say \"defaults to a memory\nheuristic\", but with a note for \"but under these conditions it is not\nused\".\n\nOr am I totally misunderstanding how it actually works (which seems\nlikely to me)?\n\n> +If the amount of memory is estimated not enough for `git repack` to\n> +run smoothly and `gc.bigPackThreshold` is not set, the largest pack\n> +will also be excluded (which is the equivalent of running `git gc`\n> +with `--keep-base-pack`).\n\nI had trouble parsing this first line. Maybe:\n\n  If the amount of memory estimated for `git repack` to run smoothly is\n  not available and ...\n\nI guess a lot of this is just being copied from elsewhere, but it's\nprobably worth cleaning it up while we're here.\n\n> --- a/Documentation/git-gc.txt\n> +++ b/Documentation/git-gc.txt\n> [...]\n> +See the `gc.auto' option in the \"CONFIGURATION\" below for how this\n> +heuristic works.\n\ns/CONFIGURATION/& section/?\n\n> +Once housekeeping is triggered by exceeding the limits of\n> +configurations options such as `gc.auto` and `gc.autoPackLimit`, all\n\ns/configurations/configuration/\n\n-Peff\n"},{"id":"371851","messageId":"20190318215107.GH29661@sigill.intra.peff.net","threadId":"50772","inReplyTo":"20190318161502.7979-1-avarab@gmail.com","subject":"Re: [PATCH 0/4] gc docs: modernize and fix the documentation","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2019-03-18T21:51:07Z","receivedAt":"2019-03-18T21:51:11Z","isPatch":true,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Mon, Mar 18, 2019 at 05:14:58PM +0100, Ævar Arnfjörð Bjarmason wrote:\n\n> This series is unrelated (and does not conflict with) my in-flight gc\n> contention series\n> (https://public-inbox.org/git/20190315155959.12390-1-avarab@gmail.com/),\n> but the \"git-gc\" docs should be updated to discuss the\n> core.filesRefLockTimeout option and how it impacts contention, see 8/8\n> in that series for context. I.e. \"you may have contention, but\n> core.filesRefLockTimeout can mitigate blah blah\".\n> \n> I was going to do that, but then thought that we should also mention\n> that on the server-side we mitigate most/all of the contention via the\n> quarantine, see \"QUARANTINE ENVIRONMENT\" in\n> git-receive-pack(1). I.e. we:\n> \n>  1. Get the temp pack\n>  2. OK it (fsck, hooks etc.)\n>  3. Move *complete* previously temp packs over\n>  4. Update the refs\n> \n> I.e. we are immune from the \"concurrently with another process\" race,\n> but of course something concurrently updating the \"server\" repo\n> without a quarantine environment may be subject to that race.\n> \n> The only problem is that the last couple of paragraphs may be\n> wrong. That's just my understanding from a brief reading of\n> 722ff7f876c (\"receive-pack: quarantine objects until pre-receive\n> accepts\", 2016-10-03) so I didn't want to include that in this\n> series. Peff (or others), any comments?\n\nI don't think the quarantine stuff should impact contention at all. It's\nonly quarantining the objects, which are the least contentious part of\nGit (because object content is idempotent, so we don't do any locking\nthere, and with two racing processes, one will just \"win\").\n\n-Peff\n"},{"id":"371854","messageId":"87k1gvespm.fsf@evledraar.gmail.com","threadId":"50772","inReplyTo":"20190318212227.GD29661@sigill.intra.peff.net","subject":"Re: [PATCH 4/4] gc docs: downplay the usefulness of --aggressive","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2019-03-18T22:13:57Z","receivedAt":"2019-03-18T22:16:11Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"\nOn Mon, Mar 18 2019, Jeff King wrote:\n\n> On Mon, Mar 18, 2019 at 01:28:24PM -0700, Jonathan Nieder wrote:\n>\n>> > +Using this option may optimize for disk space at the expense of\n>> > +runtime performance. See the `--depth` and `--window` documentation in\n>> > +linkgit:git-repack[1]. It is not recommended that this option be used\n>> > +to improve performance for a given repository without running tailored\n>> > +performance benchmarks on it. It may make things better, or worse. Not\n>> > +using this at all is the right trade-off for most users and their\n>> > +repositories.\n>>\n>> This part kind of feels like giving up.  Can we make --aggressive have\n>> good runtime read performance so we don't have to hedge this way?\n>> E.g. is this patch papering over a poor choice of --depth setting in\n>> --aggressive?\n>\n> I thought we already did that, in 07e7dbf0db (gc: default aggressive\n> depth to 50, 2016-08-11). As far as I know, \"--aggressive\" produces\n> packs with similar runtime performance.\n\n\nWhat happened here is that I'd entirely forgotten about your 07e7dbf0db\nand in skimming while writing this throught we were still picking larger\ndepth values, which we aren't.\n\nI'll fix that, and see that gc.aggressiveDepth also needs to be changed\nto note that the depth it's now using as \"aggressive\" is just the\ndefault of 50 you'd get without --aggressive.\n\n> It is possible, if it finds more deltas due to the larger window, that\n> we'd spend more time accessing those deltas. But if the chains aren't\n> long, the base cache tends to perform well, and delta reconstruction is\n> about the same cost as zlib inflating. And we have a smaller disk cache\n> footprint.\n\nI haven't tested that but suspect it won't matter. We do spend a *lot*\nmore time though, so that still needs to be noted...\n\nOn the topic of other things I may have screwed up, is this:\n\n    +The effects of this option are persistent to the extent that\n    +`gc.autoPackLimit` and friends don't cause a consolidation of existing\n    +pack(s) generated with this option.\n\nActually wrong since we don't pass -f usually, and thus a one-off\n--aggressive would live forever for the objects involved in that run no\nmatter if we later consolidate?\n\nFrom the docs it seems so, but I'd like to confirm...\n"},{"id":"371855","messageId":"87imwfesht.fsf@evledraar.gmail.com","threadId":"50772","inReplyTo":"20190318212719.GE29661@sigill.intra.peff.net","subject":"Re: [PATCH 1/4] gc docs: modernize the advice for manually running \"gc\"","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2019-03-18T22:18:38Z","receivedAt":"2019-03-18T22:18:43Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"\nOn Mon, Mar 18 2019, Jeff King wrote:\n\n> On Mon, Mar 18, 2019 at 05:14:59PM +0100, Ævar Arnfjörð Bjarmason wrote:\n>\n>> The docs have been recommending that users need to run this manually,\n>> but that hasn't been needed in practice for a long time.\n>>\n>> Let's instead have this reflect reality and say that most users don't\n>> need to run this manually at all.\n>\n> Yeah, I think this makes sense.\n>\n>> -Users are encouraged to run this task on a regular basis within\n>> -each repository to maintain good disk space utilization and good\n>> -operating performance.\n>> +Most users should not have to run this command manually. When common\n>> +porcelain operations that create objects are run, such as\n>> +linkgit:git-commit[1] and linkgit:git-fetch[1], `git gc --auto` will\n>> +be run automatically.\n>\n> This is in the description, before \"--auto\" is introduced. I wonder if\n> it is worth just describing it briefly, like:\n>\n>   When common porcelain operations that creates objects are run, they\n>   will check whether the repository has grown substantially since the\n>   last maintenance, and if so run `git gc` automatically.\n>\n> That gives a first-time reader an idea of whether they need to care\n> about this command without having to dig into what \"--auto\" is.\n\nYeah I think that's better. Also more briefly describing gc.auto=0\nwithout an example (suggesting people run that, which for most is a bad\nidea). I.e. just adding to that \"This behavior can be disabled, see\n`gc.auto` below.\"\n"},{"id":"371857","messageId":"87ftrjer8s.fsf@evledraar.gmail.com","threadId":"50772","inReplyTo":"20190318215107.GH29661@sigill.intra.peff.net","subject":"Re: [PATCH 0/4] gc docs: modernize and fix the documentation","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2019-03-18T22:45:39Z","receivedAt":"2019-03-18T22:45:44Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"\nOn Mon, Mar 18 2019, Jeff King wrote:\n\n> On Mon, Mar 18, 2019 at 05:14:58PM +0100, Ævar Arnfjörð Bjarmason wrote:\n>\n>> This series is unrelated (and does not conflict with) my in-flight gc\n>> contention series\n>> (https://public-inbox.org/git/20190315155959.12390-1-avarab@gmail.com/),\n>> but the \"git-gc\" docs should be updated to discuss the\n>> core.filesRefLockTimeout option and how it impacts contention, see 8/8\n>> in that series for context. I.e. \"you may have contention, but\n>> core.filesRefLockTimeout can mitigate blah blah\".\n>>\n>> I was going to do that, but then thought that we should also mention\n>> that on the server-side we mitigate most/all of the contention via the\n>> quarantine, see \"QUARANTINE ENVIRONMENT\" in\n>> git-receive-pack(1). I.e. we:\n>>\n>>  1. Get the temp pack\n>>  2. OK it (fsck, hooks etc.)\n>>  3. Move *complete* previously temp packs over\n>>  4. Update the refs\n>>\n>> I.e. we are immune from the \"concurrently with another process\" race,\n>> but of course something concurrently updating the \"server\" repo\n>> without a quarantine environment may be subject to that race.\n>>\n>> The only problem is that the last couple of paragraphs may be\n>> wrong. That's just my understanding from a brief reading of\n>> 722ff7f876c (\"receive-pack: quarantine objects until pre-receive\n>> accepts\", 2016-10-03) so I didn't want to include that in this\n>> series. Peff (or others), any comments?\n>\n> I don't think the quarantine stuff should impact contention at all. It's\n> only quarantining the objects, which are the least contentious part of\n> Git (because object content is idempotent, so we don't do any locking\n> there, and with two racing processes, one will just \"win\").\n\nWithout the quarantine, isn't there the race that the NOTES section\ntalks about (unless I've misread it).\n\nI.e. we have some loose object \"ABCD\" not referrred to by anything for\nthe last 2 weeks, as we're gc-ing a ref update comes in that makes it\nreferenced again. We then delete \"ABCD\" (not used!) at the same time the\nref update happens, and get corruption.\n\nWhereas the quarantine might work around since the client will have sent\nABCD with no reference pointing to it to the server in the temp pack,\nwhich we then rename in-place and then update the ref, so we don't care\nif \"ABCD\" goes away.\n\nUnless that interacts racily with the receive.unpackLimit, but then I\nhave no idea that section is trying to say...\n\nAlso, surely the part where \"NOTES\" says something to the effect of \"you\nare subject to races unless gc.auto=0\" is wrong. To the extent that\nthere's races it won't matter that you invoke \"git gc\" or \"git gc\n--auto\", it's the concurrency that matters. So if there's still races we\nshould be saying the repo needs to be locked for writes for the duration\nof the \"gc\".\n"},{"id":"371858","messageId":"87ef73er3l.fsf@evledraar.gmail.com","threadId":"50772","inReplyTo":"20190318214905.GG29661@sigill.intra.peff.net","subject":"Re: [PATCH 3/4] gc docs: de-duplicate \"OPTIONS\" and \"CONFIGURATION\"","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2019-03-18T22:48:46Z","receivedAt":"2019-03-18T22:48:52Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"\nOn Mon, Mar 18 2019, Jeff King wrote:\n\n> On Mon, Mar 18, 2019 at 05:15:01PM +0100, Ævar Arnfjörð Bjarmason wrote:\n>\n>> In an earlier commit I started including the \"gc.*\" documentation from\n>> git-config(1) in the git-gc(1) documentation. That still left us in a\n>> state where the \"--auto\" option and \"gc.auto\" were redundantly\n>> discussing the same thing.\n>>\n>> Fix that by briefly discussing how the option itself works for\n>> \"--auto\", and for the rest referring to the configuration\n>> documentation.\n>>\n>> This revealed existing blind spots in the configuration documentation,\n>> move over the documentation and reword as appropriate.\n>\n> Nice improvement. A few comments:\n>\n>> diff --git a/Documentation/config/gc.txt b/Documentation/config/gc.txt\n>> index a834a801cd6..605e14bc80b 100644\n>> --- a/Documentation/config/gc.txt\n>> +++ b/Documentation/config/gc.txt\n>> @@ -19,13 +19,27 @@ gc.auto::\n>>  \tobjects in the repository, `git gc --auto` will pack them.\n>>  \tSome Porcelain commands use this command to perform a\n>>  \tlight-weight garbage collection from time to time.  The\n>> -\tdefault value is 6700.  Setting this to 0 disables it.\n>> +\tdefault value is 6700.\n>> ++\n>> +Setting this to 0 disables not only automatic packing based on the\n>> +number of loose objects, but any other heuristic `git gc --auto` will\n>> +otherwise use to determine if there's work to do, such as\n>> +`gc.autoPackLimit`.\n>> ++\n>> +The repacking of loose objects will be performed with `git repack -d\n>> +-l`.\n>\n> I know this last sentence came from the existing documentation, but I\n> wonder if we should be more vague here. We'd pack with \"repack -dl\" when\n> we have just loose objects, and \"repack -Adl\" when we have too many\n> packs. Or \"repack -adl\" if we're pruning now, and \"--unpack-unreachable\"\n> otherwise.\n>\n> I think the point of git-gc is that you don't have to care about that\n> stuff. It works magically, and if you are implementing your own custom\n> gc scheme, then you are probably better off reading the output of\n> GIT_TRACE or looking at the source, rather than this documentation.\n\nYeah I can just drop it while I'm at it. Was just losslessly trying to\nport the existing docs.\n\n>>  gc.autoPackLimit::\n>> +\n>>  \tWhen there are more than this many packs that are not\n>\n> What's this newline for? I'm not completely opposed if that's the style\n> we want, but it seems odd that just this one has a blank between the\n> variable name and the text.\n\nMistake, will fix.\n\n>>  \tmarked with `*.keep` file in the repository, `git gc\n>>  \t--auto` consolidates them into one larger pack.  The\n>> -\tdefault\tvalue is 50.  Setting this to 0 disables it.\n>> +\tdefault value is 50.  Setting this (or `gc.auto`) to 0\n>> +\tdisables it. Packs will be consolidated using the `-A` option\n>> +\tof `git repack`.\n>\n> If we do revise the \"-d -l\" bit for the loose limit, we'd probably want\n> to adjust this to match.\n\nOr not mention it either?\n\n>> @@ -35,13 +49,18 @@ gc.bigPackThreshold::\n>>  \tIf non-zero, all packs larger than this limit are kept when\n>>  \t`git gc` is run. This is very similar to `--keep-base-pack`\n>>  \texcept that all packs that meet the threshold are kept, not\n>> -\tjust the base pack. Defaults to zero. Common unit suffixes of\n>> -\t'k', 'm', or 'g' are supported.\n>> +\tjust the base pack. Defaults to zero or a memory heuristic.\n>> +\tCommon unit suffixes of 'k', 'm', or 'g' are supported.\n>\n> I'm not sure how to read this \"or\". What's the difference between \"0\" or\n> the memory heuristic, and when is one used? Or is that what the \"if the\n> number of kept packs is more than...\" below is trying to say?\n\nThat by default we don't use gc.bigPackThreshold, unless we find that\nyou're under memory pressure. I.e. \"it's off by default, unless your\nsystem has too little memory\".\n\n> If so, I wonder if it would be simpler to say \"defaults to a memory\n> heuristic\", but with a note for \"but under these conditions it is not\n> used\".\n>\n> Or am I totally misunderstanding how it actually works (which seems\n> likely to me)?\n>\n>> +If the amount of memory is estimated not enough for `git repack` to\n>> +run smoothly and `gc.bigPackThreshold` is not set, the largest pack\n>> +will also be excluded (which is the equivalent of running `git gc`\n>> +with `--keep-base-pack`).\n>\n> I had trouble parsing this first line. Maybe:\n>\n>   If the amount of memory estimated for `git repack` to run smoothly is\n>   not available and ...\n>\n> I guess a lot of this is just being copied from elsewhere, but it's\n> probably worth cleaning it up while we're here.\n\nWill try to make it suck less.\n\n>> --- a/Documentation/git-gc.txt\n>> +++ b/Documentation/git-gc.txt\n>> [...]\n>> +See the `gc.auto' option in the \"CONFIGURATION\" below for how this\n>> +heuristic works.\n>\n> s/CONFIGURATION/& section/?\n>\n>> +Once housekeeping is triggered by exceeding the limits of\n>> +configurations options such as `gc.auto` and `gc.autoPackLimit`, all\n>\n> s/configurations/configuration/\n\n*Nod*. Thanks.\n"},{"id":"371859","messageId":"20190318234231.GJ29661@sigill.intra.peff.net","threadId":"50772","inReplyTo":"87ef73er3l.fsf@evledraar.gmail.com","subject":"Re: [PATCH 3/4] gc docs: de-duplicate \"OPTIONS\" and \"CONFIGURATION\"","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2019-03-18T23:42:32Z","receivedAt":"2019-03-18T23:42:35Z","isPatch":true,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Mon, Mar 18, 2019 at 11:48:46PM +0100, Ævar Arnfjörð Bjarmason wrote:\n\n> > I know this last sentence came from the existing documentation, but I\n> > wonder if we should be more vague here. We'd pack with \"repack -dl\" when\n> > we have just loose objects, and \"repack -Adl\" when we have too many\n> > packs. Or \"repack -adl\" if we're pruning now, and \"--unpack-unreachable\"\n> > otherwise.\n> >\n> > I think the point of git-gc is that you don't have to care about that\n> > stuff. It works magically, and if you are implementing your own custom\n> > gc scheme, then you are probably better off reading the output of\n> > GIT_TRACE or looking at the source, rather than this documentation.\n> \n> Yeah I can just drop it while I'm at it. Was just losslessly trying to\n> port the existing docs.\n\nYeah, I'm sympathetic to that (if you did drop it, you might have gotten\nthe opposite review). ;) I think it would be OK to just mention it in\nthe commit message, but I'd also be OK dropping it in a preliminary\npatch.\n\n> >>  \tmarked with `*.keep` file in the repository, `git gc\n> >>  \t--auto` consolidates them into one larger pack.  The\n> >> -\tdefault\tvalue is 50.  Setting this to 0 disables it.\n> >> +\tdefault value is 50.  Setting this (or `gc.auto`) to 0\n> >> +\tdisables it. Packs will be consolidated using the `-A` option\n> >> +\tof `git repack`.\n> >\n> > If we do revise the \"-d -l\" bit for the loose limit, we'd probably want\n> > to adjust this to match.\n> \n> Or not mention it either?\n\nYes. :)\n\n> > I'm not sure how to read this \"or\". What's the difference between \"0\" or\n> > the memory heuristic, and when is one used? Or is that what the \"if the\n> > number of kept packs is more than...\" below is trying to say?\n> \n> That by default we don't use gc.bigPackThreshold, unless we find that\n> you're under memory pressure. I.e. \"it's off by default, unless your\n> system has too little memory\".\n\nOK, I see. It might make sense to write that out more explicitly.\n\n-Peff\n"},{"id":"371860","messageId":"20190318235356.GK29661@sigill.intra.peff.net","threadId":"50772","inReplyTo":"87k1gvespm.fsf@evledraar.gmail.com","subject":"Re: [PATCH 4/4] gc docs: downplay the usefulness of --aggressive","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2019-03-18T23:53:56Z","receivedAt":"2019-03-18T23:54:00Z","isPatch":true,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Mon, Mar 18, 2019 at 11:13:57PM +0100, Ævar Arnfjörð Bjarmason wrote:\n\n> What happened here is that I'd entirely forgotten about your 07e7dbf0db\n> and in skimming while writing this throught we were still picking larger\n> depth values, which we aren't.\n> \n> I'll fix that, and see that gc.aggressiveDepth also needs to be changed\n> to note that the depth it's now using as \"aggressive\" is just the\n> default of 50 you'd get without --aggressive.\n\nYeah. I think I tweaked the documentation in that commit, but I agree\nit's probably worth calling out the subtlety that it's the same as the\ndefault.\n\n> > It is possible, if it finds more deltas due to the larger window, that\n> > we'd spend more time accessing those deltas. But if the chains aren't\n> > long, the base cache tends to perform well, and delta reconstruction is\n> > about the same cost as zlib inflating. And we have a smaller disk cache\n> > footprint.\n> \n> I haven't tested that but suspect it won't matter. We do spend a *lot*\n> more time though, so that still needs to be noted...\n\nYeah, agreed on both counts.\n\n> On the topic of other things I may have screwed up, is this:\n> \n>     +The effects of this option are persistent to the extent that\n>     +`gc.autoPackLimit` and friends don't cause a consolidation of existing\n>     +pack(s) generated with this option.\n> \n> Actually wrong since we don't pass -f usually, and thus a one-off\n> --aggressive would live forever for the objects involved in that run no\n> matter if we later consolidate?\n> \n> From the docs it seems so, but I'd like to confirm...\n\nIn general, yeah, I'd expect an --aggressive repack's effects to live on\nthrough subsequent gc's. It's not _entirely_ true, because objects from\nthat big repack may end up duplicate in another pack (e.g., due to\nthin-fixing, or just a client which sends objects we didn't need due to\npush's abbreviated negotiation). And then either:\n\n  - we may select the copy of the object from the other pack, where it's\n    a base object, and then end up looking for a new delta for it\n\n  - after the pack-mru patches from ~2016, we don't have a strict\n    ordering of the packs, which means we can see cycles in the delta\n    graph. So even if object A isn't duplicated, it may be a delta on B,\n    which deltas on C, and then the copy of C we pick is from another\n    pack where it's a delta on A. We have to break the cycle, which\n    could happen on any one of A, B, or C.\n\nI haven't done careful measurements, but I'd be surprised if those cases\nmake a significant dent, even over many gc's. What I think probably does\nmake a dent is that new objects come into the repo with whatever crappy\npacking the client did as part of the push, and you'd ideally like to\nthrow away all of their deltas and just find new good ones.\n\nI think it might help for pack-objects to have a mode that isn't quite\n\"keep the big pack\", but rather \"keep the deltas from the big pack, but\nnot other ones, but otherwise create a new big pack\". But this has\ndiverged pretty far from the point of your series. :)\n\n-Peff\n"},{"id":"371862","messageId":"20190319001829.GL29661@sigill.intra.peff.net","threadId":"50772","inReplyTo":"87ftrjer8s.fsf@evledraar.gmail.com","subject":"Re: [PATCH 0/4] gc docs: modernize and fix the documentation","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2019-03-19T00:18:29Z","receivedAt":"2019-03-19T00:18:34Z","isPatch":true,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Mon, Mar 18, 2019 at 11:45:39PM +0100, Ævar Arnfjörð Bjarmason wrote:\n\n> > I don't think the quarantine stuff should impact contention at all. It's\n> > only quarantining the objects, which are the least contentious part of\n> > Git (because object content is idempotent, so we don't do any locking\n> > there, and with two racing processes, one will just \"win\").\n> \n> Without the quarantine, isn't there the race that the NOTES section\n> talks about (unless I've misread it).\n\nAh, OK, I wasn't quite sure which documentation you were talking about.\nI see the discussion now in the \"NOTES\" section of git-gc(1).\n\n> I.e. we have some loose object \"ABCD\" not referrred to by anything for\n> the last 2 weeks, as we're gc-ing a ref update comes in that makes it\n> referenced again. We then delete \"ABCD\" (not used!) at the same time the\n> ref update happens, and get corruption.\n> \n> Whereas the quarantine might work around since the client will have sent\n> ABCD with no reference pointing to it to the server in the temp pack,\n> which we then rename in-place and then update the ref, so we don't care\n> if \"ABCD\" goes away.\n\ntl;dr I don't think quarantine impacts this, but if you really want gory\ndetails, read on.\n\nThis is a problem with or without the quarantine. It's fundamentally a\nrace because we do not atomically say \"is anybody using X? If not, we\ncan delete it\" and some other process saying \"I'd like to use X\".\n\nPushes are actually better off than most operations, because we only\nadvertise what's reachable, and the client is expected to send\neverything else. So with just a normal update-ref call, we could race\nlike this:\n\n  1. ABCD is ancient.\n\n  2. Process 1 (update-ref) wants to reference ABCD. It sees that we\n     have it.\n\n  3. Process 2 (gc/prune) sees that nobody references it. It deletes\n     ABCD.\n\n  4. Process 1 writes out the reference.\n\nThat doesn't happen with a push, because the server never would have\ntold the client that it has ABCD in the first place (so process 1 here\nis the client). That is true with or without quarantine.\n\nBut pushes aren't foolproof either. You said \"loose object ABCD not\nreferred t oby anything for the last 2 weeks\". But that's not exactly\nhow it works. It's \"object with an mtime of more than 2 weeks which is\nnot currently referenced\". So imagine a sequence like:\n\n  1. ABCD is ancient.\n\n  2. refs/heads/foo points to ABCD.\n\n  3. Server receive-pack advertises foo pointing to ABCD.\n\n  4. Simultaneous process on the server deletes refs/heads/foo (or\n     perhaps somebody force-pushes over it).\n\n  5. Client prepares and sends pack without ABCD.\n\n  6. Server receive-pack checks that yes, we still have ABCD (i.e., the\n     usual connectivity check).\n\n  7. Server gc drops ABCD, which is now unreachable (reflogs can help\n     here, if you've enabled them; but we do delete reflogs when the\n     branch is deleted).\n\n  8. Server receive-pack writes corrupt repo mentioning ABCD.\n\nThat's a lot more steps, though they might not be as implausible as you\nthink (e.g., consider moving \"refs/heads/foo\" to \"refs/heads/bar\" in a\nsingle push; that's actually a delete and an update, which is all you\nneed to race with a simultaneous gc).\n\nI have no idea how often this happens in practice. My subjective\nrecollection is that most of the race corruptions I've seen were from\nlocal operations on the server. E.g., we compute a tentative merge for\nsomebody's pull request which shares objects with an older tentative\nmerge. They click the \"merge\" button and we reference that commit, which\nis recent, but unbeknownst to us, while we were creating our new\ntentative merge, a \"gc\" was deleting the old one.\n\nWe're sometimes saved by the \"transitive freshness\" rules in d3038d22f9\n(prune: keep objects reachable from recent objects, 2014-10-15).  But\nthey're far from perfect:\n\n - some operations (like the push rename example) aren't writing new\n   objects, so the ref write _is_ the moment that gc would find out\n   something is reachable\n\n - the \"is it reachable?\" and \"no, then delete it\" steps aren't atomic.\n   Unless you want a whole-repo stop-the-world lock, somebody can\n   reference the object in between. And since it may take many seconds\n   to compute reachability, stop-the-world is not great.\n\nI think there are probably ways to make it better. Perhaps some kind of\nlockless delete-but-be-able-to-rollback scheme (but keep in mind this\nhas to be implemented no top of POSIX filesystem semantics). Or even\njust a \"compute reachability, mark for deletion, and then hold a\nstop-the-world lock briefly to double-check that our reachability is\nstill up to date\".\n\nAt least those seem plausible to me. I've never worked out the details,\nand our solution was to just stop deleting objects during routine\nmaintenance (using \"repack -adk\"). We do still occasionally prune\nmanually (e.g., when somebody writes to support to remove a confidential\nmistake).\n\nAnyway, that was more than you probably wanted to know. The short of it\nis that I don't think quarantines help (they may even make things worse\nby slightly increasing the length of the race window, though in practice\nI doubt it).\n\n> Unless that interacts racily with the receive.unpackLimit, but then I\n> have no idea that section is trying to say...\n\nNo, I don't think unpackLimit really affects it much either way.\n\n> Also, surely the part where \"NOTES\" says something to the effect of \"you\n> are subject to races unless gc.auto=0\" is wrong. To the extent that\n> there's races it won't matter that you invoke \"git gc\" or \"git gc\n> --auto\", it's the concurrency that matters. So if there's still races we\n> should be saying the repo needs to be locked for writes for the duration\n> of the \"gc\".\n\nCorrect. It's the very act of pruning that is problematic. I think the\npoint is that if you are manually running \"git gc\", you'd presumably do\nit at a time when the repository is not otherwise active.\n\n-Peff\n"},{"id":"371869","messageId":"CACsJy8DW=MPe=oU-hf2ngpMFXYexmhQyLy-W1BeKo-0gpvAh8Q@mail.gmail.com","threadId":"50772","inReplyTo":"20190318161502.7979-3-avarab@gmail.com","subject":"Re: [PATCH 2/4] gc docs: include the \"gc.*\" section from \"config\" in \"gc\"","fromName":"Duy Nguyen","fromEmail":"pclouds@gmail.com","sentAt":"2019-03-19T02:08:23Z","receivedAt":"2019-03-19T02:08:52Z","isPatch":true,"sender":{"key":"pclouds@gmail.com","avatar":"https://avatars.githubusercontent.com/u/720?v=4"},"body":"On Mon, Mar 18, 2019 at 11:15 PM Ævar Arnfjörð Bjarmason\n<avarab@gmail.com> wrote:\n>\n> Rather than duplicating the documentation for the various \"gc\" options\n> let's include the \"gc\" docs from git-config. They were mostly better\n> already, and now we don't have the same docs in two places with subtly\n> different wording.\n\nNice. I have a wip series to do this for more man pages but it's\nstill, well, wip.\n\nThere may be one thing we need to sort out to include config/*.txt,\nsometimes we mention see linkgit:git-gc[1]. When including\nconfig/gc.txt in git-gc.1, this self reference seems silly. But I\nthink we can just leave it for now.\n-- \nDuy\n"},{"id":"371884","messageId":"5cf21eee-a46f-2657-7bf3-e4963cf1c56b@kdbg.org","threadId":"50772","inReplyTo":"20190318161502.7979-5-avarab@gmail.com","subject":"Re: [PATCH 4/4] gc docs: downplay the usefulness of --aggressive","fromName":"Johannes Sixt","fromEmail":"j6t@kdbg.org","sentAt":"2019-03-19T06:54:25Z","receivedAt":"2019-03-19T06:54:31Z","isPatch":true,"sender":{"key":"j6t@kdbg.org","avatar":"https://avatars.githubusercontent.com/u/14810926?v=4"},"body":"Am 18.03.19 um 17:15 schrieb Ævar Arnfjörð Bjarmason:\n> +Using this option may optimize for disk space at the expense of\n> +runtime performance. See the `--depth` and `--window` documentation in\n> +linkgit:git-repack[1]. It is not recommended that this option be used\n> +to improve performance for a given repository without running tailored\n> +performance benchmarks on it. It may make things better, or worse. Not\n> +using this at all is the right trade-off for most users and their\n> +repositories.\n> ++\n> +The effects of this option are persistent to the extent that\n> +`gc.autoPackLimit` and friends don't cause a consolidation of existing\n> +pack(s) generated with this option.\n\nThe first paragraph talks about potential downsides. And I think that\nthe second paragraph attempts to tell me how I can back out if I'm hit\nby those downsides. But I have not the slightest idea how to read this\nsentence and know what I have to do.\n\n-- Hannes\n"},{"id":"371894","messageId":"87bm27dxhj.fsf@evledraar.gmail.com","threadId":"50772","inReplyTo":"5cf21eee-a46f-2657-7bf3-e4963cf1c56b@kdbg.org","subject":"Re: [PATCH 4/4] gc docs: downplay the usefulness of --aggressive","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2019-03-19T09:28:24Z","receivedAt":"2019-03-19T09:28:29Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"\nOn Tue, Mar 19 2019, Johannes Sixt wrote:\n\n> Am 18.03.19 um 17:15 schrieb Ævar Arnfjörð Bjarmason:\n>> +Using this option may optimize for disk space at the expense of\n>> +runtime performance. See the `--depth` and `--window` documentation in\n>> +linkgit:git-repack[1]. It is not recommended that this option be used\n>> +to improve performance for a given repository without running tailored\n>> +performance benchmarks on it. It may make things better, or worse. Not\n>> +using this at all is the right trade-off for most users and their\n>> +repositories.\n>> ++\n>> +The effects of this option are persistent to the extent that\n>> +`gc.autoPackLimit` and friends don't cause a consolidation of existing\n>> +pack(s) generated with this option.\n>\n> The first paragraph talks about potential downsides. And I think that\n> the second paragraph attempts to tell me how I can back out if I'm hit\n> by those downsides. But I have not the slightest idea how to read this\n> sentence and know what I have to do.\n\nThat's an existing issue, but I'm fine with improving the docs even\nmore, will add that :)\n\nYou need to repack non-aggressively with the -f option. Right now\nthere's nothing that exposes that from \"gc\", except setting the\n\"aggressive\" settings to the same as the default window/depth.\n"},{"id":"372162","messageId":"20190321205054.17109-1-avarab@gmail.com","threadId":"50772","inReplyTo":"20190318161502.7979-1-avarab@gmail.com","subject":"[PATCH v2 00/10] gc docs: modernize and fix the documentation","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2019-03-21T20:50:44Z","receivedAt":"2019-03-21T20:51:09Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"For v1 see: https://public-inbox.org/git/20190318161502.7979-1-avarab@gmail.com/\n\nThis addresses all the feedback it got, which includes splitting out\nvarious \"while we're at it\" fixes, and then I found/remembered some\nmore things I needed to fix.\n\nIt would still be great to have Peff submit some version of his\nhttps://public-inbox.org/git/20190319001829.GL29661@sigill.intra.peff.net/\nreply to the NOTES section sometime, but I had to stop somewhere.\n\nI also documented the fast-import caveat discussed in\nhttps://public-inbox.org/git/87o964cnn0.fsf@evledraar.gmail.com/ while\nI was at it, as promised.\n\nÆvar Arnfjörð Bjarmason (10):\n  gc docs: modernize the advice for manually running \"gc\"\n  gc docs: stop noting \"repack\" flags\n  gc docs: clean grammar for \"gc.bigPackThreshold\"\n  gc docs: include the \"gc.*\" section from \"config\" in \"gc\"\n  gc docs: re-flow the \"gc.*\" section in \"config\"\n  gc docs: note how --aggressive impacts --window & --depth\n  gc docs: downplay the usefulness of --aggressive\n  gc docs: note \"gc --aggressive\" in \"fast-import\"\n  gc docs: clarify that \"gc\" doesn't throw away referenced objects\n  gc docs: remove incorrect reference to gc.auto=0\n\n Documentation/config/gc.txt       |  34 ++++++-\n Documentation/git-fast-import.txt |   7 ++\n Documentation/git-gc.txt          | 142 ++++++++++--------------------\n 3 files changed, 84 insertions(+), 99 deletions(-)\n\nRange-diff:\n 1:  d48b9c7221 !  1:  89719142c7 gc docs: modernize the advice for manually running \"gc\"\n    @@ -3,10 +3,17 @@\n         gc docs: modernize the advice for manually running \"gc\"\n     \n         The docs have been recommending that users need to run this manually,\n    -    but that hasn't been needed in practice for a long time.\n    +    but that hasn't been needed in practice for a long time except in\n    +    exceptional circumstances.\n     \n         Let's instead have this reflect reality and say that most users don't\n    -    need to run this manually at all.\n    +    need to run this manually at all, while briefly describing the sorts\n    +    sort of cases where \"gc\" does need to be run manually.\n    +\n    +    Since we're recommending that users run this most of the and usually\n    +    don't need to tweak it, let's tone down the very prominent example of\n    +    the gc.auto=0 command. It's sufficient to point to the gc.auto\n    +    documentation below.\n     \n         Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>\n     \n    @@ -20,20 +27,24 @@\n     -Users are encouraged to run this task on a regular basis within\n     -each repository to maintain good disk space utilization and good\n     -operating performance.\n    -+Most users should not have to run this command manually. When common\n    -+porcelain operations that create objects are run, such as\n    -+linkgit:git-commit[1] and linkgit:git-fetch[1], `git gc --auto` will\n    -+be run automatically.\n    - \n    +-\n     -Some git commands may automatically run 'git gc'; see the `--auto` flag\n     -below for details. If you know what you're doing and all you want is to\n     -disable this behavior permanently without further considerations, just do:\n    -+You should only need to run `git gc` manually when adding objects to a\n    -+repository without regularly running such porcelain commands. Another\n    -+use-case is wanting to do a one-off repository optimization.\n    +-\n    +-----------------------\n    +-$ git config --global gc.auto 0\n    +-----------------------\n    ++When common porcelain operations that creates objects are run, they\n    ++will check whether the repository has grown substantially since the\n    ++last maintenance, and if so run `git gc` automatically. See `gc.auto`\n    ++below for how to disable this behavior.\n     +\n    -+If you know what you're doing and all you want is to disable automatic\n    -+runs, do:\n    ++Running `git gc` manually should only be needed when adding objects to\n    ++a repository without regularly running such porcelain commands, to do\n    ++a one-off repository optimization, or e.g. to clean up a suboptimal\n    ++mass-import. See the \"PACKFILE OPTIMIZATION\" section in\n    ++linkgit:git-fast-import[1] for more details on the import case.\n      \n    - ----------------------\n    - $ git config --global gc.auto 0\n    + OPTIONS\n    + -------\n -:  ---------- >  2:  d90a5b1b4c gc docs: stop noting \"repack\" flags\n -:  ---------- >  3:  fedd9bb886 gc docs: clean grammar for \"gc.bigPackThreshold\"\n 2:  e670d514ce !  4:  6fad05a67c gc docs: include the \"gc.*\" section from \"config\" in \"gc\"\n    @@ -34,16 +34,52 @@\n      \n      gc.auto::\n      \tWhen there are approximately more than this many loose\n    + \tobjects in the repository, `git gc --auto` will pack them.\n    + \tSome Porcelain commands use this command to perform a\n    + \tlight-weight garbage collection from time to time.  The\n    +-\tdefault value is 6700.  Setting this to 0 disables it.\n    ++\tdefault value is 6700.\n    +++\n    ++Setting this to 0 disables not only automatic packing based on the\n    ++number of loose objects, but any other heuristic `git gc --auto` will\n    ++otherwise use to determine if there's work to do, such as\n    ++`gc.autoPackLimit`.\n    + \n    + gc.autoPackLimit::\n    + \tWhen there are more than this many packs that are not\n    + \tmarked with `*.keep` file in the repository, `git gc\n    + \t--auto` consolidates them into one larger pack.  The\n    + \tdefault\tvalue is 50.  Setting this to 0 disables it.\n    ++\tSetting `gc.auto` to 0 will also disable this.\n    +++\n    ++See the `gc.bigPackThreshold` configuration variable below. When in\n    ++use, it'll affect how the auto pack limit works.\n    + \n    + gc.autoDetach::\n    + \tMake `git gc --auto` return immediately and run in background\n    +@@\n    + this configuration variable is ignored, all packs except the base pack\n    + will be repacked. After this the number of packs should go below\n    + gc.autoPackLimit and gc.bigPackThreshold should be respected again.\n    +++\n    ++If the amount of memory estimated for `git repack` to run smoothly is\n    ++not available and `gc.bigPackThreshold` is not set, the largest\n    ++pack will also be excluded (this is the equivalent of running `git gc`\n    ++with `--keep-base-pack`).\n    + \n    + gc.writeCommitGraph::\n    + \tIf true, then gc will rewrite the commit-graph file when\n     @@\n      \tWith \"<pattern>\" (e.g. \"refs/stash\")\n      \tin the middle, the setting applies only to the refs that\n      \tmatch the <pattern>.\n     ++\n    -+These types of entries are generally created as a result of using `git\n    -+commit --amend` or `git rebase` and are the commits prior to the amend\n    -+or rebase occurring. Since these changes are not part of the current\n    -+project history most users will want to expire them sooner, which is\n    -+why the default is more aggressive than `gc.reflogExpire`.\n    ++These types of entries are generally created as\n    ++a result of using `git commit --amend` or `git rebase` and are the\n    ++commits prior to the amend or rebase occurring.  Since these changes\n    ++are not part of the current project most users will want to expire\n    ++them sooner, which is why the default is more aggressive than\n    ++`gc.reflogExpire`.\n      \n      gc.rerereResolved::\n      \tRecords of conflicted merge you resolved earlier are\n    @@ -52,15 +88,38 @@\n      --- a/Documentation/git-gc.txt\n      +++ b/Documentation/git-gc.txt\n     @@\n    - repository without regularly running such porcelain commands. Another\n    - use-case is wanting to do a one-off repository optimization.\n    - \n    --If you know what you're doing and all you want is to disable automatic\n    --runs, do:\n    -+If you know what you're doing and want to disable automatic runs, do:\n    + --auto::\n    + \tWith this option, 'git gc' checks whether any housekeeping is\n    + \trequired; if not, it exits without performing any work.\n    +-\tSome git commands run `git gc --auto` after performing\n    +-\toperations that could create many loose objects. Housekeeping\n    +-\tis required if there are too many loose objects or too many\n    +-\tpacks in the repository.\n    + +\n    +-If the number of loose objects exceeds the value of the `gc.auto`\n    +-configuration variable, then all loose objects are combined into a\n    +-single pack.  Setting the value of `gc.auto`\n    +-to 0 disables automatic packing of loose objects.\n    ++See the `gc.auto' option in the \"CONFIGURATION\" section below for how\n    ++this heuristic works.\n    + +\n    +-If the number of packs exceeds the value of `gc.autoPackLimit`,\n    +-then existing packs (except those marked with a `.keep` file\n    +-or over `gc.bigPackThreshold` limit)\n    +-are consolidated into a single pack.\n    +-If the amount of memory estimated for `git repack` to run smoothly is\n    +-not available and `gc.bigPackThreshold` is not set, the largest\n    +-pack will also be excluded (this is the equivalent of running `git gc`\n    +-with `--keep-base-pack`).\n    +-Setting `gc.autoPackLimit` to 0 disables automatic consolidation of\n    +-packs.\n    +-+\n    +-If houskeeping is required due to many loose objects or packs, all\n    ++Once housekeeping is triggered by exceeding the limits of\n    ++configuration options such as `gc.auto` and `gc.autoPackLimit`, all\n    + other housekeeping tasks (e.g. rerere, working trees, reflog...) will\n    + be performed as well.\n      \n    - ----------------------\n    - $ git config --global gc.auto 0\n     @@\n      CONFIGURATION\n      -------------\n 3:  d6f1e001a4 !  5:  994e22a0d6 gc docs: de-duplicate \"OPTIONS\" and \"CONFIGURATION\"\n    @@ -1,18 +1,10 @@\n     Author: Ævar Arnfjörð Bjarmason <avarab@gmail.com>\n     \n    -    gc docs: de-duplicate \"OPTIONS\" and \"CONFIGURATION\"\n    +    gc docs: re-flow the \"gc.*\" section in \"config\"\n     \n    -    In an earlier commit I started including the \"gc.*\" documentation from\n    -    git-config(1) in the git-gc(1) documentation. That still left us in a\n    -    state where the \"--auto\" option and \"gc.auto\" were redundantly\n    -    discussing the same thing.\n    -\n    -    Fix that by briefly discussing how the option itself works for\n    -    \"--auto\", and for the rest referring to the configuration\n    -    documentation.\n    -\n    -    This revealed existing blind spots in the configuration documentation,\n    -    move over the documentation and reword as appropriate.\n    +    Re-flow the \"gc.*\" section in \"config\". A previous commit moved this\n    +    over from the \"gc\" docs, but tried to keep as many of the lines\n    +    identical to benefit from diff's move detection.\n     \n         Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>\n     \n    @@ -20,91 +12,33 @@\n      --- a/Documentation/config/gc.txt\n      +++ b/Documentation/config/gc.txt\n     @@\n    - \tobjects in the repository, `git gc --auto` will pack them.\n    - \tSome Porcelain commands use this command to perform a\n    - \tlight-weight garbage collection from time to time.  The\n    --\tdefault value is 6700.  Setting this to 0 disables it.\n    -+\tdefault value is 6700.\n    -++\n    -+Setting this to 0 disables not only automatic packing based on the\n    -+number of loose objects, but any other heuristic `git gc --auto` will\n    -+otherwise use to determine if there's work to do, such as\n    -+`gc.autoPackLimit`.\n    -++\n    -+The repacking of loose objects will be performed with `git repack -d\n    -+-l`.\n    - \n    - gc.autoPackLimit::\n    -+\n    - \tWhen there are more than this many packs that are not\n    - \tmarked with `*.keep` file in the repository, `git gc\n    - \t--auto` consolidates them into one larger pack.  The\n    --\tdefault\tvalue is 50.  Setting this to 0 disables it.\n    -+\tdefault value is 50.  Setting this (or `gc.auto`) to 0\n    -+\tdisables it. Packs will be consolidated using the `-A` option\n    -+\tof `git repack`.\n    -++\n    -+See the `gc.bigPackThreshold` configuration variable below. When in\n    -+use it'll effect how the auto pack limit works.\n    - \n    - gc.autoDetach::\n    - \tMake `git gc --auto` return immediately and run in background\n    -@@\n    - \tIf non-zero, all packs larger than this limit are kept when\n    - \t`git gc` is run. This is very similar to `--keep-base-pack`\n    - \texcept that all packs that meet the threshold are kept, not\n    --\tjust the base pack. Defaults to zero. Common unit suffixes of\n    --\t'k', 'm', or 'g' are supported.\n    -+\tjust the base pack. Defaults to zero or a memory heuristic.\n    -+\tCommon unit suffixes of 'k', 'm', or 'g' are supported.\n    - +\n    - Note that if the number of kept packs is more than gc.autoPackLimit,\n    - this configuration variable is ignored, all packs except the base pack\n    - will be repacked. After this the number of packs should go below\n      gc.autoPackLimit and gc.bigPackThreshold should be respected again.\n    -++\n    -+If the amount of memory is estimated not enough for `git repack` to\n    -+run smoothly and `gc.bigPackThreshold` is not set, the largest pack\n    -+will also be excluded (which is the equivalent of running `git gc`\n    -+with `--keep-base-pack`).\n    + +\n    + If the amount of memory estimated for `git repack` to run smoothly is\n    +-not available and `gc.bigPackThreshold` is not set, the largest\n    +-pack will also be excluded (this is the equivalent of running `git gc`\n    +-with `--keep-base-pack`).\n    ++not available and `gc.bigPackThreshold` is not set, the largest pack\n    ++will also be excluded (this is the equivalent of running `git gc` with\n    ++`--keep-base-pack`).\n      \n      gc.writeCommitGraph::\n      \tIf true, then gc will rewrite the commit-graph file when\n    -\n    - diff --git a/Documentation/git-gc.txt b/Documentation/git-gc.txt\n    - --- a/Documentation/git-gc.txt\n    - +++ b/Documentation/git-gc.txt\n     @@\n    - --auto::\n    - \tWith this option, 'git gc' checks whether any housekeeping is\n    - \trequired; if not, it exits without performing any work.\n    --\tSome git commands run `git gc --auto` after performing\n    --\toperations that could create many loose objects. Housekeeping\n    --\tis required if there are too many loose objects or too many\n    --\tpacks in the repository.\n    + \tin the middle, the setting applies only to the refs that\n    + \tmatch the <pattern>.\n      +\n    --If the number of loose objects exceeds the value of the `gc.auto`\n    --configuration variable, then all loose objects are combined into a\n    --single pack using `git repack -d -l`.  Setting the value of `gc.auto`\n    --to 0 disables automatic packing of loose objects.\n    -+See the `gc.auto' option in the \"CONFIGURATION\" below for how this\n    -+heuristic works.\n    - +\n    --If the number of packs exceeds the value of `gc.autoPackLimit`,\n    --then existing packs (except those marked with a `.keep` file\n    --or over `gc.bigPackThreshold` limit)\n    --are consolidated into a single pack by using the `-A` option of\n    --'git repack'.\n    --If the amount of memory is estimated not enough for `git repack` to\n    --run smoothly and `gc.bigPackThreshold` is not set, the largest\n    --pack will also be excluded (this is the equivalent of running `git gc`\n    --with `--keep-base-pack`).\n    --Setting `gc.autoPackLimit` to 0 disables automatic consolidation of\n    --packs.\n    --+\n    --If houskeeping is required due to many loose objects or packs, all\n    -+Once housekeeping is triggered by exceeding the limits of\n    -+configurations options such as `gc.auto` and `gc.autoPackLimit`, all\n    - other housekeeping tasks (e.g. rerere, working trees, reflog...) will\n    - be performed as well.\n    +-These types of entries are generally created as\n    +-a result of using `git commit --amend` or `git rebase` and are the\n    +-commits prior to the amend or rebase occurring.  Since these changes\n    +-are not part of the current project most users will want to expire\n    +-them sooner, which is why the default is more aggressive than\n    +-`gc.reflogExpire`.\n    ++These types of entries are generally created as a result of using `git\n    ++commit --amend` or `git rebase` and are the commits prior to the amend\n    ++or rebase occurring.  Since these changes are not part of the current\n    ++project most users will want to expire them sooner, which is why the\n    ++default is more aggressive than `gc.reflogExpire`.\n      \n    + gc.rerereResolved::\n    + \tRecords of conflicted merge you resolved earlier are\n -:  ---------- >  6:  916433ef73 gc docs: note how --aggressive impacts --window & --depth\n 4:  257aff2808 !  7:  457357b464 gc docs: downplay the usefulness of --aggressive\n    @@ -3,19 +3,17 @@\n         gc docs: downplay the usefulness of --aggressive\n     \n         The existing \"gc --aggressive\" docs come just short of recommending to\n    -    users that they run it regularly. In reality it's a waste of CPU for\n    -    most users, and may even make things actively worse. I've personally\n    -    talked to many users who've taken these docs as an advice to use this\n    -    option, and have.\n    +    users that they run it regularly. I've personally talked to many users\n    +    who've taken these docs as an advice to use this option, and have,\n    +    usually it's (mostly) a waste of time.\n     \n    -    Let's change this documentation to better reflect reality, i.e. for\n    -    most users using --aggressive is a waste of time, and may even be\n    -    actively making things worse.\n    +    So let's clarify what it really does, and let the user draw their own\n    +    conclusions.\n     \n    -    Let's also clarify the \"The effects [...] are persistent\" to clearly\n    -    note that that's true to the extent that subsequent gc's aren't going\n    -    to re-roll existing packs generated with --aggressive into a new set\n    -    of packs.\n    +    Let's also clarify the \"The effects [...] are persistent\" to\n    +    paraphrase a brief version of Jeff King's explanation at [1].\n    +\n    +    1. https://public-inbox.org/git/20190318235356.GK29661@sigill.intra.peff.net/\n     \n         Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>\n     \n    @@ -23,27 +21,45 @@\n      --- a/Documentation/git-gc.txt\n      +++ b/Documentation/git-gc.txt\n     @@\n    - --aggressive::\n    - \tUsually 'git gc' runs very quickly while providing good disk\n      \tspace utilization and performance.  This option will cause\n    --\t'git gc' to more aggressively optimize the repository at the expense\n    --\tof taking much more time.  The effects of this optimization are\n    + \t'git gc' to more aggressively optimize the repository at the expense\n    + \tof taking much more time.  The effects of this optimization are\n     -\tpersistent, so this option only needs to be used occasionally; every\n     -\tfew hundred changesets or so.\n    -+\t'git gc' to more aggressively optimize the repository to save storage space\n    -+\tat the expense of taking much more time.\n    -++\n    -+Using this option may optimize for disk space at the expense of\n    -+runtime performance. See the `--depth` and `--window` documentation in\n    -+linkgit:git-repack[1]. It is not recommended that this option be used\n    -+to improve performance for a given repository without running tailored\n    -+performance benchmarks on it. It may make things better, or worse. Not\n    -+using this at all is the right trade-off for most users and their\n    -+repositories.\n    -++\n    -+The effects of this option are persistent to the extent that\n    -+`gc.autoPackLimit` and friends don't cause a consolidation of existing\n    -+pack(s) generated with this option.\n    ++\tmostly persistent. See the \"AGGRESSIVE\" section below for details.\n      \n      --auto::\n      \tWith this option, 'git gc' checks whether any housekeeping is\n    +@@\n    + \t`.keep` files are consolidated into a single pack. When this\n    + \toption is used, `gc.bigPackThreshold` is ignored.\n    + \n    ++AGGRESSIVE\n    ++----------\n    ++\n    ++When the `--aggressive` option is supplied, linkgit:git-repack[1] will\n    ++be invoked with the `-f` flag, which in turn will pass\n    ++`--no-reuse-delta` to linkgit:git-pack-objects[1]. This will throw\n    ++away any existing deltas and re-compute them, at the expense of\n    ++spending much more time on the repacking.\n    ++\n    ++The effects of this are mostly persistent, e.g. when packs and loose\n    ++objects are coalesced into one another pack the existing deltas in\n    ++that pack might get re-used, but there are also various cases where we\n    ++might pick a sub-optimal delta from a newer pack instead.\n    ++\n    ++Furthermore, supplying `--aggressive` will tweak the `--depth` and\n    ++`--window` options passed to linkgit:git-repack[1]. See the\n    ++`gc.aggressiveDepth` and `gc.aggressiveWindow` settings below. By\n    ++using a larger window size we're more likely to find more optimal\n    ++deltas.\n    ++\n    ++It's probably not worth it to use this option on a given repository\n    ++without running tailored performance benchmarks on it. It takes a lot\n    ++more time, and the resulting space/delta optimization may or may not\n    ++be worth it. Not using this at all is the right trade-off for most\n    ++users and their repositories.\n    ++\n    + CONFIGURATION\n    + -------------\n    + \n -:  ---------- >  8:  d80a6021f5 gc docs: note \"gc --aggressive\" in \"fast-import\"\n -:  ---------- >  9:  a5d31faf6f gc docs: clarify that \"gc\" doesn't throw away referenced objects\n -:  ---------- > 10:  9fd1203ad5 gc docs: remove incorrect reference to gc.auto=0\n-- \n2.21.0.360.g471c308f928\n\n"},{"id":"372163","messageId":"20190321205054.17109-3-avarab@gmail.com","threadId":"50772","inReplyTo":"20190318161502.7979-1-avarab@gmail.com","subject":"[PATCH v2 02/10] gc docs: stop noting \"repack\" flags","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2019-03-21T20:50:46Z","receivedAt":"2019-03-21T20:51:10Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"Remove the mention of specific flags from the \"gc\" documentation, and\nleave it at describing what we'll do instead. As seen in builtin/gc.c\nwe'll use various repack flags depending on what we detect we need to\ndo, so this isn't always accurate.\n\nMore importantly, a subsequent change is about to remove all this\ndocumentation and replace it with an include of the gc.* docs in\ngit-config(1). By first changing this it's easier to reason about that\nsubsequent change.\n\nSigned-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>\n---\n Documentation/git-gc.txt | 5 ++---\n 1 file changed, 2 insertions(+), 3 deletions(-)\n\ndiff --git a/Documentation/git-gc.txt b/Documentation/git-gc.txt\nindex 774503e33d..95c3237f8e 100644\n--- a/Documentation/git-gc.txt\n+++ b/Documentation/git-gc.txt\n@@ -52,14 +52,13 @@ OPTIONS\n +\n If the number of loose objects exceeds the value of the `gc.auto`\n configuration variable, then all loose objects are combined into a\n-single pack using `git repack -d -l`.  Setting the value of `gc.auto`\n+single pack.  Setting the value of `gc.auto`\n to 0 disables automatic packing of loose objects.\n +\n If the number of packs exceeds the value of `gc.autoPackLimit`,\n then existing packs (except those marked with a `.keep` file\n or over `gc.bigPackThreshold` limit)\n-are consolidated into a single pack by using the `-A` option of\n-'git repack'.\n+are consolidated into a single pack.\n If the amount of memory is estimated not enough for `git repack` to\n run smoothly and `gc.bigPackThreshold` is not set, the largest\n pack will also be excluded (this is the equivalent of running `git gc`\n-- \n2.21.0.360.g471c308f928\n\n"},{"id":"372164","messageId":"20190321205054.17109-2-avarab@gmail.com","threadId":"50772","inReplyTo":"20190318161502.7979-1-avarab@gmail.com","subject":"[PATCH v2 01/10] gc docs: modernize the advice for manually running \"gc\"","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2019-03-21T20:50:45Z","receivedAt":"2019-03-21T20:51:10Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"The docs have been recommending that users need to run this manually,\nbut that hasn't been needed in practice for a long time except in\nexceptional circumstances.\n\nLet's instead have this reflect reality and say that most users don't\nneed to run this manually at all, while briefly describing the sorts\nsort of cases where \"gc\" does need to be run manually.\n\nSince we're recommending that users run this most of the and usually\ndon't need to tweak it, let's tone down the very prominent example of\nthe gc.auto=0 command. It's sufficient to point to the gc.auto\ndocumentation below.\n\nSigned-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>\n---\n Documentation/git-gc.txt | 21 ++++++++++-----------\n 1 file changed, 10 insertions(+), 11 deletions(-)\n\ndiff --git a/Documentation/git-gc.txt b/Documentation/git-gc.txt\nindex a7c1b0f60e..774503e33d 100644\n--- a/Documentation/git-gc.txt\n+++ b/Documentation/git-gc.txt\n@@ -20,17 +20,16 @@ created from prior invocations of 'git add', packing refs, pruning\n reflog, rerere metadata or stale working trees. May also update ancillary\n indexes such as the commit-graph.\n \n-Users are encouraged to run this task on a regular basis within\n-each repository to maintain good disk space utilization and good\n-operating performance.\n-\n-Some git commands may automatically run 'git gc'; see the `--auto` flag\n-below for details. If you know what you're doing and all you want is to\n-disable this behavior permanently without further considerations, just do:\n-\n-----------------------\n-$ git config --global gc.auto 0\n-----------------------\n+When common porcelain operations that creates objects are run, they\n+will check whether the repository has grown substantially since the\n+last maintenance, and if so run `git gc` automatically. See `gc.auto`\n+below for how to disable this behavior.\n+\n+Running `git gc` manually should only be needed when adding objects to\n+a repository without regularly running such porcelain commands, to do\n+a one-off repository optimization, or e.g. to clean up a suboptimal\n+mass-import. See the \"PACKFILE OPTIMIZATION\" section in\n+linkgit:git-fast-import[1] for more details on the import case.\n \n OPTIONS\n -------\n-- \n2.21.0.360.g471c308f928\n\n"},{"id":"372165","messageId":"20190321205054.17109-4-avarab@gmail.com","threadId":"50772","inReplyTo":"20190318161502.7979-1-avarab@gmail.com","subject":"[PATCH v2 03/10] gc docs: clean grammar for \"gc.bigPackThreshold\"","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2019-03-21T20:50:47Z","receivedAt":"2019-03-21T20:51:13Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"Clean up the grammar in the documentation for\n\"gc.bigPackThreshold\". This documentation was added in 9806f5a7bf (\"gc\n--auto: exclude base pack if not enough mem to \"repack -ad\"\",\n2018-04-15).\n\nSaying \"the amount of memory estimated for\" flows more smoothly than\nthe previous \"the amount of memory is estimated not enough\".\n\nSuggested-by: Jeff King <peff@peff.net>\nSigned-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>\n---\n Documentation/git-gc.txt | 4 ++--\n 1 file changed, 2 insertions(+), 2 deletions(-)\n\ndiff --git a/Documentation/git-gc.txt b/Documentation/git-gc.txt\nindex 95c3237f8e..c31fe581d9 100644\n--- a/Documentation/git-gc.txt\n+++ b/Documentation/git-gc.txt\n@@ -59,8 +59,8 @@ If the number of packs exceeds the value of `gc.autoPackLimit`,\n then existing packs (except those marked with a `.keep` file\n or over `gc.bigPackThreshold` limit)\n are consolidated into a single pack.\n-If the amount of memory is estimated not enough for `git repack` to\n-run smoothly and `gc.bigPackThreshold` is not set, the largest\n+If the amount of memory estimated for `git repack` to run smoothly is\n+not available and `gc.bigPackThreshold` is not set, the largest\n pack will also be excluded (this is the equivalent of running `git gc`\n with `--keep-base-pack`).\n Setting `gc.autoPackLimit` to 0 disables automatic consolidation of\n-- \n2.21.0.360.g471c308f928\n\n"},{"id":"372166","messageId":"20190321205054.17109-5-avarab@gmail.com","threadId":"50772","inReplyTo":"20190318161502.7979-1-avarab@gmail.com","subject":"[PATCH v2 04/10] gc docs: include the \"gc.*\" section from \"config\" in \"gc\"","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2019-03-21T20:50:48Z","receivedAt":"2019-03-21T20:51:14Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"Rather than duplicating the documentation for the various \"gc\" options\nlet's include the \"gc\" docs from git-config. They were mostly better\nalready, and now we don't have the same docs in two places with subtly\ndifferent wording.\n\nIn the cases where the git-gc(1) docs were saying something the \"gc\"\ndocs in git-config(1) didn't cover move the relevant section over to\nthe git-config(1) docs.\n\nSigned-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>\n---\n Documentation/config/gc.txt | 29 ++++++++++++-\n Documentation/git-gc.txt    | 86 +++----------------------------------\n 2 files changed, 35 insertions(+), 80 deletions(-)\n\ndiff --git a/Documentation/config/gc.txt b/Documentation/config/gc.txt\nindex c6fbb8a96f..a255ae67b0 100644\n--- a/Documentation/config/gc.txt\n+++ b/Documentation/config/gc.txt\n@@ -2,24 +2,39 @@ gc.aggressiveDepth::\n \tThe depth parameter used in the delta compression\n \talgorithm used by 'git gc --aggressive'.  This defaults\n \tto 50.\n++\n+See the documentation for the `--depth` option in\n+linkgit:git-repack[1] for more details.\n \n gc.aggressiveWindow::\n \tThe window size parameter used in the delta compression\n \talgorithm used by 'git gc --aggressive'.  This defaults\n \tto 250.\n++\n+See the documentation for the `--window` option in\n+linkgit:git-repack[1] for more details.\n \n gc.auto::\n \tWhen there are approximately more than this many loose\n \tobjects in the repository, `git gc --auto` will pack them.\n \tSome Porcelain commands use this command to perform a\n \tlight-weight garbage collection from time to time.  The\n-\tdefault value is 6700.  Setting this to 0 disables it.\n+\tdefault value is 6700.\n++\n+Setting this to 0 disables not only automatic packing based on the\n+number of loose objects, but any other heuristic `git gc --auto` will\n+otherwise use to determine if there's work to do, such as\n+`gc.autoPackLimit`.\n \n gc.autoPackLimit::\n \tWhen there are more than this many packs that are not\n \tmarked with `*.keep` file in the repository, `git gc\n \t--auto` consolidates them into one larger pack.  The\n \tdefault\tvalue is 50.  Setting this to 0 disables it.\n+\tSetting `gc.auto` to 0 will also disable this.\n++\n+See the `gc.bigPackThreshold` configuration variable below. When in\n+use, it'll affect how the auto pack limit works.\n \n gc.autoDetach::\n \tMake `git gc --auto` return immediately and run in background\n@@ -36,6 +51,11 @@ Note that if the number of kept packs is more than gc.autoPackLimit,\n this configuration variable is ignored, all packs except the base pack\n will be repacked. After this the number of packs should go below\n gc.autoPackLimit and gc.bigPackThreshold should be respected again.\n++\n+If the amount of memory estimated for `git repack` to run smoothly is\n+not available and `gc.bigPackThreshold` is not set, the largest\n+pack will also be excluded (this is the equivalent of running `git gc`\n+with `--keep-base-pack`).\n \n gc.writeCommitGraph::\n \tIf true, then gc will rewrite the commit-graph file when\n@@ -94,6 +114,13 @@ gc.<pattern>.reflogExpireUnreachable::\n \tWith \"<pattern>\" (e.g. \"refs/stash\")\n \tin the middle, the setting applies only to the refs that\n \tmatch the <pattern>.\n++\n+These types of entries are generally created as\n+a result of using `git commit --amend` or `git rebase` and are the\n+commits prior to the amend or rebase occurring.  Since these changes\n+are not part of the current project most users will want to expire\n+them sooner, which is why the default is more aggressive than\n+`gc.reflogExpire`.\n \n gc.rerereResolved::\n \tRecords of conflicted merge you resolved earlier are\ndiff --git a/Documentation/git-gc.txt b/Documentation/git-gc.txt\nindex c31fe581d9..ba1ff9b4cf 100644\n--- a/Documentation/git-gc.txt\n+++ b/Documentation/git-gc.txt\n@@ -45,28 +45,12 @@ OPTIONS\n --auto::\n \tWith this option, 'git gc' checks whether any housekeeping is\n \trequired; if not, it exits without performing any work.\n-\tSome git commands run `git gc --auto` after performing\n-\toperations that could create many loose objects. Housekeeping\n-\tis required if there are too many loose objects or too many\n-\tpacks in the repository.\n +\n-If the number of loose objects exceeds the value of the `gc.auto`\n-configuration variable, then all loose objects are combined into a\n-single pack.  Setting the value of `gc.auto`\n-to 0 disables automatic packing of loose objects.\n+See the `gc.auto' option in the \"CONFIGURATION\" section below for how\n+this heuristic works.\n +\n-If the number of packs exceeds the value of `gc.autoPackLimit`,\n-then existing packs (except those marked with a `.keep` file\n-or over `gc.bigPackThreshold` limit)\n-are consolidated into a single pack.\n-If the amount of memory estimated for `git repack` to run smoothly is\n-not available and `gc.bigPackThreshold` is not set, the largest\n-pack will also be excluded (this is the equivalent of running `git gc`\n-with `--keep-base-pack`).\n-Setting `gc.autoPackLimit` to 0 disables automatic consolidation of\n-packs.\n-+\n-If houskeeping is required due to many loose objects or packs, all\n+Once housekeeping is triggered by exceeding the limits of\n+configuration options such as `gc.auto` and `gc.autoPackLimit`, all\n other housekeeping tasks (e.g. rerere, working trees, reflog...) will\n be performed as well.\n \n@@ -97,66 +81,10 @@ be performed as well.\n CONFIGURATION\n -------------\n \n-The optional configuration variable `gc.reflogExpire` can be\n-set to indicate how long historical entries within each branch's\n-reflog should remain available in this repository.  The setting is\n-expressed as a length of time, for example '90 days' or '3 months'.\n-It defaults to '90 days'.\n-\n-The optional configuration variable `gc.reflogExpireUnreachable`\n-can be set to indicate how long historical reflog entries which\n-are not part of the current branch should remain available in\n-this repository.  These types of entries are generally created as\n-a result of using `git commit --amend` or `git rebase` and are the\n-commits prior to the amend or rebase occurring.  Since these changes\n-are not part of the current project most users will want to expire\n-them sooner.  This option defaults to '30 days'.\n-\n-The above two configuration variables can be given to a pattern.  For\n-example, this sets non-default expiry values only to remote-tracking\n-branches:\n-\n-------------\n-[gc \"refs/remotes/*\"]\n-\treflogExpire = never\n-\treflogExpireUnreachable = 3 days\n-------------\n-\n-The optional configuration variable `gc.rerereResolved` indicates\n-how long records of conflicted merge you resolved earlier are\n-kept.  This defaults to 60 days.\n-\n-The optional configuration variable `gc.rerereUnresolved` indicates\n-how long records of conflicted merge you have not resolved are\n-kept.  This defaults to 15 days.\n-\n-The optional configuration variable `gc.packRefs` determines if\n-'git gc' runs 'git pack-refs'. This can be set to \"notbare\" to enable\n-it within all non-bare repos or it can be set to a boolean value.\n-This defaults to true.\n-\n-The optional configuration variable `gc.writeCommitGraph` determines if\n-'git gc' should run 'git commit-graph write'. This can be set to a\n-boolean value. This defaults to false.\n-\n-The optional configuration variable `gc.aggressiveWindow` controls how\n-much time is spent optimizing the delta compression of the objects in\n-the repository when the --aggressive option is specified.  The larger\n-the value, the more time is spent optimizing the delta compression.  See\n-the documentation for the --window option in linkgit:git-repack[1] for\n-more details.  This defaults to 250.\n-\n-Similarly, the optional configuration variable `gc.aggressiveDepth`\n-controls --depth option in linkgit:git-repack[1]. This defaults to 50.\n-\n-The optional configuration variable `gc.pruneExpire` controls how old\n-the unreferenced loose objects have to be before they are pruned.  The\n-default is \"2 weeks ago\".\n-\n-Optional configuration variable `gc.worktreePruneExpire` controls how\n-old a stale working tree should be before `git worktree prune` deletes\n-it. Default is \"3 months ago\".\n+The below documentation is the same as what's found in\n+linkgit:git-config[1]:\n \n+include::config/gc.txt[]\n \n NOTES\n -----\n-- \n2.21.0.360.g471c308f928\n\n"},{"id":"372167","messageId":"20190321205054.17109-6-avarab@gmail.com","threadId":"50772","inReplyTo":"20190318161502.7979-1-avarab@gmail.com","subject":"[PATCH v2 05/10] gc docs: re-flow the \"gc.*\" section in \"config\"","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2019-03-21T20:50:49Z","receivedAt":"2019-03-21T20:51:15Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"Re-flow the \"gc.*\" section in \"config\". A previous commit moved this\nover from the \"gc\" docs, but tried to keep as many of the lines\nidentical to benefit from diff's move detection.\n\nSigned-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>\n---\n Documentation/config/gc.txt | 17 ++++++++---------\n 1 file changed, 8 insertions(+), 9 deletions(-)\n\ndiff --git a/Documentation/config/gc.txt b/Documentation/config/gc.txt\nindex a255ae67b0..3e7fc052d9 100644\n--- a/Documentation/config/gc.txt\n+++ b/Documentation/config/gc.txt\n@@ -53,9 +53,9 @@ will be repacked. After this the number of packs should go below\n gc.autoPackLimit and gc.bigPackThreshold should be respected again.\n +\n If the amount of memory estimated for `git repack` to run smoothly is\n-not available and `gc.bigPackThreshold` is not set, the largest\n-pack will also be excluded (this is the equivalent of running `git gc`\n-with `--keep-base-pack`).\n+not available and `gc.bigPackThreshold` is not set, the largest pack\n+will also be excluded (this is the equivalent of running `git gc` with\n+`--keep-base-pack`).\n \n gc.writeCommitGraph::\n \tIf true, then gc will rewrite the commit-graph file when\n@@ -115,12 +115,11 @@ gc.<pattern>.reflogExpireUnreachable::\n \tin the middle, the setting applies only to the refs that\n \tmatch the <pattern>.\n +\n-These types of entries are generally created as\n-a result of using `git commit --amend` or `git rebase` and are the\n-commits prior to the amend or rebase occurring.  Since these changes\n-are not part of the current project most users will want to expire\n-them sooner, which is why the default is more aggressive than\n-`gc.reflogExpire`.\n+These types of entries are generally created as a result of using `git\n+commit --amend` or `git rebase` and are the commits prior to the amend\n+or rebase occurring.  Since these changes are not part of the current\n+project most users will want to expire them sooner, which is why the\n+default is more aggressive than `gc.reflogExpire`.\n \n gc.rerereResolved::\n \tRecords of conflicted merge you resolved earlier are\n-- \n2.21.0.360.g471c308f928\n\n"},{"id":"372169","messageId":"20190321205054.17109-7-avarab@gmail.com","threadId":"50772","inReplyTo":"20190318161502.7979-1-avarab@gmail.com","subject":"[PATCH v2 06/10] gc docs: note how --aggressive impacts --window & --depth","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2019-03-21T20:50:50Z","receivedAt":"2019-03-21T20:51:17Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"Since 07e7dbf0db (gc: default aggressive depth to 50, 2016-08-11) we\nsomewhat confusingly use the same depth under --aggressive as we do by\ndefault.\n\nAs noted in that commit that makes sense, it was wrong to make more\ndepth the default for \"aggressive\", and thus save disk space at the\nexpense of runtime performance, which is usually the opposite of\nsomeone who'd like \"aggressive gc\" wants.\n\nBut that's left us with a mostly-redundant configuration variable, so\nlet's clearly note in its documentation that it doesn't change the\ndefault.\n\nSigned-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>\n---\n Documentation/config/gc.txt | 6 ++++--\n 1 file changed, 4 insertions(+), 2 deletions(-)\n\ndiff --git a/Documentation/config/gc.txt b/Documentation/config/gc.txt\nindex 3e7fc052d9..0daa4683f6 100644\n--- a/Documentation/config/gc.txt\n+++ b/Documentation/config/gc.txt\n@@ -1,7 +1,8 @@\n gc.aggressiveDepth::\n \tThe depth parameter used in the delta compression\n \talgorithm used by 'git gc --aggressive'.  This defaults\n-\tto 50.\n+\tto 50, which is the default for the `--depth` option when\n+\t`--aggressive` isn't in use.\n +\n See the documentation for the `--depth` option in\n linkgit:git-repack[1] for more details.\n@@ -9,7 +10,8 @@ linkgit:git-repack[1] for more details.\n gc.aggressiveWindow::\n \tThe window size parameter used in the delta compression\n \talgorithm used by 'git gc --aggressive'.  This defaults\n-\tto 250.\n+\tto 250, which is a much more aggressive window size than\n+\tthe default `--window` of 10.\n +\n See the documentation for the `--window` option in\n linkgit:git-repack[1] for more details.\n-- \n2.21.0.360.g471c308f928\n\n"},{"id":"372168","messageId":"20190321205054.17109-8-avarab@gmail.com","threadId":"50772","inReplyTo":"20190318161502.7979-1-avarab@gmail.com","subject":"[PATCH v2 07/10] gc docs: downplay the usefulness of --aggressive","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2019-03-21T20:50:51Z","receivedAt":"2019-03-21T20:51:18Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"The existing \"gc --aggressive\" docs come just short of recommending to\nusers that they run it regularly. I've personally talked to many users\nwho've taken these docs as an advice to use this option, and have,\nusually it's (mostly) a waste of time.\n\nSo let's clarify what it really does, and let the user draw their own\nconclusions.\n\nLet's also clarify the \"The effects [...] are persistent\" to\nparaphrase a brief version of Jeff King's explanation at [1].\n\n1. https://public-inbox.org/git/20190318235356.GK29661@sigill.intra.peff.net/\n\nSigned-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>\n---\n Documentation/git-gc.txt | 29 +++++++++++++++++++++++++++--\n 1 file changed, 27 insertions(+), 2 deletions(-)\n\ndiff --git a/Documentation/git-gc.txt b/Documentation/git-gc.txt\nindex ba1ff9b4cf..c50ec30c83 100644\n--- a/Documentation/git-gc.txt\n+++ b/Documentation/git-gc.txt\n@@ -39,8 +39,7 @@ OPTIONS\n \tspace utilization and performance.  This option will cause\n \t'git gc' to more aggressively optimize the repository at the expense\n \tof taking much more time.  The effects of this optimization are\n-\tpersistent, so this option only needs to be used occasionally; every\n-\tfew hundred changesets or so.\n+\tmostly persistent. See the \"AGGRESSIVE\" section below for details.\n \n --auto::\n \tWith this option, 'git gc' checks whether any housekeeping is\n@@ -78,6 +77,32 @@ be performed as well.\n \t`.keep` files are consolidated into a single pack. When this\n \toption is used, `gc.bigPackThreshold` is ignored.\n \n+AGGRESSIVE\n+----------\n+\n+When the `--aggressive` option is supplied, linkgit:git-repack[1] will\n+be invoked with the `-f` flag, which in turn will pass\n+`--no-reuse-delta` to linkgit:git-pack-objects[1]. This will throw\n+away any existing deltas and re-compute them, at the expense of\n+spending much more time on the repacking.\n+\n+The effects of this are mostly persistent, e.g. when packs and loose\n+objects are coalesced into one another pack the existing deltas in\n+that pack might get re-used, but there are also various cases where we\n+might pick a sub-optimal delta from a newer pack instead.\n+\n+Furthermore, supplying `--aggressive` will tweak the `--depth` and\n+`--window` options passed to linkgit:git-repack[1]. See the\n+`gc.aggressiveDepth` and `gc.aggressiveWindow` settings below. By\n+using a larger window size we're more likely to find more optimal\n+deltas.\n+\n+It's probably not worth it to use this option on a given repository\n+without running tailored performance benchmarks on it. It takes a lot\n+more time, and the resulting space/delta optimization may or may not\n+be worth it. Not using this at all is the right trade-off for most\n+users and their repositories.\n+\n CONFIGURATION\n -------------\n \n-- \n2.21.0.360.g471c308f928\n\n"},{"id":"372170","messageId":"20190321205054.17109-9-avarab@gmail.com","threadId":"50772","inReplyTo":"20190318161502.7979-1-avarab@gmail.com","subject":"[PATCH v2 08/10] gc docs: note \"gc --aggressive\" in \"fast-import\"","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2019-03-21T20:50:52Z","receivedAt":"2019-03-21T20:51:20Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"Amend the \"PACKFILE OPTIMIZATION\" section in \"fast-import\" to explain\nthat simply running \"git gc --aggressive\" after a \"fast-import\" should\nproperly optimize the repository. This is simpler and more effective\nthan the existing \"repack\" advice (which I'm keeping as it helps\nexplain things) because it e.g. also packs the newly imported refs.\n\nSigned-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>\n---\n Documentation/git-fast-import.txt | 7 +++++++\n 1 file changed, 7 insertions(+)\n\ndiff --git a/Documentation/git-fast-import.txt b/Documentation/git-fast-import.txt\nindex 43ab3b1637..2248755cb7 100644\n--- a/Documentation/git-fast-import.txt\n+++ b/Documentation/git-fast-import.txt\n@@ -1396,6 +1396,13 @@ deltas are suboptimal (see above) then also adding the `-f` option\n to force recomputation of all deltas can significantly reduce the\n final packfile size (30-50% smaller can be quite typical).\n \n+Instead of running `git repack` you can also run `git gc\n+--aggressive`, which will also optimize other things after an import\n+(e.g. pack loose refs). As noted in the \"AGGRESSIVE\" section in\n+linkgit:git-gc[1] the `--aggressive` option will find new deltas with\n+the `-f` option to linkgit:git-repack[1]. For the reasons elaborated\n+on above using `--aggressive` after a fast-import is one of the few\n+cases where it's known to be worthwhile.\n \n MEMORY UTILIZATION\n ------------------\n-- \n2.21.0.360.g471c308f928\n\n"},{"id":"372171","messageId":"20190321205054.17109-10-avarab@gmail.com","threadId":"50772","inReplyTo":"20190318161502.7979-1-avarab@gmail.com","subject":"[PATCH v2 09/10] gc docs: clarify that \"gc\" doesn't throw away referenced objects","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2019-03-21T20:50:53Z","receivedAt":"2019-03-21T20:51:21Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"Amend the \"NOTES\" section to fix up wording that's been with us since\n3ffb58be0a (\"doc/git-gc: add a note about what is collected\",\n2008-04-23).\n\nI can't remember when/where anymore (I think Freenode #Git), but at\nsome point I was having a conversation with someone who was convinced\nthat \"gc\" would prune things only referenced by e.g. refs/pull/*, and\npointed to this section as proof.\n\nIt turned out that they'd read the \"branches and tags\" wording here\nand thought just refs/{heads,tags}/* and refs/remotes/* etc. would be\nkept, which is what we enumerate explicitly.\n\nSo let's say \"other refs\", even though just above we say \"objects that\nare referenced anywhere in your repository\".\n\nSigned-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>\n---\n Documentation/git-gc.txt | 4 ++--\n 1 file changed, 2 insertions(+), 2 deletions(-)\n\ndiff --git a/Documentation/git-gc.txt b/Documentation/git-gc.txt\nindex c50ec30c83..dced7cde09 100644\n--- a/Documentation/git-gc.txt\n+++ b/Documentation/git-gc.txt\n@@ -119,8 +119,8 @@ anywhere in your repository. In\n particular, it will keep not only objects referenced by your current set\n of branches and tags, but also objects referenced by the index,\n remote-tracking branches, refs saved by 'git filter-branch' in\n-refs/original/, or reflogs (which may reference commits in branches\n-that were later amended or rewound).\n+refs/original/, reflogs (which may reference commits in branches\n+that were later amended or rewound), and anything else in the refs/* namespace.\n If you are expecting some objects to be deleted and they aren't, check\n all of those locations and decide whether it makes sense in your case to\n remove those references.\n-- \n2.21.0.360.g471c308f928\n\n"},{"id":"372172","messageId":"20190321205054.17109-11-avarab@gmail.com","threadId":"50772","inReplyTo":"20190318161502.7979-1-avarab@gmail.com","subject":"[PATCH v2 10/10] gc docs: remove incorrect reference to gc.auto=0","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2019-03-21T20:50:54Z","receivedAt":"2019-03-21T20:51:24Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"The chance of a repository being corrupted due to a \"gc\" has nothing\nto do with whether or not that \"gc\" was invoked via \"gc --auto\", but\nwhether there's other concurrent operations happening.\n\nThis is already noted earlier in the paragraph, so there's no reason\nto suggest this here. The user can infer from the rest of the\ndocumentation that \"gc\" will run automatically unless gc.auto=0 is\nset, and we shouldn't confuse the issue by implying that \"gc --auto\"\nis somehow more prone to produce corruption than a normal \"gc\".\n\nWell, it is in the sense that a blocking \"gc\" would stop you from\ndoing anything else in *that* particular terminal window, but users\nare likely to have another window, or to be worried about how\nconcurrent \"gc\" on a server might cause corruption.\n\nSigned-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>\n---\n Documentation/git-gc.txt | 3 +--\n 1 file changed, 1 insertion(+), 2 deletions(-)\n\ndiff --git a/Documentation/git-gc.txt b/Documentation/git-gc.txt\nindex dced7cde09..1826d7e3bb 100644\n--- a/Documentation/git-gc.txt\n+++ b/Documentation/git-gc.txt\n@@ -141,8 +141,7 @@ mitigate this problem:\n \n However, these features fall short of a complete solution, so users who\n run commands concurrently have to live with some risk of corruption (which\n-seems to be low in practice) unless they turn off automatic garbage\n-collection with 'git config gc.auto 0'.\n+seems to be low in practice).\n \n HOOKS\n -----\n-- \n2.21.0.360.g471c308f928\n\n"},{"id":"372181","messageId":"4dba2718-9e93-4f07-8f67-f79a8a9fbb0c@gmail.com","threadId":"50772","inReplyTo":"20190318213139.GF29661@sigill.intra.peff.net","subject":"Re: [PATCH 2/4] gc docs: include the \"gc.*\" section from \"config\" in \"gc\"","fromName":"Andreas Heiduk","fromEmail":"asheiduk@gmail.com","sentAt":"2019-03-21T22:11:59Z","receivedAt":"2019-03-21T22:12:05Z","isPatch":true,"sender":{"key":"asheiduk@gmail.com","avatar":"https://avatars.githubusercontent.com/u/9371344?v=4"},"body":"On 18.03.19 at 22:31 Jeff King wrote:\n> On Mon, Mar 18, 2019 at 05:15:00PM +0100, Ævar Arnfjörð Bjarmason wrote:\n> \n>> Rather than duplicating the documentation for the various \"gc\" options\n>> let's include the \"gc\" docs from git-config. They were mostly better\n>> already, and now we don't have the same docs in two places with subtly\n>> different wording.\n>>\n>> In the cases where the git-gc(1) docs were saying something the \"gc\"\n>> docs in git-config(1) didn't cover move the relevant section over to\n>> the git-config(1) docs.\n> \n> Makes sense.\n> \n> I think we lose the actual example for gc.*.reflogExpire:\n> \n>> -The above two configuration variables can be given to a pattern.  For\n>> -example, this sets non-default expiry values only to remote-tracking\n>> -branches:\n>> -\n>> -------------\n>> -[gc \"refs/remotes/*\"]\n>> -\treflogExpire = never\n>> -\treflogExpireUnreachable = 3 days\n>> -------------\n> \n> I don't actually think it's that big a loss. If we wanted to retain it,\n> though, it might make sense in the \"EXAMPLES\" section.\n\nCoincidentally I stumbled over that example a  month ago and immediately\nput `never` into my configuraton for a certain - often rebased branch\n(something like \"pu\" here).\n\n> \n> -Peff\n> \n\n"},{"id":"372204","messageId":"xmqqtvfvphv6.fsf@gitster-ct.c.googlers.com","threadId":"50772","inReplyTo":"20190321205054.17109-2-avarab@gmail.com","subject":"Re: [PATCH v2 01/10] gc docs: modernize the advice for manually running \"gc\"","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2019-03-22T06:01:49Z","receivedAt":"2019-03-22T06:01:54Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Ævar Arnfjörð Bjarmason  <avarab@gmail.com> writes:\n\n> +When common porcelain operations that creates objects are run, they\n\n\"operations that create objects are run\"?\n\n> +will check whether the repository has grown substantially since the\n> +last maintenance, and if so run `git gc` automatically. See `gc.auto`\n> +below for how to disable this behavior.\n> +\n> +Running `git gc` manually should only be needed when adding objects to\n> +a repository without regularly running such porcelain commands, to do\n> +a one-off repository optimization, or e.g. to clean up a suboptimal\n> +mass-import. See the \"PACKFILE OPTIMIZATION\" section in\n> +linkgit:git-fast-import[1] for more details on the import case.\n>  \n>  OPTIONS\n>  -------\n"},{"id":"372219","messageId":"20190322093242.5508-1-avarab@gmail.com","threadId":"50772","inReplyTo":"20190321205054.17109-1-avarab@gmail.com","subject":"[PATCH v3 00/11] gc docs: modernize the advice for manually running \"gc\"","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2019-03-22T09:32:31Z","receivedAt":"2019-03-22T09:33:02Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"Patch v3 fixes a minor grammar issue noted by Junio in\n<xmqqtvfvphv6.fsf@gitster-ct.c.googlers.com>, and another \"while I'm\nat it\" formatting error.\n\nÆvar Arnfjörð Bjarmason (11):\n  gc docs: modernize the advice for manually running \"gc\"\n  gc docs: stop noting \"repack\" flags\n  gc docs: clean grammar for \"gc.bigPackThreshold\"\n  gc docs: include the \"gc.*\" section from \"config\" in \"gc\"\n  gc docs: re-flow the \"gc.*\" section in \"config\"\n  gc docs: fix formatting for \"gc.writeCommitGraph\"\n  gc docs: note how --aggressive impacts --window & --depth\n  gc docs: downplay the usefulness of --aggressive\n  gc docs: note \"gc --aggressive\" in \"fast-import\"\n  gc docs: clarify that \"gc\" doesn't throw away referenced objects\n  gc docs: remove incorrect reference to gc.auto=0\n\n Documentation/config/gc.txt       |  38 ++++++--\n Documentation/git-fast-import.txt |   7 ++\n Documentation/git-gc.txt          | 142 ++++++++++--------------------\n 3 files changed, 86 insertions(+), 101 deletions(-)\n\nRange-diff:\n 1:  89719142c7 !  1:  a48ef8d5d8 gc docs: modernize the advice for manually running \"gc\"\n    @@ -35,7 +35,7 @@\n     -----------------------\n     -$ git config --global gc.auto 0\n     -----------------------\n    -+When common porcelain operations that creates objects are run, they\n    ++When common porcelain operations that create objects are run, they\n     +will check whether the repository has grown substantially since the\n     +last maintenance, and if so run `git gc` automatically. See `gc.auto`\n     +below for how to disable this behavior.\n 2:  d90a5b1b4c =  2:  21e66a7903 gc docs: stop noting \"repack\" flags\n 3:  fedd9bb886 =  3:  c8a1342e34 gc docs: clean grammar for \"gc.bigPackThreshold\"\n 4:  6fad05a67c =  4:  9163e2f885 gc docs: include the \"gc.*\" section from \"config\" in \"gc\"\n 5:  994e22a0d6 =  5:  8fa0e26671 gc docs: re-flow the \"gc.*\" section in \"config\"\n -:  ---------- >  6:  b70396f029 gc docs: fix formatting for \"gc.writeCommitGraph\"\n 6:  916433ef73 =  7:  04ee81a3c9 gc docs: note how --aggressive impacts --window & --depth\n 7:  457357b464 =  8:  04af0afbcf gc docs: downplay the usefulness of --aggressive\n 8:  d80a6021f5 =  9:  c35bc94416 gc docs: note \"gc --aggressive\" in \"fast-import\"\n 9:  a5d31faf6f = 10:  702f2cd2d9 gc docs: clarify that \"gc\" doesn't throw away referenced objects\n10:  9fd1203ad5 = 11:  08af3cc3ee gc docs: remove incorrect reference to gc.auto=0\n-- \n2.21.0.360.g471c308f928\n\n"},{"id":"372220","messageId":"20190322093242.5508-2-avarab@gmail.com","threadId":"50772","inReplyTo":"20190321205054.17109-1-avarab@gmail.com","subject":"[PATCH v3 01/11] gc docs: modernize the advice for manually running \"gc\"","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2019-03-22T09:32:32Z","receivedAt":"2019-03-22T09:33:06Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"The docs have been recommending that users need to run this manually,\nbut that hasn't been needed in practice for a long time except in\nexceptional circumstances.\n\nLet's instead have this reflect reality and say that most users don't\nneed to run this manually at all, while briefly describing the sorts\nsort of cases where \"gc\" does need to be run manually.\n\nSince we're recommending that users run this most of the and usually\ndon't need to tweak it, let's tone down the very prominent example of\nthe gc.auto=0 command. It's sufficient to point to the gc.auto\ndocumentation below.\n\nSigned-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>\n---\n Documentation/git-gc.txt | 21 ++++++++++-----------\n 1 file changed, 10 insertions(+), 11 deletions(-)\n\ndiff --git a/Documentation/git-gc.txt b/Documentation/git-gc.txt\nindex a7c1b0f60e..dd22eecc79 100644\n--- a/Documentation/git-gc.txt\n+++ b/Documentation/git-gc.txt\n@@ -20,17 +20,16 @@ created from prior invocations of 'git add', packing refs, pruning\n reflog, rerere metadata or stale working trees. May also update ancillary\n indexes such as the commit-graph.\n \n-Users are encouraged to run this task on a regular basis within\n-each repository to maintain good disk space utilization and good\n-operating performance.\n-\n-Some git commands may automatically run 'git gc'; see the `--auto` flag\n-below for details. If you know what you're doing and all you want is to\n-disable this behavior permanently without further considerations, just do:\n-\n-----------------------\n-$ git config --global gc.auto 0\n-----------------------\n+When common porcelain operations that create objects are run, they\n+will check whether the repository has grown substantially since the\n+last maintenance, and if so run `git gc` automatically. See `gc.auto`\n+below for how to disable this behavior.\n+\n+Running `git gc` manually should only be needed when adding objects to\n+a repository without regularly running such porcelain commands, to do\n+a one-off repository optimization, or e.g. to clean up a suboptimal\n+mass-import. See the \"PACKFILE OPTIMIZATION\" section in\n+linkgit:git-fast-import[1] for more details on the import case.\n \n OPTIONS\n -------\n-- \n2.21.0.360.g471c308f928\n\n"},{"id":"372221","messageId":"20190322093242.5508-3-avarab@gmail.com","threadId":"50772","inReplyTo":"20190321205054.17109-1-avarab@gmail.com","subject":"[PATCH v3 02/11] gc docs: stop noting \"repack\" flags","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2019-03-22T09:32:33Z","receivedAt":"2019-03-22T09:33:07Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"Remove the mention of specific flags from the \"gc\" documentation, and\nleave it at describing what we'll do instead. As seen in builtin/gc.c\nwe'll use various repack flags depending on what we detect we need to\ndo, so this isn't always accurate.\n\nMore importantly, a subsequent change is about to remove all this\ndocumentation and replace it with an include of the gc.* docs in\ngit-config(1). By first changing this it's easier to reason about that\nsubsequent change.\n\nSigned-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>\n---\n Documentation/git-gc.txt | 5 ++---\n 1 file changed, 2 insertions(+), 3 deletions(-)\n\ndiff --git a/Documentation/git-gc.txt b/Documentation/git-gc.txt\nindex dd22eecc79..c56f4f7cde 100644\n--- a/Documentation/git-gc.txt\n+++ b/Documentation/git-gc.txt\n@@ -52,14 +52,13 @@ OPTIONS\n +\n If the number of loose objects exceeds the value of the `gc.auto`\n configuration variable, then all loose objects are combined into a\n-single pack using `git repack -d -l`.  Setting the value of `gc.auto`\n+single pack.  Setting the value of `gc.auto`\n to 0 disables automatic packing of loose objects.\n +\n If the number of packs exceeds the value of `gc.autoPackLimit`,\n then existing packs (except those marked with a `.keep` file\n or over `gc.bigPackThreshold` limit)\n-are consolidated into a single pack by using the `-A` option of\n-'git repack'.\n+are consolidated into a single pack.\n If the amount of memory is estimated not enough for `git repack` to\n run smoothly and `gc.bigPackThreshold` is not set, the largest\n pack will also be excluded (this is the equivalent of running `git gc`\n-- \n2.21.0.360.g471c308f928\n\n"},{"id":"372222","messageId":"20190322093242.5508-4-avarab@gmail.com","threadId":"50772","inReplyTo":"20190321205054.17109-1-avarab@gmail.com","subject":"[PATCH v3 03/11] gc docs: clean grammar for \"gc.bigPackThreshold\"","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2019-03-22T09:32:34Z","receivedAt":"2019-03-22T09:33:10Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"Clean up the grammar in the documentation for\n\"gc.bigPackThreshold\". This documentation was added in 9806f5a7bf (\"gc\n--auto: exclude base pack if not enough mem to \"repack -ad\"\",\n2018-04-15).\n\nSaying \"the amount of memory estimated for\" flows more smoothly than\nthe previous \"the amount of memory is estimated not enough\".\n\nSuggested-by: Jeff King <peff@peff.net>\nSigned-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>\n---\n Documentation/git-gc.txt | 4 ++--\n 1 file changed, 2 insertions(+), 2 deletions(-)\n\ndiff --git a/Documentation/git-gc.txt b/Documentation/git-gc.txt\nindex c56f4f7cde..66386439b7 100644\n--- a/Documentation/git-gc.txt\n+++ b/Documentation/git-gc.txt\n@@ -59,8 +59,8 @@ If the number of packs exceeds the value of `gc.autoPackLimit`,\n then existing packs (except those marked with a `.keep` file\n or over `gc.bigPackThreshold` limit)\n are consolidated into a single pack.\n-If the amount of memory is estimated not enough for `git repack` to\n-run smoothly and `gc.bigPackThreshold` is not set, the largest\n+If the amount of memory estimated for `git repack` to run smoothly is\n+not available and `gc.bigPackThreshold` is not set, the largest\n pack will also be excluded (this is the equivalent of running `git gc`\n with `--keep-base-pack`).\n Setting `gc.autoPackLimit` to 0 disables automatic consolidation of\n-- \n2.21.0.360.g471c308f928\n\n"},{"id":"372223","messageId":"20190322093242.5508-7-avarab@gmail.com","threadId":"50772","inReplyTo":"20190321205054.17109-1-avarab@gmail.com","subject":"[PATCH v3 06/11] gc docs: fix formatting for \"gc.writeCommitGraph\"","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2019-03-22T09:32:37Z","receivedAt":"2019-03-22T09:33:12Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"Change the AsciiDoc formatting so that an example of \"gc --auto\" isn't\nrendered as \"git-gc(1) --auto\", but as \"git gc --auto\". This is\nconsistent with the rest of the links and command examples in this\ndocumentation.\n\nThe formatting I'm changing was initially introduced in\nd5d5d7b641 (\"gc: automatically write commit-graph files\", 2018-06-27).\n\nSigned-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>\n---\n Documentation/config/gc.txt | 4 ++--\n 1 file changed, 2 insertions(+), 2 deletions(-)\n\ndiff --git a/Documentation/config/gc.txt b/Documentation/config/gc.txt\nindex 3e7fc052d9..56918a5008 100644\n--- a/Documentation/config/gc.txt\n+++ b/Documentation/config/gc.txt\n@@ -59,8 +59,8 @@ will also be excluded (this is the equivalent of running `git gc` with\n \n gc.writeCommitGraph::\n \tIf true, then gc will rewrite the commit-graph file when\n-\tlinkgit:git-gc[1] is run. When using linkgit:git-gc[1]\n-\t'--auto' the commit-graph will be updated if housekeeping is\n+\tlinkgit:git-gc[1] is run. When using `git gc --auto`\n+\tthe commit-graph will be updated if housekeeping is\n \trequired. Default is false. See linkgit:git-commit-graph[1]\n \tfor details.\n \n-- \n2.21.0.360.g471c308f928\n\n"},{"id":"372224","messageId":"20190322093242.5508-6-avarab@gmail.com","threadId":"50772","inReplyTo":"20190321205054.17109-1-avarab@gmail.com","subject":"[PATCH v3 05/11] gc docs: re-flow the \"gc.*\" section in \"config\"","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2019-03-22T09:32:36Z","receivedAt":"2019-03-22T09:33:13Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"Re-flow the \"gc.*\" section in \"config\". A previous commit moved this\nover from the \"gc\" docs, but tried to keep as many of the lines\nidentical to benefit from diff's move detection.\n\nSigned-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>\n---\n Documentation/config/gc.txt | 17 ++++++++---------\n 1 file changed, 8 insertions(+), 9 deletions(-)\n\ndiff --git a/Documentation/config/gc.txt b/Documentation/config/gc.txt\nindex a255ae67b0..3e7fc052d9 100644\n--- a/Documentation/config/gc.txt\n+++ b/Documentation/config/gc.txt\n@@ -53,9 +53,9 @@ will be repacked. After this the number of packs should go below\n gc.autoPackLimit and gc.bigPackThreshold should be respected again.\n +\n If the amount of memory estimated for `git repack` to run smoothly is\n-not available and `gc.bigPackThreshold` is not set, the largest\n-pack will also be excluded (this is the equivalent of running `git gc`\n-with `--keep-base-pack`).\n+not available and `gc.bigPackThreshold` is not set, the largest pack\n+will also be excluded (this is the equivalent of running `git gc` with\n+`--keep-base-pack`).\n \n gc.writeCommitGraph::\n \tIf true, then gc will rewrite the commit-graph file when\n@@ -115,12 +115,11 @@ gc.<pattern>.reflogExpireUnreachable::\n \tin the middle, the setting applies only to the refs that\n \tmatch the <pattern>.\n +\n-These types of entries are generally created as\n-a result of using `git commit --amend` or `git rebase` and are the\n-commits prior to the amend or rebase occurring.  Since these changes\n-are not part of the current project most users will want to expire\n-them sooner, which is why the default is more aggressive than\n-`gc.reflogExpire`.\n+These types of entries are generally created as a result of using `git\n+commit --amend` or `git rebase` and are the commits prior to the amend\n+or rebase occurring.  Since these changes are not part of the current\n+project most users will want to expire them sooner, which is why the\n+default is more aggressive than `gc.reflogExpire`.\n \n gc.rerereResolved::\n \tRecords of conflicted merge you resolved earlier are\n-- \n2.21.0.360.g471c308f928\n\n"},{"id":"372225","messageId":"20190322093242.5508-8-avarab@gmail.com","threadId":"50772","inReplyTo":"20190321205054.17109-1-avarab@gmail.com","subject":"[PATCH v3 07/11] gc docs: note how --aggressive impacts --window & --depth","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2019-03-22T09:32:38Z","receivedAt":"2019-03-22T09:33:15Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"Since 07e7dbf0db (gc: default aggressive depth to 50, 2016-08-11) we\nsomewhat confusingly use the same depth under --aggressive as we do by\ndefault.\n\nAs noted in that commit that makes sense, it was wrong to make more\ndepth the default for \"aggressive\", and thus save disk space at the\nexpense of runtime performance, which is usually the opposite of\nsomeone who'd like \"aggressive gc\" wants.\n\nBut that's left us with a mostly-redundant configuration variable, so\nlet's clearly note in its documentation that it doesn't change the\ndefault.\n\nSigned-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>\n---\n Documentation/config/gc.txt | 6 ++++--\n 1 file changed, 4 insertions(+), 2 deletions(-)\n\ndiff --git a/Documentation/config/gc.txt b/Documentation/config/gc.txt\nindex 56918a5008..f732fe5bfd 100644\n--- a/Documentation/config/gc.txt\n+++ b/Documentation/config/gc.txt\n@@ -1,7 +1,8 @@\n gc.aggressiveDepth::\n \tThe depth parameter used in the delta compression\n \talgorithm used by 'git gc --aggressive'.  This defaults\n-\tto 50.\n+\tto 50, which is the default for the `--depth` option when\n+\t`--aggressive` isn't in use.\n +\n See the documentation for the `--depth` option in\n linkgit:git-repack[1] for more details.\n@@ -9,7 +10,8 @@ linkgit:git-repack[1] for more details.\n gc.aggressiveWindow::\n \tThe window size parameter used in the delta compression\n \talgorithm used by 'git gc --aggressive'.  This defaults\n-\tto 250.\n+\tto 250, which is a much more aggressive window size than\n+\tthe default `--window` of 10.\n +\n See the documentation for the `--window` option in\n linkgit:git-repack[1] for more details.\n-- \n2.21.0.360.g471c308f928\n\n"},{"id":"372226","messageId":"20190322093242.5508-5-avarab@gmail.com","threadId":"50772","inReplyTo":"20190321205054.17109-1-avarab@gmail.com","subject":"[PATCH v3 04/11] gc docs: include the \"gc.*\" section from \"config\" in \"gc\"","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2019-03-22T09:32:35Z","receivedAt":"2019-03-22T09:33:15Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"Rather than duplicating the documentation for the various \"gc\" options\nlet's include the \"gc\" docs from git-config. They were mostly better\nalready, and now we don't have the same docs in two places with subtly\ndifferent wording.\n\nIn the cases where the git-gc(1) docs were saying something the \"gc\"\ndocs in git-config(1) didn't cover move the relevant section over to\nthe git-config(1) docs.\n\nSigned-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>\n---\n Documentation/config/gc.txt | 29 ++++++++++++-\n Documentation/git-gc.txt    | 86 +++----------------------------------\n 2 files changed, 35 insertions(+), 80 deletions(-)\n\ndiff --git a/Documentation/config/gc.txt b/Documentation/config/gc.txt\nindex c6fbb8a96f..a255ae67b0 100644\n--- a/Documentation/config/gc.txt\n+++ b/Documentation/config/gc.txt\n@@ -2,24 +2,39 @@ gc.aggressiveDepth::\n \tThe depth parameter used in the delta compression\n \talgorithm used by 'git gc --aggressive'.  This defaults\n \tto 50.\n++\n+See the documentation for the `--depth` option in\n+linkgit:git-repack[1] for more details.\n \n gc.aggressiveWindow::\n \tThe window size parameter used in the delta compression\n \talgorithm used by 'git gc --aggressive'.  This defaults\n \tto 250.\n++\n+See the documentation for the `--window` option in\n+linkgit:git-repack[1] for more details.\n \n gc.auto::\n \tWhen there are approximately more than this many loose\n \tobjects in the repository, `git gc --auto` will pack them.\n \tSome Porcelain commands use this command to perform a\n \tlight-weight garbage collection from time to time.  The\n-\tdefault value is 6700.  Setting this to 0 disables it.\n+\tdefault value is 6700.\n++\n+Setting this to 0 disables not only automatic packing based on the\n+number of loose objects, but any other heuristic `git gc --auto` will\n+otherwise use to determine if there's work to do, such as\n+`gc.autoPackLimit`.\n \n gc.autoPackLimit::\n \tWhen there are more than this many packs that are not\n \tmarked with `*.keep` file in the repository, `git gc\n \t--auto` consolidates them into one larger pack.  The\n \tdefault\tvalue is 50.  Setting this to 0 disables it.\n+\tSetting `gc.auto` to 0 will also disable this.\n++\n+See the `gc.bigPackThreshold` configuration variable below. When in\n+use, it'll affect how the auto pack limit works.\n \n gc.autoDetach::\n \tMake `git gc --auto` return immediately and run in background\n@@ -36,6 +51,11 @@ Note that if the number of kept packs is more than gc.autoPackLimit,\n this configuration variable is ignored, all packs except the base pack\n will be repacked. After this the number of packs should go below\n gc.autoPackLimit and gc.bigPackThreshold should be respected again.\n++\n+If the amount of memory estimated for `git repack` to run smoothly is\n+not available and `gc.bigPackThreshold` is not set, the largest\n+pack will also be excluded (this is the equivalent of running `git gc`\n+with `--keep-base-pack`).\n \n gc.writeCommitGraph::\n \tIf true, then gc will rewrite the commit-graph file when\n@@ -94,6 +114,13 @@ gc.<pattern>.reflogExpireUnreachable::\n \tWith \"<pattern>\" (e.g. \"refs/stash\")\n \tin the middle, the setting applies only to the refs that\n \tmatch the <pattern>.\n++\n+These types of entries are generally created as\n+a result of using `git commit --amend` or `git rebase` and are the\n+commits prior to the amend or rebase occurring.  Since these changes\n+are not part of the current project most users will want to expire\n+them sooner, which is why the default is more aggressive than\n+`gc.reflogExpire`.\n \n gc.rerereResolved::\n \tRecords of conflicted merge you resolved earlier are\ndiff --git a/Documentation/git-gc.txt b/Documentation/git-gc.txt\nindex 66386439b7..c037a46b09 100644\n--- a/Documentation/git-gc.txt\n+++ b/Documentation/git-gc.txt\n@@ -45,28 +45,12 @@ OPTIONS\n --auto::\n \tWith this option, 'git gc' checks whether any housekeeping is\n \trequired; if not, it exits without performing any work.\n-\tSome git commands run `git gc --auto` after performing\n-\toperations that could create many loose objects. Housekeeping\n-\tis required if there are too many loose objects or too many\n-\tpacks in the repository.\n +\n-If the number of loose objects exceeds the value of the `gc.auto`\n-configuration variable, then all loose objects are combined into a\n-single pack.  Setting the value of `gc.auto`\n-to 0 disables automatic packing of loose objects.\n+See the `gc.auto' option in the \"CONFIGURATION\" section below for how\n+this heuristic works.\n +\n-If the number of packs exceeds the value of `gc.autoPackLimit`,\n-then existing packs (except those marked with a `.keep` file\n-or over `gc.bigPackThreshold` limit)\n-are consolidated into a single pack.\n-If the amount of memory estimated for `git repack` to run smoothly is\n-not available and `gc.bigPackThreshold` is not set, the largest\n-pack will also be excluded (this is the equivalent of running `git gc`\n-with `--keep-base-pack`).\n-Setting `gc.autoPackLimit` to 0 disables automatic consolidation of\n-packs.\n-+\n-If houskeeping is required due to many loose objects or packs, all\n+Once housekeeping is triggered by exceeding the limits of\n+configuration options such as `gc.auto` and `gc.autoPackLimit`, all\n other housekeeping tasks (e.g. rerere, working trees, reflog...) will\n be performed as well.\n \n@@ -97,66 +81,10 @@ be performed as well.\n CONFIGURATION\n -------------\n \n-The optional configuration variable `gc.reflogExpire` can be\n-set to indicate how long historical entries within each branch's\n-reflog should remain available in this repository.  The setting is\n-expressed as a length of time, for example '90 days' or '3 months'.\n-It defaults to '90 days'.\n-\n-The optional configuration variable `gc.reflogExpireUnreachable`\n-can be set to indicate how long historical reflog entries which\n-are not part of the current branch should remain available in\n-this repository.  These types of entries are generally created as\n-a result of using `git commit --amend` or `git rebase` and are the\n-commits prior to the amend or rebase occurring.  Since these changes\n-are not part of the current project most users will want to expire\n-them sooner.  This option defaults to '30 days'.\n-\n-The above two configuration variables can be given to a pattern.  For\n-example, this sets non-default expiry values only to remote-tracking\n-branches:\n-\n-------------\n-[gc \"refs/remotes/*\"]\n-\treflogExpire = never\n-\treflogExpireUnreachable = 3 days\n-------------\n-\n-The optional configuration variable `gc.rerereResolved` indicates\n-how long records of conflicted merge you resolved earlier are\n-kept.  This defaults to 60 days.\n-\n-The optional configuration variable `gc.rerereUnresolved` indicates\n-how long records of conflicted merge you have not resolved are\n-kept.  This defaults to 15 days.\n-\n-The optional configuration variable `gc.packRefs` determines if\n-'git gc' runs 'git pack-refs'. This can be set to \"notbare\" to enable\n-it within all non-bare repos or it can be set to a boolean value.\n-This defaults to true.\n-\n-The optional configuration variable `gc.writeCommitGraph` determines if\n-'git gc' should run 'git commit-graph write'. This can be set to a\n-boolean value. This defaults to false.\n-\n-The optional configuration variable `gc.aggressiveWindow` controls how\n-much time is spent optimizing the delta compression of the objects in\n-the repository when the --aggressive option is specified.  The larger\n-the value, the more time is spent optimizing the delta compression.  See\n-the documentation for the --window option in linkgit:git-repack[1] for\n-more details.  This defaults to 250.\n-\n-Similarly, the optional configuration variable `gc.aggressiveDepth`\n-controls --depth option in linkgit:git-repack[1]. This defaults to 50.\n-\n-The optional configuration variable `gc.pruneExpire` controls how old\n-the unreferenced loose objects have to be before they are pruned.  The\n-default is \"2 weeks ago\".\n-\n-Optional configuration variable `gc.worktreePruneExpire` controls how\n-old a stale working tree should be before `git worktree prune` deletes\n-it. Default is \"3 months ago\".\n+The below documentation is the same as what's found in\n+linkgit:git-config[1]:\n \n+include::config/gc.txt[]\n \n NOTES\n -----\n-- \n2.21.0.360.g471c308f928\n\n"},{"id":"372227","messageId":"20190322093242.5508-11-avarab@gmail.com","threadId":"50772","inReplyTo":"20190321205054.17109-1-avarab@gmail.com","subject":"[PATCH v3 10/11] gc docs: clarify that \"gc\" doesn't throw away referenced objects","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2019-03-22T09:32:41Z","receivedAt":"2019-03-22T09:33:18Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"Amend the \"NOTES\" section to fix up wording that's been with us since\n3ffb58be0a (\"doc/git-gc: add a note about what is collected\",\n2008-04-23).\n\nI can't remember when/where anymore (I think Freenode #Git), but at\nsome point I was having a conversation with someone who was convinced\nthat \"gc\" would prune things only referenced by e.g. refs/pull/*, and\npointed to this section as proof.\n\nIt turned out that they'd read the \"branches and tags\" wording here\nand thought just refs/{heads,tags}/* and refs/remotes/* etc. would be\nkept, which is what we enumerate explicitly.\n\nSo let's say \"other refs\", even though just above we say \"objects that\nare referenced anywhere in your repository\".\n\nSigned-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>\n---\n Documentation/git-gc.txt | 4 ++--\n 1 file changed, 2 insertions(+), 2 deletions(-)\n\ndiff --git a/Documentation/git-gc.txt b/Documentation/git-gc.txt\nindex 165f05e999..49aec5435b 100644\n--- a/Documentation/git-gc.txt\n+++ b/Documentation/git-gc.txt\n@@ -119,8 +119,8 @@ anywhere in your repository. In\n particular, it will keep not only objects referenced by your current set\n of branches and tags, but also objects referenced by the index,\n remote-tracking branches, refs saved by 'git filter-branch' in\n-refs/original/, or reflogs (which may reference commits in branches\n-that were later amended or rewound).\n+refs/original/, reflogs (which may reference commits in branches\n+that were later amended or rewound), and anything else in the refs/* namespace.\n If you are expecting some objects to be deleted and they aren't, check\n all of those locations and decide whether it makes sense in your case to\n remove those references.\n-- \n2.21.0.360.g471c308f928\n\n"},{"id":"372228","messageId":"20190322093242.5508-9-avarab@gmail.com","threadId":"50772","inReplyTo":"20190321205054.17109-1-avarab@gmail.com","subject":"[PATCH v3 08/11] gc docs: downplay the usefulness of --aggressive","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2019-03-22T09:32:39Z","receivedAt":"2019-03-22T09:33:19Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"The existing \"gc --aggressive\" docs come just short of recommending to\nusers that they run it regularly. I've personally talked to many users\nwho've taken these docs as an advice to use this option, and have,\nusually it's (mostly) a waste of time.\n\nSo let's clarify what it really does, and let the user draw their own\nconclusions.\n\nLet's also clarify the \"The effects [...] are persistent\" to\nparaphrase a brief version of Jeff King's explanation at [1].\n\n1. https://public-inbox.org/git/20190318235356.GK29661@sigill.intra.peff.net/\n\nSigned-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>\n---\n Documentation/git-gc.txt | 29 +++++++++++++++++++++++++++--\n 1 file changed, 27 insertions(+), 2 deletions(-)\n\ndiff --git a/Documentation/git-gc.txt b/Documentation/git-gc.txt\nindex c037a46b09..165f05e999 100644\n--- a/Documentation/git-gc.txt\n+++ b/Documentation/git-gc.txt\n@@ -39,8 +39,7 @@ OPTIONS\n \tspace utilization and performance.  This option will cause\n \t'git gc' to more aggressively optimize the repository at the expense\n \tof taking much more time.  The effects of this optimization are\n-\tpersistent, so this option only needs to be used occasionally; every\n-\tfew hundred changesets or so.\n+\tmostly persistent. See the \"AGGRESSIVE\" section below for details.\n \n --auto::\n \tWith this option, 'git gc' checks whether any housekeeping is\n@@ -78,6 +77,32 @@ be performed as well.\n \t`.keep` files are consolidated into a single pack. When this\n \toption is used, `gc.bigPackThreshold` is ignored.\n \n+AGGRESSIVE\n+----------\n+\n+When the `--aggressive` option is supplied, linkgit:git-repack[1] will\n+be invoked with the `-f` flag, which in turn will pass\n+`--no-reuse-delta` to linkgit:git-pack-objects[1]. This will throw\n+away any existing deltas and re-compute them, at the expense of\n+spending much more time on the repacking.\n+\n+The effects of this are mostly persistent, e.g. when packs and loose\n+objects are coalesced into one another pack the existing deltas in\n+that pack might get re-used, but there are also various cases where we\n+might pick a sub-optimal delta from a newer pack instead.\n+\n+Furthermore, supplying `--aggressive` will tweak the `--depth` and\n+`--window` options passed to linkgit:git-repack[1]. See the\n+`gc.aggressiveDepth` and `gc.aggressiveWindow` settings below. By\n+using a larger window size we're more likely to find more optimal\n+deltas.\n+\n+It's probably not worth it to use this option on a given repository\n+without running tailored performance benchmarks on it. It takes a lot\n+more time, and the resulting space/delta optimization may or may not\n+be worth it. Not using this at all is the right trade-off for most\n+users and their repositories.\n+\n CONFIGURATION\n -------------\n \n-- \n2.21.0.360.g471c308f928\n\n"},{"id":"372229","messageId":"20190322093242.5508-12-avarab@gmail.com","threadId":"50772","inReplyTo":"20190321205054.17109-1-avarab@gmail.com","subject":"[PATCH v3 11/11] gc docs: remove incorrect reference to gc.auto=0","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2019-03-22T09:32:42Z","receivedAt":"2019-03-22T09:33:20Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"The chance of a repository being corrupted due to a \"gc\" has nothing\nto do with whether or not that \"gc\" was invoked via \"gc --auto\", but\nwhether there's other concurrent operations happening.\n\nThis is already noted earlier in the paragraph, so there's no reason\nto suggest this here. The user can infer from the rest of the\ndocumentation that \"gc\" will run automatically unless gc.auto=0 is\nset, and we shouldn't confuse the issue by implying that \"gc --auto\"\nis somehow more prone to produce corruption than a normal \"gc\".\n\nWell, it is in the sense that a blocking \"gc\" would stop you from\ndoing anything else in *that* particular terminal window, but users\nare likely to have another window, or to be worried about how\nconcurrent \"gc\" on a server might cause corruption.\n\nSigned-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>\n---\n Documentation/git-gc.txt | 3 +--\n 1 file changed, 1 insertion(+), 2 deletions(-)\n\ndiff --git a/Documentation/git-gc.txt b/Documentation/git-gc.txt\nindex 49aec5435b..2af503bdb1 100644\n--- a/Documentation/git-gc.txt\n+++ b/Documentation/git-gc.txt\n@@ -141,8 +141,7 @@ mitigate this problem:\n \n However, these features fall short of a complete solution, so users who\n run commands concurrently have to live with some risk of corruption (which\n-seems to be low in practice) unless they turn off automatic garbage\n-collection with 'git config gc.auto 0'.\n+seems to be low in practice).\n \n HOOKS\n -----\n-- \n2.21.0.360.g471c308f928\n\n"},{"id":"372230","messageId":"20190322093242.5508-10-avarab@gmail.com","threadId":"50772","inReplyTo":"20190321205054.17109-1-avarab@gmail.com","subject":"[PATCH v3 09/11] gc docs: note \"gc --aggressive\" in \"fast-import\"","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2019-03-22T09:32:40Z","receivedAt":"2019-03-22T09:33:21Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"Amend the \"PACKFILE OPTIMIZATION\" section in \"fast-import\" to explain\nthat simply running \"git gc --aggressive\" after a \"fast-import\" should\nproperly optimize the repository. This is simpler and more effective\nthan the existing \"repack\" advice (which I'm keeping as it helps\nexplain things) because it e.g. also packs the newly imported refs.\n\nSigned-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>\n---\n Documentation/git-fast-import.txt | 7 +++++++\n 1 file changed, 7 insertions(+)\n\ndiff --git a/Documentation/git-fast-import.txt b/Documentation/git-fast-import.txt\nindex 43ab3b1637..2248755cb7 100644\n--- a/Documentation/git-fast-import.txt\n+++ b/Documentation/git-fast-import.txt\n@@ -1396,6 +1396,13 @@ deltas are suboptimal (see above) then also adding the `-f` option\n to force recomputation of all deltas can significantly reduce the\n final packfile size (30-50% smaller can be quite typical).\n \n+Instead of running `git repack` you can also run `git gc\n+--aggressive`, which will also optimize other things after an import\n+(e.g. pack loose refs). As noted in the \"AGGRESSIVE\" section in\n+linkgit:git-gc[1] the `--aggressive` option will find new deltas with\n+the `-f` option to linkgit:git-repack[1]. For the reasons elaborated\n+on above using `--aggressive` after a fast-import is one of the few\n+cases where it's known to be worthwhile.\n \n MEMORY UTILIZATION\n ------------------\n-- \n2.21.0.360.g471c308f928\n\n"},{"id":"372809","messageId":"20190330180415.GC4047@pobox.com","threadId":"50772","inReplyTo":"20190322093242.5508-5-avarab@gmail.com","subject":"Re: [PATCH v3 04/11] gc docs: include the \"gc.*\" section from \"config\" in \"gc\"","fromName":"Todd Zullinger","fromEmail":"tmz@pobox.com","sentAt":"2019-03-30T18:04:15Z","receivedAt":"2019-03-30T18:04:26Z","isPatch":true,"sender":{"key":"tmz@pobox.com","avatar":"https://avatars.githubusercontent.com/u/806319?v=4"},"body":"Hi Ævar,\n\nÆvar Arnfjörð Bjarmason wrote:\n> diff --git a/Documentation/git-gc.txt b/Documentation/git-gc.txt\n> index 66386439b7..c037a46b09 100644\n> --- a/Documentation/git-gc.txt\n> +++ b/Documentation/git-gc.txt\n> @@ -45,28 +45,12 @@ OPTIONS\n>  --auto::\n>  \tWith this option, 'git gc' checks whether any housekeeping is\n>  \trequired; if not, it exits without performing any work.\n> -\tSome git commands run `git gc --auto` after performing\n> -\toperations that could create many loose objects. Housekeeping\n> -\tis required if there are too many loose objects or too many\n> -\tpacks in the repository.\n>  +\n> -If the number of loose objects exceeds the value of the `gc.auto`\n> -configuration variable, then all loose objects are combined into a\n> -single pack.  Setting the value of `gc.auto`\n> -to 0 disables automatic packing of loose objects.\n> +See the `gc.auto' option in the \"CONFIGURATION\" section below for how\n> +this heuristic works.\n\nDid you want this \"gc.auto\" to use the differing left and\nright accent/quote characters (which asciidoc renders as\nsingle-quotes and asciidoctor as double-quotes) or should\nthe closing \"'\" instead be \"`\" to render \"gc.auto\" as\nmonospaced text?\n\nI suspect it's the latter, as that matches most of the other\nvariable names in the docs.\n\nI noticed this while comparing the output from asciidoc and\nasciidoctor.  I've got a few similar changes queued up as\nminor fixes to lower the diff between asciidoc/tor but I\nwanted to check whether you intended this one before I sent\na patch to correct it. :)\n\nThanks,\n\n-- \nTodd\n"},{"id":"373355","messageId":"20190407195217.3607-1-avarab@gmail.com","threadId":"50772","inReplyTo":"20190322093242.5508-5-avarab@gmail.com","subject":"[PATCH v4 00/11] gc docs: modernize and fix the documentation","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2019-04-07T19:52:06Z","receivedAt":"2019-04-07T19:52:48Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"v4 fixes a misbalanced quote noted by Todd Zullinger in\n<20190330180415.GC4047@pobox.com>, and makes this equivalent to the\npost-squash version sitting in gitster/ab/gc-docs now.\n\nÆvar Arnfjörð Bjarmason (11):\n  gc docs: modernize the advice for manually running \"gc\"\n  gc docs: stop noting \"repack\" flags\n  gc docs: clean grammar for \"gc.bigPackThreshold\"\n  gc docs: include the \"gc.*\" section from \"config\" in \"gc\"\n  gc docs: re-flow the \"gc.*\" section in \"config\"\n  gc docs: fix formatting for \"gc.writeCommitGraph\"\n  gc docs: note how --aggressive impacts --window & --depth\n  gc docs: downplay the usefulness of --aggressive\n  gc docs: note \"gc --aggressive\" in \"fast-import\"\n  gc docs: clarify that \"gc\" doesn't throw away referenced objects\n  gc docs: remove incorrect reference to gc.auto=0\n\n Documentation/config/gc.txt       |  38 ++++++--\n Documentation/git-fast-import.txt |   7 ++\n Documentation/git-gc.txt          | 142 ++++++++++--------------------\n 3 files changed, 86 insertions(+), 101 deletions(-)\n\nRange-diff:\n 1:  a48ef8d5d8 =  1:  a48ef8d5d8 gc docs: modernize the advice for manually running \"gc\"\n 2:  21e66a7903 =  2:  21e66a7903 gc docs: stop noting \"repack\" flags\n 3:  c8a1342e34 =  3:  c8a1342e34 gc docs: clean grammar for \"gc.bigPackThreshold\"\n 4:  9163e2f885 !  4:  f54ef80e69 gc docs: include the \"gc.*\" section from \"config\" in \"gc\"\n    @@ -100,7 +100,7 @@\n     -configuration variable, then all loose objects are combined into a\n     -single pack.  Setting the value of `gc.auto`\n     -to 0 disables automatic packing of loose objects.\n    -+See the `gc.auto' option in the \"CONFIGURATION\" section below for how\n    ++See the `gc.auto` option in the \"CONFIGURATION\" section below for how\n     +this heuristic works.\n      +\n     -If the number of packs exceeds the value of `gc.autoPackLimit`,\n 5:  8fa0e26671 =  5:  4ea4cf885a gc docs: re-flow the \"gc.*\" section in \"config\"\n 6:  b70396f029 =  6:  ae5755278f gc docs: fix formatting for \"gc.writeCommitGraph\"\n 7:  04ee81a3c9 =  7:  fc3bd0d5f4 gc docs: note how --aggressive impacts --window & --depth\n 8:  04af0afbcf =  8:  7cff026e58 gc docs: downplay the usefulness of --aggressive\n 9:  c35bc94416 =  9:  64617c43f6 gc docs: note \"gc --aggressive\" in \"fast-import\"\n10:  702f2cd2d9 = 10:  84e5c669eb gc docs: clarify that \"gc\" doesn't throw away referenced objects\n11:  08af3cc3ee = 11:  6a027d25a7 gc docs: remove incorrect reference to gc.auto=0\n-- \n2.21.0.392.gf8f6787159e\n\n"},{"id":"373356","messageId":"20190407195217.3607-2-avarab@gmail.com","threadId":"50772","inReplyTo":"20190322093242.5508-5-avarab@gmail.com","subject":"[PATCH v4 01/11] gc docs: modernize the advice for manually running \"gc\"","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2019-04-07T19:52:07Z","receivedAt":"2019-04-07T19:52:48Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"The docs have been recommending that users need to run this manually,\nbut that hasn't been needed in practice for a long time except in\nexceptional circumstances.\n\nLet's instead have this reflect reality and say that most users don't\nneed to run this manually at all, while briefly describing the sorts\nsort of cases where \"gc\" does need to be run manually.\n\nSince we're recommending that users run this most of the and usually\ndon't need to tweak it, let's tone down the very prominent example of\nthe gc.auto=0 command. It's sufficient to point to the gc.auto\ndocumentation below.\n\nSigned-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>\n---\n Documentation/git-gc.txt | 21 ++++++++++-----------\n 1 file changed, 10 insertions(+), 11 deletions(-)\n\ndiff --git a/Documentation/git-gc.txt b/Documentation/git-gc.txt\nindex a7c1b0f60e..dd22eecc79 100644\n--- a/Documentation/git-gc.txt\n+++ b/Documentation/git-gc.txt\n@@ -20,17 +20,16 @@ created from prior invocations of 'git add', packing refs, pruning\n reflog, rerere metadata or stale working trees. May also update ancillary\n indexes such as the commit-graph.\n \n-Users are encouraged to run this task on a regular basis within\n-each repository to maintain good disk space utilization and good\n-operating performance.\n-\n-Some git commands may automatically run 'git gc'; see the `--auto` flag\n-below for details. If you know what you're doing and all you want is to\n-disable this behavior permanently without further considerations, just do:\n-\n-----------------------\n-$ git config --global gc.auto 0\n-----------------------\n+When common porcelain operations that create objects are run, they\n+will check whether the repository has grown substantially since the\n+last maintenance, and if so run `git gc` automatically. See `gc.auto`\n+below for how to disable this behavior.\n+\n+Running `git gc` manually should only be needed when adding objects to\n+a repository without regularly running such porcelain commands, to do\n+a one-off repository optimization, or e.g. to clean up a suboptimal\n+mass-import. See the \"PACKFILE OPTIMIZATION\" section in\n+linkgit:git-fast-import[1] for more details on the import case.\n \n OPTIONS\n -------\n-- \n2.21.0.392.gf8f6787159e\n\n"},{"id":"373357","messageId":"20190407195217.3607-3-avarab@gmail.com","threadId":"50772","inReplyTo":"20190322093242.5508-5-avarab@gmail.com","subject":"[PATCH v4 02/11] gc docs: stop noting \"repack\" flags","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2019-04-07T19:52:08Z","receivedAt":"2019-04-07T19:52:48Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"Remove the mention of specific flags from the \"gc\" documentation, and\nleave it at describing what we'll do instead. As seen in builtin/gc.c\nwe'll use various repack flags depending on what we detect we need to\ndo, so this isn't always accurate.\n\nMore importantly, a subsequent change is about to remove all this\ndocumentation and replace it with an include of the gc.* docs in\ngit-config(1). By first changing this it's easier to reason about that\nsubsequent change.\n\nSigned-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>\n---\n Documentation/git-gc.txt | 5 ++---\n 1 file changed, 2 insertions(+), 3 deletions(-)\n\ndiff --git a/Documentation/git-gc.txt b/Documentation/git-gc.txt\nindex dd22eecc79..c56f4f7cde 100644\n--- a/Documentation/git-gc.txt\n+++ b/Documentation/git-gc.txt\n@@ -52,14 +52,13 @@ OPTIONS\n +\n If the number of loose objects exceeds the value of the `gc.auto`\n configuration variable, then all loose objects are combined into a\n-single pack using `git repack -d -l`.  Setting the value of `gc.auto`\n+single pack.  Setting the value of `gc.auto`\n to 0 disables automatic packing of loose objects.\n +\n If the number of packs exceeds the value of `gc.autoPackLimit`,\n then existing packs (except those marked with a `.keep` file\n or over `gc.bigPackThreshold` limit)\n-are consolidated into a single pack by using the `-A` option of\n-'git repack'.\n+are consolidated into a single pack.\n If the amount of memory is estimated not enough for `git repack` to\n run smoothly and `gc.bigPackThreshold` is not set, the largest\n pack will also be excluded (this is the equivalent of running `git gc`\n-- \n2.21.0.392.gf8f6787159e\n\n"},{"id":"373358","messageId":"20190407195217.3607-4-avarab@gmail.com","threadId":"50772","inReplyTo":"20190322093242.5508-5-avarab@gmail.com","subject":"[PATCH v4 03/11] gc docs: clean grammar for \"gc.bigPackThreshold\"","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2019-04-07T19:52:09Z","receivedAt":"2019-04-07T19:52:48Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"Clean up the grammar in the documentation for\n\"gc.bigPackThreshold\". This documentation was added in 9806f5a7bf (\"gc\n--auto: exclude base pack if not enough mem to \"repack -ad\"\",\n2018-04-15).\n\nSaying \"the amount of memory estimated for\" flows more smoothly than\nthe previous \"the amount of memory is estimated not enough\".\n\nSuggested-by: Jeff King <peff@peff.net>\nSigned-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>\n---\n Documentation/git-gc.txt | 4 ++--\n 1 file changed, 2 insertions(+), 2 deletions(-)\n\ndiff --git a/Documentation/git-gc.txt b/Documentation/git-gc.txt\nindex c56f4f7cde..66386439b7 100644\n--- a/Documentation/git-gc.txt\n+++ b/Documentation/git-gc.txt\n@@ -59,8 +59,8 @@ If the number of packs exceeds the value of `gc.autoPackLimit`,\n then existing packs (except those marked with a `.keep` file\n or over `gc.bigPackThreshold` limit)\n are consolidated into a single pack.\n-If the amount of memory is estimated not enough for `git repack` to\n-run smoothly and `gc.bigPackThreshold` is not set, the largest\n+If the amount of memory estimated for `git repack` to run smoothly is\n+not available and `gc.bigPackThreshold` is not set, the largest\n pack will also be excluded (this is the equivalent of running `git gc`\n with `--keep-base-pack`).\n Setting `gc.autoPackLimit` to 0 disables automatic consolidation of\n-- \n2.21.0.392.gf8f6787159e\n\n"},{"id":"373359","messageId":"20190407195217.3607-5-avarab@gmail.com","threadId":"50772","inReplyTo":"20190322093242.5508-5-avarab@gmail.com","subject":"[PATCH v4 04/11] gc docs: include the \"gc.*\" section from \"config\" in \"gc\"","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2019-04-07T19:52:10Z","receivedAt":"2019-04-07T19:52:48Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"Rather than duplicating the documentation for the various \"gc\" options\nlet's include the \"gc\" docs from git-config. They were mostly better\nalready, and now we don't have the same docs in two places with subtly\ndifferent wording.\n\nIn the cases where the git-gc(1) docs were saying something the \"gc\"\ndocs in git-config(1) didn't cover move the relevant section over to\nthe git-config(1) docs.\n\nSigned-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>\n---\n Documentation/config/gc.txt | 29 ++++++++++++-\n Documentation/git-gc.txt    | 86 +++----------------------------------\n 2 files changed, 35 insertions(+), 80 deletions(-)\n\ndiff --git a/Documentation/config/gc.txt b/Documentation/config/gc.txt\nindex c6fbb8a96f..a255ae67b0 100644\n--- a/Documentation/config/gc.txt\n+++ b/Documentation/config/gc.txt\n@@ -2,24 +2,39 @@ gc.aggressiveDepth::\n \tThe depth parameter used in the delta compression\n \talgorithm used by 'git gc --aggressive'.  This defaults\n \tto 50.\n++\n+See the documentation for the `--depth` option in\n+linkgit:git-repack[1] for more details.\n \n gc.aggressiveWindow::\n \tThe window size parameter used in the delta compression\n \talgorithm used by 'git gc --aggressive'.  This defaults\n \tto 250.\n++\n+See the documentation for the `--window` option in\n+linkgit:git-repack[1] for more details.\n \n gc.auto::\n \tWhen there are approximately more than this many loose\n \tobjects in the repository, `git gc --auto` will pack them.\n \tSome Porcelain commands use this command to perform a\n \tlight-weight garbage collection from time to time.  The\n-\tdefault value is 6700.  Setting this to 0 disables it.\n+\tdefault value is 6700.\n++\n+Setting this to 0 disables not only automatic packing based on the\n+number of loose objects, but any other heuristic `git gc --auto` will\n+otherwise use to determine if there's work to do, such as\n+`gc.autoPackLimit`.\n \n gc.autoPackLimit::\n \tWhen there are more than this many packs that are not\n \tmarked with `*.keep` file in the repository, `git gc\n \t--auto` consolidates them into one larger pack.  The\n \tdefault\tvalue is 50.  Setting this to 0 disables it.\n+\tSetting `gc.auto` to 0 will also disable this.\n++\n+See the `gc.bigPackThreshold` configuration variable below. When in\n+use, it'll affect how the auto pack limit works.\n \n gc.autoDetach::\n \tMake `git gc --auto` return immediately and run in background\n@@ -36,6 +51,11 @@ Note that if the number of kept packs is more than gc.autoPackLimit,\n this configuration variable is ignored, all packs except the base pack\n will be repacked. After this the number of packs should go below\n gc.autoPackLimit and gc.bigPackThreshold should be respected again.\n++\n+If the amount of memory estimated for `git repack` to run smoothly is\n+not available and `gc.bigPackThreshold` is not set, the largest\n+pack will also be excluded (this is the equivalent of running `git gc`\n+with `--keep-base-pack`).\n \n gc.writeCommitGraph::\n \tIf true, then gc will rewrite the commit-graph file when\n@@ -94,6 +114,13 @@ gc.<pattern>.reflogExpireUnreachable::\n \tWith \"<pattern>\" (e.g. \"refs/stash\")\n \tin the middle, the setting applies only to the refs that\n \tmatch the <pattern>.\n++\n+These types of entries are generally created as\n+a result of using `git commit --amend` or `git rebase` and are the\n+commits prior to the amend or rebase occurring.  Since these changes\n+are not part of the current project most users will want to expire\n+them sooner, which is why the default is more aggressive than\n+`gc.reflogExpire`.\n \n gc.rerereResolved::\n \tRecords of conflicted merge you resolved earlier are\ndiff --git a/Documentation/git-gc.txt b/Documentation/git-gc.txt\nindex 66386439b7..37c4d26a76 100644\n--- a/Documentation/git-gc.txt\n+++ b/Documentation/git-gc.txt\n@@ -45,28 +45,12 @@ OPTIONS\n --auto::\n \tWith this option, 'git gc' checks whether any housekeeping is\n \trequired; if not, it exits without performing any work.\n-\tSome git commands run `git gc --auto` after performing\n-\toperations that could create many loose objects. Housekeeping\n-\tis required if there are too many loose objects or too many\n-\tpacks in the repository.\n +\n-If the number of loose objects exceeds the value of the `gc.auto`\n-configuration variable, then all loose objects are combined into a\n-single pack.  Setting the value of `gc.auto`\n-to 0 disables automatic packing of loose objects.\n+See the `gc.auto` option in the \"CONFIGURATION\" section below for how\n+this heuristic works.\n +\n-If the number of packs exceeds the value of `gc.autoPackLimit`,\n-then existing packs (except those marked with a `.keep` file\n-or over `gc.bigPackThreshold` limit)\n-are consolidated into a single pack.\n-If the amount of memory estimated for `git repack` to run smoothly is\n-not available and `gc.bigPackThreshold` is not set, the largest\n-pack will also be excluded (this is the equivalent of running `git gc`\n-with `--keep-base-pack`).\n-Setting `gc.autoPackLimit` to 0 disables automatic consolidation of\n-packs.\n-+\n-If houskeeping is required due to many loose objects or packs, all\n+Once housekeeping is triggered by exceeding the limits of\n+configuration options such as `gc.auto` and `gc.autoPackLimit`, all\n other housekeeping tasks (e.g. rerere, working trees, reflog...) will\n be performed as well.\n \n@@ -97,66 +81,10 @@ be performed as well.\n CONFIGURATION\n -------------\n \n-The optional configuration variable `gc.reflogExpire` can be\n-set to indicate how long historical entries within each branch's\n-reflog should remain available in this repository.  The setting is\n-expressed as a length of time, for example '90 days' or '3 months'.\n-It defaults to '90 days'.\n-\n-The optional configuration variable `gc.reflogExpireUnreachable`\n-can be set to indicate how long historical reflog entries which\n-are not part of the current branch should remain available in\n-this repository.  These types of entries are generally created as\n-a result of using `git commit --amend` or `git rebase` and are the\n-commits prior to the amend or rebase occurring.  Since these changes\n-are not part of the current project most users will want to expire\n-them sooner.  This option defaults to '30 days'.\n-\n-The above two configuration variables can be given to a pattern.  For\n-example, this sets non-default expiry values only to remote-tracking\n-branches:\n-\n-------------\n-[gc \"refs/remotes/*\"]\n-\treflogExpire = never\n-\treflogExpireUnreachable = 3 days\n-------------\n-\n-The optional configuration variable `gc.rerereResolved` indicates\n-how long records of conflicted merge you resolved earlier are\n-kept.  This defaults to 60 days.\n-\n-The optional configuration variable `gc.rerereUnresolved` indicates\n-how long records of conflicted merge you have not resolved are\n-kept.  This defaults to 15 days.\n-\n-The optional configuration variable `gc.packRefs` determines if\n-'git gc' runs 'git pack-refs'. This can be set to \"notbare\" to enable\n-it within all non-bare repos or it can be set to a boolean value.\n-This defaults to true.\n-\n-The optional configuration variable `gc.writeCommitGraph` determines if\n-'git gc' should run 'git commit-graph write'. This can be set to a\n-boolean value. This defaults to false.\n-\n-The optional configuration variable `gc.aggressiveWindow` controls how\n-much time is spent optimizing the delta compression of the objects in\n-the repository when the --aggressive option is specified.  The larger\n-the value, the more time is spent optimizing the delta compression.  See\n-the documentation for the --window option in linkgit:git-repack[1] for\n-more details.  This defaults to 250.\n-\n-Similarly, the optional configuration variable `gc.aggressiveDepth`\n-controls --depth option in linkgit:git-repack[1]. This defaults to 50.\n-\n-The optional configuration variable `gc.pruneExpire` controls how old\n-the unreferenced loose objects have to be before they are pruned.  The\n-default is \"2 weeks ago\".\n-\n-Optional configuration variable `gc.worktreePruneExpire` controls how\n-old a stale working tree should be before `git worktree prune` deletes\n-it. Default is \"3 months ago\".\n+The below documentation is the same as what's found in\n+linkgit:git-config[1]:\n \n+include::config/gc.txt[]\n \n NOTES\n -----\n-- \n2.21.0.392.gf8f6787159e\n\n"},{"id":"373360","messageId":"20190407195217.3607-6-avarab@gmail.com","threadId":"50772","inReplyTo":"20190322093242.5508-5-avarab@gmail.com","subject":"[PATCH v4 05/11] gc docs: re-flow the \"gc.*\" section in \"config\"","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2019-04-07T19:52:11Z","receivedAt":"2019-04-07T19:52:48Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"Re-flow the \"gc.*\" section in \"config\". A previous commit moved this\nover from the \"gc\" docs, but tried to keep as many of the lines\nidentical to benefit from diff's move detection.\n\nSigned-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>\n---\n Documentation/config/gc.txt | 17 ++++++++---------\n 1 file changed, 8 insertions(+), 9 deletions(-)\n\ndiff --git a/Documentation/config/gc.txt b/Documentation/config/gc.txt\nindex a255ae67b0..3e7fc052d9 100644\n--- a/Documentation/config/gc.txt\n+++ b/Documentation/config/gc.txt\n@@ -53,9 +53,9 @@ will be repacked. After this the number of packs should go below\n gc.autoPackLimit and gc.bigPackThreshold should be respected again.\n +\n If the amount of memory estimated for `git repack` to run smoothly is\n-not available and `gc.bigPackThreshold` is not set, the largest\n-pack will also be excluded (this is the equivalent of running `git gc`\n-with `--keep-base-pack`).\n+not available and `gc.bigPackThreshold` is not set, the largest pack\n+will also be excluded (this is the equivalent of running `git gc` with\n+`--keep-base-pack`).\n \n gc.writeCommitGraph::\n \tIf true, then gc will rewrite the commit-graph file when\n@@ -115,12 +115,11 @@ gc.<pattern>.reflogExpireUnreachable::\n \tin the middle, the setting applies only to the refs that\n \tmatch the <pattern>.\n +\n-These types of entries are generally created as\n-a result of using `git commit --amend` or `git rebase` and are the\n-commits prior to the amend or rebase occurring.  Since these changes\n-are not part of the current project most users will want to expire\n-them sooner, which is why the default is more aggressive than\n-`gc.reflogExpire`.\n+These types of entries are generally created as a result of using `git\n+commit --amend` or `git rebase` and are the commits prior to the amend\n+or rebase occurring.  Since these changes are not part of the current\n+project most users will want to expire them sooner, which is why the\n+default is more aggressive than `gc.reflogExpire`.\n \n gc.rerereResolved::\n \tRecords of conflicted merge you resolved earlier are\n-- \n2.21.0.392.gf8f6787159e\n\n"},{"id":"373361","messageId":"20190407195217.3607-7-avarab@gmail.com","threadId":"50772","inReplyTo":"20190322093242.5508-5-avarab@gmail.com","subject":"[PATCH v4 06/11] gc docs: fix formatting for \"gc.writeCommitGraph\"","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2019-04-07T19:52:12Z","receivedAt":"2019-04-07T19:52:48Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"Change the AsciiDoc formatting so that an example of \"gc --auto\" isn't\nrendered as \"git-gc(1) --auto\", but as \"git gc --auto\". This is\nconsistent with the rest of the links and command examples in this\ndocumentation.\n\nThe formatting I'm changing was initially introduced in\nd5d5d7b641 (\"gc: automatically write commit-graph files\", 2018-06-27).\n\nSigned-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>\n---\n Documentation/config/gc.txt | 4 ++--\n 1 file changed, 2 insertions(+), 2 deletions(-)\n\ndiff --git a/Documentation/config/gc.txt b/Documentation/config/gc.txt\nindex 3e7fc052d9..56918a5008 100644\n--- a/Documentation/config/gc.txt\n+++ b/Documentation/config/gc.txt\n@@ -59,8 +59,8 @@ will also be excluded (this is the equivalent of running `git gc` with\n \n gc.writeCommitGraph::\n \tIf true, then gc will rewrite the commit-graph file when\n-\tlinkgit:git-gc[1] is run. When using linkgit:git-gc[1]\n-\t'--auto' the commit-graph will be updated if housekeeping is\n+\tlinkgit:git-gc[1] is run. When using `git gc --auto`\n+\tthe commit-graph will be updated if housekeeping is\n \trequired. Default is false. See linkgit:git-commit-graph[1]\n \tfor details.\n \n-- \n2.21.0.392.gf8f6787159e\n\n"},{"id":"373362","messageId":"20190407195217.3607-8-avarab@gmail.com","threadId":"50772","inReplyTo":"20190322093242.5508-5-avarab@gmail.com","subject":"[PATCH v4 07/11] gc docs: note how --aggressive impacts --window & --depth","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2019-04-07T19:52:13Z","receivedAt":"2019-04-07T19:52:50Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"Since 07e7dbf0db (gc: default aggressive depth to 50, 2016-08-11) we\nsomewhat confusingly use the same depth under --aggressive as we do by\ndefault.\n\nAs noted in that commit that makes sense, it was wrong to make more\ndepth the default for \"aggressive\", and thus save disk space at the\nexpense of runtime performance, which is usually the opposite of\nsomeone who'd like \"aggressive gc\" wants.\n\nBut that's left us with a mostly-redundant configuration variable, so\nlet's clearly note in its documentation that it doesn't change the\ndefault.\n\nSigned-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>\n---\n Documentation/config/gc.txt | 6 ++++--\n 1 file changed, 4 insertions(+), 2 deletions(-)\n\ndiff --git a/Documentation/config/gc.txt b/Documentation/config/gc.txt\nindex 56918a5008..f732fe5bfd 100644\n--- a/Documentation/config/gc.txt\n+++ b/Documentation/config/gc.txt\n@@ -1,7 +1,8 @@\n gc.aggressiveDepth::\n \tThe depth parameter used in the delta compression\n \talgorithm used by 'git gc --aggressive'.  This defaults\n-\tto 50.\n+\tto 50, which is the default for the `--depth` option when\n+\t`--aggressive` isn't in use.\n +\n See the documentation for the `--depth` option in\n linkgit:git-repack[1] for more details.\n@@ -9,7 +10,8 @@ linkgit:git-repack[1] for more details.\n gc.aggressiveWindow::\n \tThe window size parameter used in the delta compression\n \talgorithm used by 'git gc --aggressive'.  This defaults\n-\tto 250.\n+\tto 250, which is a much more aggressive window size than\n+\tthe default `--window` of 10.\n +\n See the documentation for the `--window` option in\n linkgit:git-repack[1] for more details.\n-- \n2.21.0.392.gf8f6787159e\n\n"},{"id":"373363","messageId":"20190407195217.3607-10-avarab@gmail.com","threadId":"50772","inReplyTo":"20190322093242.5508-5-avarab@gmail.com","subject":"[PATCH v4 09/11] gc docs: note \"gc --aggressive\" in \"fast-import\"","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2019-04-07T19:52:15Z","receivedAt":"2019-04-07T19:52:52Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"Amend the \"PACKFILE OPTIMIZATION\" section in \"fast-import\" to explain\nthat simply running \"git gc --aggressive\" after a \"fast-import\" should\nproperly optimize the repository. This is simpler and more effective\nthan the existing \"repack\" advice (which I'm keeping as it helps\nexplain things) because it e.g. also packs the newly imported refs.\n\nSigned-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>\n---\n Documentation/git-fast-import.txt | 7 +++++++\n 1 file changed, 7 insertions(+)\n\ndiff --git a/Documentation/git-fast-import.txt b/Documentation/git-fast-import.txt\nindex 43ab3b1637..2248755cb7 100644\n--- a/Documentation/git-fast-import.txt\n+++ b/Documentation/git-fast-import.txt\n@@ -1396,6 +1396,13 @@ deltas are suboptimal (see above) then also adding the `-f` option\n to force recomputation of all deltas can significantly reduce the\n final packfile size (30-50% smaller can be quite typical).\n \n+Instead of running `git repack` you can also run `git gc\n+--aggressive`, which will also optimize other things after an import\n+(e.g. pack loose refs). As noted in the \"AGGRESSIVE\" section in\n+linkgit:git-gc[1] the `--aggressive` option will find new deltas with\n+the `-f` option to linkgit:git-repack[1]. For the reasons elaborated\n+on above using `--aggressive` after a fast-import is one of the few\n+cases where it's known to be worthwhile.\n \n MEMORY UTILIZATION\n ------------------\n-- \n2.21.0.392.gf8f6787159e\n\n"},{"id":"373364","messageId":"20190407195217.3607-11-avarab@gmail.com","threadId":"50772","inReplyTo":"20190322093242.5508-5-avarab@gmail.com","subject":"[PATCH v4 10/11] gc docs: clarify that \"gc\" doesn't throw away referenced objects","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2019-04-07T19:52:16Z","receivedAt":"2019-04-07T19:52:55Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"Amend the \"NOTES\" section to fix up wording that's been with us since\n3ffb58be0a (\"doc/git-gc: add a note about what is collected\",\n2008-04-23).\n\nI can't remember when/where anymore (I think Freenode #Git), but at\nsome point I was having a conversation with someone who was convinced\nthat \"gc\" would prune things only referenced by e.g. refs/pull/*, and\npointed to this section as proof.\n\nIt turned out that they'd read the \"branches and tags\" wording here\nand thought just refs/{heads,tags}/* and refs/remotes/* etc. would be\nkept, which is what we enumerate explicitly.\n\nSo let's say \"other refs\", even though just above we say \"objects that\nare referenced anywhere in your repository\".\n\nSigned-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>\n---\n Documentation/git-gc.txt | 4 ++--\n 1 file changed, 2 insertions(+), 2 deletions(-)\n\ndiff --git a/Documentation/git-gc.txt b/Documentation/git-gc.txt\nindex 5e80f306e7..9cdae588fb 100644\n--- a/Documentation/git-gc.txt\n+++ b/Documentation/git-gc.txt\n@@ -119,8 +119,8 @@ anywhere in your repository. In\n particular, it will keep not only objects referenced by your current set\n of branches and tags, but also objects referenced by the index,\n remote-tracking branches, refs saved by 'git filter-branch' in\n-refs/original/, or reflogs (which may reference commits in branches\n-that were later amended or rewound).\n+refs/original/, reflogs (which may reference commits in branches\n+that were later amended or rewound), and anything else in the refs/* namespace.\n If you are expecting some objects to be deleted and they aren't, check\n all of those locations and decide whether it makes sense in your case to\n remove those references.\n-- \n2.21.0.392.gf8f6787159e\n\n"},{"id":"373365","messageId":"20190407195217.3607-12-avarab@gmail.com","threadId":"50772","inReplyTo":"20190322093242.5508-5-avarab@gmail.com","subject":"[PATCH v4 11/11] gc docs: remove incorrect reference to gc.auto=0","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2019-04-07T19:52:17Z","receivedAt":"2019-04-07T19:52:55Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"The chance of a repository being corrupted due to a \"gc\" has nothing\nto do with whether or not that \"gc\" was invoked via \"gc --auto\", but\nwhether there's other concurrent operations happening.\n\nThis is already noted earlier in the paragraph, so there's no reason\nto suggest this here. The user can infer from the rest of the\ndocumentation that \"gc\" will run automatically unless gc.auto=0 is\nset, and we shouldn't confuse the issue by implying that \"gc --auto\"\nis somehow more prone to produce corruption than a normal \"gc\".\n\nWell, it is in the sense that a blocking \"gc\" would stop you from\ndoing anything else in *that* particular terminal window, but users\nare likely to have another window, or to be worried about how\nconcurrent \"gc\" on a server might cause corruption.\n\nSigned-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>\n---\n Documentation/git-gc.txt | 3 +--\n 1 file changed, 1 insertion(+), 2 deletions(-)\n\ndiff --git a/Documentation/git-gc.txt b/Documentation/git-gc.txt\nindex 9cdae588fb..247f765604 100644\n--- a/Documentation/git-gc.txt\n+++ b/Documentation/git-gc.txt\n@@ -141,8 +141,7 @@ mitigate this problem:\n \n However, these features fall short of a complete solution, so users who\n run commands concurrently have to live with some risk of corruption (which\n-seems to be low in practice) unless they turn off automatic garbage\n-collection with 'git config gc.auto 0'.\n+seems to be low in practice).\n \n HOOKS\n -----\n-- \n2.21.0.392.gf8f6787159e\n\n"},{"id":"373366","messageId":"20190407195217.3607-9-avarab@gmail.com","threadId":"50772","inReplyTo":"20190322093242.5508-5-avarab@gmail.com","subject":"[PATCH v4 08/11] gc docs: downplay the usefulness of --aggressive","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2019-04-07T19:52:14Z","receivedAt":"2019-04-07T19:52:56Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"The existing \"gc --aggressive\" docs come just short of recommending to\nusers that they run it regularly. I've personally talked to many users\nwho've taken these docs as an advice to use this option, and have,\nusually it's (mostly) a waste of time.\n\nSo let's clarify what it really does, and let the user draw their own\nconclusions.\n\nLet's also clarify the \"The effects [...] are persistent\" to\nparaphrase a brief version of Jeff King's explanation at [1].\n\n1. https://public-inbox.org/git/20190318235356.GK29661@sigill.intra.peff.net/\n\nSigned-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>\n---\n Documentation/git-gc.txt | 29 +++++++++++++++++++++++++++--\n 1 file changed, 27 insertions(+), 2 deletions(-)\n\ndiff --git a/Documentation/git-gc.txt b/Documentation/git-gc.txt\nindex 37c4d26a76..5e80f306e7 100644\n--- a/Documentation/git-gc.txt\n+++ b/Documentation/git-gc.txt\n@@ -39,8 +39,7 @@ OPTIONS\n \tspace utilization and performance.  This option will cause\n \t'git gc' to more aggressively optimize the repository at the expense\n \tof taking much more time.  The effects of this optimization are\n-\tpersistent, so this option only needs to be used occasionally; every\n-\tfew hundred changesets or so.\n+\tmostly persistent. See the \"AGGRESSIVE\" section below for details.\n \n --auto::\n \tWith this option, 'git gc' checks whether any housekeeping is\n@@ -78,6 +77,32 @@ be performed as well.\n \t`.keep` files are consolidated into a single pack. When this\n \toption is used, `gc.bigPackThreshold` is ignored.\n \n+AGGRESSIVE\n+----------\n+\n+When the `--aggressive` option is supplied, linkgit:git-repack[1] will\n+be invoked with the `-f` flag, which in turn will pass\n+`--no-reuse-delta` to linkgit:git-pack-objects[1]. This will throw\n+away any existing deltas and re-compute them, at the expense of\n+spending much more time on the repacking.\n+\n+The effects of this are mostly persistent, e.g. when packs and loose\n+objects are coalesced into one another pack the existing deltas in\n+that pack might get re-used, but there are also various cases where we\n+might pick a sub-optimal delta from a newer pack instead.\n+\n+Furthermore, supplying `--aggressive` will tweak the `--depth` and\n+`--window` options passed to linkgit:git-repack[1]. See the\n+`gc.aggressiveDepth` and `gc.aggressiveWindow` settings below. By\n+using a larger window size we're more likely to find more optimal\n+deltas.\n+\n+It's probably not worth it to use this option on a given repository\n+without running tailored performance benchmarks on it. It takes a lot\n+more time, and the resulting space/delta optimization may or may not\n+be worth it. Not using this at all is the right trade-off for most\n+users and their repositories.\n+\n CONFIGURATION\n -------------\n \n-- \n2.21.0.392.gf8f6787159e\n\n"},{"id":"374942","messageId":"878svjj4t5.fsf@evledraar.gmail.com","threadId":"50772","inReplyTo":"20190319001829.GL29661@sigill.intra.peff.net","subject":"Re: [PATCH 0/4] gc docs: modernize and fix the documentation","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2019-05-06T09:44:06Z","receivedAt":"2019-05-06T09:44:15Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"\nOn Tue, Mar 19 2019, Jeff King wrote:\n\n> On Mon, Mar 18, 2019 at 11:45:39PM +0100, Ævar Arnfjörð Bjarmason wrote:\n>\n>> > I don't think the quarantine stuff should impact contention at all. It's\n>> > only quarantining the objects, which are the least contentious part of\n>> > Git (because object content is idempotent, so we don't do any locking\n>> > there, and with two racing processes, one will just \"win\").\n>>\n>> Without the quarantine, isn't there the race that the NOTES section\n>> talks about (unless I've misread it).\n>\n> Ah, OK, I wasn't quite sure which documentation you were talking about.\n> I see the discussion now in the \"NOTES\" section of git-gc(1).\n>\n>> I.e. we have some loose object \"ABCD\" not referrred to by anything for\n>> the last 2 weeks, as we're gc-ing a ref update comes in that makes it\n>> referenced again. We then delete \"ABCD\" (not used!) at the same time the\n>> ref update happens, and get corruption.\n>>\n>> Whereas the quarantine might work around since the client will have sent\n>> ABCD with no reference pointing to it to the server in the temp pack,\n>> which we then rename in-place and then update the ref, so we don't care\n>> if \"ABCD\" goes away.\n>\n> tl;dr I don't think quarantine impacts this, but if you really want gory\n> details, read on.\n>\n> This is a problem with or without the quarantine. It's fundamentally a\n> race because we do not atomically say \"is anybody using X? If not, we\n> can delete it\" and some other process saying \"I'd like to use X\".\n>\n> Pushes are actually better off than most operations, because we only\n> advertise what's reachable, and the client is expected to send\n> everything else. So with just a normal update-ref call, we could race\n> like this:\n>\n>   1. ABCD is ancient.\n>\n>   2. Process 1 (update-ref) wants to reference ABCD. It sees that we\n>      have it.\n>\n>   3. Process 2 (gc/prune) sees that nobody references it. It deletes\n>      ABCD.\n>\n>   4. Process 1 writes out the reference.\n>\n> That doesn't happen with a push, because the server never would have\n> told the client that it has ABCD in the first place (so process 1 here\n> is the client). That is true with or without quarantine.\n>\n> But pushes aren't foolproof either. You said \"loose object ABCD not\n> referred t oby anything for the last 2 weeks\". But that's not exactly\n> how it works. It's \"object with an mtime of more than 2 weeks which is\n> not currently referenced\". So imagine a sequence like:\n>\n>   1. ABCD is ancient.\n>\n>   2. refs/heads/foo points to ABCD.\n>\n>   3. Server receive-pack advertises foo pointing to ABCD.\n>\n>   4. Simultaneous process on the server deletes refs/heads/foo (or\n>      perhaps somebody force-pushes over it).\n>\n>   5. Client prepares and sends pack without ABCD.\n>\n>   6. Server receive-pack checks that yes, we still have ABCD (i.e., the\n>      usual connectivity check).\n>\n>   7. Server gc drops ABCD, which is now unreachable (reflogs can help\n>      here, if you've enabled them; but we do delete reflogs when the\n>      branch is deleted).\n>\n>   8. Server receive-pack writes corrupt repo mentioning ABCD.\n>\n> That's a lot more steps, though they might not be as implausible as you\n> think (e.g., consider moving \"refs/heads/foo\" to \"refs/heads/bar\" in a\n> single push; that's actually a delete and an update, which is all you\n> need to race with a simultaneous gc).\n>\n> I have no idea how often this happens in practice. My subjective\n> recollection is that most of the race corruptions I've seen were from\n> local operations on the server. E.g., we compute a tentative merge for\n> somebody's pull request which shares objects with an older tentative\n> merge. They click the \"merge\" button and we reference that commit, which\n> is recent, but unbeknownst to us, while we were creating our new\n> tentative merge, a \"gc\" was deleting the old one.\n>\n> We're sometimes saved by the \"transitive freshness\" rules in d3038d22f9\n> (prune: keep objects reachable from recent objects, 2014-10-15).  But\n> they're far from perfect:\n>\n>  - some operations (like the push rename example) aren't writing new\n>    objects, so the ref write _is_ the moment that gc would find out\n>    something is reachable\n>\n>  - the \"is it reachable?\" and \"no, then delete it\" steps aren't atomic.\n>    Unless you want a whole-repo stop-the-world lock, somebody can\n>    reference the object in between. And since it may take many seconds\n>    to compute reachability, stop-the-world is not great.\n>\n> I think there are probably ways to make it better. Perhaps some kind of\n> lockless delete-but-be-able-to-rollback scheme (but keep in mind this\n> has to be implemented no top of POSIX filesystem semantics). Or even\n> just a \"compute reachability, mark for deletion, and then hold a\n> stop-the-world lock briefly to double-check that our reachability is\n> still up to date\".\n>\n> At least those seem plausible to me. I've never worked out the details,\n> and our solution was to just stop deleting objects during routine\n> maintenance (using \"repack -adk\"). We do still occasionally prune\n> manually (e.g., when somebody writes to support to remove a confidential\n> mistake).\n>\n> Anyway, that was more than you probably wanted to know. The short of it\n> is that I don't think quarantines help (they may even make things worse\n> by slightly increasing the length of the race window, though in practice\n> I doubt it).\n>\n>> Unless that interacts racily with the receive.unpackLimit, but then I\n>> have no idea that section is trying to say...\n>\n> No, I don't think unpackLimit really affects it much either way.\n>\n>> Also, surely the part where \"NOTES\" says something to the effect of \"you\n>> are subject to races unless gc.auto=0\" is wrong. To the extent that\n>> there's races it won't matter that you invoke \"git gc\" or \"git gc\n>> --auto\", it's the concurrency that matters. So if there's still races we\n>> should be saying the repo needs to be locked for writes for the duration\n>> of the \"gc\".\n>\n> Correct. It's the very act of pruning that is problematic. I think the\n> point is that if you are manually running \"git gc\", you'd presumably do\n> it at a time when the repository is not otherwise active.\n\nThanks for that E-Mail. I'm hoping to get around to another set of \"gc\ndoc\" updates and will hopefully be able to steal liberally from it.\n\nMaybe there's some case I haven't thought of that makes this stupid, but\nI wonder if something like a \"gc quarantine\" might be a fix fo both of\nthe the issues you noted above.\n\nI.e. it seems to me that the main issue is that we conflate \"mtime 2\nweeks old because it's unreferenced for 2 weeks\" v.s. \"mtime 2 weeks old\nbecause we haven't gotten around to a 'gc'\".\n\nSo in such a \"gc quarantine\" mode when we discover an object/pack that's\nunreachable/purely made up of unreachable objects we'd move the relevant\nloose object/\"loose\" pack to such a quarantine, which would just be\n.git/unreferenced-objects/{??,pack}/ or whatever.\n\nAFAICT both cases you mentioned above would be mitigated by this because\nwe'd no longer conflate \"haven't gc'd this yet and it's 2 weeks old\"\nv.s. \"hasn't been referenced in 2 weeks\".\n\nI started looking at this initially because I was wondering if the\n--keep-unreachable mode you modified in e26a8c4721 (\"repack: extend\n--keep-unreachable to loose objects\", 2016-06-13) could be made to write\nout such \"unreferenced\" objects into their *own* pack, so we could\ndelete them all at once as a batch, and wouldn't create the \"ref\nexplosions\" mentioned in [1].\n\nBut of course without an accompanying quarantine described above doing\nthat would just make this race condition worse.\n\n1. https://public-inbox.org/git/87fu6bmr0j.fsf@evledraar.gmail.com/\n"},{"id":"374996","messageId":"20190507075158.GG28060@sigill.intra.peff.net","threadId":"50772","inReplyTo":"878svjj4t5.fsf@evledraar.gmail.com","subject":"Re: [PATCH 0/4] gc docs: modernize and fix the documentation","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2019-05-07T07:51:58Z","receivedAt":"2019-05-07T07:52:02Z","isPatch":true,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Mon, May 06, 2019 at 11:44:06AM +0200, Ævar Arnfjörð Bjarmason wrote:\n\n> Maybe there's some case I haven't thought of that makes this stupid, but\n> I wonder if something like a \"gc quarantine\" might be a fix fo both of\n> the the issues you noted above.\n> \n> I.e. it seems to me that the main issue is that we conflate \"mtime 2\n> weeks old because it's unreferenced for 2 weeks\" v.s. \"mtime 2 weeks old\n> because we haven't gotten around to a 'gc'\".\n> \n> So in such a \"gc quarantine\" mode when we discover an object/pack that's\n> unreachable/purely made up of unreachable objects we'd move the relevant\n> loose object/\"loose\" pack to such a quarantine, which would just be\n> .git/unreferenced-objects/{??,pack}/ or whatever.\n> \n> AFAICT both cases you mentioned above would be mitigated by this because\n> we'd no longer conflate \"haven't gc'd this yet and it's 2 weeks old\"\n> v.s. \"hasn't been referenced in 2 weeks\".\n\nMichael Haggerty and I have (off-list) discussed variations on that, but\nit opens up a lot of new issues.  Moving something into quarantine isn't\natomic. So you've still corrupted the repo, but now it's recoverable by\nreaching into the quarantine. Who notices that the repo is corrupt, and\nhow? When do we expire objects from quarantine?\n\nI think the heart of the issue is really the lack of atomicity in the\noperations. You need some way to mark \"I am using this now\" in a way\nthat cannot race with \"looks like nobody is using this, so I'll delete\nit\".\n\nAnd ideally without traversing large bits of the graph on the writing\nside, and without requiring any stop-the-world locks during pruning.\n\n> I started looking at this initially because I was wondering if the\n> --keep-unreachable mode you modified in e26a8c4721 (\"repack: extend\n> --keep-unreachable to loose objects\", 2016-06-13) could be made to write\n> out such \"unreferenced\" objects into their *own* pack, so we could\n> delete them all at once as a batch, and wouldn't create the \"ref\n> explosions\" mentioned in [1].\n> \n> But of course without an accompanying quarantine described above doing\n> that would just make this race condition worse.\n\nI'm not sure it really makes it worse. The pack would have the same\nmtime as the loose objects would.\n\n-Peff\n"},{"id":"375272","messageId":"8736lnxlig.fsf@evledraar.gmail.com","threadId":"50772","inReplyTo":"20190507075158.GG28060@sigill.intra.peff.net","subject":"Re: [PATCH 0/4] gc docs: modernize and fix the documentation","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2019-05-09T23:20:55Z","receivedAt":"2019-05-09T23:21:02Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"\nOn Tue, May 07 2019, Jeff King wrote:\n\n> On Mon, May 06, 2019 at 11:44:06AM +0200, Ævar Arnfjörð Bjarmason wrote:\n>\n>> Maybe there's some case I haven't thought of that makes this stupid, but\n>> I wonder if something like a \"gc quarantine\" might be a fix fo both of\n>> the the issues you noted above.\n>>\n>> I.e. it seems to me that the main issue is that we conflate \"mtime 2\n>> weeks old because it's unreferenced for 2 weeks\" v.s. \"mtime 2 weeks old\n>> because we haven't gotten around to a 'gc'\".\n>>\n>> So in such a \"gc quarantine\" mode when we discover an object/pack that's\n>> unreachable/purely made up of unreachable objects we'd move the relevant\n>> loose object/\"loose\" pack to such a quarantine, which would just be\n>> .git/unreferenced-objects/{??,pack}/ or whatever.\n>>\n>> AFAICT both cases you mentioned above would be mitigated by this because\n>> we'd no longer conflate \"haven't gc'd this yet and it's 2 weeks old\"\n>> v.s. \"hasn't been referenced in 2 weeks\".\n>\n> Michael Haggerty and I have (off-list) discussed variations on that, but\n> it opens up a lot of new issues.  Moving something into quarantine isn't\n> atomic. So you've still corrupted the repo, but now it's recoverable by\n> reaching into the quarantine. Who notices that the repo is corrupt, and\n> how? When do we expire objects from quarantine?\n>\n> I think the heart of the issue is really the lack of atomicity in the\n> operations. You need some way to mark \"I am using this now\" in a way\n> that cannot race with \"looks like nobody is using this, so I'll delete\n> it\".\n>\n> And ideally without traversing large bits of the graph on the writing\n> side, and without requiring any stop-the-world locks during pruning.\n\nI was thinking (but realize now that I didn't articulate) that the \"gc\nquarantine\" would be another \"alternate\" implementing a copy-on-write\n\"lockless delete-but-be-able-to-rollback scheme\" as you put it.\n\nSo \"gc\" would decide (racily) what's unreachable, but instead of\nunlink()-ing it would \"mv\" the loose object/pack into the\n\"unreferenced-objects\" quarantine.\n\nThen in your example #1 \"wants to reference ABCD. It sees that we have\nit.\" would race on the \"other side\". I.e. maybe ABCD was *just* moved to\nthe quarantine. But in that case we'd move it back, which would bump the\nmtime and thus make it ineligible for expiry.\n\nSimilarly for example #2, the \"ABCD is ancient\" would be moved, but then\npromptely moved back on the next GC as we notice ABCD has been\nre-referenced.\n\nMaybe it's just the same problem all over again, but I don't see how\nyet.\n\nAside from that, I have a hunch that while it's theoretically true that\nyou can at any time re-reference some loose blob/tree/commit again, that\nthe likelyhood of that in practice goes down as it ages, since a user is\nlikely to e.g. re-push or rename some branch they pushed last week, not\nlast year.\n\nHence the mention of creating \"unreferenced packs\" with some new\n--keep-unreachable mode. Since we'd pack those together they wouldn't\ncreate the \"ref explosion\" problem we have with the loose refs, and thus\nyou could afford to keep them longer (even though the deltas would be\nshittier).\n\nWhereas now you either need --keep-unreachable (keep stuff forever) or a\nmore aggressive gc.pruneExpire if you'd like to not end up with a\nginormous amount of loose objects.\n\n>> I started looking at this initially because I was wondering if the\n>> --keep-unreachable mode you modified in e26a8c4721 (\"repack: extend\n>> --keep-unreachable to loose objects\", 2016-06-13) could be made to write\n>> out such \"unreferenced\" objects into their *own* pack, so we could\n>> delete them all at once as a batch, and wouldn't create the \"ref\n>> explosions\" mentioned in [1].\n>>\n>> But of course without an accompanying quarantine described above doing\n>> that would just make this race condition worse.\n>\n> I'm not sure it really makes it worse. The pack would have the same\n> mtime as the loose objects would.\n"},{"id":"379637","messageId":"20190731042601.GA26559@sigill.intra.peff.net","threadId":"50772","inReplyTo":"8736lnxlig.fsf@evledraar.gmail.com","subject":"Re: [PATCH 0/4] gc docs: modernize and fix the documentation","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2019-07-31T04:26:02Z","receivedAt":"2019-07-31T04:26:05Z","isPatch":true,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Fri, May 10, 2019 at 01:20:55AM +0200, Ævar Arnfjörð Bjarmason wrote:\n\n> > Michael Haggerty and I have (off-list) discussed variations on that, but\n> > it opens up a lot of new issues.  Moving something into quarantine isn't\n> > atomic. So you've still corrupted the repo, but now it's recoverable by\n> > reaching into the quarantine. Who notices that the repo is corrupt, and\n> > how? When do we expire objects from quarantine?\n> >\n> > I think the heart of the issue is really the lack of atomicity in the\n> > operations. You need some way to mark \"I am using this now\" in a way\n> > that cannot race with \"looks like nobody is using this, so I'll delete\n> > it\".\n> >\n> > And ideally without traversing large bits of the graph on the writing\n> > side, and without requiring any stop-the-world locks during pruning.\n> \n> I was thinking (but realize now that I didn't articulate) that the \"gc\n> quarantine\" would be another \"alternate\" implementing a copy-on-write\n> \"lockless delete-but-be-able-to-rollback scheme\" as you put it.\n> \n> So \"gc\" would decide (racily) what's unreachable, but instead of\n> unlink()-ing it would \"mv\" the loose object/pack into the\n> \"unreferenced-objects\" quarantine.\n> \n> Then in your example #1 \"wants to reference ABCD. It sees that we have\n> it.\" would race on the \"other side\". I.e. maybe ABCD was *just* moved to\n> the quarantine. But in that case we'd move it back, which would bump the\n> mtime and thus make it ineligible for expiry.\n\nI think this is basically the same as the current freshening scheme,\nthough. In general, you can replace \"move it back\" with \"update its\nmtime\". Neither is atomic with respect to other operations.\n\nIt does seem like the twist is that \"gc\" is supposed to do the \"move it\nback\" step (and it's also the thing expiring, if we assume that there's\nonly one gc running at a time). But again, how do we know somebody isn't\nreferencing it _right now_ while we're deciding whether to move it back?\n\nI think there are lots of solutions you can come up with if you have\natomicity. But fundamentally it isn't there in the way we handle updates\nnow. You could imagine something like a shared/unique lock where anybody\nupdating a ref takes the \"shared\" side, and multiple entities can hold\nit at once. But somebody pruning takes the \"unique\" side and excludes\neverybody else, stopping ref updates during the prune (which you'd\nobviously want to do in a way that you hold the lock for as short as\npossible; say, optimistically check reachability without the lock, then\ntake the lock and check to see if anything has changed).\n\n(By shared/unique I basically mean a reader/writer lock, but I didn't\nwant to use those terms in the paragraph since both holders are\nwriting).\n\nIt is tricky to find out when to hold the shared lock, though. It's\n_not_ just a ref write, for example. When you accept a push, you'd want\nto hold the lock while you are checking that you have all of the\nnecessary objects to write the ref. For something like \"git commit\" it's\neven harder, because we implicitly rely on state created by commands run\nover the course of hours or days (e.g., \"git add\" to put a blob in the\nindex and maybe create the tree via cache-tree, then a commit to\nreference it, and finally the ref write; each step adds state which the\nnext step relies on).\n\n> Aside from that, I have a hunch that while it's theoretically true that\n> you can at any time re-reference some loose blob/tree/commit again, that\n> the likelyhood of that in practice goes down as it ages, since a user is\n> likely to e.g. re-push or rename some branch they pushed last week, not\n> last year.\n>\n> Hence the mention of creating \"unreferenced packs\" with some new\n> --keep-unreachable mode. Since we'd pack those together they wouldn't\n> create the \"ref explosion\" problem we have with the loose refs, and thus\n> you could afford to keep them longer (even though the deltas would be\n> shittier).\n\nYeah, that may make it less likely (and we'd like those unreferenced\npacks for other reasons anyway, so it's certainly worth a shot). But the\nwhole race is kind of unlikely in the first place. If you have enough\nrepositories, you see it eventually. ;)\n\n-Peff\n"},{"id":"379655","messageId":"87pnlq5x8h.fsf@evledraar.gmail.com","threadId":"50772","inReplyTo":"20190731042601.GA26559@sigill.intra.peff.net","subject":"Re: [PATCH 0/4] gc docs: modernize and fix the documentation","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2019-07-31T10:12:14Z","receivedAt":"2019-07-31T10:12:20Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"\nOn Wed, Jul 31 2019, Jeff King wrote:\n\n> On Fri, May 10, 2019 at 01:20:55AM +0200, Ævar Arnfjörð Bjarmason wrote:\n>\n>> > Michael Haggerty and I have (off-list) discussed variations on that, but\n>> > it opens up a lot of new issues.  Moving something into quarantine isn't\n>> > atomic. So you've still corrupted the repo, but now it's recoverable by\n>> > reaching into the quarantine. Who notices that the repo is corrupt, and\n>> > how? When do we expire objects from quarantine?\n>> >\n>> > I think the heart of the issue is really the lack of atomicity in the\n>> > operations. You need some way to mark \"I am using this now\" in a way\n>> > that cannot race with \"looks like nobody is using this, so I'll delete\n>> > it\".\n>> >\n>> > And ideally without traversing large bits of the graph on the writing\n>> > side, and without requiring any stop-the-world locks during pruning.\n>>\n>> I was thinking (but realize now that I didn't articulate) that the \"gc\n>> quarantine\" would be another \"alternate\" implementing a copy-on-write\n>> \"lockless delete-but-be-able-to-rollback scheme\" as you put it.\n>>\n>> So \"gc\" would decide (racily) what's unreachable, but instead of\n>> unlink()-ing it would \"mv\" the loose object/pack into the\n>> \"unreferenced-objects\" quarantine.\n>>\n>> Then in your example #1 \"wants to reference ABCD. It sees that we have\n>> it.\" would race on the \"other side\". I.e. maybe ABCD was *just* moved to\n>> the quarantine. But in that case we'd move it back, which would bump the\n>> mtime and thus make it ineligible for expiry.\n>\n> I think this is basically the same as the current freshening scheme,\n> though. In general, you can replace \"move it back\" with \"update its\n> mtime\". Neither is atomic with respect to other operations.\n>\n> It does seem like the twist is that \"gc\" is supposed to do the \"move it\n> back\" step (and it's also the thing expiring, if we assume that there's\n> only one gc running at a time). But again, how do we know somebody isn't\n> referencing it _right now_ while we're deciding whether to move it back?\n\nThe twist is to create a \"quarantine\" area of the ref store you can't\nread any objects from without copying them to the \"main\" area (git-gc\nitself would be an exception).\n\nHence step #2 and #6, respectively, in your examples in\nhttps://public-inbox.org/git/20190319001829.GL29661@sigill.intra.peff.net/\nwould have update-ref/receive-pack fail to find \"ABCD\" in the \"main\"\nstore due to the exact same race we have now with mtimes & gc, then fall\nback to the \"quarantine\" and (this is the important part) immediately\ncopy it back to the \"main\" store.\n\nIOW yes, you'd have the exact same race you have now with the initial\nmove to the quarantine. You'd have ref updates & gc racing and\n\"unreachable\" things would be moved to the quarantine, but really the\njust became reachable again.\n\nThe difference is that instead of unlinking that unreachable object we\nmove it to the quarantine, so the next \"gc\" (which is what would delete\nit) would notice it's reachable and move it to the \"main\" area before\nproceeding, *and* anything that \"faults\" back to reading the\n\"quarantine\" would do the same.\n\n> I think there are lots of solutions you can come up with if you have\n> atomicity. But fundamentally it isn't there in the way we handle updates\n> now. You could imagine something like a shared/unique lock where anybody\n> updating a ref takes the \"shared\" side, and multiple entities can hold\n> it at once. But somebody pruning takes the \"unique\" side and excludes\n> everybody else, stopping ref updates during the prune (which you'd\n> obviously want to do in a way that you hold the lock for as short as\n> possible; say, optimistically check reachability without the lock, then\n> take the lock and check to see if anything has changed).\n>\n> (By shared/unique I basically mean a reader/writer lock, but I didn't\n> want to use those terms in the paragraph since both holders are\n> writing).\n>\n> It is tricky to find out when to hold the shared lock, though. It's\n> _not_ just a ref write, for example. When you accept a push, you'd want\n> to hold the lock while you are checking that you have all of the\n> necessary objects to write the ref. For something like \"git commit\" it's\n> even harder, because we implicitly rely on state created by commands run\n> over the course of hours or days (e.g., \"git add\" to put a blob in the\n> index and maybe create the tree via cache-tree, then a commit to\n> reference it, and finally the ref write; each step adds state which the\n> next step relies on).\n\nI don't think this sort of approach would require any global locks, but\nit would be vulnerable to operations that take longer than the\n\"main->quarantine->unlink()\" cycle takes. E.g. a \"hash-object\" that\ntakes a month before the subsequent \"write-tree\" etc.\n\nAll of the above written with the previously stated \"I may be missing\nsomething\" caveat etc. :)\n"}]}