{"thread":{"id":"61089","subject":"repositoryformatversion and extensions","startedAt":"2024-03-11T11:55:07Z","lastAt":"2024-03-27T10:08:22Z","messageCount":3,"participants":["Edward Thomson","Patrick Steinhardt","Jeff King"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"490354","messageId":"CA+WKDT0fAASYE_ZhSxATv2NUnYceOP-_EF1kqKNFSOWU4QdPVQ@mail.gmail.com","threadId":"61089","inReplyTo":null,"subject":"repositoryformatversion and extensions","fromName":"Edward Thomson","fromEmail":"ethomson@edwardthomson.com","sentAt":"2024-03-11T11:54:55Z","receivedAt":"2024-03-11T11:55:07Z","isPatch":false,"sender":{"key":"ethomson@edwardthomson.com","avatar":"https://avatars.githubusercontent.com/u/1130014?v=4"},"body":"While looking at worktree configuration, I realized that the\nextensions.worktreeconfig configuration option was honored even when\ncore.repositoryformatversion was unset, or explicitly set to 0.\n\nRe-reading the docs for repository-version, it doesn't explicitly state\nthat extensions _require_ repositoryformatversion >= 1, it instead\nstates that:\n\n> When reading the core.repositoryformatversion variable, a git\n> implementation which supports version 1 MUST also read any configuration\n> keys found in the extensions section of the configuration file.\n\nDespite that, at least one extension (objectformat) does require\ncore.repositoryformatversion >= 1. Though, do note that objectformat\nis not mentioned in the repository-version documentation.\n\nWhat other extensions require repository format version >= 1? Is\ncore.objectformat the outlier here?\n\nIs there documentation on which extensions are supported at which\nrepository format versions?\n\nAre there other extensions that are not documented in repository-version?\n\nCheers-\nEd\n"},{"id":"491113","messageId":"Zfv-kpa4CJJvop2j@tanuki","threadId":"61089","inReplyTo":"CA+WKDT0fAASYE_ZhSxATv2NUnYceOP-_EF1kqKNFSOWU4QdPVQ@mail.gmail.com","subject":"Re: repositoryformatversion and extensions","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2024-03-21T09:32:11Z","receivedAt":"2024-03-21T09:32:16Z","isPatch":false,"sender":{"key":"ps@pks.im","avatar":"https://avatars.githubusercontent.com/u/4056630?v=4"},"body":"On Mon, Mar 11, 2024 at 11:54:55AM +0000, Edward Thomson wrote:\n> While looking at worktree configuration, I realized that the\n> extensions.worktreeconfig configuration option was honored even when\n> core.repositoryformatversion was unset, or explicitly set to 0.\n> \n> Re-reading the docs for repository-version, it doesn't explicitly state\n> that extensions _require_ repositoryformatversion >= 1, it instead\n> states that:\n> \n> > When reading the core.repositoryformatversion variable, a git\n> > implementation which supports version 1 MUST also read any configuration\n> > keys found in the extensions section of the configuration file.\n\nYeah, indeed. As far as I understand, the major difference between\nversion 1 and version 0 is that version 1 asks the Git client to bail\nout as soon as there is any extension that isn't understood by the\nclient. To me this behaviour is somewhat surprising because it doesn't\nallow for in-between states where some extensions are mandatory whereas\nothers aren't. It would have been better to have a mechanism that\ndisregards the version but makes each extension itself highlight whether\nit is mandatory or not.\n\n> Despite that, at least one extension (objectformat) does require\n> core.repositoryformatversion >= 1. Though, do note that objectformat\n> is not mentioned in the repository-version documentation.\n> \n> What other extensions require repository format version >= 1? Is\n> core.objectformat the outlier here?\n\nIn any case, there are two extensions that make us set version 1 right\nnow: `objectFormat` and `refStorage`. For the latter it is quite\nimportant for any client to bail out if it's not understood by them so\nthat we don't accidentally start to write e.g. loose refs in a reftable\nrepository.\n\n> Is there documentation on which extensions are supported at which\n> repository format versions?\n\nNot that I'd know of. Right now, it includes \"noop-v1\", \"objectFormat\"\nand \"refStorage\". But I only derive that info from `handle_extension()`\nin \"setup.c\".\n\n> Are there other extensions that are not documented in repository-version?\n\nThe only extension I'm aware of that is missing is \"noop-v1\". That\nextension is mostly just used for testing purposes though, so it's of\nrather dubious value to document it. On the other hand, the \"noop\"\nextension is documented, as well.\n\nIn any case, the policy of the Git project is that any new extension\nshould only ever be understood when the repository format version is >=\n1 from now on. At least, that's what the code tells us:\n\n```\n/*\n * Do not add new extensions to this function. It handles extensions which are\n * respected even in v0-format repositories for historical compatibility.\n */\nstatic enum extension_result handle_extension_v0(const char *var,\n\t\t\t\t\t\t const char *value,\n\t\t\t\t\t\t const char *ext,\n\t\t\t\t\t\t struct repository_format *data)\n```\n\nSo it's probably safe to assume that every new extension should be at\nleast v1 from now on.\n\nPatrick\n"},{"id":"491680","messageId":"20240327100819.GB857435@coredump.intra.peff.net","threadId":"61089","inReplyTo":"Zfv-kpa4CJJvop2j@tanuki","subject":"Re: repositoryformatversion and extensions","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2024-03-27T10:08:19Z","receivedAt":"2024-03-27T10:08:22Z","isPatch":false,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Thu, Mar 21, 2024 at 10:32:11AM +0100, Patrick Steinhardt wrote:\n\n> > > When reading the core.repositoryformatversion variable, a git\n> > > implementation which supports version 1 MUST also read any configuration\n> > > keys found in the extensions section of the configuration file.\n> \n> Yeah, indeed. As far as I understand, the major difference between\n> version 1 and version 0 is that version 1 asks the Git client to bail\n> out as soon as there is any extension that isn't understood by the\n> client. To me this behaviour is somewhat surprising because it doesn't\n> allow for in-between states where some extensions are mandatory whereas\n> others aren't. It would have been better to have a mechanism that\n> disregards the version but makes each extension itself highlight whether\n> it is mandatory or not.\n\nThe point of bumping \"0\" to \"1\" at all was that older versions of Git,\nwhich did not understand extensions at all, would continue to bail. So\nyes, I do think it's unfortunate that we do not insist that the version\nis set to \"1\" when parsing extensions, but it is a bug for any writer to\nset an extension and _not_ bump the version. Versions of Git older than\nv2.6.3 would quietly ignore them.\n\nAnd you're correct that it does not allow for non-mandatory extensions.\nBut then, that is the whole point of the extensions.* mechanism: it is\nto make sure that versions of Git that do not understand the particular\nextension refuse to run. If something isn't mandatory, then it should\nnot be in extensions.* at all. It should be regular config, with the\nknowledge that older versions may ignore it.\n\nAt least that was how it was all originally designed. I think later\nextensions did not always correctly set the repository version. We tried\nto fix that later, but ran into compatibility issues. See 11664196ac\n(Revert \"check_repository_format_gently(): refuse extensions for old\nrepositories\", 2020-07-15).\n\nSo that splits us our extensions into two sets:\n\n  - ones that existed at the time of 11664196ac, which are allowed with\n    v0 for compatibility. These are found in handle_extension_v0() in\n    Git's code\n\n  - everything else that is added should do things correctly, and should\n    go into handle_extension_v1()\n\nSo getting back to Ed's question...\n\n> > Despite that, at least one extension (objectformat) does require\n> > core.repositoryformatversion >= 1. Though, do note that objectformat\n> > is not mentioned in the repository-version documentation.\n> > \n> > What other extensions require repository format version >= 1? Is\n> > core.objectformat the outlier here?\n\n...the answer is that it's the other way around. Those older extensions\nare the outliers. But since we don't add extensions often, for a long\ntime it was the only one that did things right. ;) And now refstorage\ndoes it right, too.\n\nIf we add more extensions, they will also behave that way.\n\n> In any case, the policy of the Git project is that any new extension\n> should only ever be understood when the repository format version is >=\n> 1 from now on. At least, that's what the code tells us:\n> \n> ```\n> /*\n>  * Do not add new extensions to this function. It handles extensions which are\n>  * respected even in v0-format repositories for historical compatibility.\n>  */\n> static enum extension_result handle_extension_v0(const char *var,\n> \t\t\t\t\t\t const char *value,\n> \t\t\t\t\t\t const char *ext,\n> \t\t\t\t\t\t struct repository_format *data)\n> ```\n> \n> So it's probably safe to assume that every new extension should be at\n> least v1 from now on.\n\nRight, exactly.\n\nIf you really want more gory details, I think the discussion is all in:\n\n  https://lore.kernel.org/git/pull.675.git.1594677321039.gitgitgadget@gmail.com/\n\n-Peff\n"}]}