{"thread":{"id":"62005","subject":"Improvement: `git-maintenance` to allow configuring of remotes to fetch","startedAt":"2024-08-26T11:51:27Z","lastAt":"2024-09-04T18:10:48Z","messageCount":10,"participants":["Shubham Kanodia","Patrick Steinhardt","Junio C Hamano"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"501669","messageId":"CAG=Um+1wTbXn_RN+LOCrpZpSNR_QF582PszxNyhz5anVHtBp+w@mail.gmail.com","threadId":"62005","inReplyTo":null,"subject":"Improvement: `git-maintenance` to allow configuring of remotes to fetch","fromName":"Shubham Kanodia","fromEmail":"shubhamsizzles@gmail.com","sentAt":"2024-08-26T11:50:50Z","receivedAt":"2024-08-26T11:51:27Z","isPatch":false,"sender":{"key":"shubhamsizzles@gmail.com","avatar":"https://gravatar.com/avatar/cf63177ba437ab5a64944e88d08cf42b6342d8134884aa8e3120a52fd0cbf66c?d=mp&s=160"},"body":"`git-maintenance` aims to improve the health & performance of\nespecially large git repositories.\nA part of git maintenance tasks is `prefetch`.\n\nHowever, at the moment, it prefetches all objects from all branches —\n\n> The prefetch task updates the object directory with the latest objects from all registered remotes.\n\nThis seems non-optimal for repositories where users use the main\nrepository to fork branches (instead of personal forks) as the remote\ncould have thousands of refs and objects from branches that a user may\nnot care about, leading to a very large prefetch dir.\n\nFor large repositories, it might instead be helpful to only fetch\nobjects on `main` or `master` because those are the ones most likely\nto be used by users.\n\nTo this effect, git maintenance should perhaps expose a\nmaintenance.prefetch.refs setting that subsets prefetching only to\ncertain references.\n\nIs this something that has been raised before / is in works by any\nchance or something that would be an acceptable contribution?\n\n-- \nRegards,\nShubham Kanodia\n"},{"id":"501758","messageId":"Zs8KzG0vzCEDvkvx@tanuki","threadId":"62005","inReplyTo":"CAG=Um+1wTbXn_RN+LOCrpZpSNR_QF582PszxNyhz5anVHtBp+w@mail.gmail.com","subject":"Re: Improvement: `git-maintenance` to allow configuring of remotes to fetch","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2024-08-28T11:32:34Z","receivedAt":"2024-08-28T11:32:39Z","isPatch":false,"sender":{"key":"ps@pks.im","avatar":"https://avatars.githubusercontent.com/u/4056630?v=4"},"body":"On Mon, Aug 26, 2024 at 05:20:50PM +0530, Shubham Kanodia wrote:\n> `git-maintenance` aims to improve the health & performance of\n> especially large git repositories.\n> A part of git maintenance tasks is `prefetch`.\n> \n> However, at the moment, it prefetches all objects from all branches —\n> \n> > The prefetch task updates the object directory with the latest objects from all registered remotes.\n> \n> This seems non-optimal for repositories where users use the main\n> repository to fork branches (instead of personal forks) as the remote\n> could have thousands of refs and objects from branches that a user may\n> not care about, leading to a very large prefetch dir.\n> \n> For large repositories, it might instead be helpful to only fetch\n> objects on `main` or `master` because those are the ones most likely\n> to be used by users.\n> \n> To this effect, git maintenance should perhaps expose a\n> maintenance.prefetch.refs setting that subsets prefetching only to\n> certain references.\n\nI think that this makes sense as an addition. We would have to think\nabout things like globbing and whatnot, such that we can say e.g. to\nprefetch only tags, only branches or only a subset thereof. We may even\ndecide to use refspecs here, which could be the most flexible way of\nconfiguring this. We already use the default refspecs anyway -- the only\nthing we change is that we prefix the refspec destination with\n\"refs/prefetch/\". So allowing the user to override the refspec used for\nprefetching makes sense.\n\nIt opens up another question though: if you've got multiple remotes in a\nrepository, do we also want to make it configurable which of these\nshould be fetched from? And if you do, do you want to allow configuring\nrefs to fetch per remote? I guess the answer to both of these questions\nwould be \"yes\".\n\n> Is this something that has been raised before / is in works by any\n> chance or something that would be an acceptable contribution?\n\nI'm not aware of any discussion around this, and think that this feature\ndoes make sense indeed.\n\nPatrick\n"},{"id":"501780","messageId":"xmqq4j7438yc.fsf@gitster.g","threadId":"62005","inReplyTo":"Zs8KzG0vzCEDvkvx@tanuki","subject":"Re: Improvement: `git-maintenance` to allow configuring of remotes to fetch","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2024-08-28T16:31:39Z","receivedAt":"2024-08-28T16:31:42Z","isPatch":false,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Patrick Steinhardt <ps@pks.im> writes:\n\n> I'm not aware of any discussion around this...\n\nI do not think so, either.\n\nI agree that it makes as much sense to limit prefetches to a subset\nof remotes as it makes sense to limit to certain hierarchies (e.g.\nexcluding refs/changes/ or even limiting to refs/heads/seen and\nnothing else).\n"},{"id":"501991","messageId":"CAG=Um+2OQofcfo3vjvPJEAUht5cGg0LnPAx54SWUPETgkRACPQ@mail.gmail.com","threadId":"62005","inReplyTo":"xmqq4j7438yc.fsf@gitster.g","subject":"Re: Improvement: `git-maintenance` to allow configuring of remotes to fetch","fromName":"Shubham Kanodia","fromEmail":"shubhamsizzles@gmail.com","sentAt":"2024-09-02T15:46:24Z","receivedAt":"2024-09-02T15:47:02Z","isPatch":false,"sender":{"key":"shubhamsizzles@gmail.com","avatar":"https://gravatar.com/avatar/cf63177ba437ab5a64944e88d08cf42b6342d8134884aa8e3120a52fd0cbf66c?d=mp&s=160"},"body":"> Patrick Steinhardt <ps@pks.im> writes:\n>\n> > I'm not aware of any discussion around this...\n>\n> I do not think so, either.\n>\n> I agree that it makes as much sense to limit prefetches to a subset\n> of remotes as it makes sense to limit to certain hierarchies (e.g.\n> excluding refs/changes/ or even limiting to refs/heads/seen and\n> nothing else).\n\nI'm seeking advice on the configuration option structure for this\nfeature. The typical config format for maintenance tasks seems to be\nas follows:\n\n`maintenance.<task-name>.<option>`\n\nA natural extension of this for the prefetch task could be:\n\n```\ngit config maintenance.prefetch.<remote-name>.refs refs/heads/master\n```\n\nIn this structure, the 'refs' value represents only the source part of\na refspec, and both remote and refs can be configured.\nSpecifying a full refspec isn't necessary since the --prefetch option\nmay override the destination anyway.\n\nWhile I've successfully implemented this approach, I'm open to\nsuggestions for alternative configuration options. My concerns are:\n\n1. Most Git configurations are nested up to three levels deep, whereas\nthis proposal introduces a fourth level.\n2. This configuration appears in the config file as:\n\n```\n[maintenance \"prefetch.origin\"]\n       refs = refs/heads/master\n```\nwhich might look odd?\n\nAlso, hopefully my mail is formatted better this time!\n"},{"id":"501997","messageId":"ZtacHCuql0pX3V2u@tanuki","threadId":"62005","inReplyTo":"CAG=Um+2OQofcfo3vjvPJEAUht5cGg0LnPAx54SWUPETgkRACPQ@mail.gmail.com","subject":"Re: Improvement: `git-maintenance` to allow configuring of remotes to fetch","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2024-09-03T05:18:20Z","receivedAt":"2024-09-03T05:18:27Z","isPatch":false,"sender":{"key":"ps@pks.im","avatar":"https://avatars.githubusercontent.com/u/4056630?v=4"},"body":"On Mon, Sep 02, 2024 at 09:16:24PM +0530, Shubham Kanodia wrote:\n> > Patrick Steinhardt <ps@pks.im> writes:\n> >\n> > > I'm not aware of any discussion around this...\n> >\n> > I do not think so, either.\n> >\n> > I agree that it makes as much sense to limit prefetches to a subset\n> > of remotes as it makes sense to limit to certain hierarchies (e.g.\n> > excluding refs/changes/ or even limiting to refs/heads/seen and\n> > nothing else).\n> \n> I'm seeking advice on the configuration option structure for this\n> feature. The typical config format for maintenance tasks seems to be\n> as follows:\n> \n> `maintenance.<task-name>.<option>`\n> \n> A natural extension of this for the prefetch task could be:\n> \n> ```\n> git config maintenance.prefetch.<remote-name>.refs refs/heads/master\n> ```\n> \n> In this structure, the 'refs' value represents only the source part of\n> a refspec, and both remote and refs can be configured.\n> Specifying a full refspec isn't necessary since the --prefetch option\n> may override the destination anyway.\n> \n> While I've successfully implemented this approach, I'm open to\n> suggestions for alternative configuration options. My concerns are:\n> \n> 1. Most Git configurations are nested up to three levels deep, whereas\n> this proposal introduces a fourth level.\n> 2. This configuration appears in the config file as:\n> \n> ```\n> [maintenance \"prefetch.origin\"]\n>        refs = refs/heads/master\n> ```\n> which might look odd?\n\nAgreed, it does. To me, the most natural way to configure this would be\nas part of the remotes themselves:\n\n```\n[remote \"origin\"]\n    url = https://example.com/repo.git\n    fetch = +refs/heads/*:refs/remotes/origin/*\n    # Whether or not the prefetch task shall fatch this repository.\n    # Defaults to `true`.\n    prefetch = true\n    # An arbitrary number of refspecs used by the prefetch task.\n    # Overrides the fetch refspec if given, otherwise we fall back to\n    # using the fetch refspec.\n    prefetchRefspec = +refs/heads/main:refs/remotes/origin/main\n```\n\nThe prefetch refspec would be rewritten by git-maintenance(1) such that\nthe destination part (the right-hand side of the refspec) is prefixed\nwith `refs/prefetch/`, same as the fetch refspec would be changed in\nthis way.\n\nAn alternative would be to _not_ rewrite the prefetch refspec at all and\nthus allow the user to prefetch into arbitrary hierarchies. But I'm a\nbit worried that this might cause users to misconfigure prefetches by\naccident, causing it to overwrite their usual set of refs.\n\n> Also, hopefully my mail is formatted better this time!\n\nIt is, thanks!\n\nPatrick\n"},{"id":"501998","messageId":"CAG=Um+17JYyqC6n0gU3GSNaVz1PaB1U50M1hy87zObB+Rc03Qg@mail.gmail.com","threadId":"62005","inReplyTo":"ZtacHCuql0pX3V2u@tanuki","subject":"Re: Improvement: `git-maintenance` to allow configuring of remotes to fetch","fromName":"Shubham Kanodia","fromEmail":"shubhamsizzles@gmail.com","sentAt":"2024-09-03T06:01:04Z","receivedAt":"2024-09-03T06:01:41Z","isPatch":false,"sender":{"key":"shubhamsizzles@gmail.com","avatar":"https://gravatar.com/avatar/cf63177ba437ab5a64944e88d08cf42b6342d8134884aa8e3120a52fd0cbf66c?d=mp&s=160"},"body":"On Tue, Sep 3, 2024 at 10:48 AM Patrick Steinhardt <ps@pks.im> wrote:\n>\n> On Mon, Sep 02, 2024 at 09:16:24PM +0530, Shubham Kanodia wrote:\n> > > Patrick Steinhardt <ps@pks.im> writes:\n> > >\n> > > > I'm not aware of any discussion around this...\n> > >\n> > > I do not think so, either.\n> > >\n> > > I agree that it makes as much sense to limit prefetches to a subset\n> > > of remotes as it makes sense to limit to certain hierarchies (e.g.\n> > > excluding refs/changes/ or even limiting to refs/heads/seen and\n> > > nothing else).\n> >\n> > I'm seeking advice on the configuration option structure for this\n> > feature. The typical config format for maintenance tasks seems to be\n> > as follows:\n> >\n> > `maintenance.<task-name>.<option>`\n> >\n> > A natural extension of this for the prefetch task could be:\n> >\n> > ```\n> > git config maintenance.prefetch.<remote-name>.refs refs/heads/master\n> > ```\n> >\n> > In this structure, the 'refs' value represents only the source part of\n> > a refspec, and both remote and refs can be configured.\n> > Specifying a full refspec isn't necessary since the --prefetch option\n> > may override the destination anyway.\n> >\n> > While I've successfully implemented this approach, I'm open to\n> > suggestions for alternative configuration options. My concerns are:\n> >\n> > 1. Most Git configurations are nested up to three levels deep, whereas\n> > this proposal introduces a fourth level.\n> > 2. This configuration appears in the config file as:\n> >\n> > ```\n> > [maintenance \"prefetch.origin\"]\n> >        refs = refs/heads/master\n> > ```\n> > which might look odd?\n>\n> Agreed, it does. To me, the most natural way to configure this would be\n> as part of the remotes themselves:\n>\n> ```\n> [remote \"origin\"]\n>     url = https://example.com/repo.git\n>     fetch = +refs/heads/*:refs/remotes/origin/*\n>     # Whether or not the prefetch task shall fatch this repository.\n>     # Defaults to `true`.\n>     prefetch = true\n>     # An arbitrary number of refspecs used by the prefetch task.\n>     # Overrides the fetch refspec if given, otherwise we fall back to\n>     # using the fetch refspec.\n>     prefetchRefspec = +refs/heads/main:refs/remotes/origin/main\n> ```\n>\n> The prefetch refspec would be rewritten by git-maintenance(1) such that\n> the destination part (the right-hand side of the refspec) is prefixed\n> with `refs/prefetch/`, same as the fetch refspec would be changed in\n> this way.\n>\n> An alternative would be to _not_ rewrite the prefetch refspec at all and\n> thus allow the user to prefetch into arbitrary hierarchies. But I'm a\n> bit worried that this might cause users to misconfigure prefetches by\n> accident, causing it to overwrite their usual set of refs.\n>\n> > Also, hopefully my mail is formatted better this time!\n>\n> It is, thanks!\n>\n> Patrick\n\nInteresting. I guess if we put this in `remote.*` instead of\n`maintenance.*` what's unclear then is if this setting should also be\nrespected by `git fetch --prefetch`\nwhen used outside the context of a maintenance task — since that'd\nprobably be a bigger change.\n\nFor instance, the `skipFetchAll` remote config option seems to apply\nto prefetches (within maintenance & outside) and normal fetches.\n\nAdditionally, we'd need to discuss what to do with backward compatibility:\n\nIf we were designing maintenance prefetch right now, it'd probably\nmake sense not to fetch it for any remote / refspec by default unless\nexplicitly enabled since\nfetching all refs can be a waste of space in more cases than not.\n\nHowever, since the current behaviour already fetches all remotes and\nrefs, I don't know if breaking this is something we could do? If not,\nthe behavior would be —\n\n1. If none of the remotes specify a `prefetch: true`, then prefetch\nall remotes and refs (backwards compat)\n2. If at least one of the remotes specifies `prefetch: true` then only\nthat remote should be fetched\n3. Two-tier `fetch` / `prefetchRefSpec` hierarchy to decide which refs\nto fetch (we can decide on the name later as `fetch` and\n`prefetchRefSpec` seem asymmetrical)\n"},{"id":"502022","messageId":"ZtbbejUrUO2tdsUQ@pks.im","threadId":"62005","inReplyTo":"CAG=Um+17JYyqC6n0gU3GSNaVz1PaB1U50M1hy87zObB+Rc03Qg@mail.gmail.com","subject":"Re: Improvement: `git-maintenance` to allow configuring of remotes to fetch","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2024-09-03T09:48:47Z","receivedAt":"2024-09-03T09:48:53Z","isPatch":false,"sender":{"key":"ps@pks.im","avatar":"https://avatars.githubusercontent.com/u/4056630?v=4"},"body":"On Tue, Sep 03, 2024 at 11:31:04AM +0530, Shubham Kanodia wrote:\n> On Tue, Sep 3, 2024 at 10:48 AM Patrick Steinhardt <ps@pks.im> wrote:\n> > On Mon, Sep 02, 2024 at 09:16:24PM +0530, Shubham Kanodia wrote:\n> > > > Patrick Steinhardt <ps@pks.im> writes:\n> > > >\n> > > > > I'm not aware of any discussion around this...\n> > > >\n> > > > I do not think so, either.\n> > > >\n> > > > I agree that it makes as much sense to limit prefetches to a subset\n> > > > of remotes as it makes sense to limit to certain hierarchies (e.g.\n> > > > excluding refs/changes/ or even limiting to refs/heads/seen and\n> > > > nothing else).\n> > >\n> > > I'm seeking advice on the configuration option structure for this\n> > > feature. The typical config format for maintenance tasks seems to be\n> > > as follows:\n> > >\n> > > `maintenance.<task-name>.<option>`\n> > >\n> > > A natural extension of this for the prefetch task could be:\n> > >\n> > > ```\n> > > git config maintenance.prefetch.<remote-name>.refs refs/heads/master\n> > > ```\n> > >\n> > > In this structure, the 'refs' value represents only the source part of\n> > > a refspec, and both remote and refs can be configured.\n> > > Specifying a full refspec isn't necessary since the --prefetch option\n> > > may override the destination anyway.\n> > >\n> > > While I've successfully implemented this approach, I'm open to\n> > > suggestions for alternative configuration options. My concerns are:\n> > >\n> > > 1. Most Git configurations are nested up to three levels deep, whereas\n> > > this proposal introduces a fourth level.\n> > > 2. This configuration appears in the config file as:\n> > >\n> > > ```\n> > > [maintenance \"prefetch.origin\"]\n> > >        refs = refs/heads/master\n> > > ```\n> > > which might look odd?\n> >\n> > Agreed, it does. To me, the most natural way to configure this would be\n> > as part of the remotes themselves:\n> >\n> > ```\n> > [remote \"origin\"]\n> >     url = https://example.com/repo.git\n> >     fetch = +refs/heads/*:refs/remotes/origin/*\n> >     # Whether or not the prefetch task shall fatch this repository.\n> >     # Defaults to `true`.\n> >     prefetch = true\n> >     # An arbitrary number of refspecs used by the prefetch task.\n> >     # Overrides the fetch refspec if given, otherwise we fall back to\n> >     # using the fetch refspec.\n> >     prefetchRefspec = +refs/heads/main:refs/remotes/origin/main\n> > ```\n> >\n> > The prefetch refspec would be rewritten by git-maintenance(1) such that\n> > the destination part (the right-hand side of the refspec) is prefixed\n> > with `refs/prefetch/`, same as the fetch refspec would be changed in\n> > this way.\n> >\n> > An alternative would be to _not_ rewrite the prefetch refspec at all and\n> > thus allow the user to prefetch into arbitrary hierarchies. But I'm a\n> > bit worried that this might cause users to misconfigure prefetches by\n> > accident, causing it to overwrite their usual set of refs.\n> >\n> > > Also, hopefully my mail is formatted better this time!\n> >\n> > It is, thanks!\n> >\n> > Patrick\n> \n> Interesting. I guess if we put this in `remote.*` instead of\n> `maintenance.*` what's unclear then is if this setting should also be\n> respected by `git fetch --prefetch`\n> when used outside the context of a maintenance task — since that'd\n> probably be a bigger change.\n\nI would've expected it to already do, given that it is explicitly\ndocumented in git-fetch(1) to be for git-maintenance(1). I don't have\nenough knowledge around the prefetching task though to be able to tell\nwhether it should or shouldn't.\n\n> For instance, the `skipFetchAll` remote config option seems to apply\n> to prefetches (within maintenance & outside) and normal fetches.\n> \n> Additionally, we'd need to discuss what to do with backward compatibility:\n> \n> If we were designing maintenance prefetch right now, it'd probably\n> make sense not to fetch it for any remote / refspec by default unless\n> explicitly enabled since\n> fetching all refs can be a waste of space in more cases than not.\n> \n> However, since the current behaviour already fetches all remotes and\n> refs, I don't know if breaking this is something we could do? If not,\n> the behavior would be —\n> \n> 1. If none of the remotes specify a `prefetch: true`, then prefetch\n> all remotes and refs (backwards compat)\n> 2. If at least one of the remotes specifies `prefetch: true` then only\n> that remote should be fetched\n> 3. Two-tier `fetch` / `prefetchRefSpec` hierarchy to decide which refs\n> to fetch (we can decide on the name later as `fetch` and\n> `prefetchRefSpec` seem asymmetrical)\n\nI don't think we should change the current default. As mentioned in my\nmail, `prefetch` should default to `true` such that the behaviour as we\nhave it right now is unchanged. Thus any remote that doesn't have it\ngets fetched by default, and to disable fetching a specific remote you\nwould have to set `remote.<name>.prefetch = false`.\n\nWhile the second item, namely change the default to `false` when there\nis at least one `prefetch` config, might be something to consider. But\nin the end I think it would lead to confusing behaviour, so I'd not go\nthere personally.\n\nPatrick\n"},{"id":"502044","messageId":"xmqq5xrcn2k1.fsf@gitster.g","threadId":"62005","inReplyTo":"ZtacHCuql0pX3V2u@tanuki","subject":"Re: Improvement: `git-maintenance` to allow configuring of remotes to fetch","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2024-09-03T16:07:42Z","receivedAt":"2024-09-03T16:07:46Z","isPatch":false,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Patrick Steinhardt <ps@pks.im> writes:\n\n> The prefetch refspec would be rewritten by git-maintenance(1) such that\n> the destination part (the right-hand side of the refspec) is prefixed\n> with `refs/prefetch/`, same as the fetch refspec would be changed in\n> this way.\n>\n> An alternative would be to _not_ rewrite the prefetch refspec at all and\n> thus allow the user to prefetch into arbitrary hierarchies. But I'm a\n> bit worried that this might cause users to misconfigure prefetches by\n> accident, causing it to overwrite their usual set of refs.\n\nI agree that it is the right place to configure this as attributes\nto remotes.  It would make it handy if we could give a catch-all\nconfiguration, though.  For example:\n\n [remote \"origin\"]\n\tprefetch = true\n\tprefetchref = refs/heads/* refs/tags/*\n [remote \"*\"]\n\tprefetch = false\n\nmay toggle prefetch off for all remotes, except that the tags and\nthe local branches of the remote \"origin\" are prefetched.  Instead\nof a multi-value configuration variable (like remote.*.fetch) where\nwe need to worry about clearing convention, we can use a regular\n\"last one wins\" variable that is whitespace separated patterns, as\nsuch a pattern can never have a whitespace in it.\n\nAs you too agree with the position to consider \"prefetch\" should be\ninvisible to the end-users, we should not allow users to specify the\nfull refspec at all, or if it is forced or not with \"+\" prefix.\nOnly accept a set of patterns to match, and keep it opaque where in\nthe local refs/* hierarchy they are stored.  It is an implementation\ndetail that the users should not have to know about and rely on.\n\nThanks.\n"},{"id":"502150","messageId":"CAG=Um+3QtRkNcSq68iOWW=HWJytDReaikAfjX6MrgOY5n56gFg@mail.gmail.com","threadId":"62005","inReplyTo":"xmqq5xrcn2k1.fsf@gitster.g","subject":"Re: Improvement: `git-maintenance` to allow configuring of remotes to fetch","fromName":"Shubham Kanodia","fromEmail":"shubhamsizzles@gmail.com","sentAt":"2024-09-04T16:29:29Z","receivedAt":"2024-09-04T16:30:06Z","isPatch":false,"sender":{"key":"shubhamsizzles@gmail.com","avatar":"https://gravatar.com/avatar/cf63177ba437ab5a64944e88d08cf42b6342d8134884aa8e3120a52fd0cbf66c?d=mp&s=160"},"body":"On Tue, Sep 3, 2024 at 9:37 PM Junio C Hamano <gitster@pobox.com> wrote:\n>\n> Patrick Steinhardt <ps@pks.im> writes:\n>\n> > The prefetch refspec would be rewritten by git-maintenance(1) such that\n> > the destination part (the right-hand side of the refspec) is prefixed\n> > with `refs/prefetch/`, same as the fetch refspec would be changed in\n> > this way.\n> >\n> > An alternative would be to _not_ rewrite the prefetch refspec at all and\n> > thus allow the user to prefetch into arbitrary hierarchies. But I'm a\n> > bit worried that this might cause users to misconfigure prefetches by\n> > accident, causing it to overwrite their usual set of refs.\n>\n> I agree that it is the right place to configure this as attributes\n> to remotes.  It would make it handy if we could give a catch-all\n> configuration, though.  For example:\n>\n>  [remote \"origin\"]\n>         prefetch = true\n>         prefetchref = refs/heads/* refs/tags/*\n>  [remote \"*\"]\n>         prefetch = false\n>\n> may toggle prefetch off for all remotes, except that the tags and\n> the local branches of the remote \"origin\" are prefetched.  Instead\n> of a multi-value configuration variable (like remote.*.fetch) where\n> we need to worry about clearing convention, we can use a regular\n> \"last one wins\" variable that is whitespace separated patterns, as\n> such a pattern can never have a whitespace in it.\n>\n> As you too agree with the position to consider \"prefetch\" should be\n> invisible to the end-users, we should not allow users to specify the\n> full refspec at all, or if it is forced or not with \"+\" prefix.\n> Only accept a set of patterns to match, and keep it opaque where in\n> the local refs/* hierarchy they are stored.  It is an implementation\n> detail that the users should not have to know about and rely on.\n>\n> Thanks.\n\nRight so I have good direction to start working on this. I'm going to\ntackle this in a few parts since this is my first contribution —\n\n1. adding a boolean `prefetch` attribute to `remote` — allows control\nover remotes that prefetch\n2. adding `prefetchref` — allows control over refs per remote.\n3. Check if adding a ` [remote \"*\"]` catchall is a possibility\n\nFor the boolean `prefetch` value the —\n1. The default would be assumed to be `true`\n2. No changes in behavior to any variation of  `git fetch --prefetch`\ncommand itself.\nIf no remote is supplied to `git fetch` then it means default remote,\nor if `--all` is used, it means all remotes and `--prefetch` for both\nalways\nprefetches irrespective of the config value.\n\nIf I understand the codebase well, the changes would primarily affect\n`remote.c` and `gc.c`.\n\nAlso much thanks for the feedback both of you've given thus far.\n"},{"id":"502159","messageId":"CAG=Um+0EBDREtFxW8Fd3ktQ-ZtwAmhuiur1Y-J_BtW7u3HhM-A@mail.gmail.com","threadId":"62005","inReplyTo":"CAG=Um+3QtRkNcSq68iOWW=HWJytDReaikAfjX6MrgOY5n56gFg@mail.gmail.com","subject":"Re: Improvement: `git-maintenance` to allow configuring of remotes to fetch","fromName":"Shubham Kanodia","fromEmail":"shubhamsizzles@gmail.com","sentAt":"2024-09-04T18:10:11Z","receivedAt":"2024-09-04T18:10:48Z","isPatch":false,"sender":{"key":"shubhamsizzles@gmail.com","avatar":"https://gravatar.com/avatar/cf63177ba437ab5a64944e88d08cf42b6342d8134884aa8e3120a52fd0cbf66c?d=mp&s=160"},"body":"On Wed, Sep 4, 2024 at 9:59 PM Shubham Kanodia <shubhamsizzles@gmail.com> wrote:\n>\n> On Tue, Sep 3, 2024 at 9:37 PM Junio C Hamano <gitster@pobox.com> wrote:\n> >\n> > Patrick Steinhardt <ps@pks.im> writes:\n> >\n> > > The prefetch refspec would be rewritten by git-maintenance(1) such that\n> > > the destination part (the right-hand side of the refspec) is prefixed\n> > > with `refs/prefetch/`, same as the fetch refspec would be changed in\n> > > this way.\n> > >\n> > > An alternative would be to _not_ rewrite the prefetch refspec at all and\n> > > thus allow the user to prefetch into arbitrary hierarchies. But I'm a\n> > > bit worried that this might cause users to misconfigure prefetches by\n> > > accident, causing it to overwrite their usual set of refs.\n> >\n> > I agree that it is the right place to configure this as attributes\n> > to remotes.  It would make it handy if we could give a catch-all\n> > configuration, though.  For example:\n> >\n> >  [remote \"origin\"]\n> >         prefetch = true\n> >         prefetchref = refs/heads/* refs/tags/*\n> >  [remote \"*\"]\n> >         prefetch = false\n> >\n> > may toggle prefetch off for all remotes, except that the tags and\n> > the local branches of the remote \"origin\" are prefetched.  Instead\n> > of a multi-value configuration variable (like remote.*.fetch) where\n> > we need to worry about clearing convention, we can use a regular\n> > \"last one wins\" variable that is whitespace separated patterns, as\n> > such a pattern can never have a whitespace in it.\n> >\n> > As you too agree with the position to consider \"prefetch\" should be\n> > invisible to the end-users, we should not allow users to specify the\n> > full refspec at all, or if it is forced or not with \"+\" prefix.\n> > Only accept a set of patterns to match, and keep it opaque where in\n> > the local refs/* hierarchy they are stored.  It is an implementation\n> > detail that the users should not have to know about and rely on.\n> >\n> > Thanks.\n>\n> Right so I have good direction to start working on this. I'm going to\n> tackle this in a few parts since this is my first contribution —\n>\n> 1. adding a boolean `prefetch` attribute to `remote` — allows control\n> over remotes that prefetch\n> 2. adding `prefetchref` — allows control over refs per remote.\n> 3. Check if adding a ` [remote \"*\"]` catchall is a possibility\n>\n> For the boolean `prefetch` value the —\n> 1. The default would be assumed to be `true`\n> 2. No changes in behavior to any variation of  `git fetch --prefetch`\n> command itself.\n> If no remote is supplied to `git fetch` then it means default remote,\n> or if `--all` is used, it means all remotes and `--prefetch` for both\n> always\n> prefetches irrespective of the config value.\n>\n> If I understand the codebase well, the changes would primarily affect\n> `remote.c` and `gc.c`.\n>\n> Also much thanks for the feedback both of you've given thus far.\n\nI've published my first patch using gitgitgadget. I'm still figuring\nout what the best way to link to that is but\nI'll add both for now:\n\nhttps://lore.kernel.org/git/pull.1779.git.1725472799637.gitgitgadget@gmail.com/T/#u\nhttps://github.com/gitgitgadget/git/pull/1779\n\n-- \nRegards,\nShubham Kanodia\n"}]}