{"thread":{"id":"59536","subject":"Negative patterns in cone mode","startedAt":"2023-04-04T12:00:27Z","lastAt":"2023-04-08T21:52:19Z","messageCount":3,"participants":["Brown, Chris","Rudy Rigot"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"474774","messageId":"GVXPR10MB8199D186532A7A285CCF1538B9939@GVXPR10MB8199.EURPRD10.PROD.OUTLOOK.COM","threadId":"59536","inReplyTo":null,"subject":"Negative patterns in cone mode","fromName":"Brown, Chris","fromEmail":"chris.c.brown@siemens.com","sentAt":"2023-04-04T12:00:18Z","receivedAt":"2023-04-04T12:00:27Z","isPatch":false,"sender":{"key":"chris.c.brown@siemens.com","avatar":null},"body":"Hi all,\n\nI'm facing an issue with negative patterns in cone mode. I can't tell from the docs or git code if I misunderstand the usage, am trying something not supported, or if there is a bug.\n\nWhat I want to do is exclude a small number of directories (and their subdirectories) from a large repo, without explicitly declaring all the other directories that I wish to keep.\n\nMy directory structure is:\nroot\n^----A\n        ^----B\n        ^----C\n\nMy sparse-checkout file is:\n/*\n!/A/B/\n\nI've set up an MRE:\n1. git clone --sparse https://github.com/peakschris/sparse.git\n2. cd sparse\n3. copy sparse-checkout.txt .git\\info\\sparse-checkout\n4. git sparse-checkout reapply\n\nOutput:\ngit sparse-checkout reapply\nwarning: unrecognized negative pattern: '/A/B'\nwarning: disabling cone pattern matching\nwarning: unrecognized negative pattern: '/A/B'\nwarning: disabling cone pattern matching\n\nThanks! Chris\n\n\n"},{"id":"475036","messageId":"CANaDLWJ4XSFUULc4PGen_trsyJ1_K1qoufisoxgpjCfMhoDjKQ@mail.gmail.com","threadId":"59536","inReplyTo":"GVXPR10MB8199D186532A7A285CCF1538B9939@GVXPR10MB8199.EURPRD10.PROD.OUTLOOK.COM","subject":"Re: Negative patterns in cone mode","fromName":"Rudy Rigot","fromEmail":"rudy.rigot@gmail.com","sentAt":"2023-04-08T15:43:27Z","receivedAt":"2023-04-08T15:43:44Z","isPatch":false,"sender":{"key":"rudy.rigot@gmail.com","avatar":"https://avatars.githubusercontent.com/u/552279?v=4"},"body":"Hi,\n\n> I'm facing an issue with negative patterns in cone mode.\n> I can't tell from the docs or git code if I misunderstand the\n> usage, am trying something not supported, or if there is a\n> bug.\n\nMy understanding so far, and I would appreciate if someone can\ncorrect me if I'm wrong, is that the point of cone mode is that there\ncan't be negative patterns, and everything is a positive rule, so\nthe match search can stop as soon as a positive rule is found.\n\nMy understanding has been that it was designed with the use\ncase in mind of large mono-repos made of several independent\napplications, of which a given developer only needs a few. For\ninstance, if I am an iOS developer, I will configure my sparse\ncheckout to have the back-end code and the iOS code, but not\nthe front-end code and the Android code.\n\nI don't know if that's accurate because I'm not as well-versed\nabout it as I should be, so I would appreciate if someone could\ncorrect my understanding. It is the chief reason we are\nsticking with non-cone mode with our massive monolith at\nSalesforce: it is not a mono-repo of independent applications,\nbut one massive monolith of which only a few (very large) files\nare not needed for all devs.\n\nThanks in advance for anyone who may have insights.\n"},{"id":"475039","messageId":"GVXPR10MB8199695A0BD5D7A404CCCFD4B9979@GVXPR10MB8199.EURPRD10.PROD.OUTLOOK.COM","threadId":"59536","inReplyTo":"CANaDLWJ4XSFUULc4PGen_trsyJ1_K1qoufisoxgpjCfMhoDjKQ@mail.gmail.com","subject":"RE: Negative patterns in cone mode","fromName":"Brown, Chris","fromEmail":"chris.c.brown@siemens.com","sentAt":"2023-04-08T21:52:09Z","receivedAt":"2023-04-08T21:52:19Z","isPatch":false,"sender":{"key":"chris.c.brown@siemens.com","avatar":null},"body":"Thanks for the inputs. I came to a similar conclusion after poring over the docs. We have a similar situation to you; by excluding ~50 directories not required at build time we can avoid 18GB of files on disk and reduce the file count by 2x. I ended up writing a python script that uses git ls-tree and then converts a few negative patterns specified by the developer into a huge set of positive patterns for all directories in the tree *except* those that should be excluded. The performance is good; the python script takes around 1s, and then allows the sparse checkouts to operate in cone mode which works in seconds. This is great compared to non-cone-mode processing which takes several minutes to sparse-checkout the same directories expressed directly as negative patterns.\n\nThis suggests to me that cone mode *could* be enhanced to natively support a restricted type of negative pattern (exclude this directory and all subdirs) without performance overhead.\n\nThe problem with my script is that it is quite complex, generates thousands of positive patterns, and I am not yet 100% convinced that the complexity is worth it over simply paying the cost to download the monorepo.\n\nChris\n\n-----Original Message-----\nFrom: Rudy Rigot <rudy.rigot@gmail.com> \nSent: 08 April 2023 16:43\nTo: Brown, Chris (DI SW LCS CF) <chris.c.brown@siemens.com>\nCc: git@vger.kernel.org\nSubject: Re: Negative patterns in cone mode\n\nHi,\n\n> I'm facing an issue with negative patterns in cone mode.\n> I can't tell from the docs or git code if I misunderstand the usage, \n> am trying something not supported, or if there is a bug.\n\nMy understanding so far, and I would appreciate if someone can correct me if I'm wrong, is that the point of cone mode is that there can't be negative patterns, and everything is a positive rule, so the match search can stop as soon as a positive rule is found.\n\nMy understanding has been that it was designed with the use case in mind of large mono-repos made of several independent applications, of which a given developer only needs a few. For instance, if I am an iOS developer, I will configure my sparse checkout to have the back-end code and the iOS code, but not the front-end code and the Android code.\n\nI don't know if that's accurate because I'm not as well-versed about it as I should be, so I would appreciate if someone could correct my understanding. It is the chief reason we are sticking with non-cone mode with our massive monolith at\nSalesforce: it is not a mono-repo of independent applications, but one massive monolith of which only a few (very large) files are not needed for all devs.\n\nThanks in advance for anyone who may have insights.\n"}]}