{"thread":{"id":"39270","subject":"Please consider extending .gitignore to support setting of a file size limit","startedAt":"2015-05-07T21:14:26Z","lastAt":"2015-05-08T22:13:59Z","messageCount":7,"participants":["Roger Mendes","Junio C Hamano","Thomas Koch","Kevin Daudt","Fredrik Gustafsson","Philip Oakley","Stefan Beller"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"260784","messageId":"CAC+AdWRpJnt2PN1YcB7VoHcKsO86_DhE1mOB5Ls9bWaEd_XKXQ@mail.gmail.com","threadId":"39270","inReplyTo":null,"subject":"Please consider extending .gitignore to support setting of a file size limit","fromName":"Roger Mendes","fromEmail":"sproger1@gmail.com","sentAt":"2015-05-07T21:14:26Z","receivedAt":"2015-05-07T21:14:26Z","isPatch":false,"sender":{"key":"sproger1@gmail.com","avatar":null},"body":"Allow .gitignore to support setting a file size limit so that all\nfiles over a certain size will by automatically ignored when\nperforming git add, commit -a...;\nExclusions to this size limit for files can be allowed by specifying\nexceptions '!' syntax that already exists. It probably should be\nconsidered to have a default limit already built into git of say 100MB\n(used by github) which can be changed in the .gitignore file.\n\nThis would avoid accidental commits of large files which are known to\nbe poorly supported.\n\nThis seems like a fairly simple feature to add, I'm not sure why no\none has done this or thought of it given the number of complaints,\nquestions... with large file handling.\n\nWe can ignore by file name attribute, why not by file size?\n\nYes, I know I can add pre-commit scripts... to workaround this issue,\nbut there should be a better way (or maybe I'm missing something?).\n\n\nAccidental commits of large files is a common mistake, and often\nresults in the commit taking an extremely long time to process and\nsometimes if you kill the process, then you end up with corrupt git\nrepo (at least in jgit, which is used by Netbeans).\n\nIMHO this would be a very useful feature.\n"},{"id":"260786","messageId":"xmqqy4l0m5wf.fsf@gitster.dls.corp.google.com","threadId":"39270","inReplyTo":"CAC+AdWRpJnt2PN1YcB7VoHcKsO86_DhE1mOB5Ls9bWaEd_XKXQ@mail.gmail.com","subject":"Re: Please consider extending .gitignore to support setting of a file size limit","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2015-05-07T21:31:12Z","receivedAt":"2015-05-07T21:31:12Z","isPatch":false,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Roger Mendes <sproger1@gmail.com> writes:\n\n> Allow .gitignore to support setting a file size limit so that all\n> files over a certain size will by automatically ignored when\n> performing git add, commit -a...;\n\nNo.\n\nI would not say that it would not make sense to have a method to\ntell your Git to reject an attempt to add certain files to the\nindex, but .gitignore is make that decision based on paths and\nis a bad place to do any other decision, so take the above \"No\"\nas \"No, we do not consider butchering .gitignore for that\", not\nas \"No, you shouldn't base your decision on file size\".\n\nAlso, once the path is tracked, then \"git add\" will keep tracking it\nunless and until you \"git rm --cached\" it.  So you may \"git add\" a\n1kB file, grow the file to 500MB and then the next \"git add\" should\nregister the updated contents to the index.  Whatever you do when\ndesigning that new feature, you should need to take that into\naccount.\n\nHaving said all that, why is it that you do not want to track\ncertain files that happen to be large?  Don't the existing large\nfile support facility work for you (or perhaps you are not using\nthem)?  It might be that your time is better spent on improving\nthat, instead of working around the issue, no?\n"},{"id":"260797","messageId":"201505080844.40002.thomas@koch.ro","threadId":"39270","inReplyTo":"xmqqy4l0m5wf.fsf@gitster.dls.corp.google.com","subject":"Re: Please consider extending .gitignore to support setting of a file size limit","fromName":"Thomas Koch","fromEmail":"thomas@koch.ro","sentAt":"2015-05-08T06:44:39Z","receivedAt":"2015-05-08T06:44:39Z","isPatch":false,"sender":{"key":"thomas@koch.ro","avatar":null},"body":"On Thursday, May 07, 2015 11:31:12 PM Junio C Hamano wrote:\n> Roger Mendes <sproger1@gmail.com> writes:\n> > Allow .gitignore to support setting a file size limit so that all\n> > files over a certain size will by automatically ignored when\n> > performing git add, commit -a...;\n> \n> No.\n\nPlease have a look into \"git annex\". It provides large file support without\nblowing up the repository size entirely and has facilities to decide what to\ndo with files based on multiple criterias like file size.\n\nMaybe some features from git annex have matured enough to be proposed\nfor inclusion into git?\n\nRegards, Thomas Koch\n"},{"id":"260798","messageId":"20150508065459.GB31516@vps892.directvps.nl","threadId":"39270","inReplyTo":"xmqqy4l0m5wf.fsf@gitster.dls.corp.google.com","subject":"Re: Please consider extending .gitignore to support setting of a file size limit","fromName":"Kevin Daudt","fromEmail":"me@ikke.info","sentAt":"2015-05-08T06:54:59Z","receivedAt":"2015-05-08T06:54:59Z","isPatch":false,"sender":{"key":"me@ikke.info","avatar":"https://avatars.githubusercontent.com/u/135698?v=4"},"body":"On Thu, May 07, 2015 at 02:31:12PM -0700, Junio C Hamano wrote:\n> Roger Mendes <sproger1@gmail.com> writes:\n> \n> Having said all that, why is it that you do not want to track\n> certain files that happen to be large?  Don't the existing large\n> file support facility work for you (or perhaps you are not using\n> them)?  It might be that your time is better spent on improving\n> that, instead of working around the issue, no?\n> \n\nMy experience in #git is that people want to prevent other people\naccidentelly adding large files, resulting in all sorts of problems,\nwhich takes a lot of effort to fix if you're not an experienced user.\n"},{"id":"260799","messageId":"20150508070906.GC8248@paksenarrion.iveqy.com","threadId":"39270","inReplyTo":"20150508065459.GB31516@vps892.directvps.nl","subject":"Re: Please consider extending .gitignore to support setting of a file size limit","fromName":"Fredrik Gustafsson","fromEmail":"iveqy@iveqy.com","sentAt":"2015-05-08T07:09:06Z","receivedAt":"2015-05-08T07:09:06Z","isPatch":false,"sender":{"key":"iveqy@iveqy.com","avatar":"https://avatars.githubusercontent.com/u/761743?v=4"},"body":"On Fri, May 08, 2015 at 08:54:59AM +0200, Kevin Daudt wrote:\n> My experience in #git is that people want to prevent other people\n> accidentelly adding large files, resulting in all sorts of problems,\n> which takes a lot of effort to fix if you're not an experienced user.\n\nI second this. In #git@freenode we often get people trying to reduce\nrepo size since they or a collegue accidental added a huge binary file.\n\nThey do a filter branch and remove that file. However then they need to\ndo a forced push. And the person (the collegue) that didn't had enough\ngit experience to understand why it was a mistake to add a big file, now\nends up with having to sync with a rewritten remote.\n\nSo I can really see that there's a demand for such a feature. However, I\nwonder if this shouldn't just be a pre-recieve hook?\n\nIf we want to inform the user, we might just add a pre-commit hook for\nthis.\n\nThe problem with pre-commit hooks is distribution of them. As it is with\nmaking this a configuration option. There's really no good way to share\nconfiguration and hooks between git repos today as I know of. (Yes, I'm\naware of this beeing a hard problem due to security concerns).\n\n-- \nFredrik Gustafsson\n\nphone: +46 733-608274\ne-mail: iveqy@iveqy.com\nwebsite: http://www.iveqy.com\n"},{"id":"260873","messageId":"682DB2BD13D143CEB606BF9C8E4E55D5@PhilipOakley","threadId":"39270","inReplyTo":"20150508065459.GB31516@vps892.directvps.nl","subject":"Re: Please consider extending .gitignore to support setting of a file size limit","fromName":"Philip Oakley","fromEmail":"philipoakley@iee.org","sentAt":"2015-05-08T21:55:17Z","receivedAt":"2015-05-08T21:55:17Z","isPatch":false,"sender":{"key":"philipoakley@iee.email","avatar":"https://avatars.githubusercontent.com/u/914343?v=4"},"body":"From: \"Kevin Daudt\" <me@ikke.info>\n> On Thu, May 07, 2015 at 02:31:12PM -0700, Junio C Hamano wrote:\n>> Roger Mendes <sproger1@gmail.com> writes:\n>>\n>> Having said all that, why is it that you do not want to track\n>> certain files that happen to be large?  Don't the existing large\n>> file support facility work for you (or perhaps you are not using\n>> them)?  It might be that your time is better spent on improving\n>> that, instead of working around the issue, no?\n>>\n>\n> My experience in #git is that people want to prevent other people\n> accidentelly adding large files, resulting in all sorts of problems,\n> which takes a lot of effort to fix if you're not an experienced user.\n> --\nI'd also support this option.\n\nOne issue is to locate a suitable method/character code for the 'magic \npathspec' that would define a size limit that can be used for the 'git \nadd .' etc., and possibly also for sparse checkout (definitely an area \nwhere a size limit may be appropriate for some users).\n\nThe ability to extend the ignore method could also, in the future be \nused for a possible narrow clone (when desired).\n--\nPhilip \n"},{"id":"260874","messageId":"CAGZ79kazifa82bob5RvL4gGarkKympb-aLDBmhT=y7ddTj-aTg@mail.gmail.com","threadId":"39270","inReplyTo":"20150508070906.GC8248@paksenarrion.iveqy.com","subject":"Re: Please consider extending .gitignore to support setting of a file size limit","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2015-05-08T22:13:59Z","receivedAt":"2015-05-08T22:13:59Z","isPatch":false,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"On Fri, May 8, 2015 at 12:09 AM, Fredrik Gustafsson <iveqy@iveqy.com> wrote:\n> On Fri, May 08, 2015 at 08:54:59AM +0200, Kevin Daudt wrote:\n>> My experience in #git is that people want to prevent other people\n>> accidentelly adding large files, resulting in all sorts of problems,\n>> which takes a lot of effort to fix if you're not an experienced user.\n>\n> I second this. In #git@freenode we often get people trying to reduce\n> repo size since they or a collegue accidental added a huge binary file.\n>\n> They do a filter branch and remove that file. However then they need to\n> do a forced push. And the person (the collegue) that didn't had enough\n> git experience to understand why it was a mistake to add a big file, now\n> ends up with having to sync with a rewritten remote.\n>\n> So I can really see that there's a demand for such a feature. However, I\n> wonder if this shouldn't just be a pre-recieve hook?\n>\n> If we want to inform the user, we might just add a pre-commit hook for\n> this.\n>\n> The problem with pre-commit hooks is distribution of them. As it is with\n> making this a configuration option. There's really no good way to share\n> configuration and hooks between git repos today as I know of. (Yes, I'm\n> aware of this beeing a hard problem due to security concerns).\n>\n\nAny problem in computer science can be solved with another layer of indirection\nthey said. So:\n\nMaybe we can just distribute some good defaults (or your internal distribution\nfor git can ship adapted versions of the default). And instead of just running\na hook, maybe the clone/fetch could tell the client to enable these\npredefined hooks.\nso maybe we'd need a fetch capability \"recommended-hooks\" which could look like\n\n    recommended-hooks=<name>,<sha1>\n\nThe name would indicate what hook you're interested in such as\n.git/hooks/pre-commit-warn-huge-size.sample and the sha1 would be the\nsha1 of that default hook to ensure nobody has tampered with that script.\n\nI am not sure if we need the sha1, maybe that's overengineered. I mean trusting\na default script is as much as trusting the git binary on your system, isn't it?\n"}]}