{"thread":{"id":"59324","subject":"gitattributesLarge: .gitattributes too large to parse","startedAt":"2023-02-28T16:09:24Z","lastAt":"2023-03-01T09:14:10Z","messageCount":4,"participants":["Danny Smit","Taylor Blau","Jeff King"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"472849","messageId":"CACoz=-x6kVGMq1oCq8-5k2Hb82DCqi94_EmX6w4_9uGXnwBLaQ@mail.gmail.com","threadId":"59324","inReplyTo":null,"subject":"gitattributesLarge: .gitattributes too large to parse","fromName":"Danny Smit","fromEmail":"danny.smit.0@gmail.com","sentAt":"2023-02-28T16:09:01Z","receivedAt":"2023-02-28T16:09:24Z","isPatch":false,"sender":{"key":"danny.smit.0@gmail.com","avatar":null},"body":"Hello everyone,\n\nI'm running into a problem with git fsck and the .gitattributes file.\nWith more recent git versions, it reports the following error on my\nbare git repository:\n\n$ git --version\ngit version 2.39.2\n\n$ git -C bare_repo fsck\nChecking object directories: 100% (256/256), done.\nerror in blob 70dc06c1e2e79d8cfa4fb67007edcbb8c941d7e0:\ngitattributesLarge: .gitattributes too large to parse\nerror in blob 7f2a61db90e023cc2a3b180203b7298cd971250d:\ngitattributesLarge: .gitattributes too large to parse\nChecking objects: 100% (33216/33216), done.\nVerifying commits in commit graph: 100% (2024/2024), done.\n\nThe files seems to be around 1.5MB in size:\n\n$ git -C bare_repo cat-file -s 70dc06c1e2e79d8cfa4fb67007edcbb8c941d7e0\n1579407\n$ git -C bare_repo cat-file -s 7f2a61db90e023cc2a3b180203b7298cd971250d\n1579652\n\nWith a cloned repository, the error is not shown:\n\n$ git -C cloned_repo fsck\nChecking object directories: 100% (256/256), done.\nChecking objects: 100% (33158/33158), done.\n\nI couldn't find a lot of documentation about the size limitations of\nthe .gitattributes file, but I did find the change that seems to have\nintroduced it: https://github.com/git/git/commit/27ab4784d5c9e24345b9f5b443609cbe527c51f9\nThe change describes that the file needs to be smaller than 100MB, which it is.\n\nTested on archlinux (rolling release), but also with docker to quickly\nverify different versions with a completely clean git configuration:\nhttps://hub.docker.com/r/bitnami/git\n\nThe error seems to occur with git version 2.39.2 2.38.3, but not in 2.38.2.\n\nWhy is git showing this message if the file isn't too big?\nIs there a way to get rid of the message, without updating/deleting\nthe file and having to rewrite the history in git?\n\nRegards,\nDanny\n"},{"id":"472859","messageId":"Y/5SyNUbVfKFMxsf@nand.local","threadId":"59324","inReplyTo":"CACoz=-x6kVGMq1oCq8-5k2Hb82DCqi94_EmX6w4_9uGXnwBLaQ@mail.gmail.com","subject":"Re: gitattributesLarge: .gitattributes too large to parse","fromName":"Taylor Blau","fromEmail":"me@ttaylorr.com","sentAt":"2023-02-28T19:15:20Z","receivedAt":"2023-02-28T19:15:38Z","isPatch":false,"sender":{"key":"me@ttaylorr.com","avatar":"https://avatars.githubusercontent.com/u/301000140?v=4"},"body":"Hi Danny,\n\nOn Tue, Feb 28, 2023 at 05:09:01PM +0100, Danny Smit wrote:\n> I couldn't find a lot of documentation about the size limitations of\n> the .gitattributes file, but I did find the change that seems to have\n> introduced it: https://github.com/git/git/commit/27ab4784d5c9e24345b9f5b443609cbe527c51f9\n> The change describes that the file needs to be smaller than 100MB, which it is.\n\nIt's interesting that you can cause `fsck` to produce an error in the\nbare repository but not in the non-bare one. Do you have\n`fsck.gitattributesLarge` set to anything in the non-bare repository?\nAre the affected objects in the `fsck.skipList`?\n\nLooking at 27ab4784d5, the comment there says:\n\n    if (!buf || size > ATTR_MAX_FILE_SIZE) {\n      /*\n       * A missing buffer here is a sign that the caller found the\n       * blob too gigantic to load into memory. Let's just consider\n       * that an error.\n       */\n      return report(options, oid, OBJ_BLOB,\n                    FSCK_MSG_GITATTRIBUTES_LARGE,\n                    \".gitattributes too large to parse\");\n    }\n\n...so it's possible that the caller indeed found the blob too large to\nload into memory, which would cause us to emit the \".gitattributes too\nlarge to parse\" fsck error without a .gitattributes file that actually\nexceeds 100 MiB in size.\n\n> The error seems to occur with git version 2.39.2 2.38.3, but not in\n> 2.38.2.\n\nIndeed, the change there 27ab4784d5 (fsck: implement checks for\ngitattributes, 2022-12-01) exists in v2.38.3 and newer, as well as\nv2.39.1 and newer. v2.38.2 was released before 27ab4784d5 existed.\n\n> Is there a way to get rid of the message, without updating/deleting\n> the file and having to rewrite the history in git?\n\nYou can suppress this message with:\n\n    $ git config fsck.gitattributesLarge ignore\n\nThanks,\nTaylor\n"},{"id":"472861","messageId":"Y/5yj8uAEM3Pz7T7@coredump.intra.peff.net","threadId":"59324","inReplyTo":"Y/5SyNUbVfKFMxsf@nand.local","subject":"Re: gitattributesLarge: .gitattributes too large to parse","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2023-02-28T21:30:55Z","receivedAt":"2023-02-28T21:31:01Z","isPatch":false,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Tue, Feb 28, 2023 at 02:15:20PM -0500, Taylor Blau wrote:\n\n> On Tue, Feb 28, 2023 at 05:09:01PM +0100, Danny Smit wrote:\n> > I couldn't find a lot of documentation about the size limitations of\n> > the .gitattributes file, but I did find the change that seems to have\n> > introduced it: https://github.com/git/git/commit/27ab4784d5c9e24345b9f5b443609cbe527c51f9\n> > The change describes that the file needs to be smaller than 100MB, which it is.\n> \n> It's interesting that you can cause `fsck` to produce an error in the\n> bare repository but not in the non-bare one. Do you have\n> `fsck.gitattributesLarge` set to anything in the non-bare repository?\n> Are the affected objects in the `fsck.skipList`?\n> \n> Looking at 27ab4784d5, the comment there says:\n> \n>     if (!buf || size > ATTR_MAX_FILE_SIZE) {\n>       /*\n>        * A missing buffer here is a sign that the caller found the\n>        * blob too gigantic to load into memory. Let's just consider\n>        * that an error.\n>        */\n>       return report(options, oid, OBJ_BLOB,\n>                     FSCK_MSG_GITATTRIBUTES_LARGE,\n>                     \".gitattributes too large to parse\");\n>     }\n> \n> ...so it's possible that the caller indeed found the blob too large to\n> load into memory, which would cause us to emit the \".gitattributes too\n> large to parse\" fsck error without a .gitattributes file that actually\n> exceeds 100 MiB in size.\n\nI think that \"!buf\" case would also trigger if the size exceeded\ncore.bigFileThreshold. It might be worth checking for that, too.\n\n-Peff\n"},{"id":"472883","messageId":"CACoz=-zOhoe=gQ_0Jr=rbdBvh=tQhLQ=Hfmb4Z=ZePpsaWcTHw@mail.gmail.com","threadId":"59324","inReplyTo":"Y/5yj8uAEM3Pz7T7@coredump.intra.peff.net","subject":"Re: gitattributesLarge: .gitattributes too large to parse","fromName":"Danny Smit","fromEmail":"danny.smit.0@gmail.com","sentAt":"2023-03-01T09:13:18Z","receivedAt":"2023-03-01T09:14:10Z","isPatch":false,"sender":{"key":"danny.smit.0@gmail.com","avatar":null},"body":"On Tue, Feb 28, 2023 at 10:30 PM Jeff King <peff@peff.net> wrote:\n>\n> On Tue, Feb 28, 2023 at 02:15:20PM -0500, Taylor Blau wrote:\n> > You can suppress this message with:\n> >\n> >    $ git config fsck.gitattributesLarge ignore\n\nThanks, that is useful to know.\n\n> >\n> > It's interesting that you can cause `fsck` to produce an error in the\n> > bare repository but not in the non-bare one. Do you have\n> > `fsck.gitattributesLarge` set to anything in the non-bare repository?\n> > Are the affected objects in the `fsck.skipList`?\n\nThe configuration in the non-bare repository is almost clean, except\nfor the remotes and branch, it only contains the following:\n\n$ git config -l --local\ncore.repositoryformatversion=0\ncore.filemode=true\ncore.bare=false\ncore.logallrefupdates=true\n\nand:\n\n$ git config -l\ncolor.ui=true\ncore.autocrlf=input\ncore.bare=false\ncore.filemode=true\ncore.logallrefupdates=true\ncore.repositoryformatversion=0\nfsck.gitattributeslarge=ignore\npull.rebase=true\npush.autosetupremote=true\nstatus.showuntrackedfiles=normal\n\nSo there is nothing in `fsck.skipList`.\n\n> > Looking at 27ab4784d5, the comment there says:\n> >\n> >     if (!buf || size > ATTR_MAX_FILE_SIZE) {\n> >       /*\n> >        * A missing buffer here is a sign that the caller found the\n> >        * blob too gigantic to load into memory. Let's just consider\n> >        * that an error.\n> >        */\n> >       return report(options, oid, OBJ_BLOB,\n> >                     FSCK_MSG_GITATTRIBUTES_LARGE,\n> >                     \".gitattributes too large to parse\");\n> >     }\n> >\n> > ...so it's possible that the caller indeed found the blob too large to\n> > load into memory, which would cause us to emit the \".gitattributes too\n> > large to parse\" fsck error without a .gitattributes file that actually\n> > exceeds 100 MiB in size.\n>\n> I think that \"!buf\" case would also trigger if the size exceeded\n> core.bigFileThreshold. It might be worth checking for that, too.\n\nAs shown above `core.bigFileThreshold` unset is in the local git configuration.\n\nHowever, I just found the following configuration in the `config` file\nin the  bare repository, where the `core.bigFileThreshold` is defined\nwith a value of 1m. That seems likely to be the cause:\n\n    [core]\n            repositoryformatversion = 0\n            filemode = true\n            bare = true\n            bigFileThreshold = 1m\n            logallrefupdates = true\n            autocrlf = false\n            eol = lf\n            symlinks = true\n    [gc]\n            autodetach = false\n            auto = 0\n\nAfter changing the value here, the error is gone.\n\nThanks for your help!\nDanny\n"}]}