gitattributesLarge: .gitattributes too large to parse

Hello everyone,

I'm running into a problem with git fsck and the .gitattributes file. With more recent git versions, it reports the following error on my bare git repository:

$ git --version git version 2.39.2

$ git -C bare_repo fsck Checking object directories: 100% (256/256), done. error in blob 70dc06c1e2e79d8cfa4fb67007edcbb8c941d7e0: gitattributesLarge: .gitattributes too large to parse error in blob 7f2a61db90e023cc2a3b180203b7298cd971250d: gitattributesLarge: .gitattributes too large to parse Checking objects: 100% (33216/33216), done. Verifying commits in commit graph: 100% (2024/2024), done.

The files seems to be around 1.5MB in size:

$ git -C bare_repo cat-file -s 70dc06c1e2e79d8cfa4fb67007edcbb8c941d7e0 1579407 $ git -C bare_repo cat-file -s 7f2a61db90e023cc2a3b180203b7298cd971250d 1579652

With a cloned repository, the error is not shown:

$ git -C cloned_repo fsck Checking object directories: 100% (256/256), done. Checking objects: 100% (33158/33158), done.

I couldn't find a lot of documentation about the size limitations of the .gitattributes file, but I did find the change that seems to have introduced it: https://github.com/git/git/commit/27ab4784d5c9e24345b9f5b443609cbe527c51f9 The change describes that the file needs to be smaller than 100MB, which it is.

Tested on archlinux (rolling release), but also with docker to quickly verify different versions with a completely clean git configuration: https://hub.docker.com/r/bitnami/git

The error seems to occur with git version 2.39.2 2.38.3, but not in 2.38.2.

Why is git showing this message if the file isn't too big? Is there a way to get rid of the message, without updating/deleting the file and having to rewrite the history in git?

Regards, Danny

Hi Danny,

On Tue, Feb 28, 2023 at 05:09:01PM +0100, Danny Smit wrote:

It's interesting that you can cause `fsck` to produce an error in the bare repository but not in the non-bare one. Do you have `fsck.gitattributesLarge` set to anything in the non-bare repository? Are the affected objects in the `fsck.skipList`?

Looking at 27ab4784d5, the comment there says:

    if (!buf || size > ATTR_MAX_FILE_SIZE) {
      /*
       * A missing buffer here is a sign that the caller found the
       * blob too gigantic to load into memory. Let's just consider
       * that an error.
       */
      return report(options, oid, OBJ_BLOB,
                    FSCK_MSG_GITATTRIBUTES_LARGE,
                    ".gitattributes too large to parse");
    }

...so it's possible that the caller indeed found the blob too large to load into memory, which would cause us to emit the ".gitattributes too large to parse" fsck error without a .gitattributes file that actually exceeds 100 MiB in size.

Indeed, the change there 27ab4784d5 (fsck: implement checks for gitattributes, 2022-12-01) exists in v2.38.3 and newer, as well as v2.39.1 and newer. v2.38.2 was released before 27ab4784d5 existed.

You can suppress this message with:

    $ git config fsck.gitattributesLarge ignore

Thanks, Taylor

On Tue, Feb 28, 2023 at 02:15:20PM -0500, Taylor Blau wrote:

I think that "!buf" case would also trigger if the size exceeded core.bigFileThreshold. It might be worth checking for that, too.

-Peff

On Tue, Feb 28, 2023 at 10:30 PM Jeff King <peff@peff.net> wrote:

Thanks, that is useful to know.

The configuration in the non-bare repository is almost clean, except for the remotes and branch, it only contains the following:

$ git config -l --local core.repositoryformatversion=0 core.filemode=true core.bare=false core.logallrefupdates=true

and:

$ git config -l color.ui=true core.autocrlf=input core.bare=false core.filemode=true core.logallrefupdates=true core.repositoryformatversion=0 fsck.gitattributeslarge=ignore pull.rebase=true push.autosetupremote=true status.showuntrackedfiles=normal

So there is nothing in `fsck.skipList`.

As shown above `core.bigFileThreshold` unset is in the local git configuration.

However, I just found the following configuration in the `config` file in the bare repository, where the `core.bigFileThreshold` is defined with a value of 1m. That seems likely to be the cause:

    [core]
            repositoryformatversion = 0
            filemode = true
            bare = true
            bigFileThreshold = 1m
            logallrefupdates = true
            autocrlf = false
            eol = lf
            symlinks = true
    [gc]
            autodetach = false
            auto = 0

After changing the value here, the error is gone.

Thanks for your help! Danny