git support for "xattrs" (extended filesystem attributes)?

Hi everyone :)

I'm evaluating and testing extended attributes (xattrs) for professional archival collection use. I couldn't find any proper documentation/information on how (well) git supports including xattrs in version control.

I assume behavior may be similar to any other property-change of files/folders? Would it be possible to have bit-proof preservation of all xattrs? (even larger ones).

I'm grateful for any insights, pointers or comments. :D

Thanks in advance, Peter B.

"Peter B." <pb@das-werkstatt.com> writes:

Git tracks contents, pathnames where these contents are stored, and the executable bit. Nothing else. Not even the owner/group of each contents, or time of creation or the last modification.

I do not think Git is even aware of xattr at all.

On Tue, Nov 26, 2024 at 08:40:32PM +0100, Peter B. wrote:

As Junio noted, Git does not store most metadata at all (and most other version control systems are the same).

There are some third party tools to save the information in a separate file within the history and restore it on checkout. I haven't used them myself and some of them are quite old, so YMMV:

  - https://etckeeper.branchable.com/

  - https://github.com/przemoc/metastore

-Peff

Jeff King <peff@peff.net> writes:

Ah, you're right of course.  We are extensible that way.

On 2024-11-26 at 19:40:32, Peter B. wrote:

As others have mentioned, Git doesn't store this. There are a couple ways that one could store this information, depending on what you want to store.

One way is to store the metadata in the `.gitattributes` file if you're storing one or two well-known xattrs and the values are well stored as text in the file. You can then use `git ls-attr` and a `post-checkout` hook[0] to set those attributes appropriately.

Another way is to store the data as part of an mtree(1) file in the repository. mtree is a file format that comes from the BSDs that can contain a wide variety of data, including file type, uid and gid, text user and group, permissions, size, mtime, cryptographic hashes, and a wide variety of other data. mtree utilities can usually generate a manifest based on the state of the file system, validate that files match a manifest, and usually update existing files such that they match a manifest. There's a Go-based version[1] which does support storing xattrs. Again, you can use a `post-checkout` hook here.

Since mtree is a key-value format, you can also extend the value with other data. I use a similar format in my dotfiles to specify install location, for instance, so I recommend this as a good way to store this data.

[0] Git LFS uses post-checkout hooks to adjust the read-only bit for files which are lockable but not locked, so this is generally a robust method. [1] https://github.com/vbatts/go-mtree

Thank you everyone for your input! This is very good now know.

I'll take a look at your suggestions, especially mtree and metastore, as they seem very useful in general.

Thanks and have a great day, Peter

On 11/27/24 02:29, brian m. carlson wrote:

I just installed gomtree, and read my xattrs: looks very promising! Thank you for the suggestion. :)

I'll read up on "post-checkout hooks" to see how I can integrate this in my git setup.

In case anyone's curious, here's an example of gomtree's output text for a file annotated using `yt-dlp --xattr`:

``` Universal\040Sequence\040\13320vPbH6UWIc].mp4 xattr.user.xdg.referrer.url=aHR0cHM6Ly93d3cueW91dHViZS5jb20vd2F0Y2g/dj0yMHZQYkg2VVdJYw== xattr.user.dublincore.title=VW5pdmVyc2FsIFNlcXVlbmNl xattr.user.dublincore.date=MjAxNi0wOS0xNA== xattr.user.dublincore.contributor=Q29uc3BpcmFjeQ== xattr.user.dublincore.format=Mjk4IC0gMTI4MHg1NDAgKDcyMHA2MCkrMTQwIC0gYXVkaW8gb25seSAobWVkaXVtKQ== xattr.user.dublincore.description=VmlkZW8gcmVjb3JkaW5nIG9mIHRoZSBmaW5hbCB2ZXJzaW9uIG9mICJVbml2ZXJzYWwgU2VxdWVuY2UiLCBhIDY0IEtCeXRlIHJlYWx0aW1lIGRlbW8gYnkgQ29uc3BpcmFjeS4gRmlyc3QgcHJlc2VudGVkIGF0IHRoZSBGdW5jdGlvbiAyMDE2IGRlbW9wYXJ0eSwgd2hlcmUgaXQgcmVhY2hlZCAxc3QgcGxhY2UuCgpEb3dubG9hZCB0aGUgcmVhbHRpbWUgdmVyc2lvbiBhbmQgc291bmR0cmFjayBhdApodHRwOi8vY29uc3BpcmFjeS5odS9yZWxlYXNlLzY0ay91bml2ZXJzYWxfc2VxdWVuY2UvCgpFbmdpbmUgY29kZSBhbmQgZWZmZWN0cyBieQpCT1lDCgpWaXN1YWxzIGFuZCBkaXJlY3Rpb24gYnkKWk9PTQoKU3ludGggYW5kIG11c2ljIGJ5CkdBUkdBSgoKaHR0cDovL3d3dy5jb25zcGlyYWN5Lmh1Cmh0dHA6Ly9mYWNlYm9vay5jb20vY29uc3BpcmFjeS5odQpodHRwOi8vdHdpdHRlci5jb20vY29uc3BpcmFjeWh1CgojY29uc3BpcmFjeSAjZGVtb3NjZW5lICM2NGsgI2ludHJvICNyZWFsdGltZSAjaHVuZ2FyeQ== ```

Values are base64-encoded. Nice.

Regards, Peter B.