# git support for "xattrs" (extended filesystem attributes)? 7 messages from 2024-11-26 to 2024-11-27. Participants: Peter B., Junio C Hamano, Jeff King, brian m. carlson. Thread: https://gitlist.dev/t/62549 ## Peter B., 2024-11-26 19:40 Subject: git support for "xattrs" (extended filesystem attributes)? Message-ID: <5b4c09a9-64bb-e672-e604-120563fc1ad6@das-werkstatt.com> URL: https://gitlist.dev/e/5b4c09a9-64bb-e672-e604-120563fc1ad6%40das-werkstatt.com ``` Hi everyone :) I'm evaluating and testing extended attributes (xattrs) for professional archival collection use. I couldn't find any proper documentation/information on how (well) git supports including xattrs in version control. I assume behavior may be similar to any other property-change of files/folders? Would it be possible to have bit-proof preservation of all xattrs? (even larger ones). I'm grateful for any insights, pointers or comments. :D Thanks in advance, Peter B. ``` ## Junio C Hamano, 2024-11-26 19:47 Subject: Re: git support for "xattrs" (extended filesystem attributes)? Message-ID: URL: https://gitlist.dev/e/xmqqr06xwzdc.fsf%40gitster.g In-Reply-To: <5b4c09a9-64bb-e672-e604-120563fc1ad6@das-werkstatt.com> ``` "Peter B." writes: > I'm evaluating and testing extended attributes (xattrs) for > professional archival collection use. > I couldn't find any proper documentation/information on how (well) git > supports including xattrs in version control. Git tracks contents, pathnames where these contents are stored, and the executable bit. Nothing else. Not even the owner/group of each contents, or time of creation or the last modification. I do not think Git is even aware of xattr at all. ``` ## Jeff King, 2024-11-26 22:41 Subject: Re: git support for "xattrs" (extended filesystem attributes)? Message-ID: <20241126224124.GB77402@coredump.intra.peff.net> URL: https://gitlist.dev/e/20241126224124.GB77402%40coredump.intra.peff.net In-Reply-To: <5b4c09a9-64bb-e672-e604-120563fc1ad6@das-werkstatt.com> ``` On Tue, Nov 26, 2024 at 08:40:32PM +0100, Peter B. wrote: > I'm evaluating and testing extended attributes (xattrs) for professional > archival collection use. > I couldn't find any proper documentation/information on how (well) git > supports including xattrs in version control. As Junio noted, Git does not store most metadata at all (and most other version control systems are the same). There are some third party tools to save the information in a separate file within the history and restore it on checkout. I haven't used them myself and some of them are quite old, so YMMV: - https://etckeeper.branchable.com/ - https://github.com/przemoc/metastore -Peff ``` ## Junio C Hamano, 2024-11-27 00:00 Subject: Re: git support for "xattrs" (extended filesystem attributes)? Message-ID: URL: https://gitlist.dev/e/xmqq4j3twnof.fsf%40gitster.g In-Reply-To: <20241126224124.GB77402@coredump.intra.peff.net> ``` Jeff King writes: > There are some third party tools to save the information in a separate > file within the history and restore it on checkout. Ah, you're right of course. We are extensible that way. > I haven't used them > myself and some of them are quite old, so YMMV: > > - https://etckeeper.branchable.com/ > > - https://github.com/przemoc/metastore ``` ## brian m. carlson, 2024-11-27 01:29 Subject: Re: git support for "xattrs" (extended filesystem attributes)? Message-ID: URL: https://gitlist.dev/e/Z0Z2DVuR2PiN-oxy%40tapette.crustytoothpaste.net In-Reply-To: <5b4c09a9-64bb-e672-e604-120563fc1ad6@das-werkstatt.com> ``` On 2024-11-26 at 19:40:32, Peter B. wrote: > Hi everyone :) > > I'm evaluating and testing extended attributes (xattrs) for professional > archival collection use. > I couldn't find any proper documentation/information on how (well) git > supports including xattrs in version control. > > I assume behavior may be similar to any other property-change of > files/folders? > Would it be possible to have bit-proof preservation of all xattrs? (even > larger ones). As others have mentioned, Git doesn't store this. There are a couple ways that one could store this information, depending on what you want to store. One way is to store the metadata in the `.gitattributes` file if you're storing one or two well-known xattrs and the values are well stored as text in the file. You can then use `git ls-attr` and a `post-checkout` hook[0] to set those attributes appropriately. Another way is to store the data as part of an mtree(1) file in the repository. mtree is a file format that comes from the BSDs that can contain a wide variety of data, including file type, uid and gid, text user and group, permissions, size, mtime, cryptographic hashes, and a wide variety of other data. mtree utilities can usually generate a manifest based on the state of the file system, validate that files match a manifest, and usually update existing files such that they match a manifest. There's a Go-based version[1] which does support storing xattrs. Again, you can use a `post-checkout` hook here. Since mtree is a key-value format, you can also extend the value with other data. I use a similar format in my dotfiles to specify install location, for instance, so I recommend this as a good way to store this data. [0] Git LFS uses post-checkout hooks to adjust the read-only bit for files which are lockable but not locked, so this is generally a robust method. [1] https://github.com/vbatts/go-mtree -- brian m. carlson (they/them or he/him) Toronto, Ontario, CA ``` ## Peter B., 2024-11-27 07:59 Subject: Re: git support for "xattrs" (extended filesystem attributes)? Message-ID: <7746de08-2c4e-48bb-0a5f-4e6412a1f643@das-werkstatt.com> URL: https://gitlist.dev/e/7746de08-2c4e-48bb-0a5f-4e6412a1f643%40das-werkstatt.com In-Reply-To: ``` Thank you everyone for your input! This is very good now know. I'll take a look at your suggestions, especially mtree and metastore, as they seem very useful in general. Thanks and have a great day, Peter ``` ## Peter B., 2024-11-27 20:52 Subject: Re: git support for "xattrs" (extended filesystem attributes)? Message-ID: URL: https://gitlist.dev/e/f58d0ca9-305f-50e5-83f7-7939d310b8fc%40das-werkstatt.com In-Reply-To: ``` On 11/27/24 02:29, brian m. carlson wrote: > [...] There's a Go-based version[1] which does support storing > xattrs. Again, you can use a `post-checkout` hook here. > > Since mtree is a key-value format, you can also extend the value with > other data. I use a similar format in my dotfiles to specify install > location, for instance, so I recommend this as a good way to store this > data. I just installed gomtree, and read my xattrs: looks very promising! Thank you for the suggestion. :) I'll read up on "post-checkout hooks" to see how I can integrate this in my git setup. In case anyone's curious, here's an example of gomtree's output text for a file annotated using `yt-dlp --xattr`: ˋˋˋ Universal\040Sequence\040\13320vPbH6UWIc].mp4 xattr.user.xdg.referrer.url=aHR0cHM6Ly93d3cueW91dHViZS5jb20vd2F0Y2g/dj0yMHZQYkg2VVdJYw== xattr.user.dublincore.title=VW5pdmVyc2FsIFNlcXVlbmNl xattr.user.dublincore.date=MjAxNi0wOS0xNA== xattr.user.dublincore.contributor=Q29uc3BpcmFjeQ== xattr.user.dublincore.format=Mjk4IC0gMTI4MHg1NDAgKDcyMHA2MCkrMTQwIC0gYXVkaW8gb25seSAobWVkaXVtKQ== xattr.user.dublincore.description=VmlkZW8gcmVjb3JkaW5nIG9mIHRoZSBmaW5hbCB2ZXJzaW9uIG9mICJVbml2ZXJzYWwgU2VxdWVuY2UiLCBhIDY0IEtCeXRlIHJlYWx0aW1lIGRlbW8gYnkgQ29uc3BpcmFjeS4gRmlyc3QgcHJlc2VudGVkIGF0IHRoZSBGdW5jdGlvbiAyMDE2IGRlbW9wYXJ0eSwgd2hlcmUgaXQgcmVhY2hlZCAxc3QgcGxhY2UuCgpEb3dubG9hZCB0aGUgcmVhbHRpbWUgdmVyc2lvbiBhbmQgc291bmR0cmFjayBhdApodHRwOi8vY29uc3BpcmFjeS5odS9yZWxlYXNlLzY0ay91bml2ZXJzYWxfc2VxdWVuY2UvCgpFbmdpbmUgY29kZSBhbmQgZWZmZWN0cyBieQpCT1lDCgpWaXN1YWxzIGFuZCBkaXJlY3Rpb24gYnkKWk9PTQoKU3ludGggYW5kIG11c2ljIGJ5CkdBUkdBSgoKaHR0cDovL3d3dy5jb25zcGlyYWN5Lmh1Cmh0dHA6Ly9mYWNlYm9vay5jb20vY29uc3BpcmFjeS5odQpodHRwOi8vdHdpdHRlci5jb20vY29uc3BpcmFjeWh1CgojY29uc3BpcmFjeSAjZGVtb3NjZW5lICM2NGsgI2ludHJvICNyZWFsdGltZSAjaHVuZ2FyeQ== ˋˋˋ Values are base64-encoded. Nice. Regards, Peter B. ```