Re: Stability of git-archive, breaking (?) the Github universe, and a possible solution

If the goal is stable tar.gz files, Debian has a very nice soution called pristine-tar[1]. This you to store a tar.gz image which in a very efficient way, by leveraging the objects in the git repository.

[1] https://manpages.debian.org/unstable/pristine-tar/pristine-tar.1.en.html

The data is stored on the pristine-tar branch, and is quite efficient:

% git show --stat pristine-tar
commit 56dded989c9e0c852b8af9ae72ffe94270bfd34a (origin/pristine-tar, github/pristine-tar, pristine-tar)
Author: Theodore Ts'o <tytso@mit.edu>
Date:   Thu Dec 30 01:06:13 2021 -0500

    pristine-tar data for e2fsprogs_1.46.5.orig.tar.gz

 e2fsprogs_1.46.5.orig.tar.gz.asc   |  11 +++++++++++
 e2fsprogs_1.46.5.orig.tar.gz.delta | Bin 0 -> 59034 bytes
 e2fsprogs_1.46.5.orig.tar.gz.id    |   1 +
 3 files changed, 12 insertions(+)

And this allows me to reproduce the original tar.gz file, along with a GPG signature file, which is about 9 megabytes. The *.id file contains the git commit from which the tar file was generated, and this is what allows the *.delta file to be as small as it is.

% pristine-tar checkout e2fsprogs_1.46.5.orig.tar.gz -s e2fsprogs_1.46.5.orig.tar.gz.asc pristine-tar: successfully generated e2fsprogs_1.46.5.orig.tar.gz pristine-tar: successfully generated e2fsprogs_1.46.5.orig.tar.gz.asc

% ls -sh e2fsprogs_1.46.5.orig.tar.gz* 9.1M e2fsprogs_1.46.5.orig.tar.gz 4.0K e2fsprogs_1.46.5.orig.tar.gz.asc

% gpg e2fsprogs_1.46.5.orig.tar.gz.asc
gpg: WARNING: no command supplied.  Trying to guess what you mean ...
gpg: assuming signed data in 'e2fsprogs_1.46.5.orig.tar.gz'
gpg: Signature made Thu 30 Dec 2021 01:02:52 AM EST
gpg:                using RSA key 2B69B954DBFE0879288137C9F2F95956950D81A3
gpg: Good signature from "Theodore Ts'o <tytso@mit.edu>" [ultimate]
gpg:                 aka "Theodore Ts'o <tytso@debian.org>" [ultimate]
gpg:                 aka "Theodore Ts'o <tytso@google.com>" [ultimate]
Primary key fingerprint: 3AB0 57B7 E78D 945C 8C55  91FB D36F 769B C118 04F0
     Subkey fingerprint: 2B69 B954 DBFE 0879 2881  37C9 F2F9 5956 950D 81A3

This is currently a Debian special, and while its functionality was designed to work well with Debian packaging workflows, but it's a general tool that could be used in multiple contexts, not just for Debian packaging.

If I recall correctly, pristine-tar is currently in maintenance mode, and I suspect if someone was interested in investing time into making pristine-tar more portable to other OS's, including MacOS and Windows, and maybe potentially even integrating into git directly, the current maintainer of pristine-tar might be quite happy to let other people give the code more TLC.

						- Ted