git/list[1] front-page[2] threads[3] people[4] search[5] about
 

[PATCH 0/9] git archive: use gzip again by default, document output stabilty

From
Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Date
Feb 2, 2023, 09:32 UTC
Message-ID
<cover-0.9-00000000000-20230202T093212Z-avarab@gmail.com>
In-Reply-To
<230131.86357rrtsg.gmgdl@evledraar.gmail.com>

As reported in https://lore.kernel.org/git/a812a664-67ea-c0ba-599f-cb79e2d96694@gmail.com/ changing the default "tgz" output method of from "gzip(1)" to our internal "git archive gzip" (using zlib ) broke things for users in the wild that assume that the "git archive" output is stable, most notably GitHub: https://github.com/orgs/community/discussions/45830

Leaving aside the larger question of whether we're going to promise output stability for "git archive" in general, the motivation for that change was to have a working compression method on systems that lacked a gzip(1).

As the disruption of changing the default isn't worth it, let's use gzip(1) again by default, and only fall back on the new "git archive gzip" if it isn't available.

The later parts of this series then document and test for the output stability of the command.

We're not promising anything new there, except that we now promise that we're going to use "gzip" as the default compressor, but that it's up to that command to be stable, should the user desire output stability.

The documentation discusses the various caveats involved, suggests alternatives to checksumming compressed archives, but in the end notes what's been the policy so far: We're not promising that the "tar" output is going to be stable.

The early parts of this series (1-2/9) are clean-up for existing config drift, as later in the series we'll otherwise need to change the divergent config documentation in two places.

CI & branch for this at: https://github.com/avar/git/tree/avar/archive-internal-gzip-not-the-default

Ævar Arnfjörð Bjarmason (9):
  archive & tar config docs: de-duplicate configuration section
  git config docs: document "tar.<format>.{command,remote}"
  archiver API: make the "flags" in "struct archiver" an enum
  archive: omit the shell for built-in "command" filters
  archive-tar.c: move internal gzip implementation to a function
  archive: use "gzip -cn" for stability, not "git archive gzip"
  test-lib.sh: add a lazy GZIP prerequisite
  archive tests: test for "gzip -cn" and "git archive gzip" stability
  git archive docs: document output non-stability
 Documentation/config/tar.txt           | 29 +++++++-
 Documentation/git-archive.txt          | 96 +++++++++++++++++++-------
 archive-tar.c                          | 78 ++++++++++++++-------
 archive.h                              | 11 +--
 t/t5000-tar-tree.sh                    |  2 -
 t/t5005-archive-stability.sh           | 70 +++++++++++++++++++
 t/t5562-http-backend-content-length.sh |  2 -
 t/test-lib.sh                          |  4 ++
 8 files changed, 231 insertions(+), 61 deletions(-)
 create mode 100755 t/t5005-archive-stability.sh
-- 
2.39.1.1392.g63e6d408230
Previous: Eli SchwartzNext: Ævar Arnfjörð Bjarmason
Message 4 of 57 in “Stability of git-archive, breaking (?) the Github universe, and a possible solution”
  1. Eli SchwartzJan 31, 2023
  2. Ævar Arnfjörð BjarmasonJan 31, 2023
  3. Eli SchwartzJan 31, 2023
  4. 0/9 git archive: use gzip again by default, document output stabiltyÆvar Arnfjörð Bjarmason, Feb 2, 2023
  5. 1/9 archive & tar config docs: de-duplicate configuration sectionÆvar Arnfjörð Bjarmason, Feb 2, 2023
  6. 2/9 git config docs: document "tar.<format>.{command,remote}"Ævar Arnfjörð Bjarmason, Feb 2, 2023
  7. 3/9 archiver API: make the "flags" in "struct archiver" an enumÆvar Arnfjörð Bjarmason, Feb 2, 2023
  8. 4/9 archive: omit the shell for built-in "command" filtersÆvar Arnfjörð Bjarmason, Feb 2, 2023
  9. 5/9 archive-tar.c: move internal gzip implementation to a functionÆvar Arnfjörð Bjarmason, Feb 2, 2023
  10. 6/9 archive: use "gzip -cn" for stability, not "git archive gzip"Ævar Arnfjörð Bjarmason, Feb 2, 2023
  11. 7/9 test-lib.sh: add a lazy GZIP prerequisiteÆvar Arnfjörð Bjarmason, Feb 2, 2023
  12. 8/9 archive tests: test for "gzip -cn" and "git archive gzip" stabilityÆvar Arnfjörð Bjarmason, Feb 2, 2023
  13. 9/9 git archive docs: document output non-stabilityÆvar Arnfjörð Bjarmason, Feb 2, 2023
  14. brian m. carlsonFeb 2, 2023
  15. Ævar Arnfjörð BjarmasonFeb 2, 2023
  16. Junio C HamanoFeb 2, 2023
  17. brian m. carlsonFeb 4, 2023
  18. Phillip WoodFeb 2, 2023
  19. Junio C HamanoFeb 2, 2023
  20. Raymond E. PascoFeb 2, 2023
  21. archive: document output stability concernsRaymond E. Pasco, Feb 3, 2023
  22. Ævar Arnfjörð BjarmasonFeb 3, 2023
  23. Phillip WoodFeb 6, 2023
  24. Theodore Ts'oFeb 3, 2023
  25. Junio C HamanoFeb 2, 2023
  26. René ScharfeFeb 4, 2023
  27. Ævar Arnfjörð BjarmasonFeb 5, 2023
  28. René ScharfeFeb 12, 2023
  29. brian m. carlsonJan 31, 2023
  30. Ævar Arnfjörð BjarmasonJan 31, 2023
  31. Konstantin RyabitsevJan 31, 2023
  32. brian m. carlsonJan 31, 2023
  33. Ævar Arnfjörð BjarmasonFeb 1, 2023
  34. demerphqFeb 1, 2023
  35. Michal SuchánekFeb 1, 2023
  36. demerphqFeb 1, 2023
  37. Ævar Arnfjörð BjarmasonFeb 1, 2023
  38. demerphqFeb 1, 2023
  39. Theodore Ts'oFeb 1, 2023
  40. Joey HessFeb 2, 2023
  41. Theodore Ts'oFeb 3, 2023
  42. Ævar Arnfjörð BjarmasonFeb 3, 2023
  43. Raymond E. PascoFeb 1, 2023
  44. brian m. carlsonFeb 1, 2023
  45. Junio C HamanoFeb 1, 2023
  46. brian m. carlsonFeb 2, 2023
  47. rsbecker@nexbridge.comFeb 2, 2023
  48. Ævar Arnfjörð BjarmasonFeb 3, 2023
  49. Ævar Arnfjörð BjarmasonFeb 2, 2023
  50. Eli SchwartzJan 31, 2023
  51. Konstantin RyabitsevJan 31, 2023
  52. Eli SchwartzJan 31, 2023
  53. Konstantin RyabitsevJan 31, 2023
  54. Michal SuchánekJan 31, 2023
  55. brian m. carlsonFeb 1, 2023
  56. Ævar Arnfjörð BjarmasonFeb 1, 2023
  57. brian m. carlsonFeb 1, 2023

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.