git/list[1] front-page[2] threads[3] people[4] search[5] about
 

[PATCH v3 3/4] archive-tar: write extended headers for far-future mtime

From
Jeff King <peff@peff.net>
Date
Jun 23, 2016, 23:21 UTC
Message-ID
<20160623232112.GC3668@sigill.intra.peff.net>
In-Reply-To
<20160623231512.GA27683@sigill.intra.peff.net>

The ustar format represents timestamps as seconds since the epoch, but only has room to store 11 octal digits. To express anything larger, we need to use an extended header. This is exactly the same case we fixed for the size field in the previous commit, and the solution here follows the same pattern.

This is even mentioned as an issue in f2f0267 (archive-tar: use xsnprintf for trivial formatting, 2015-09-24), but since it only affected things far in the future, it wasn't deemed worth dealing with. But note that my calculations claiming thousands of years were off there; because our xsnprintf produces a NUL byte, we only have until the year 2242 to fix this.

Given that this is just around the corner (geologically speaking, anyway), and because it's easy to fix, let's just make it work. Unlike the previous fix for "size", where we had to write an individual extended header for each file, we can write one global header (since we have only one mtime for the whole archive).

There's a slight bit of trickiness there. We may already be writing a global header with a "comment" field for the commit sha1. So we need to write our new field into the same header. To do this, we push the decision of whether to write such a header down into write_global_extended_header(), which will now assemble the header as it sees fit, and will return early if we have nothing to write (in practice, we'll only have a large mtime if it comes from a commit, but this makes it also work if you set your system clock ahead such that time() returns a huge value).

Note that we don't (and never did) handle negative timestamps (i.e., before 1970). This would probably not be too hard to support in the same way, but since git does not support negative timestamps at all, I didn't bother here.

After writing the extended header, we munge the timestamp in the ustar headers to the maximum-allowable size. This is wrong, but it's the least-wrong thing we can provide to a tar implementation that doesn't understand pax headers (it's also what GNU tar does).

Helped-by: René Scharfe <l.s.r@web.de>
Signed-off-by: Jeff King <peff@peff.net>
---
 archive-tar.c       | 16 +++++++++++++---
 t/t5000-tar-tree.sh |  4 ++--
 2 files changed, 15 insertions(+), 5 deletions(-)
diff --git a/archive-tar.c b/archive-tar.c
index 274bdfa..0bb164c 100644
--- a/archive-tar.c
+++ b/archive-tar.c
@@ -317,7 +317,18 @@ static int write_global_extended_header(struct archiver_args *args)
 	unsigned int mode;
 	int err = 0;
 
-	strbuf_append_ext_header(&ext_header, "comment", sha1_to_hex(sha1), 40);
+	if (sha1)
+		strbuf_append_ext_header(&ext_header, "comment",
+					 sha1_to_hex(sha1), 40);
+	if (args->time > 077777777777UL) {
+		strbuf_append_ext_header_uint(&ext_header, "mtime",
+					      args->time);
+		args->time = 077777777777UL;
+	}
+
+	if (!ext_header.len)
+		return 0;
+
 	memset(&header, 0, sizeof(header));
 	*header.typeflag = TYPEFLAG_GLOBAL_HEADER;
 	mode = 0100666;
@@ -402,8 +413,7 @@ static int write_tar_archive(const struct archiver *ar,
 {
 	int err = 0;
 
-	if (args->commit_sha1)
-		err = write_global_extended_header(args);
+	err = write_global_extended_header(args);
 	if (!err)
 		err = write_archive_entries(args, write_tar_entry);
 	if (!err)
diff --git a/t/t5000-tar-tree.sh b/t/t5000-tar-tree.sh
index 79dbc88..07e0bdc 100755
--- a/t/t5000-tar-tree.sh
+++ b/t/t5000-tar-tree.sh
@@ -382,11 +382,11 @@ test_expect_success 'set up repository with far-future commit' '
 		git commit -m "tempori parendum"
 '
 
-test_expect_failure 'generate tar with future mtime' '
+test_expect_success 'generate tar with future mtime' '
 	git archive HEAD >future.tar
 '
 
-test_expect_failure TAR_HUGE 'tar can encode dates far in future' '
+test_expect_success TAR_HUGE 'tar can encode dates far in future' '
 	echo 4147 >expect &&
 	tar_info future.tar | cut -d" " -f2 >actual &&
 	test_cmp expect actual
-- 
2.9.0.217.g096ca68
Previous: Jeff KingNext: Junio C Hamano
Message 55 of 61 in “friendlier handling of overflows in archive-tar”
  1. 0/2 friendlier handling of overflows in archive-tarJeff King, Jun 16, 2016
  2. 1/2 archive-tar: write extended headers for file sizes >= 8GBJeff King, Jun 16, 2016
  3. René ScharfeJun 20, 2016
  4. Jeff KingJun 21, 2016
  5. Jeff KingJun 21, 2016
  6. René ScharfeJun 21, 2016
  7. René ScharfeJun 21, 2016
  8. Jeff KingJun 21, 2016
  9. René ScharfeJun 22, 2016
  10. Jeff KingJun 21, 2016
  11. René ScharfeJun 22, 2016
  12. Jeff KingJun 23, 2016
  13. René ScharfeJun 21, 2016
  14. Robin H. JohnsonJun 21, 2016
  15. Jeff KingJun 21, 2016
  16. 2/2 archive-tar: write extended headers for far-future mtimeJeff King, Jun 16, 2016
  17. René ScharfeJun 20, 2016
  18. René ScharfeJun 22, 2016
  19. Jeff KingJun 23, 2016
  20. René ScharfeJun 23, 2016
  21. Jeff KingJun 23, 2016
  22. Junio C HamanoJun 16, 2016
  23. Jeff KingJun 21, 2016
  24. 1/2 archive-tar: write extended headers for file sizes >= 8GBJeff King, Jun 21, 2016
  25. 2/2 archive-tar: write extended headers for far-future mtimeJeff King, Jun 21, 2016
  26. Junio C HamanoJun 21, 2016
  27. friendlier handling of overflows in archive-tarJeff King, Jun 23, 2016
  28. 1/4 t5000: test tar files that overflow ustar headersJeff King, Jun 23, 2016
  29. Jeff KingJun 23, 2016
  30. Johannes SixtJun 24, 2016
  31. Jeff KingJun 24, 2016
  32. Johannes SixtJun 24, 2016
  33. 0/4 portable signal-checking in testsJeff King, Jun 24, 2016
  34. 1/4 tests: factor portable signal check out of t0005Jeff King, Jun 24, 2016
  35. Johannes SixtJun 24, 2016
  36. Jeff KingJun 24, 2016
  37. Johannes SixtJun 24, 2016
  38. 2/4 t0005: use test_match_signal as appropriateJeff King, Jun 24, 2016
  39. 3/4 test_must_fail: use test_match_signalJeff King, Jun 24, 2016
  40. 4/4 t/lib-git-daemon: use test_match_signalJeff King, Jun 24, 2016
  41. Jeff KingJun 24, 2016
  42. Junio C HamanoJun 24, 2016
  43. Jeff KingJun 24, 2016
  44. Junio C HamanoJun 24, 2016
  45. Eric SunshineJun 24, 2016
  46. Jeff KingJun 24, 2016
  47. Jeff KingJun 24, 2016
  48. Junio C HamanoJun 24, 2016
  49. Jeff KingJun 24, 2016
  50. 2/4 archive-tar: write extended headers for file sizes >= 8GBJeff King, Jun 23, 2016
  51. Junio C HamanoJun 24, 2016
  52. Jeff KingJun 24, 2016
  53. Junio C HamanoJun 24, 2016
  54. Jeff KingJun 24, 2016
  55. 3/4 archive-tar: write extended headers for far-future mtimeJeff King, Jun 23, 2016
  56. Junio C HamanoJun 24, 2016
  57. Jeff KingJun 24, 2016
  58. 4/4 archive-tar: drop return valueJeff King, Jun 23, 2016
  59. Remi Galan AlfonsoJun 24, 2016
  60. Jeff KingJun 24, 2016
  61. Junio C HamanoJun 24, 2016

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.