git/list[1] front-page[2] threads[3] people[4] search[5] about
 

[PATCHv4 07/10] pack-objects: Allow --max-pack-size to be used together with --stdout

From
Johan Herland <johan@herland.net>
Date
May 23, 2011, 00:52 UTC
Message-ID
<1306111923-16859-8-git-send-email-johan@herland.net>
In-Reply-To
<1306111923-16859-1-git-send-email-johan@herland.net>

Currently we refuse combining --max-pack-size with --stdout since there's no way to make multiple packs when the pack is written to stdout. However, we want to be able to limit the maximum size of the pack created by --stdout (and abort pack-objects if we are unable to meet that limit).

Therefore, when used together with --stdout, we reinterpret --max-pack-size to indicate the maximum pack size which - if exceeded - will cause pack-objects to abort with an error message.

Signed-off-by: Johan Herland <johan@herland.net>
---
 Documentation/git-pack-objects.txt |    3 ++
 builtin/pack-objects.c             |    9 ++++---
 t/t5300-pack-object.sh             |   43 ++++++++++++++++++++++++++++++++++++
 3 files changed, 51 insertions(+), 4 deletions(-)
diff --git a/Documentation/git-pack-objects.txt b/Documentation/git-pack-objects.txt
index e43904e..24cf975 100644
--- a/Documentation/git-pack-objects.txt
+++ b/Documentation/git-pack-objects.txt
@@ -112,6 +112,9 @@ base-name::
 	If specified,  multiple packfiles may be created.
 	The default is unlimited, unless the config variable
 	`pack.packSizeLimit` is set.
++
+When used together with --stdout, the command will fail with an error
+message if the pack output exceeds the given limit.
 
 --max-commit-count=<n>::
 	This option is only useful together with --stdout.
diff --git a/builtin/pack-objects.c b/builtin/pack-objects.c
index f0fc187..e226053 100644
--- a/builtin/pack-objects.c
+++ b/builtin/pack-objects.c
@@ -229,7 +229,7 @@ static unsigned long write_object(struct sha1file *f,
 
 	if (!entry->delta)
 		usable_delta = 0;	/* no delta */
-	else if (!pack_size_limit)
+	else if (!pack_size_limit || pack_to_stdout)
 	       usable_delta = 1;	/* unlimited packfile */
 	else if (entry->delta->idx.offset == (off_t)-1)
 		usable_delta = 0;	/* base was written to another pack */
@@ -478,6 +478,9 @@ static void write_pack_file(void)
 		 * If so, rewrite it like in fast-import
 		 */
 		if (pack_to_stdout) {
+			if (nr_written != nr_remaining)
+				die("unable to make pack within the pack size"
+				    " limit (%lu bytes)", pack_size_limit);
 			sha1close(f, sha1, CSUM_CLOSE);
 		} else if (nr_written == nr_remaining) {
 			sha1close(f, sha1, CSUM_FSYNC);
@@ -2327,9 +2330,7 @@ int cmd_pack_objects(int argc, const char **argv, const char *prefix)
 
 	if (!pack_to_stdout && !pack_size_limit)
 		pack_size_limit = pack_size_limit_cfg;
-	if (pack_to_stdout && pack_size_limit)
-		die("--max-pack-size cannot be used to build a pack for transfer.");
-	if (pack_size_limit && pack_size_limit < 1024*1024) {
+	if (!pack_to_stdout && pack_size_limit && pack_size_limit < 1024*1024) {
 		warning("minimum pack size limit is 1 MiB");
 		pack_size_limit = 1024*1024;
 	}
diff --git a/t/t5300-pack-object.sh b/t/t5300-pack-object.sh
index 80df631..46c1214 100755
--- a/t/t5300-pack-object.sh
+++ b/t/t5300-pack-object.sh
@@ -396,6 +396,49 @@ test_expect_success 'verify resulting packs' '
 	git verify-pack test-11-*.pack
 '
 
+test_expect_success '--stdout ignores pack.packSizeLimit' '
+	git pack-objects --stdout <obj-list >test-12.pack &&
+	git index-pack --strict test-12.pack
+'
+
+test_expect_success 'verify resulting pack' '
+	git verify-pack test-12.pack
+'
+
+test_expect_success 'honor --max-pack-size' '
+	git config --unset pack.packSizeLimit &&
+	packname_13=$(git pack-objects --max-pack-size=3m test-13 <obj-list) &&
+	test 2 = $(ls test-13-*.pack | wc -l)
+'
+
+test_expect_success 'verify resulting packs' '
+	git verify-pack test-13-*.pack
+'
+
+test_expect_success 'tolerate --max-pack-size smaller than biggest object' '
+	packname_14=$(git pack-objects --max-pack-size=1 test-14 <obj-list) &&
+	test 5 = $(ls test-14-*.pack | wc -l)
+'
+
+test_expect_success 'verify resulting packs' '
+	git verify-pack test-14-*.pack
+'
+
+test_expect_success '--stdout works with large enough --max-pack-size' '
+	git pack-objects --stdout --max-pack-size=10m <obj-list >test-15.pack &&
+	git index-pack --strict test-15.pack
+'
+
+test_expect_success 'verify resulting pack' '
+	git verify-pack test-15.pack
+'
+
+test_expect_success '--stdout fails when pack exceeds --max-pack-size' '
+	test_must_fail git pack-objects --stdout --max-pack-size=1 <obj-list >test-16.pack 2>errs &&
+	test_must_fail git index-pack --strict test-16.pack &&
+	grep -q "pack size limit" errs
+'
+
 test_expect_success 'make a few more commits' '
 	git reset --hard $commit &&
 	echo "change" > file &&
-- 
1.7.5.rc1.3.g4d7b
Previous: Johan HerlandNext: Junio C Hamano
Message 16 of 26 in “[PATCHv4 00/10] Push limits”
  1. Johan HerlandMay 23, 2011
  2. 01/10 Update technical docs to reflect side-band-64k capability in receive-packJohan Herland, May 23, 2011
  3. 02/10 send-pack: Attempt to retrieve remote status even if pack-objects failsJohan Herland, May 23, 2011
  4. Junio C HamanoMay 23, 2011
  5. Johan HerlandMay 23, 2011
  6. 03/10 Tighten rules for matching server capabilities in server_supports()Johan Herland, May 23, 2011
  7. 04/10 receive-pack: Prepare for addition of the new 'limit-*' family of capabilitiesJohan Herland, May 23, 2011
  8. Junio C HamanoMay 23, 2011
  9. Johan HerlandMay 24, 2011
  10. 05/10 pack-objects: Teach new option --max-commit-count, limiting #commits in packJohan Herland, May 23, 2011
  11. Junio C HamanoMay 23, 2011
  12. Johan HerlandMay 24, 2011
  13. 06/10 send-pack/receive-pack: Allow server to refuse pushes with too many commitsJohan Herland, May 23, 2011
  14. Junio C HamanoMay 23, 2011
  15. Johan HerlandMay 24, 2011
  16. 07/10 pack-objects: Allow --max-pack-size to be used together with --stdoutJohan Herland, May 23, 2011
  17. Junio C HamanoMay 24, 2011
  18. Johan HerlandMay 24, 2011
  19. 08/10 send-pack/receive-pack: Allow server to refuse pushing too large packsJohan Herland, May 23, 2011
  20. Junio C HamanoMay 24, 2011
  21. 09/10 pack-objects: Estimate pack size; abort early if pack size limit is exceededJohan Herland, May 23, 2011
  22. Shawn PearceMay 23, 2011
  23. Johan HerlandMay 23, 2011
  24. Junio C HamanoMay 24, 2011
  25. Johan HerlandMay 24, 2011
  26. 10/10 receive-pack: Allow server to refuse pushes with too many objectsJohan Herland, May 23, 2011

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.