git/list[1] front-page[2] threads[3] people[4] search[5] about
 

[PATCHv3 6/6] refs.c: enable large transactions

From
Stefan Beller <sbeller@google.com>
Date
Jan 23, 2015, 20:04 UTC
Message-ID
<1422043442-30676-7-git-send-email-sbeller@google.com>
In-Reply-To
<1422043442-30676-1-git-send-email-sbeller@google.com>

By closing the file descriptors after creating the lock file we are not limiting the size of the transaction by the number of available file descriptors.

When closing the file descriptors early, we also need to write the values in early, if we don't want to reopen the files.

Signed-off-by: Stefan Beller <sbeller@google.com>
---
Notes:
    version3:
    * Do not reopen the files after closing them. Make sure we have
      written all necessary information before closing the file.
      Doing it that way enabled us to drop the patch "[PATCH 4/6]
      refs.c: Have a write_in_full_to_lock_file wrapping write_in_full"
 refs.c                | 15 ++++++++++++---
 t/t1400-update-ref.sh |  4 ++--
 2 files changed, 14 insertions(+), 5 deletions(-)
diff --git a/refs.c b/refs.c
index 1bfc84b..2594b23 100644
--- a/refs.c
+++ b/refs.c
@@ -3752,6 +3752,17 @@ int ref_transaction_commit(struct ref_transaction *transaction,
 				    update->refname);
 			goto cleanup;
 		}
+		if (!is_null_sha1(update->new_sha1)) {
+			if (write_ref_sha1(update->lock, update->new_sha1,
+					   update->msg)) {
+				strbuf_addf(err, "Cannot write to the ref lock '%s'.",
+					    update->refname);
+				ret = TRANSACTION_GENERIC_ERROR;
+				goto cleanup;
+			}
+		}
+		/* Do not keep all lock files open at the same time. */
+		close_ref(update->lock);
 	}
 
 	/* Perform updates first so live commits remain referenced */
@@ -3759,9 +3770,7 @@ int ref_transaction_commit(struct ref_transaction *transaction,
 		struct ref_update *update = updates[i];
 
 		if (!is_null_sha1(update->new_sha1)) {
-			if (write_ref_sha1(update->lock, update->new_sha1,
-					   update->msg)
-			    || commit_ref(update->lock, update->new_sha1)) {
+			if (commit_ref(update->lock, update->new_sha1)) {
 				strbuf_addf(err, "Cannot update the ref '%s'.",
 					    update->refname);
 				ret = TRANSACTION_GENERIC_ERROR;
diff --git a/t/t1400-update-ref.sh b/t/t1400-update-ref.sh
index 47d2fe9..c593a1d 100755
--- a/t/t1400-update-ref.sh
+++ b/t/t1400-update-ref.sh
@@ -979,7 +979,7 @@ run_with_limited_open_files () {
 
 test_lazy_prereq ULIMIT_FILE_DESCRIPTORS 'run_with_limited_open_files true'
 
-test_expect_failure ULIMIT_FILE_DESCRIPTORS 'large transaction creating branches does not burst open file limit' '
+test_expect_success ULIMIT_FILE_DESCRIPTORS 'large transaction creating branches does not burst open file limit' '
 (
 	for i in $(test_seq 33)
 	do
@@ -990,7 +990,7 @@ test_expect_failure ULIMIT_FILE_DESCRIPTORS 'large transaction creating branches
 )
 '
 
-test_expect_failure ULIMIT_FILE_DESCRIPTORS 'large transaction deleting branches does not burst open file limit' '
+test_expect_success ULIMIT_FILE_DESCRIPTORS 'large transaction deleting branches does not burst open file limit' '
 (
 	for i in $(test_seq 33)
 	do

-- 
2.2.1.62.g3f15098
Previous: Junio C HamanoNext: Junio C Hamano
Message 37 of 45 in “[PATCHv2 0/5] Fix bug in large transactions”
  1. Stefan BellerJan 22, 2015
  2. 1/5 update-ref: test handling large transactions properlyStefan Beller, Jan 22, 2015
  3. Michael HaggertyJan 22, 2015
  4. Jeff KingJan 22, 2015
  5. 2/5 t7004: rename ULIMIT test prerequisite to ULIMIT_STACK_SIZEStefan Beller, Jan 22, 2015
  6. 3/5 refs.c: remove lock_fd from struct ref_lockStefan Beller, Jan 22, 2015
  7. 4/5 refs.c: have a write_sha1_to_lock_file wrapperStefan Beller, Jan 22, 2015
  8. 5/5 refs.c: enable large transactionsStefan Beller, Jan 22, 2015
  9. Michael HaggertyJan 22, 2015
  10. Jeff KingJan 22, 2015
  11. Michael HaggertyJan 22, 2015
  12. Stefan BellerJan 22, 2015
  13. 0/5 So you dislike the sequence of system calls?Stefan Beller, Jan 22, 2015
  14. 1/5 fixup for "refs.c: enable large transactions"Stefan Beller, Jan 22, 2015
  15. 2/5 refs.c: remove unlock_ref from write_ref_sha1Stefan Beller, Jan 22, 2015
  16. 3/5 refs.c: move static functions to close and commit refsStefan Beller, Jan 22, 2015
  17. 4/5 refs.c: remove committing the ref from write_ref_sha1Stefan Beller, Jan 22, 2015
  18. 5/5 refs.c: write values to lock files early for committingStefan Beller, Jan 22, 2015
  19. Ramsay JonesJan 22, 2015
  20. Stefan BellerJan 22, 2015
  21. Ramsay JonesJan 22, 2015
  22. Ramsay JonesJan 22, 2015
  23. Stefan BellerJan 22, 2015
  24. Ramsay JonesJan 22, 2015
  25. Michael HaggertyJan 22, 2015
  26. 0/6 Fix bug in large transactionsStefan Beller, Jan 23, 2015
  27. 1/6 update-ref: test handling large transactions properlyStefan Beller, Jan 23, 2015
  28. 2/6 t7004: rename ULIMIT test prerequisite to ULIMIT_STACK_SIZEStefan Beller, Jan 23, 2015
  29. 3/6 refs.c: remove lock_fd from struct ref_lockStefan Beller, Jan 23, 2015
  30. 4/6 refs.c: move static functions to close and commit refsStefan Beller, Jan 23, 2015
  31. 5/6 refs.c: remove unlock_ref and commit_ref from write_ref_sha1Stefan Beller, Jan 23, 2015
  32. Junio C HamanoJan 23, 2015
  33. Stefan BellerJan 24, 2015
  34. Junio C HamanoJan 24, 2015
  35. Stefan BellerJan 24, 2015
  36. Junio C HamanoJan 24, 2015
  37. 6/6 refs.c: enable large transactionsStefan Beller, Jan 23, 2015
  38. Junio C HamanoJan 24, 2015
  39. Stefan BellerJan 24, 2015
  40. Junio C HamanoJan 24, 2015
  41. Stefan BellerJan 26, 2015
  42. refs.c: clean up write_ref_sha1 returnsStefan Beller, Jan 26, 2015
  43. Junio C HamanoJan 27, 2015
  44. Stefan BellerJan 28, 2015
  45. Junio C HamanoJan 27, 2015

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.