git/list[1] front-page[2] threads[3] people[4] search[5] about
 

[PATCHv3 8/8] t3302-notes-index-expensive: Speed up create_repo()

From
Johan Herland <johan@herland.net>
Date
Jul 29, 2009, 02:25 UTC
Message-ID
<1248834326-31488-9-git-send-email-johan@herland.net>
In-Reply-To
<1248834326-31488-1-git-send-email-johan@herland.net>

Creating repos with 10/100/1000/10000 commits and notes takes a lot of time. However, using git-fast-import to do the job is a lot more efficient than using plumbing commands to do the same.

This patch decreases the overall run-time of this test on my machine from ~3 to ~1 minutes.

Signed-off-by: Johan Herland <johan@herland.net>
---
 t/t3302-notes-index-expensive.sh |   74 ++++++++++++++++++++++++--------------
 1 files changed, 47 insertions(+), 27 deletions(-)
diff --git a/t/t3302-notes-index-expensive.sh b/t/t3302-notes-index-expensive.sh
index 0ef3e95..ee84fc4 100755
--- a/t/t3302-notes-index-expensive.sh
+++ b/t/t3302-notes-index-expensive.sh
@@ -16,30 +16,50 @@ test -z "$GIT_NOTES_TIMING_TESTS" && {
 create_repo () {
 	number_of_commits=$1
 	nr=0
-	parent=
 	test -d .git || {
 	git init &&
-	tree=$(git write-tree) &&
-	while [ $nr -lt $number_of_commits ]; do
-		test_tick &&
-		commit=$(echo $nr | git commit-tree $tree $parent) ||
-			return
-		parent="-p $commit"
-		nr=$(($nr+1))
-	done &&
-	git update-ref refs/heads/master $commit &&
-	{
-		GIT_INDEX_FILE=.git/temp; export GIT_INDEX_FILE;
-		git rev-list HEAD | cat -n | sed "s/^[ 	][ 	]*/ /g" |
-		while read nr sha1; do
-			blob=$(echo note $nr | git hash-object -w --stdin) &&
-			echo $sha1 | sed "s/^/0644 $blob 0	/"
-		done | git update-index --index-info &&
-		tree=$(git write-tree) &&
+	(
+		while [ $nr -lt $number_of_commits ]; do
+			nr=$(($nr+1))
+			mark=$(($nr+$nr))
+			notemark=$(($mark+1))
+			test_tick &&
+			cat <<INPUT_END &&
+commit refs/heads/master
+mark :$mark
+committer $GIT_COMMITTER_NAME <$GIT_COMMITTER_EMAIL> $GIT_COMMITTER_DATE
+data <<COMMIT
+commit #$nr
+COMMIT
+
+M 644 inline file
+data <<EOF
+file in commit #$nr
+EOF
+
+blob
+mark :$notemark
+data <<EOF
+note for commit #$nr
+EOF
+
+INPUT_END
+
+			echo "N :$notemark :$mark" >> note_commit
+		done &&
 		test_tick &&
-		commit=$(echo notes | git commit-tree $tree) &&
-		git update-ref refs/notes/commits $commit
-	} &&
+		cat <<INPUT_END &&
+commit refs/notes/commits
+committer $GIT_COMMITTER_NAME <$GIT_COMMITTER_EMAIL> $GIT_COMMITTER_DATE
+data <<COMMIT
+notes
+COMMIT
+
+INPUT_END
+
+		cat note_commit
+	) |
+	git fast-import --quiet &&
 	git config core.notesRef refs/notes/commits
 	}
 }
@@ -48,13 +68,13 @@ test_notes () {
 	count=$1 &&
 	git config core.notesRef refs/notes/commits &&
 	git log | grep "^    " > output &&
-	i=1 &&
-	while [ $i -le $count ]; do
-		echo "    $(($count-$i))" &&
-		echo "    note $i" &&
-		i=$(($i+1));
+	i=$count &&
+	while [ $i -gt 0 ]; do
+		echo "    commit #$i" &&
+		echo "    note for commit #$i" &&
+		i=$(($i-1));
 	done > expect &&
-	git diff expect output
+	test_cmp expect output
 }
 
 cat > time_notes << \EOF
-- 
1.6.4.rc3.138.ga6b98.dirty
Previous: Johan HerlandNext: Johannes Schindelin
Message 29 of 30 in “[PATCHv3 0/8] RESEND: git notes”
  1. Johan HerlandJul 29, 2009
  2. 1/8 Introduce commit notesJohan Herland, Jul 29, 2009
  3. Alex RiesenJul 29, 2009
  4. Johannes SchindelinJul 29, 2009
  5. Johan HerlandJul 30, 2009
  6. Johannes SchindelinJul 30, 2009
  7. Johan HerlandJul 30, 2009
  8. 2/8 Add a script to edit/inspect notesJohan Herland, Jul 29, 2009
  9. 3/8 Speed up git notes lookupJohan Herland, Jul 29, 2009
  10. 4/8 Add an expensive test for git-notesJohan Herland, Jul 29, 2009
  11. 5/8 Teach "-m <msg>" and "-F <file>" to "git notes edit"Johan Herland, Jul 29, 2009
  12. Thomas RastJul 29, 2009
  13. Johan HerlandJul 30, 2009
  14. 6/8 First draft of notes tree parser with support for fanout subtreesJohan Herland, Jul 29, 2009
  15. Johannes SchindelinJul 29, 2009
  16. Testing performance of the notes lookup code (Was: [PATCHv3 6/8] First draft of notes tree parser with support for fanout subtrees)Johan Herland, Jul 30, 2009
  17. [RFC] First draft of 256-tree structure for storing notesJohan Herland, Aug 1, 2009
  18. [RFC] Store subtree entries in the same hash map as the note entriesJohan Herland, Aug 13, 2009
  19. [RFC] Use a 16-tree instead of a 256-tree for storing notesJohan Herland, Aug 26, 2009
  20. Alex RiesenAug 26, 2009
  21. Johan HerlandAug 26, 2009
  22. Alex RiesenAug 26, 2009
  23. Andreas EricssonAug 26, 2009
  24. Johan HerlandAug 26, 2009
  25. Johannes SchindelinAug 27, 2009
  26. 7/8 fast-import: Add support for importing commit notesJohan Herland, Jul 29, 2009
  27. Shawn O. PearceJul 29, 2009
  28. Johan HerlandJul 30, 2009
  29. 8/8 t3302-notes-index-expensive: Speed up create_repo()Johan Herland, Jul 29, 2009
  30. Johannes SchindelinJul 29, 2009

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.