git/list[1] front-page[2] threads[3] people[4] search[5] about
 

[JGIT PATCH 08/10] Scan for new packs if GIT_DIR/objects/pack has been modified

From
Shawn O. Pearce <spearce@spearce.org>
Date
Apr 21, 2009, 01:21 UTC
Message-ID
<1240276872-17893-9-git-send-email-spearce@spearce.org>
In-Reply-To
<1240276872-17893-8-git-send-email-spearce@spearce.org>

If GIT_DIR/objects/pack directory has changed then one or more packs may have been added to the directory, or removed from it, due to a concurrent application performing a repack. In such cases we need to reload the list of available packs before conducting a search.

As packs are infrequently created or modified relative to searches, we only scan for new packs if our search through existing packs has turned up no results.

Transports that create packs locally through IndexPack automatically register their new PackFile, so even those cases will typically not require scanning the object directory again.

As misses are fairly common during IndexPack (due to its automatic collision detection logic) we avoid scanning for packs unless the GIT_DIR/objects/pack directory has been modified since the last scan of it.

Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
---
 .../src/org/spearce/jgit/lib/ObjectDatabase.java   |   15 ++++++++++++++-
 .../src/org/spearce/jgit/lib/ObjectDirectory.java  |   13 +++++++++++++
 2 files changed, 27 insertions(+), 1 deletions(-)
diff --git a/org.spearce.jgit/src/org/spearce/jgit/lib/ObjectDatabase.java b/org.spearce.jgit/src/org/spearce/jgit/lib/ObjectDatabase.java
index ed1290f..ec228a1 100644
--- a/org.spearce.jgit/src/org/spearce/jgit/lib/ObjectDatabase.java
+++ b/org.spearce.jgit/src/org/spearce/jgit/lib/ObjectDatabase.java
@@ -137,7 +137,7 @@ private final boolean hasObjectImpl1(final AnyObjectId objectId) {
 				return true;
 			}
 		}
-		return false;
+		return tryAgain1() && hasObject1(objectId);
 	}
 
 	private final boolean hasObjectImpl2(final String objectId) {
@@ -216,6 +216,12 @@ private ObjectLoader openObjectImpl1(final WindowCursor curs,
 				return ldr;
 			}
 		}
+		if (tryAgain1()) {
+			ldr = openObject1(curs, objectId);
+			if (ldr != null) {
+				return ldr;
+			}
+		}
 		return null;
 	}
 
@@ -311,6 +317,13 @@ void openObjectInAllPacks1(Collection<PackedObjectLoader> out,
 	}
 
 	/**
+	 * @return true if the fast-half search should be tried again.
+	 */
+	protected boolean tryAgain1() {
+		return false;
+	}
+
+	/**
 	 * Get the alternate databases known to this database.
 	 * 
 	 * @return the alternate list. Never null, but may be an empty array.
diff --git a/org.spearce.jgit/src/org/spearce/jgit/lib/ObjectDirectory.java b/org.spearce.jgit/src/org/spearce/jgit/lib/ObjectDirectory.java
index 36f221e..d29c63a 100644
--- a/org.spearce.jgit/src/org/spearce/jgit/lib/ObjectDirectory.java
+++ b/org.spearce.jgit/src/org/spearce/jgit/lib/ObjectDirectory.java
@@ -75,6 +75,8 @@
 
 	private final AtomicReference<PackFile[]> packList;
 
+	private volatile long packDirectoryLastModified;
+
 	/**
 	 * Initialize a reference to an on-disk object directory.
 	 * 
@@ -257,6 +259,16 @@ protected ObjectLoader openObject2(final WindowCursor curs,
 		}
 	}
 
+	@Override
+	protected boolean tryAgain1() {
+		final PackFile[] old = packList.get();
+		if (packDirectoryLastModified <= packDirectory.lastModified()) {
+			scanPacks(old);
+			return true;
+		}
+		return false;
+	}
+
 	private void insertPack(final PackFile pf) {
 		PackFile[] o, n;
 		do {
@@ -389,6 +401,7 @@ synchronized (packList) {
 	}
 
 	private String[] listPackIdx() {
+		packDirectoryLastModified = packDirectory.lastModified();
 		final String[] idxList = packDirectory.list(new FilenameFilter() {
 			public boolean accept(final File baseDir, final String n) {
 				// Must match "pack-[0-9a-f]{40}.idx" to be an index.
-- 
1.6.3.rc1.188.ga02b
Previous: Shawn O. PearceNext: Shawn O. Pearce
Message 9 of 17 in “Object access improvements”
  1. 00/10 Object access improvementsShawn O. Pearce, Apr 21, 2009
  2. 01/10 Safely handle closing an already closed WindowedFileShawn O. Pearce, Apr 21, 2009
  3. 02/10 Change empty tree test case to use a temporary repositoryShawn O. Pearce, Apr 21, 2009
  4. 03/10 Replace hand-coded read fully loop with NB.readFullyShawn O. Pearce, Apr 21, 2009
  5. 04/10 Clear the reverse index when closing a PackFileShawn O. Pearce, Apr 21, 2009
  6. 05/10 Introduce a new exception type for an in-place pack modificationShawn O. Pearce, Apr 21, 2009
  7. 06/10 Refactor object database access with new abstractionShawn O. Pearce, Apr 21, 2009
  8. 07/10 Rescan packs if a pack is modified in-place (part 1)Shawn O. Pearce, Apr 21, 2009
  9. 08/10 Scan for new packs if GIT_DIR/objects/pack has been modifiedShawn O. Pearce, Apr 21, 2009
  10. 09/10 Add test cases for loading new (or replaced) pack filesShawn O. Pearce, Apr 21, 2009
  11. 10/10 BROKEN TEST: ObjectLoader stays valid across repacksShawn O. Pearce, Apr 21, 2009
  12. Robin RosenbergApr 21, 2009
  13. Shawn O. PearceApr 22, 2009
  14. Shawn O. PearceApr 22, 2009
  15. 09/10 Add test cases for loading new (or replaced) pack filesShawn O. Pearce, Apr 21, 2009
  16. Shawn O. PearceApr 21, 2009
  17. 08/10 Scan for new packs if GIT_DIR/objects/pack has been modifiedShawn O. Pearce, Apr 21, 2009

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.