git/list[1] front-page[2] threads[3] people[4] search[5] about
 

[JGIT PATCH 04/14] Fix DirCache's skip over null byte padding when reading a DIRC file

From
Shawn O. Pearce <spearce@spearce.org>
Date
Aug 18, 2008, 23:53 UTC
Message-ID
<1219103602-32222-5-git-send-email-spearce@spearce.org>
In-Reply-To
<1219103602-32222-4-git-send-email-spearce@spearce.org>

Sometimes we hit EOFException while reading from a 'DIRC' file with the new DirCache API. This was caused by BufferedInputStream.skip skipping only part of the range we asked it to skip if the range we asked it to skip spanned over the end of the current buffer block. Two skip requests are necessary in this case: one to force the stream to skip to the end of the buffer, and another to skip over data in the source stream before reading the next buffer block into memory.

NB.skipFully handles this by abstracting the necessary loop into a utility function, much like NB.readFully handles the necessary read loop to ensure we read a full block of data.

DirCacheEntry and DirCache both need to use this routine to skip over the parts of the DIRC file they do not wish to read.

Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
---
 .../src/org/spearce/jgit/dircache/DirCache.java    |    2 +-
 .../org/spearce/jgit/dircache/DirCacheEntry.java   |    2 +-
 org.spearce.jgit/src/org/spearce/jgit/util/NB.java |   27 ++++++++++++++++++++
 3 files changed, 29 insertions(+), 2 deletions(-)
diff --git a/org.spearce.jgit/src/org/spearce/jgit/dircache/DirCache.java b/org.spearce.jgit/src/org/spearce/jgit/dircache/DirCache.java
index 995942c..76657c4 100644
--- a/org.spearce.jgit/src/org/spearce/jgit/dircache/DirCache.java
+++ b/org.spearce.jgit/src/org/spearce/jgit/dircache/DirCache.java
@@ -370,7 +370,7 @@ private void readFrom(final FileInputStream inStream) throws IOException,
 					// a performance optimization. Since we do not
 					// understand it, we can safely skip past it.
 					//
-					in.skip(NB.decodeInt32(hdr, 4));
+					NB.skipFully(in, NB.decodeUInt32(hdr, 4));
 				} else {
 					// The extension is not an optimization and is
 					// _required_ to understand this index format.
diff --git a/org.spearce.jgit/src/org/spearce/jgit/dircache/DirCacheEntry.java b/org.spearce.jgit/src/org/spearce/jgit/dircache/DirCacheEntry.java
index bcf5596..011bc16 100644
--- a/org.spearce.jgit/src/org/spearce/jgit/dircache/DirCacheEntry.java
+++ b/org.spearce.jgit/src/org/spearce/jgit/dircache/DirCacheEntry.java
@@ -116,7 +116,7 @@ DirCacheEntry(final byte[] sharedInfo, final int infoAt,
 		final int actLen = INFO_LEN + pathLen;
 		final int expLen = (actLen + 8) & ~7;
 		if (actLen != expLen)
-			in.skip(expLen - actLen);
+			NB.skipFully(in, expLen - actLen);
 	}
 
 	/**
diff --git a/org.spearce.jgit/src/org/spearce/jgit/util/NB.java b/org.spearce.jgit/src/org/spearce/jgit/util/NB.java
index fa13354..759caf5 100644
--- a/org.spearce.jgit/src/org/spearce/jgit/util/NB.java
+++ b/org.spearce.jgit/src/org/spearce/jgit/util/NB.java
@@ -71,6 +71,33 @@ public static void readFully(final InputStream fd, final byte[] dst,
 	}
 
 	/**
+	 * Skip an entire region of an input stream.
+	 * <p>
+	 * The input stream's position is moved forward by the number of requested
+	 * bytes, discarding them from the input. This method does not return until
+	 * the exact number of bytes requested has been skipped.
+	 * 
+	 * @param fd
+	 *            the stream to skip bytes from.
+	 * @param toSkip
+	 *            total number of bytes to be discarded. Must be >= 0.
+	 * @throws EOFException
+	 *             the stream ended before the requested number of bytes were
+	 *             skipped.
+	 * @throws IOException
+	 *             there was an error reading from the stream.
+	 */
+	public static void skipFully(final InputStream fd, long toSkip)
+			throws IOException {
+		while (toSkip > 0) {
+			final long r = fd.skip(toSkip);
+			if (r <= 0)
+				throw new EOFException("Short skip of block");
+			toSkip -= r;
+		}
+	}
+
+	/**
 	 * Compare a 32 bit unsigned integer stored in a 32 bit signed integer.
 	 * <p>
 	 * This function performs an unsigned compare operation, even though Java
-- 
1.6.0.87.g2858d
Previous: Shawn O. PearceNext: Shawn O. Pearce
Message 5 of 20 in “TreeWalk D/F conflict detection”
  1. 00/14 TreeWalk D/F conflict detectionShawn O. Pearce, Aug 18, 2008
  2. 01/14 Detect path names which overflow the name length field in the indexShawn O. Pearce, Aug 18, 2008
  3. 02/14 Fix NB.decodeUInt16 to correctly handle the high byteShawn O. Pearce, Aug 18, 2008
  4. 03/14 Add test cases for NB.encode and NB.decode family of routinesShawn O. Pearce, Aug 18, 2008
  5. 04/14 Fix DirCache's skip over null byte padding when reading a DIRC fileShawn O. Pearce, Aug 18, 2008
  6. 05/14 Fix usage of assertEquals in DirCacheIteratorTestShawn O. Pearce, Aug 18, 2008
  7. 06/14 Refactor AbstractTreeIterator.pathCompare to force another modeShawn O. Pearce, Aug 18, 2008
  8. 07/14 Micro-optimize AbstractTreeIterator.pathCompareShawn O. Pearce, Aug 18, 2008
  9. 08/14 Optimize path comparsion within subtrees during TreeWalkShawn O. Pearce, Aug 18, 2008
  10. 09/14 Refactor AbstractTreeIterator semantics to start on first entryShawn O. Pearce, Aug 18, 2008
  11. 10/14 Make all AbstractTreeIterator implementations bi-directionalShawn O. Pearce, Aug 18, 2008
  12. 11/14 Expose beginning of iterator indication from AbstractTreeIteratorShawn O. Pearce, Aug 18, 2008
  13. 12/14 Allow application code to set ObjectIds in DirCacheEntryShawn O. Pearce, Aug 18, 2008
  14. 13/14 Create NameConflictTreeWalk to transparently detect D/F conflictsShawn O. Pearce, Aug 18, 2008
  15. 14/14 Add test case for NameConflictTreeWalkShawn O. Pearce, Aug 18, 2008
  16. Junio C HamanoAug 19, 2008
  17. Robin RosenbergAug 19, 2008
  18. Shawn O. PearceAug 19, 2008
  19. David WoodhouseAug 19, 2008
  20. Shawn O. PearceAug 19, 2008

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.