[EGIT PATCH 00/20] PackWriter, first usable attempt

From: Marek Zawirski <marek.zawirski@gmail.com>
Date: Jun 15, 2008, 21:45 UTC
Message-ID: <1213566349-25395-1-git-send-email-marek.zawirski@gmail.com>

Studying made me busy last week, but I'm back: with another GSoC series, adding some usable feature this time.

At first, some stuff was still missing to produce packs, mostly raw-data access related and ObjectWalk related.

Finally, we've got some support for pack writing! It's not that power that C git version offers, but something usable. Delta generation is not supported. Although we can reuse deltas and objects, and support all other (I hope) options of git-pack-objects directly or indirectly, most importantly --thin.

Pack writing and some other features are tested, seem to work.

This implementation of packing is not a very valuable thing directly (achieving efficient storage), however it's a base for enhancements and can be used for sending packs over net (with some assumptions). It's more a "repacking" than "packing" tool.

So... I'm switching now to push implementation. If time allows, delta-algorithms will be added later.

Robin, this series is based on master of egit.git when I saw it last time before repo.or.cz went down (9354293) ;) I'll add packwriter branch to my repo when server is up.

Marek Zawirski (20):
  Fix typo in PackIndexV2
  Integer versions of copyRawTo() and fromRaw() in ObjectId
  Add openObjectInAllPacks() to Repository, exposing packed objects
    storage
  WindowedFile fragments copying: copyToStream()
  Reverse pack index implementation: PackReverseIndex
  Tests for PackReverseIndex
  Refactor PackIndexV2 - extract binarySearchLevelTwo()
  CRC32 support for PackIndex
  CRC32 PackIndex tests
  Format PackedObjectLoader class
  Format UnpackedObjectLoader class
  Format DeltaOfsPackedObjectLoader class
  Raw-data operations in ObjectLoaders and PackFile
  Add hasRevSort() in RevWalk for faster sorting strategy checking
  Refactor getRevSort() calls to hasRevSort()
  Support for RevSort.BOUNDARY in ObjectWalk
  Rename confusing objects field in ObjectWalk
  New CountingOutputStream class - stream decorator
  Simplified implementation of pack creation: PackWriter
  PackWriter test suite

 .../tst/org/spearce/jgit/lib/PackIndexTest.java    |   10 +
 .../tst/org/spearce/jgit/lib/PackIndexV1Test.java  |   19 +
 .../tst/org/spearce/jgit/lib/PackIndexV2Test.java  |   30 +
 .../org/spearce/jgit/lib/PackReverseIndexTest.java |  115 +++
 .../tst/org/spearce/jgit/lib/PackWriterTest.java   |  454 ++++++++++
 org.spearce.jgit.test/tst/pack-huge.idx            |  Bin 0 -> 2368 bytes
 .../src/org/spearce/jgit/lib/AnyObjectId.java      |   16 +
 .../jgit/lib/DeltaOfsPackedObjectLoader.java       |   24 +-
 .../spearce/jgit/lib/DeltaPackedObjectLoader.java  |    9 +-
 .../jgit/lib/DeltaRefPackedObjectLoader.java       |   15 +-
 .../src/org/spearce/jgit/lib/ObjectId.java         |   26 +
 .../src/org/spearce/jgit/lib/ObjectLoader.java     |   24 +
 .../src/org/spearce/jgit/lib/PackFile.java         |   85 ++-
 .../src/org/spearce/jgit/lib/PackIndex.java        |   23 +
 .../src/org/spearce/jgit/lib/PackIndexV1.java      |   10 +
 .../src/org/spearce/jgit/lib/PackIndexV2.java      |   73 +-
 .../src/org/spearce/jgit/lib/PackReverseIndex.java |  179 ++++
 .../src/org/spearce/jgit/lib/PackWriter.java       |  882 ++++++++++++++++++++
 .../org/spearce/jgit/lib/PackedObjectLoader.java   |   47 +-
 .../src/org/spearce/jgit/lib/Repository.java       |   42 +
 .../org/spearce/jgit/lib/UnpackedObjectLoader.java |   26 +-
 .../spearce/jgit/lib/WholePackedObjectLoader.java  |   20 +-
 .../src/org/spearce/jgit/lib/WindowedFile.java     |   43 +
 .../src/org/spearce/jgit/revwalk/ObjectWalk.java   |   40 +-
 .../src/org/spearce/jgit/revwalk/RevSort.java      |    5 +-
 .../src/org/spearce/jgit/revwalk/RevWalk.java      |   11 +
 .../org/spearce/jgit/revwalk/StartGenerator.java   |   14 +-
 .../spearce/jgit/util/CountingOutputStream.java    |   89 ++
 28 files changed, 2258 insertions(+), 73 deletions(-)
 create mode 100644 org.spearce.jgit.test/tst/org/spearce/jgit/lib/PackReverseIndexTest.java
 create mode 100644 org.spearce.jgit.test/tst/org/spearce/jgit/lib/PackWriterTest.java
 create mode 100644 org.spearce.jgit.test/tst/pack-huge.idx
 create mode 100644 org.spearce.jgit/src/org/spearce/jgit/lib/PackReverseIndex.java
 create mode 100644 org.spearce.jgit/src/org/spearce/jgit/lib/PackWriter.java
 create mode 100644 org.spearce.jgit/src/org/spearce/jgit/util/CountingOutputStream.java