git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: [EGIT PATCH 00/20] PackWriter, first usable attempt

From
Shawn O. Pearce <spearce@spearce.org>
Date
Jun 16, 2008, 05:19 UTC
Message-ID
<20080616051927.GV11793@spearce.org>
In-Reply-To
<1213566349-25395-1-git-send-email-marek.zawirski@gmail.com>
Marek Zawirski <marek.zawirski@gmail.com> wrote:
> At first, some stuff was still missing to produce packs, mostly
> raw-data access related and ObjectWalk related.

I'm glad it turned out to be so little missing actually. Reusing ObjectWalk saved a lot of code in the pack writer, and for the most part our existing data access structures were already well organized.

It is too early to say how the performance is going to work, but
object packing with delta reuse can be difficult and I'm happy to
see that our abstractions more-or-less supported it.  Tuning can
come later, once we better understand the code, and have something
for end-users to complain (or praise) about.
 
Show 12 quoted lines
> Finally, we've got some support for pack writing! It's not that
> power that C git version offers, but something usable. Delta
> generation is not supported. Although we can reuse deltas and objects,
> and support all other (I hope) options of git-pack-objects directly or
> indirectly, most importantly --thin.
> 
> Pack writing and some other features are tested, seem to work.
> 
> This implementation of packing is not a very valuable thing directly
> (achieving efficient storage), however it's a base for enhancements
> and can be used for sending packs over net (with some assumptions).
> It's more a "repacking" than "packing" tool.

Yup. The critical part here is jgit can now format a pack file, which means we can now actually implement native push over the local pipe (to fork+exec'd git-receive-pack) or over SSH. That is one of the major missing features in the Eclipse plugin, so this is a huge milestone for us. Thank you Marek.

> So... I'm switching now to push implementation. If time allows,
> delta-algorithms will be added later.
Yay.

Native push protocol support at this point is much more important than delta generation. Although delta generation is one of the key features that makes git so damn efficient it is pointless if we cannot actually communicate with a remote repository to send them our changes. Early adopters of the push support coming from this plugin can at least use it on local area networks, where bandwidth is not (usually) a limiting factor.

>  28 files changed, 2258 insertions(+), 73 deletions(-)
Nice to see it didn't take that much code either.
-- 
Shawn.
Previous: Shawn O. PearceNext: Marek Zawirski
Message 28 of 29 in “PackWriter, first usable attempt”
  1. 00/20 PackWriter, first usable attemptMarek Zawirski, Jun 15, 2008
  2. 01/20 Fix typo in PackIndexV2Marek Zawirski, Jun 15, 2008
  3. 02/20 Integer versions of copyRawTo() and fromRaw() in ObjectIdMarek Zawirski, Jun 15, 2008
  4. 03/20 Add openObjectInAllPacks() to Repository, exposing packed objects storageMarek Zawirski, Jun 15, 2008
  5. 04/20 WindowedFile fragments copying: copyToStream()Marek Zawirski, Jun 15, 2008
  6. 05/20 Reverse pack index implementation: PackReverseIndexMarek Zawirski, Jun 15, 2008
  7. 06/20 Tests for PackReverseIndexMarek Zawirski, Jun 15, 2008
  8. 07/20 Refactor PackIndexV2 - extract binarySearchLevelTwo()Marek Zawirski, Jun 15, 2008
  9. 08/20 CRC32 support for PackIndexMarek Zawirski, Jun 15, 2008
  10. 09/20 CRC32 PackIndex testsMarek Zawirski, Jun 15, 2008
  11. 10/20 Format PackedObjectLoader classMarek Zawirski, Jun 15, 2008
  12. 11/20 Format UnpackedObjectLoader classMarek Zawirski, Jun 15, 2008
  13. 12/20 Format DeltaOfsPackedObjectLoader classMarek Zawirski, Jun 15, 2008
  14. 13/20 Raw-data operations in ObjectLoaders and PackFileMarek Zawirski, Jun 15, 2008
  15. 14/20 Add hasRevSort() in RevWalk for faster sorting strategy checkingMarek Zawirski, Jun 15, 2008
  16. 15/20 Refactor getRevSort() calls to hasRevSort()Marek Zawirski, Jun 15, 2008
  17. 16/20 Support for RevSort.BOUNDARY in ObjectWalkMarek Zawirski, Jun 15, 2008
  18. 17/20 Rename confusing objects field in ObjectWalkMarek Zawirski, Jun 15, 2008
  19. 18/20 New CountingOutputStream class - stream decoratorMarek Zawirski, Jun 15, 2008
  20. 19/20 Simplified implementation of pack creation: PackWriterMarek Zawirski, Jun 15, 2008
  21. 20/20 PackWriter test suiteMarek Zawirski, Jun 15, 2008
  22. 21/20 Make isBetterDeltaReuseLoader() static in PackWriterMarek Zawirski, Jun 17, 2008
  23. Robin RosenbergJun 17, 2008
  24. Marek ZawirskiJun 19, 2008
  25. Shawn O. PearceJun 16, 2008
  26. Marek ZawirskiJun 16, 2008
  27. Shawn O. PearceJun 17, 2008
  28. Shawn O. PearceJun 16, 2008
  29. Marek ZawirskiJun 16, 2008

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.