git/list[1] front-page[2] threads[3] people[4] search[5] about
 

[PATCH 0/5] support reading and writing uncompressed loose object

From
LYLiu Yubao <yubao.liu@gmail.com>
Date
Dec 2, 2008, 01:48 UTC
Message-ID
<493493ED.8090903@gmail.com>
In-Reply-To
<7voczws3np.fsf@gitster.siamese.dyndns.org>
Hi,
In original implementation, git stores loose object like this:
    loose object = deflate(typename + <space> + size + data)
The patches below add support to read and write uncompressed loose
object:
    loose object = typename + <space> + size + data
The cons and pros to use uncompressed loose object:
cons
    * old git can't read these uncompressed loose objects
      (I think it's not a big problem because old git can read
       pack files generated by new git)
    * uncompressed loose objects occupy more disk space
      (I also think it's not a big problem because loose objects
       aren't too many in general)
pros
    * avoid compressing and uncompressing loose objects that are likely
      frequently used when coding/merging with git add/diff/diff --cached/
      merge/rebase/log.
    * the code to read and write uncompressed loose objects is
      simpler, although there are now more code paths for compatibility.
    * better to share loose objects among multiple git processes because
      sha1 files can be used directly after mmapped. The original git
      uncompresses loose objects into heap memory area so that they
      can't be shared by other processes.
     (NOTICE: The patches below doesn't use mmapped sha1 files directly
      because I find parse_object() requires a buffer terminated with
      zero.)
    * easy to grep objects in .git/objects  (...stupid use case :-)

If these patches are worth being included into upstream branch, I will add a new config variable core.uncompressedLooseObject.

Explanation to the patches:
1) avoid parse_sha1_header() accessing memory out of bound
  Just for more safety, no inflateInit() to detect errors for
  uncompressed loose objects.
2) don't die immediately when convert an invalid type name
  So we can fall back to compressed loose objects.
3) optimize parse_sha1_header() a little by detecting object type
  To quickly detect whether it seems an uncompressed loose object.
4) support reading uncompressed loose object
  The new feature.
5) support writing uncompressed loose object
  The new feature, need a git-config variable yet.

The patches are generated against git-1.6.1-rc, I have run the test cases and it seems ok.

 object.c    |   14 +++++++++++++-
 object.h    |    1 +
 sha1_file.c |   58 +++++++++++++++++++++++++++++++++++++++++++++-------------
 3 files changed, 59 insertions(+), 14 deletions(-)
Previous: Liu YubaoNext: Liu Yubao
Message 8 of 27 in “two questions about the format of loose object”
  1. Liu YubaoDec 1, 2008
  2. Junio C HamanoDec 1, 2008
  3. Liu YubaoDec 1, 2008
  4. Jakub NarebskiDec 1, 2008
  5. Liu YubaoDec 2, 2008
  6. Shawn O. PearceDec 1, 2008
  7. Liu YubaoDec 2, 2008
  8. 0/5 support reading and writing uncompressed loose objectLiu Yubao, Dec 2, 2008
  9. 1/5 avoid parse_sha1_header() accessing memory out of boundLiu Yubao, Dec 2, 2008
  10. Shawn O. PearceDec 2, 2008
  11. Liu YubaoDec 3, 2008
  12. 2/5 don't die immediately when convert an invalid type nameLiu Yubao, Dec 2, 2008
  13. 3/5 optimize parse_sha1_header() a little by detecting object typeLiu Yubao, Dec 2, 2008
  14. Shawn O. PearceDec 2, 2008
  15. Liu YubaoDec 3, 2008
  16. 4/5 support reading uncompressed loose objectLiu Yubao, Dec 2, 2008
  17. Shawn O. PearceDec 2, 2008
  18. Liu YubaoDec 3, 2008
  19. 5/5 support writing uncompressed loose objectLiu Yubao, Dec 2, 2008
  20. Shawn O. PearceDec 2, 2008
  21. Liu YubaoDec 3, 2008
  22. Liu YubaoDec 2, 2008
  23. Nick AndrewDec 1, 2008
  24. Liu YubaoDec 2, 2008
  25. Shawn O. PearceDec 1, 2008
  26. Liu YubaoDec 2, 2008
  27. Nicolas PitreDec 4, 2008

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.