git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Multiblobs

From
Sergio Callegari <sergio.callegari@gmail.com>
Date
Apr 28, 2010, 15:12 UTC
Message-ID
<loom.20100428T164432-954@post.gmane.org>
Hi,

it happened to me to read an older post by Jeff King about "multiblobs" (http://kerneltrap.org/mailarchive/git/2008/4/6/1360014) and I was wandering whether the idea has been abandoned for some reason or just put on hold.

Apparently, this would marvellously help on
- storing large binary blobs (the split could happen with a rolling checksum
approach)
- storing "structured files", such as the many zip-based file formats
(Opendocument, Docx, Jar files, zip files themselves), tars (including
compressed tars), pdfs, etc, whose number is rising day after day...
- storing binary files with textual tags, where the tags could go on a separate
blob, greatly simplifying their readout without any need for caching them on a
note tree.
- etc...
Furthermore, this could also
- help the management of upstream trees. This could be simplified since the
"pristine tree" distributed as a tar.gz file and the exploded repo could share
their blobs making commands such as pristine-tree unnecessary.
- help projects such as bup that currently need to provide split mechanisms of
their own.
- be used to add "different representations" to objects... for instance, when
storing a pdf one could use a fake split to store in a separate blob the
corresponding text, making the git-diff of pdfs almost instantaneous.
>From Jeff's post, I guess that the major issue could be that the same file could

get a different sha1 as a multiblob versus a regular blob, but maybe it could be possible to make the multiblob take the same sha1 of the "equivalent plain blob" rather than its real hash.

For the moment, I am just very curious about the idea and the possible pros and cons... can someone (maybe Jeff himself) tell me a little more? Also I wonder about the two possibilities (implement it in git vs implement it "on top of" git).

Sergio
Next: Avery Pennarun
Message 1 of 21 in “Multiblobs”
  1. Sergio CallegariApr 28, 2010
  2. Avery PennarunApr 28, 2010
  3. Sergio CallegariApr 28, 2010
  4. Avery PennarunApr 28, 2010
  5. Michael WittenApr 28, 2010
  6. SergioApr 28, 2010
  7. Avery PennarunApr 29, 2010
  8. Peter KreftingApr 29, 2010
  9. Avery PennarunApr 29, 2010
  10. Peter KreftingApr 30, 2010
  11. Avery PennarunApr 30, 2010
  12. Hervé CauwelierApr 30, 2010
  13. Avery PennarunApr 30, 2010
  14. Michael WittenApr 30, 2010
  15. Hervé CauwelierApr 30, 2010
  16. Geert BoschApr 28, 2010
  17. Mike HommeyApr 29, 2010
  18. Jeff KingMay 6, 2010
  19. Sergio CallegariMay 6, 2010
  20. Jeff KingMay 10, 2010
  21. Sergio CallegariMay 10, 2010

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.