git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: Problem with large files on different OSes

From
Linus Torvalds <torvalds@linux-foundation.org>
Date
May 28, 2009, 19:49 UTC
Message-ID
<alpine.LFD.2.01.0905281246330.3435@localhost.localdomain>
In-Reply-To
<20090528194348.GH13499@coredump.intra.peff.net>
On Thu, 28 May 2009, Jeff King wrote:
Show 12 quoted lines
> 
> > So my "fixed chunk" approach would be nice in that if you have this kind 
> > of "chunkblob" entry, in the tree (and index) it would literally be one 
> > entry, and look like that:
> > 
> >    100644 chunkblob <sha1>
> 
> But if I am understanding you correctly, you _are_ proposing to munge
> the git data structure here. Which means that pre-chunkblob trees will
> point to the raw blob, and then post-chunkblob trees will point to the
> chunked representation. And that means not being able to use the sha-1
> to see that they eventually point to the same content.

Yes. If we were to do this, and people have large chunks, then once you start using the chunkblob (for lack of a better word) model, you'll see the same object with two different SHA1's. But it's a one-time (and one-way - since once it's a chunkblob, older models can't touch it) thing, it can never cause any long-term confusion.

(We'll end up with something similar if somebody ever breaks SHA-1 enough for us to care - the logical way to handle it is likely to just accept the SHA512-160 object name "aliases")

			Linus
Previous: Jeff KingNext: Nicolas Pitre
Message 27 of 31 in “Problem with large files on different OSes”
  1. Christopher JeffersonMay 27, 2009
  2. Andreas EricssonMay 27, 2009
  3. Christopher JeffersonMay 27, 2009
  4. John TapsellMay 27, 2009
  5. Christopher JeffersonMay 27, 2009
  6. John TapsellMay 27, 2009
  7. Tomas CarneckyMay 27, 2009
  8. Christopher JeffersonMay 27, 2009
  9. Andreas EricssonMay 27, 2009
  10. Jakub NarebskiMay 27, 2009
  11. Linus TorvaldsMay 27, 2009
  12. Linus TorvaldsMay 27, 2009
  13. Christopher JeffersonMay 27, 2009
  14. Jakub NarebskiMay 27, 2009
  15. Nicolas PitreMay 27, 2009
  16. Jeff KingMay 27, 2009
  17. Linus TorvaldsMay 27, 2009
  18. Alan Manuel GloriaMay 27, 2009
  19. Linus TorvaldsMay 28, 2009
  20. Nicolas PitreMay 28, 2009
  21. Eric RaibleMay 28, 2009
  22. Shawn O. PearceMay 28, 2009
  23. Eric RaibleMay 28, 2009
  24. Andreas EricssonMay 28, 2009
  25. Nicolas PitreMay 28, 2009
  26. Jeff KingMay 28, 2009
  27. Linus TorvaldsMay 28, 2009
  28. Nicolas PitreMay 27, 2009
  29. Jeff KingMay 28, 2009
  30. Nicolas PitreMay 28, 2009
  31. Jeff KingMay 28, 2009

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.