git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: large files and low memory

From
EWEnrico Weigelt <weigelt@metux.de>
Date
Oct 5, 2010, 00:59 UTC
Message-ID
<20101005005902.GC2768@nibiru.local>
In-Reply-To
<AANLkTi=nzyRgiBL07f2oeyjdUc8XnaKbJEJ+k8_g9rQ1@mail.gmail.com>
* Shawn Pearce <spearce@spearce.org> wrote:
Show 7 quoted lines
> Laziness.  Git originally assumed it would only be used for smaller
> source files written by humans.  Its easier to write the code as a
> single malloc'd buffer than to stream it.  We'd like to fix it, but
> its harder than it sounds.  Today we copy the file into a buffer
> before we deflate and compute the SHA-1 as this prevents us from
> getting into a consistency error when the file is modified between
> these two stages.
hmm, perhaps copy it to a temporary file, if it's too large ?
cu
-- 
----------------------------------------------------------------------
 Enrico Weigelt, metux IT service -- http://www.metux.de/

 phone:  +49 36207 519931  email: weigelt@metux.de
 mobile: +49 151 27565287  icq:   210169427         skype: nekrad666
----------------------------------------------------------------------
 Embedded-Linux / Portierung / Opensource-QM / Verteilte Systeme
----------------------------------------------------------------------
Previous: Shawn PearceNext: Enrico Weigelt
Message 5 of 29 in “large files and low memory”
  1. Enrico WeigeltOct 4, 2010
  2. Shawn PearceOct 4, 2010
  3. Joshua JensenOct 4, 2010
  4. Shawn PearceOct 4, 2010
  5. Enrico WeigeltOct 5, 2010
  6. Enrico WeigeltOct 5, 2010
  7. Matthieu MoyOct 5, 2010
  8. Enrico WeigeltOct 5, 2010
  9. Alex RiesenOct 5, 2010
  10. Matthieu MoyOct 5, 2010
  11. Nguyen Thai Ngoc DuyOct 5, 2010
  12. Junio C HamanoOct 5, 2010
  13. Nguyen Thai Ngoc DuyOct 5, 2010
  14. Nicolas PitreOct 5, 2010
  15. Jonathan NiederOct 4, 2010
  16. Shawn PearceOct 4, 2010
  17. Jonathan NiederOct 4, 2010
  18. Nguyen Thai Ngoc DuyOct 5, 2010
  19. Nicolas PitreOct 5, 2010
  20. Jonathan NiederOct 5, 2010
  21. Nicolas PitreOct 5, 2010
  22. Enrico WeigeltOct 5, 2010
  23. Ævar Arnfjörð BjarmasonOct 5, 2010
  24. Jonathan NiederOct 5, 2010
  25. Enrico WeigeltOct 5, 2010
  26. Jonathan NiederOct 5, 2010
  27. Enrico WeigeltOct 5, 2010
  28. Nicolas PitreOct 5, 2010
  29. Enrico WeigeltOct 5, 2010

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.