git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: git with large files...

From
KLKalle Launiala <kalle.launiala@gmail.com>
Date
Jul 22, 2012, 09:37 UTC
Message-ID
<CAHLTmjmGs5NR96krBozdD2AaZ3NXM2E-FfH-pV6BxwOVsojX5g@mail.gmail.com>
In-Reply-To
<CA+EOSBm2o+9j0Ykpe01tQmTGApZeJA36mJ=Kt8bNAywfpbODPw@mail.gmail.com>

While my example isn't about puppet, it's about another templating technology - that is by technical design hence also completely open source. We have a completely modulirized and completely distributed deployment scenario that is based on git.

The fundamental core how git addresses the solution is explained here
- especially on the visualization.
http://abstractiondev.wordpress.com/git-based-distribution/

I'd like to add a point of view that will underline git behaviour as a core requirement on distributed depoloyment - if there is any issue with big files, that's a good concern to address.

While this can be well justified and argued to benefit for possibly platform specific deployment tools such as rpm or msi modules, the audit trail of distributed software (the core requirement for source code andyway) still remains. If there is a "big file" that is part of the audit process, it's consistency needs to be guaranteed. And there is no way around it but the git way of doing complete secure hashed history trail.

This is not "nice to have" feature, but critical requirement to deploy embedded software for automated machinery to suffice safety regulations. Hospital, aviation, nuclear device software all apply here. Distributed software processes and projects all benefit from this - and trying to "cut short" some part of the complete audit trail of development will cause larger pain points elsewhere.

So is there real problem with big files performance or storage files? Anything else than "uncommon" slowness on calculating and comparing large file sha1 hashes?

Kalle
Previous: Elia PintoNext: Nick
Message 9 of 15 in “git with large files...”
  1. Darek BridgesJul 20, 2012
  2. Randal L. SchwartzJul 20, 2012
  3. Martin LanghoffJul 20, 2012
  4. David AguilarJul 21, 2012
  5. Elia PintoJul 21, 2012
  6. Martin LanghoffJul 21, 2012
  7. Elia PintoJul 21, 2012
  8. Elia PintoJul 22, 2012
  9. Kalle LaunialaJul 22, 2012
  10. NickJul 21, 2012
  11. Martin LanghoffJul 21, 2012
  12. Junio C HamanoJul 22, 2012
  13. Sitaram ChamartyJul 23, 2012
  14. Kalle LaunialaJul 23, 2012
  15. Drew NorthupJul 29, 2012

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.