git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: GIT as binary repository

From
EWEnrico Weigelt <weigelt@metux.de>
Date
Oct 21, 2010, 19:19 UTC
Message-ID
<20101021191925.GE28700@nibiru.local>
In-Reply-To
<2EBB46ACFCD6CF48B45E2B1865BD02338037D36B4E@GVW1098EXB.americas.hpqcorp.net>
* Wilson, Kevin Lee (OpenView Engineer) <kevin.l.wilson@hp.com> wrote:
Hi,
Show 5 quoted lines
> We are investigating the use of GIT as a binary repository solution.
> Our larger files are near 800MB and the total checked out repo size
> is about 3 GB the repo size in SVN is more like 20-30GB, if we could
> prune the history prior to MR, we could get these sizes down
> considerably. This binary repo is really for our super project build. 
What exactly do you need such large binary objects in an git repo ?

IMHO, Git isn't made for large files. I've noticed this when doing git-based mail archives on an old P3 box w/ 256MB physical memory. I had to split mbox'es to maildirs.

Perhaps you would like to have a look at some pure object store like venti ? (It's not distributed yet, but I'm currently working on an distributed successor, called Nebulon, which will also support strong encryption, on-demand replication, etc).

> I also have some questions, about how the workflow would be for
> getting all of the changes merged from several different teams
> into the one repository would operate. 

IMHO, there should be some dedicated release manager role, which is responsible for merging finished branches into the mainline (eg. similar that Linus does for the official Linux tree).

BUT: you perhaps should think carefully, whether you need everything
in one big repo. Perhaps a bunch of smaller ones (eg. having separate
modules in the own repos/trees) would fit better.
cu
-- 
----------------------------------------------------------------------
 Enrico Weigelt, metux IT service -- http://www.metux.de/

 phone:  +49 36207 519931  email: weigelt@metux.de
 mobile: +49 151 27565287  icq:   210169427         skype: nekrad666
----------------------------------------------------------------------
 Embedded-Linux / Portierung / Opensource-QM / Verteilte Systeme
----------------------------------------------------------------------
Previous: Wilson, Kevin Lee (OpenView Engineer)
Message 9 of 9 in “GIT as binary repository”
  1. Wilson, Kevin Lee (OpenView Engineer)Oct 21, 2010
  2. Tay Ray ChuanOct 21, 2010
  3. Enrico WeigeltOct 21, 2010
  4. Shawn PearceOct 21, 2010
  5. Enrico WeigeltOct 22, 2010
  6. Shawn PearceOct 22, 2010
  7. Shawn PearceOct 21, 2010
  8. Wilson, Kevin Lee (OpenView Engineer)Oct 21, 2010
  9. Enrico WeigeltOct 21, 2010

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.