{"thread":{"id":"22809","subject":"Roadmap to better handle big files?","startedAt":"2010-02-24T23:00:33Z","lastAt":"2010-02-25T18:06:10Z","messageCount":5,"participants":["Nick Triantos","Nicolas Pitre","Jakub Narebski","Joshua Jensen"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"135625","messageId":"B85968F5-E7C2-499D-A8BE-0160BA575F10@perceptivepixel.com","threadId":"22809","inReplyTo":null,"subject":"Roadmap to better handle big files?","fromName":"Nick Triantos","fromEmail":"nick@perceptivepixel.com","sentAt":"2010-02-24T23:00:33Z","receivedAt":"2010-02-24T23:00:33Z","isPatch":false,"sender":{"key":"nick@perceptivepixel.com","avatar":null},"body":"Hi,\n\nIs there any planned functionality to better support large files in git? (> 100MB / file)\n\nWe've been happily using git but we now have some files which we'd very much like to have under the same version control as our source code, and some of those files have been as large as 450MB/file. We are looking at chunking the file up before commiting it to git, but is there any plan to better support chunking of these files during repacks or other operations? Right now, it appears either the whole file, or the whole collection of files in a commit (not sure which) can need to be resident in memory up to twice, from reading various places on the web. Our poor 32-bit server is barfing on this. We are going to put more RAM and a 64bit OS on the machine, but this still seems like an unnecessary design decision.\n\nthanks very much,\n-Nick\n"},{"id":"135636","messageId":"alpine.LFD.2.00.1002241837510.1946@xanadu.home","threadId":"22809","inReplyTo":"B85968F5-E7C2-499D-A8BE-0160BA575F10@perceptivepixel.com","subject":"Re: Roadmap to better handle big files?","fromName":"Nicolas Pitre","fromEmail":"nico@fluxnic.net","sentAt":"2010-02-24T23:39:22Z","receivedAt":"2010-02-24T23:39:22Z","isPatch":false,"sender":{"key":"nico@fluxnic.net","avatar":"https://avatars.githubusercontent.com/u/702790?v=4"},"body":"On Wed, 24 Feb 2010, Nick Triantos wrote:\n\n> Hi,\n> \n> Is there any planned functionality to better support large files in git? (> 100MB / file)\n\nYes. It's just a matter of available time to implement it.\n\n\nNicolas\n"},{"id":"135638","messageId":"m3fx4qmbwr.fsf@localhost.localdomain","threadId":"22809","inReplyTo":"B85968F5-E7C2-499D-A8BE-0160BA575F10@perceptivepixel.com","subject":"Re: Roadmap to better handle big files?","fromName":"Jakub Narebski","fromEmail":"jnareb@gmail.com","sentAt":"2010-02-24T23:51:00Z","receivedAt":"2010-02-24T23:51:00Z","isPatch":false,"sender":{"key":"jnareb@gmail.com","avatar":"https://avatars.githubusercontent.com/u/2706?v=4"},"body":"Nick Triantos writes:\n\n> Is there any planned functionality to better support large files in\n> git? (> 100MB / file)\n> \n> We've been happily using git but we now have some files which we'd\n> very much like to have under the same version control as our source\n> code, and some of those files have been as large as 450MB/file. We\n> are looking at chunking the file up before commiting it to git, but\n> is there any plan to better support chunking of these files during\n> repacks or other operations? Right now, it appears either the whole\n> file, or the whole collection of files in a commit (not sure which)\n> can need to be resident in memory up to twice, from reading various\n> places on the web. Our poor 32-bit server is barfing on this. We\n> are going to put more RAM and a 64bit OS on the machine, but this\n> still seems like an unnecessary design decision.\n\nGit has a roadmap???\n\nMore seriously, take a look at git-bigfiles project (fork):\nhttp://caca.zoy.org/wiki/git-bigfiles\n\nHTH\n-- \nJakub Narebski\nPoland\nShadeHawk on #git\n"},{"id":"135639","messageId":"2009C5FE-F0B7-4D64-BF5C-04087E17EDF1@perceptivepixel.com","threadId":"22809","inReplyTo":"m3fx4qmbwr.fsf@localhost.localdomain","subject":"Re: Roadmap to better handle big files?","fromName":"Nick Triantos","fromEmail":"nick@perceptivepixel.com","sentAt":"2010-02-25T00:02:16Z","receivedAt":"2010-02-25T00:02:16Z","isPatch":false,"sender":{"key":"nick@perceptivepixel.com","avatar":null},"body":"Thanks. I had looked at that project, but the logo being a piece of poop sort of scared me away from it (and it looked to be very early on in their design work so far)...\n\nthanks!\n-Nick\n\nOn Feb 24, 2010, at 3:51 PM, Jakub Narebski wrote:\n\n> Nick Triantos writes:\n> \n>> Is there any planned functionality to better support large files in\n>> git? (> 100MB / file)\n>> \n>> We've been happily using git but we now have some files which we'd\n>> very much like to have under the same version control as our source\n>> code, and some of those files have been as large as 450MB/file. We\n>> are looking at chunking the file up before commiting it to git, but\n>> is there any plan to better support chunking of these files during\n>> repacks or other operations? Right now, it appears either the whole\n>> file, or the whole collection of files in a commit (not sure which)\n>> can need to be resident in memory up to twice, from reading various\n>> places on the web. Our poor 32-bit server is barfing on this. We\n>> are going to put more RAM and a 64bit OS on the machine, but this\n>> still seems like an unnecessary design decision.\n> \n> Git has a roadmap???\n> \n> More seriously, take a look at git-bigfiles project (fork):\n> http://caca.zoy.org/wiki/git-bigfiles\n> \n> HTH\n> --\n> Jakub Narebski\n> Poland\n> ShadeHawk on #git\n"},{"id":"135686","messageId":"4B86BC12.9080201@workspacewhiz.com","threadId":"22809","inReplyTo":"B85968F5-E7C2-499D-A8BE-0160BA575F10@perceptivepixel.com","subject":"Re: Roadmap to better handle big files?","fromName":"Joshua Jensen","fromEmail":"jjensen@workspacewhiz.com","sentAt":"2010-02-25T18:06:10Z","receivedAt":"2010-02-25T18:06:10Z","isPatch":false,"sender":{"key":"jjensen@workspacewhiz.com","avatar":"https://avatars.githubusercontent.com/u/111687?v=4"},"body":"----- Original Message -----\nFrom: Nick Triantos\nDate: 2/24/2010 4:00 PM\n> Is there any planned functionality to better support large files in git? (> 100MB / file)\n> \nI once used Git alternates to point to a network share filled with the \nreally large files hashed into a .git/objects directory. It worked, \nalthough it was slower than having the entire repository locally.\n\nJosh\n"}]}