{"thread":{"id":"25810","subject":"vcs for hefty video and graphics files","startedAt":"2010-11-22T18:09:14Z","lastAt":"2010-11-25T17:34:41Z","messageCount":7,"participants":["Harry Putnam","Philippe Lhoste","Jonathan Nieder","Jakub Narebski","Stefan Monnier"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"156331","messageId":"877hg55iyd.fsf@newsguy.com","threadId":"25810","inReplyTo":null,"subject":"vcs for hefty video and graphics files","fromName":"Harry Putnam","fromEmail":"reader@newsguy.com","sentAt":"2010-11-22T18:09:14Z","receivedAt":"2010-11-22T18:09:14Z","isPatch":false,"sender":{"key":"reader@newsguy.com","avatar":null},"body":"I hope not to appear to be sneaking in a war of systems, and may rate\na bit a high in windbag factor since I've asked a very similar\nquestion here but quite a while ago\n\nI'd like to briefly describe my projected usage of a versioning system\nand see what people here think in terms what system best suits to that\nusage.\n\nI'm a light weight semi-professional videographer but I have enough\ngoing on to feel a need to keep track of and be able to rollback\nversions of a project as it is being worked on\n\nIt would involve on any one projects something like 15 to 60 GB of\nstuff to keep up with. Large numbers of images and a dozen or 2 dozen\nvideo files. All in some stake of compression depending on the codex.\n\nI'd like to keep a few version of say an Adobe Premier project with\nall the attendant files that play a role in it.\n\nDitto for sets of Adobe After effects and attendant files. And many\nmany images and compilation in various states of cutting or whatever\nediting.\n\nAnd then the whole project including the various Premier and After\neffects sets and piles of captured video tape. All in DV-avi format.\n\nSo versions inside versions and then repeated for different projects.\n\nEach of which may have more than 1 version.\n\nBut cutting to the chase... assorted video and graphics files\namounting to between 15 - 60 GB in any one project\n\nEach project would only run a month or 2 months at the most and then\nall but the final delivered version would be deleted. That version\nmight be keep for a yr or so.\n\nI have some experience with cvs over quite a few yrs but still only a\nhomeboy user and not very skilled or knowledgeable about cvs and even\nless with mercurial or git, where I do have some tiny bit of\nexperience too.\n\nWhich of the main contenders: cvs subversion mercurial git bizarre \nMaybe a few more I don't know about, would be the best candidate for\nthe usage and user described\n"},{"id":"156363","messageId":"icg5ia$5an$1@dough.gmane.org","threadId":"25810","inReplyTo":"877hg55iyd.fsf@newsguy.com","subject":"Re: vcs for hefty video and graphics files","fromName":"Philippe Lhoste","fromEmail":"philho@gmx.net","sentAt":"2010-11-23T10:37:46Z","receivedAt":"2010-11-23T10:37:46Z","isPatch":false,"sender":{"key":"philho@gmx.net","avatar":null},"body":"On 22/11/2010 19:09, Harry Putnam wrote:\n> Which of the main contenders: cvs subversion mercurial git bizarre\n> Maybe a few more I don't know about, would be the best candidate for\n> the usage and user described\n\nbizarre? Never heard of this VCS before...\n\n > Each project would only run a month or 2 months at the most and then\n > all but the final delivered version would be deleted. That version\n > might be keep for a yr or so.\n\nMaybe it is pure heresy, but since all you want is to keep temporarily several very big \nand nearly incompressible files where diffs (or deltas?) are probably not significant, I \nwould advance that a VCS won't be very useful here.\nAdvantages of VCS are (among others):\n- make delta of changes to keep as little data as possible\n- compress this data (?)\n- keep changes indefinitely to be sure to have them when we need them\n- share and merge (changes from somebody else, or you elsewhere)\nUnless I missed something, these advantages doesn't seem to apply there.\n\nSome game makers keep track of their (large) binary files, along with the rest of the \nproject (source code). Rarely in isolation.\nPerforce and PlasticSCM both boast superior support of these files, I won't comment on \nthese allegations (over other VCS), just having no experience here.\n\nSomehow, in your case, the good old way of keeping copies renamed to keep the version (or \nkept in specific directories) might work for you... Perhaps along with a small text file \nwith comments on content of each file.\n\nPS.: I don't see why you included Tomcat list...\n\n-- \nPhilippe Lhoste\n-- (near) Paris -- France\n-- http://Phi.Lho.free.fr\n-- -- -- -- -- -- -- -- -- -- -- -- -- --\n"},{"id":"156401","messageId":"87oc9g2eti.fsf@newsguy.com","threadId":"25810","inReplyTo":"icg5ia$5an$1@dough.gmane.org","subject":"Re: vcs for hefty video and graphics files","fromName":"Harry Putnam","fromEmail":"reader@newsguy.com","sentAt":"2010-11-23T16:19:05Z","receivedAt":"2010-11-23T16:19:05Z","isPatch":false,"sender":{"key":"reader@newsguy.com","avatar":null},"body":"Philippe Lhoste writes:\n\n> Somehow, in your case, the good old way of keeping copies renamed to\n> keep the version (or kept in specific directories) might work for\n> you... Perhaps along with a small text file with comments on content\n> of each file.\n\nThanks for the input... that method has sort of worked so far but\nsomewhat tedious\n\n> PS.: I don't see why you included Tomcat list...\n\nAll I can say about that is that it was NOT my doing.\nI only included bazaar cvs mercurial, git and subversion.\n\nIts a mystery to me too. I hadn't noticed until you mentioned it.\nI removed it from this reply.\n\nA copy of my OP is kept on my home machine. It shows exactly what I\nsent out, I'm not sure what it goes thru when it gets to gmane server:\n\n From: Harry Putnam \n Subject: vcs for hefty video and graphics files\n Newsgroups: gmane.comp.version-control.mercurial.general,\n gmane.comp.version-control.bazaar-ng.general,\n gmane.comp.version-control.git,\n gmane.comp.version-control.subversion.user,\n gmane.comp.version-control.cvs.general\n Date: Mon, 22 Nov 2010 12:09:14 -0600\n Message-ID: <877hg55iyd.fsf@newsguy.com>\n\n \n"},{"id":"156447","messageId":"20101123204149.GA2373@burratino","threadId":"25810","inReplyTo":"icg5ia$5an$1@dough.gmane.org","subject":"Re: vcs for hefty video and graphics files","fromName":"Jonathan Nieder","fromEmail":"jrnieder@gmail.com","sentAt":"2010-11-23T20:41:50Z","receivedAt":"2010-11-23T20:41:50Z","isPatch":false,"sender":{"key":"jrnieder@gmail.com","avatar":"https://avatars.githubusercontent.com/u/281595?v=4"},"body":"Philippe Lhoste wrote:\n\n> Some game makers keep track of their (large) binary files, along\n> with the rest of the project (source code). Rarely in isolation.\n> Perforce and PlasticSCM both boast superior support of these files,\n> I won't comment on these allegations (over other VCS), just having\n> no experience here.\n\nOne small thing to add: for this use case, you might like git-annex[1].\n\nIt is a shame no single mailing list to follow up to was mentioned.\nHopefully the reply will get to Harry, anyway.\n\nRegards,\nJonathan\n\n[1] http://git-annex.branchable.com/\n"},{"id":"156511","messageId":"m3r5ea4a5y.fsf@localhost.localdomain","threadId":"25810","inReplyTo":"20101123204149.GA2373@burratino","subject":"Re: vcs for hefty video and graphics files","fromName":"Jakub Narebski","fromEmail":"jnareb@gmail.com","sentAt":"2010-11-24T22:42:11Z","receivedAt":"2010-11-24T22:42:11Z","isPatch":false,"sender":{"key":"jnareb@gmail.com","avatar":"https://avatars.githubusercontent.com/u/2706?v=4"},"body":"Jonathan Nieder writes:\n> Philippe Lhoste wrote:\n> \n> > Some game makers keep track of their (large) binary files, along\n> > with the rest of the project (source code). Rarely in isolation.\n> > Perforce and PlasticSCM both boast superior support of these files,\n> > I won't comment on these allegations (over other VCS), just having\n> > no experience here.\n> \n> One small thing to add: for this use case, you might like git-annex[1].\n> \n> It is a shame no single mailing list to follow up to was mentioned.\n> Hopefully the reply will get to Harry, anyway.\n> \n> Regards,\n> Jonathan\n> \n> [1] http://git-annex.branchable.com/\n\nJonathan, could you add git-annex to \"Interfaces, frontends, and tools\"\npage on git wiki:\n https://git.wiki.kernel.org/index.php/InterfacesFrontendsAndTools\n\nDoes git-annex use git-replace?\n\n-- \nJakub Narebski\nPoland\nShadeHawk on #git\n"},{"id":"156516","messageId":"20101125022106.GB13274@burratino","threadId":"25810","inReplyTo":"m3r5ea4a5y.fsf@localhost.localdomain","subject":"Re: vcs for hefty video and graphics files","fromName":"Jonathan Nieder","fromEmail":"jrnieder@gmail.com","sentAt":"2010-11-25T02:21:06Z","receivedAt":"2010-11-25T02:21:06Z","isPatch":false,"sender":{"key":"jrnieder@gmail.com","avatar":"https://avatars.githubusercontent.com/u/281595?v=4"},"body":"Jakub Narebski wrote:\n\n> Jonathan, could you add git-annex to \"Interfaces, frontends, and tools\"\n> page on git wiki:\n> https://git.wiki.kernel.org/index.php/InterfacesFrontendsAndTools\n\nOk, done. Please feel free to improve the description.\n\n> Does git-annex use git-replace?\n\nNo, just plain symlinks. See http://git-annex.branchable.com/walkthrough/\nto get a feel for it.\n\nRegards,\nJonathan\n"},{"id":"156570","messageId":"jwvr5e91fj2.fsf-monnier+gmane.comp.version-control.bazaar-ng.general@gnu.org","threadId":"25810","inReplyTo":"877hg55iyd.fsf@newsguy.com","subject":"Re: vcs for hefty video and graphics files","fromName":"Stefan Monnier","fromEmail":"monnier@iro.umontreal.ca","sentAt":"2010-11-25T17:34:41Z","receivedAt":"2010-11-25T17:34:41Z","isPatch":false,"sender":{"key":"monnier@iro.umontreal.ca","avatar":null},"body":"> It would involve on any one projects something like 15 to 60 GB of\n> stuff to keep up with. Large numbers of images and a dozen or 2 dozen\n> video files. All in some stake of compression depending on the codex.\n\nFWIW, for a VCS to do a good job on this kind of problem, you'd need\nto use a representation that lends itself to it.\n\nI.e. regardless of what you end up doing, I would recommend you contact\nthe mailing-list of Free Software that can do the kind of video\nmanipulation you want to do, and tell them that you'd need their tool to\nrepresent a project in such a way that it has a bunch of big-files that\nare almost never modified (containing \"binary data\" such as the source\nimages and audio recordings, say) along with a few other smaller files\nthat mostly contain instructions about how to use the big-files to\ngenerate the desired output.\n\nSuch a representation should lead to very good support from most VCSs.\nE.g. If these small files use a clean text representation, then a VCS\nmight even be able to do useful merges between different branches of\na project (as long as the branches share the same big-files).\n\n\n Stefan\n"}]}