{"thread":{"id":"26789","subject":"About Summer of code idea -- better big-file support","startedAt":"2011-03-19T13:15:40Z","lastAt":"2011-03-20T12:33:50Z","messageCount":5,"participants":["徐迪","Nguyen Thai Ngoc Duy"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"163745","messageId":"AANLkTi=Z_4cGBSPQW2YVuj=dd8mcO4YQ0G2owknt7vfw@mail.gmail.com","threadId":"26789","inReplyTo":null,"subject":"About Summer of code idea -- better big-file support","fromName":"徐迪","fromEmail":"xudifsd@gmail.com","sentAt":"2011-03-19T13:15:40Z","receivedAt":"2011-03-19T13:15:40Z","isPatch":false,"sender":{"key":"xudifsd@gmail.com","avatar":"https://gravatar.com/avatar/260814589d486b8dba6ff105f193d831bf9de466d688a20c53f7a516c7ffe146?d=mp&s=160"},"body":"Hi, there\n\nI am wondering start implementing the idea about better big-file support\nduring summer of code.\nHere are my idea: often gigantic files is media file, such as rmvb, swf,\npdf, dll, etc. And those file themselves are already being compressed, as we\nall know, git uses zlib to compress all the object in repository, and it is\nthe common sense that if we use zlib to compress those file, the compress\nratio will be unbelievable low, from my test if we compress 521MiB mkv file\nit will be 520MiB after compressed, how ridiculous it is!\nAlso I test that if we use \"git hash-object\" to calculate SHA-1, and copy\nall content to repository, it will costs 1 minutes, once we use \"git\nhash-object -w\" it will costs 2 minutes. So we sacrificed a lot, but gained\na little.\nSo, how do you guys think about that we can test the file type, and then\ndecide to use zlib or not?\nCheer\nDi\n"},{"id":"163746","messageId":"AANLkTikHigPhaWkiozsurbxjvR5WkAGW9LUtFRT9tK8=@mail.gmail.com","threadId":"26789","inReplyTo":"AANLkTi=Z_4cGBSPQW2YVuj=dd8mcO4YQ0G2owknt7vfw@mail.gmail.com","subject":"Re: About Summer of code idea -- better big-file support","fromName":"Nguyen Thai Ngoc Duy","fromEmail":"pclouds@gmail.com","sentAt":"2011-03-19T13:48:25Z","receivedAt":"2011-03-19T13:48:25Z","isPatch":false,"sender":{"key":"pclouds@gmail.com","avatar":"https://avatars.githubusercontent.com/u/720?v=4"},"body":"2011/3/19 徐迪 :\n> So, how do you guys think about that we can test the file type, and then\n> decide to use zlib or not?\n\nWe already have core.bigFileThreshold, which can be used to determine\nstorage type (traditional one with zlib, or some fancy ones for big\nfiles). If that's not enough, .gitattributes can be used to mark\ncertain files as 'big'.\n\nSee below link for recent discussions regarding big file support:\n\nhttp://thread.gmane.org/gmane.comp.version-control.git/165389/focus=168956\n\nOn the store of big files, bup (at github) model might fit in as well\nif large files change a small portion regularly.\n-- \nDuy\n"},{"id":"163747","messageId":"AANLkTikp5GS0gnqLVgZY4E0wBDmQBp1iftOON3MSMHea@mail.gmail.com","threadId":"26789","inReplyTo":"AANLkTikHigPhaWkiozsurbxjvR5WkAGW9LUtFRT9tK8=@mail.gmail.com","subject":"Re: About Summer of code idea -- better big-file support","fromName":"徐迪","fromEmail":"xudifsd@gmail.com","sentAt":"2011-03-19T14:28:11Z","receivedAt":"2011-03-19T14:28:11Z","isPatch":false,"sender":{"key":"xudifsd@gmail.com","avatar":"https://gravatar.com/avatar/260814589d486b8dba6ff105f193d831bf9de466d688a20c53f7a516c7ffe146?d=mp&s=160"},"body":"2011/3/19, Nguyen Thai Ngoc Duy :\n> 2011/3/19 徐迪 :\n>> So, how do you guys think about that we can test the file type, and then\n>> decide to use zlib or not?\n>\n> We already have core.bigFileThreshold, which can be used to determine\n> storage type (traditional one with zlib, or some fancy ones for big\n> files). If that's not enough, .gitattributes can be used to mark\n> certain files as 'big'.\n>\n> See below link for recent discussions regarding big file support:\n>\n> http://thread.gmane.org/gmane.comp.version-control.git/165389/focus=168956\n>\n> On the store of big files, bup (at github) model might fit in as well\n> if large files change a small portion regularly.\n> --\n> Duy\n>\nBut, why those features haven't merged into upstream developement?\nSorry for this question, but I'm fresh here.\n"},{"id":"163750","messageId":"AANLkTimH9pgO7JxN3vOcN=Kb3QGpbM7OO4bMpL2eAs=3@mail.gmail.com","threadId":"26789","inReplyTo":"AANLkTikp5GS0gnqLVgZY4E0wBDmQBp1iftOON3MSMHea@mail.gmail.com","subject":"Re: About Summer of code idea -- better big-file support","fromName":"Nguyen Thai Ngoc Duy","fromEmail":"pclouds@gmail.com","sentAt":"2011-03-19T15:13:17Z","receivedAt":"2011-03-19T15:13:17Z","isPatch":false,"sender":{"key":"pclouds@gmail.com","avatar":"https://avatars.githubusercontent.com/u/720?v=4"},"body":"2011/3/19 徐迪 :\n> 2011/3/19, Nguyen Thai Ngoc Duy :\n>> See below link for recent discussions regarding big file support:\n>>\n>> http://thread.gmane.org/gmane.comp.version-control.git/165389/focus=168956\n>>\n>> On the store of big files, bup (at github) model might fit in as well\n>> if large files change a small portion regularly.\n>>\n> But, why those features haven't merged into upstream developement?\n> Sorry for this question, but I'm fresh here.\n\nThis is just my guess. For specific apps, you should pull their authors in.\n\nAll apps experiment different ways of dealing with big files and work\nin certain situations. None is generic enough to be merged upstream.\nTake bup as an example, it aims at big files _only_. A file is treated\nas a directory of many smaller pieces. In order to get it merged, we\nneed to separate those big files from the rest because it's just too\ndifferent in handling them.\n-- \nDuy\n"},{"id":"163815","messageId":"AANLkTimTWLqkaFkiruqZ8FU8y3YX9C4ujPOPTTJzdRtk@mail.gmail.com","threadId":"26789","inReplyTo":"AANLkTimH9pgO7JxN3vOcN=Kb3QGpbM7OO4bMpL2eAs=3@mail.gmail.com","subject":"Re: About Summer of code idea -- better big-file support","fromName":"徐迪","fromEmail":"xudifsd@gmail.com","sentAt":"2011-03-20T12:33:50Z","receivedAt":"2011-03-20T12:33:50Z","isPatch":false,"sender":{"key":"xudifsd@gmail.com","avatar":"https://gravatar.com/avatar/260814589d486b8dba6ff105f193d831bf9de466d688a20c53f7a516c7ffe146?d=mp&s=160"},"body":"2011/3/19 Nguyen Thai Ngoc Duy :\n> 2011/3/19 徐迪 :\n>> 2011/3/19, Nguyen Thai Ngoc Duy :\n>>> See below link for recent discussions regarding big file support:\n>>>\n>>> http://thread.gmane.org/gmane.comp.version-control.git/165389/focus=168956\n>>>\n>>> On the store of big files, bup (at github) model might fit in as well\n>>> if large files change a small portion regularly.\n>>>\n>> But, why those features haven't merged into upstream developement?\n>> Sorry for this question, but I'm fresh here.\n>\n> This is just my guess. For specific apps, you should pull their authors in.\n>\n> All apps experiment different ways of dealing with big files and work\n> in certain situations. None is generic enough to be merged upstream.\n> Take bup as an example, it aims at big files _only_. A file is treated\n> as a directory of many smaller pieces. In order to get it merged, we\n> need to separate those big files from the rest because it's just too\n> different in handling them.\n> --\n> Duy\n>\n\nThanks for that timely link, I think I should do some more research now :P\n"}]}