{"thread":{"id":"25724","subject":"Incremental object transfer","startedAt":"2010-11-12T10:34:27Z","lastAt":"2010-11-13T18:31:36Z","messageCount":3,"participants":["Enrico Weigelt","Jonathan Nieder","Jakub Narebski"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"155771","messageId":"20101112103427.GA29057@nibiru.local","threadId":"25724","inReplyTo":null,"subject":"Incremental object transfer","fromName":"Enrico Weigelt","fromEmail":"weigelt@metux.de","sentAt":"2010-11-12T10:34:27Z","receivedAt":"2010-11-12T10:34:27Z","isPatch":false,"sender":{"key":"weigelt@metux.de","avatar":null},"body":"\nHi folks,\n\n\nas already said some time ago, I'm using git to back up maildirs\non a machine w/ relatively low ram. The biggest problem for now\nis the initial push (maybe later larger subsequent pushes could\nbe also affected too): it takes quite a long time to get everything\npacked, and if the connection breaks (the box is sitting behind\na dynamic-IP DSL link), everything has to be restarted :(\n\nSo my idea is to incrementally transfer objects in smaller packs,\ndisable gc on remote side and update refs when some commit is\ncomplete.\n\nIs there any way to do this ?\n\n\nthx\n-- \n----------------------------------------------------------------------\n Enrico Weigelt, metux IT service -- http://www.metux.de/\n\n phone:  +49 36207 519931  email: weigelt@metux.de\n mobile: +49 151 27565287  icq:   210169427         skype: nekrad666\n----------------------------------------------------------------------\n Embedded-Linux / Portierung / Opensource-QM / Verteilte Systeme\n----------------------------------------------------------------------\n"},{"id":"155787","messageId":"20101112160933.GC15141@burratino","threadId":"25724","inReplyTo":"20101112103427.GA29057@nibiru.local","subject":"Re: Incremental object transfer","fromName":"Jonathan Nieder","fromEmail":"jrnieder@gmail.com","sentAt":"2010-11-12T16:09:33Z","receivedAt":"2010-11-12T16:09:33Z","isPatch":false,"sender":{"key":"jrnieder@gmail.com","avatar":"https://avatars.githubusercontent.com/u/281595?v=4"},"body":"Enrico Weigelt wrote:\n\n> as already said some time ago, I'm using git to back up maildirs\n> on a machine w/ relatively low ram. The biggest problem for now\n> is the initial push (maybe later larger subsequent pushes could\n> be also affected too): it takes quite a long time to get everything\n> packed, and if the connection breaks (the box is sitting behind\n> a dynamic-IP DSL link), everything has to be restarted :(\n\nMaybe use \"git bundle\" and then transfer by rsync?\n"},{"id":"155830","messageId":"m31v6pf54h.fsf@localhost.localdomain","threadId":"25724","inReplyTo":"20101112103427.GA29057@nibiru.local","subject":"Re: Incremental object transfer","fromName":"Jakub Narebski","fromEmail":"jnareb@gmail.com","sentAt":"2010-11-13T18:31:36Z","receivedAt":"2010-11-13T18:31:36Z","isPatch":false,"sender":{"key":"jnareb@gmail.com","avatar":"https://avatars.githubusercontent.com/u/2706?v=4"},"body":"Enrico Weigelt <weigelt@metux.de> writes:\n\n> As already said some time ago, I'm using git to back up maildirs\n> on a machine w/ relatively low ram. The biggest problem for now\n> is the initial push (maybe later larger subsequent pushes could\n> be also affected too): it takes quite a long time to get everything\n> packed, and if the connection breaks (the box is sitting behind\n> a dynamic-IP DSL link), everything has to be restarted :(\n> \n> So my idea is to incrementally transfer objects in smaller packs,\n> disable gc on remote side and update refs when some commit is\n> complete.\n> \n> Is there any way to do this ?\n\nThis would work only for \"dumb\" transports: \"dumb\" HTTP transport and\ndeprecated rsync transport.\n\n\"Smart\" transport (e.g. git://, SSH, new \"smart\" HTTP transport) all\ncreate packs to send on the fly.  Those packs would rarely be\nbyte-for-byte the same, even if both server and client have the same\nobjects, unless perhaps on single code (unthreaded).\n\nThere were discussion about resumable download, by examining what got\ndownloaded in partial pack; but this is a hard problem.\n\nSo I would recommend either creating a bundle (see git-bundle\nmanpage), which can be resumably downloaded via HTTP, FTP, P2P - it is\nan ordinary file.  Or you can try cloning via rsync (but the\nrepository should be quiescent).\n\n-- \nJakub Narebski\nPoland\nShadeHawk on #git\n"}]}