{"thread":{"id":"24970","subject":"usage for git-p2p (was: git pack/unpack over bittorrent - works!)","startedAt":"2010-09-04T11:54:54Z","lastAt":"2010-09-04T19:29:47Z","messageCount":4,"participants":["Luke Kenneth Casson Leighton","Artur Skawina","Joey Hess"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"149830","messageId":"AANLkTin0-Zjy7Chvntf2pNj5iCQ-4Y5u=bu8r7DSejeu@mail.gmail.com","threadId":"24970","inReplyTo":null,"subject":"usage for git-p2p (was: git pack/unpack over bittorrent - works!)","fromName":"Luke Kenneth Casson Leighton","fromEmail":"luke.leighton@gmail.com","sentAt":"2010-09-04T11:54:54Z","receivedAt":"2010-09-04T11:54:54Z","isPatch":false,"sender":{"key":"luke.leighton@gmail.com","avatar":null},"body":"On Sat, Sep 4, 2010 at 2:52 AM, Artur Skawina <art.08.09@gmail.com> wrote:\n> Hmm, taking a few steps back, what is the expected usage of git-p2p?\n> Note it's a bit of a trick question; what i'm really asking is what _else_,\n> other than pulling/tracking Linus' kernel tree will/can be done with it?\n\n i'm _so_ glad you asked :)  please note - for all of these i'm keenly\naware that GPG signing is required (and nicolas has pointed out that\nyou only need a 20-byte hash to be securely transferred by some OOB\nmechanism)\n\n * distribution of large mailing lists and reduction of load on SMTP\nservers by adding in a firebreak layer where the SMTP data goes into a\ngit repository first, then is extracted on the other side.  users may\nchoose to download the mailing list using git over a peer-to-peer\nnetwork and then use the exact same software that the mailing list\nserver is running, thus reducing the load on the web front-end.  if\nyou then choose also to allow users to GPG-sign commits to the same\ngit repository, you have the possibility of reducing the load on the\nSMTP servers *as well*.  _and_ a group of users can operate in\n\"offline\" mode whilst still being able to collaborate (by sharing the\ngit repo between themselves on an isolated LAN) _and_ when one of them\ngoes back \"online\", all their commits sync up, and the rest of the\nworld becomes aware of that group's offline discussion.\n\n * distributed wikis.  there are lots of wikis now using git: ikiwiki\nis one, and moinmoin can also be configured to use git.  so, wikis\nhave already solved the \"conflict\" problem, in particular i know that\njoey is a smart bunny and has solved conflict resolution in ikiwiki.\nthat makes it possible to simply turn an \"ordinary\" wiki into a\n\"peer-to-peer\" wiki by merely inserting git-p2p underneath.\n\n * distributed bugtrackers.  the one that i know of which i believe\nwould work straight away is the ikiwiki \"bug\" plugin.  the rest i need\nto investigate: i know of at least one that's based on git that i\nhaven't looked at for a loong while (ditrack) - but the principle is\nvery straightforward: create bugs by UUID, commit, distribute.  if you\nwant a \"central\" numbering scheme then all that's needed is for a\n\"central\" service to allocate a \"central\" number - a symlink to the\nUUID would do - and you're done.\n\n * of course there's source management for other projects other than\nlinux-2.6 :)\n\nthere's plenty more ideas out there, but the primary ones i'm\ninterested in are the ones where \"traditional\" software development\ntools are being used which are utterly dependent on the client-server\nparadigm.  for reasons i won't explain as it's not relevant to this\ntechnical discussion list, this is making me feel a bit twitchy [free\nsoftware development being dependent on the client-server paradigm,\nthat is].\n\nl.\n"},{"id":"149844","messageId":"4C8260EC.9000404@gmail.com","threadId":"24970","inReplyTo":"AANLkTin0-Zjy7Chvntf2pNj5iCQ-4Y5u=bu8r7DSejeu@mail.gmail.com","subject":"Re: usage for git-p2p (was: git pack/unpack over bittorrent - works!)","fromName":"Artur Skawina","fromEmail":"art.08.09@gmail.com","sentAt":"2010-09-04T15:08:28Z","receivedAt":"2010-09-04T15:08:28Z","isPatch":false,"sender":{"key":"art.08.09@gmail.com","avatar":null},"body":"On 09/04/10 13:54, Luke Kenneth Casson Leighton wrote:\n> On Sat, Sep 4, 2010 at 2:52 AM, Artur Skawina <art.08.09@gmail.com> wrote:\n>> Hmm, taking a few steps back, what is the expected usage of git-p2p?\n>> Note it's a bit of a trick question; what i'm really asking is what _else_,\n>> other than pulling/tracking Linus' kernel tree will/can be done with it?\n> \n>  i'm _so_ glad you asked :)  please note - for all of these i'm keenly\n> aware that GPG signing is required (and nicolas has pointed out that\n> you only need a 20-byte hash to be securely transferred by some OOB\n> mechanism)\n> \n>  * distribution of large mailing lists and reduction of load on SMTP\n\n>  * distributed wikis.  there are lots of wikis now using git: ikiwiki\n\n>  * distributed bugtrackers.  the one that i know of which i believe\n\n>  * of course there's source management for other projects other than\n> linux-2.6 :)\n\nI'm not sure that git fits well any of the above; it may be convenient\nto reuse git as it already exists and can be more or less easily plugged\nin, but it's probably not the best backend choice. The one that might\nmap relatively closely to git is wikis, but there i'm not sure something\nmore scalable wouldn't be better (eg how would wikipedia map onto git..)\n\n\nThe one other use case for git-p2p that i can see is distributing the\nobject store over a LAN. But it's really about a _remote_ object store,\nnot so much a distributed one.\nIOW, having git-checkout trying to reach \"/tmp/.git-unix/G0\" and then any\nremote object store in addition to \".git/objects\" would be very useful.\nBut, do you actually want to spread the objects on _every_ developer\ndesktop on the LAN? Wouldn't it much more make sense to have two or three\ndedicated \"object servers\", which would give you enough redundancy and,\nbecause of the higher chances of finding the required objects in cache,\nalso perform better?\n\nFor the case of an ad-hoc group of developers, the protocol described in\nmy previous email would work well, they only need to find one peer and\nthen can move objects back and forth in the swarm.\n\n\nOne of the advantages that git-p2p could bring is better packing -- instead\nof everyone running git-repack with a limited window, each peer in the swarm\ncould better use its resources by looking much deeper for the perfect delta\ncandidate, but only for a subset of the objects, and then share the results\nwith the rest. As a \"git-repack -a -f\" can eg shrink the gcc.git repo by ~15%\n(~100M) the savings could be noticeable.\n\nartur\n"},{"id":"149846","messageId":"AANLkTi=_cXwHfDrDFx+1dG_6ZiHkrb8GObv72zwH3z3b@mail.gmail.com","threadId":"24970","inReplyTo":"4C8260EC.9000404@gmail.com","subject":"Re: usage for git-p2p (was: git pack/unpack over bittorrent - works!)","fromName":"Luke Kenneth Casson Leighton","fromEmail":"luke.leighton@gmail.com","sentAt":"2010-09-04T15:21:07Z","receivedAt":"2010-09-04T15:21:07Z","isPatch":false,"sender":{"key":"luke.leighton@gmail.com","avatar":null},"body":"On Sat, Sep 4, 2010 at 4:08 PM, Artur Skawina <art.08.09@gmail.com> wrote:\n> The one other use case for git-p2p that i can see is distributing the\n> object store over a LAN.\n\n ok - then you should help with the \"git hive\" effort that casey's\nwriting, as he is focussing on writing an extension to git which does\nexactly that.  the discussion on this thread is focussed on extending\ngit to work out on the wider internet, not just the case of optimised\ntransfers on a LAN segment.\n\nl.\n"},{"id":"149860","messageId":"20100904192947.GA15621@gnu.kitenet.net","threadId":"24970","inReplyTo":"AANLkTin0-Zjy7Chvntf2pNj5iCQ-4Y5u=bu8r7DSejeu@mail.gmail.com","subject":"Re: usage for git-p2p (was: git pack/unpack over bittorrent - works!)","fromName":"Joey Hess","fromEmail":"joey@kitenet.net","sentAt":"2010-09-04T19:29:47Z","receivedAt":"2010-09-04T19:29:47Z","isPatch":false,"sender":{"key":"joey@kitenet.net","avatar":"https://avatars.githubusercontent.com/u/16392?v=4"},"body":"Luke Kenneth Casson Leighton wrote:\n> ikiwiki is one, and moinmoin can also be configured to use git. Wikis\n> have already solved the \"conflict\" problem, in particular i know that\n> joey is a smart bunny and has solved conflict resolution in ikiwiki.\n\nMore accurately, I have entirely punted conflict resolution onto the\nunderlying VCS, such as git, and taken advantage of its generally\nexcellent merging behavior to avoid the most common classes of things\nthat result in conflicts/locking on other wikis (like two users editing\ndifferent parts of the same page concurrently).\n\nTwo branches of a wiki with committers to both can still of course\nresult in conflicts. In this case, it might make sense to avoid conflict\nresolution during merging, and make the conflicted page be committed\ncontaining both versions and some sort of \"conflict here!\" warning\ndirective. Then the wiki software can provide an web interface for\ncleaning up the conflict later. A custom merge driver could do something\nlike that, although there's the problem that not all files in the wiki\nwill be wiki pages, or the conflict could involve conflicting renames of\npages. (Another approach would be to use the \"ours\" merge strategy.)\n\n-- \nsee shy jo, lazy bunny\n"}]}