{"thread":{"id":"25855","subject":"http://tech.slashdot.org/comments.pl?sid=1885890&cid=34358134","startedAt":"2010-11-27T17:33:07Z","lastAt":"2010-11-30T10:08:36Z","messageCount":10,"participants":["Luke Kenneth Casson Leighton","Ævar Arnfjörð Bjarmason","Jonathan Nieder","Will Palmer","Nguyen Thai Ngoc Duy","J.H.","Jan Krüger"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"156732","messageId":"AANLkTinTsn4PP8VxJX=pUOYKtoybCxqB0+-p9kNRGMj8@mail.gmail.com","threadId":"25855","inReplyTo":null,"subject":"http://tech.slashdot.org/comments.pl?sid=1885890&cid=34358134","fromName":"Luke Kenneth Casson Leighton","fromEmail":"luke.leighton@gmail.com","sentAt":"2010-11-27T17:33:07Z","receivedAt":"2010-11-27T17:33:07Z","isPatch":false,"sender":{"key":"luke.leighton@gmail.com","avatar":null},"body":"On Wed, Sep 22, 2010 at 11:20 AM, Luke Kenneth Casson Leighton\n wrote:\n> i trust that nobody will make the comment \"but source code couldn't\n> possibly be considered dangerous enough to block using ACTA\" and this\n> might go some way towards explaining one - not all - one i repeat just\n> one of the factors why i believe that free software development\n> _needs_ tools which can side-step single-point-of-failure locations\n> that can be blocked by stupid fucking \"agreements\" like ACTA.\n\n i trust that the above subject-line comment, or more specifically its\nconstituent links, begins to hint at why i believe it is important\nthat git-over-p2p be given a higher priority than it is at present.\nyou can easily join the dots to see where this government insanity is\ngoing: it won't be long before this insanity is utilised to take\ncontrol of free software web sites just because they happen to be\nhosting tools which MIGHT be utilised \"for copyright infringment\".\nit's bad enough that the DMCA exists and can be used to intimidate\nsourceforget into removing rtmpdump - now imagine if someone decided\nto take control of sourceforget ok actually yeah maybe that's not so\ngreat a loss after all but it's the PRINCIPLE damnit :)\n\n l.\n"},{"id":"156737","messageId":"AANLkTim0FeCE94R1zacOxGiEP8vZRSoDqNuNRUotnd9B@mail.gmail.com","threadId":"25855","inReplyTo":"AANLkTinTsn4PP8VxJX=pUOYKtoybCxqB0+-p9kNRGMj8@mail.gmail.com","subject":"Re: http://tech.slashdot.org/comments.pl?sid=1885890&cid=34358134","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2010-11-27T18:36:20Z","receivedAt":"2010-11-27T18:36:20Z","isPatch":false,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"On Sat, Nov 27, 2010 at 18:33, Luke Kenneth Casson Leighton\n wrote:\n\n> I believe it is important that git-over-p2p be given a higher\n> priority than it is at present.\n\nWe give \"priority\" to the stuff people submit patches for. There isn't\na git-over-p2p because nobody is working on it, would you like to work\non it?\n\nI'm also not convinced that this is actually needed. It's trivial to\nset up a p2p-like network by just emulating a darknet by manually\nadding remotes & fetching/pushing.\n\nThat's not viable for the stuff that usually gets distributed on p2p\nnetworks, but it sure is viable when you have people working on the\nsame codebase. Which would be the use case for a git-over-p2p client,\nright?\n"},{"id":"156739","messageId":"AANLkTima6meFsovFS-15X7CMTD53n=kkvueKrOeN4Yd4@mail.gmail.com","threadId":"25855","inReplyTo":"AANLkTim0FeCE94R1zacOxGiEP8vZRSoDqNuNRUotnd9B@mail.gmail.com","subject":"Re: http://tech.slashdot.org/comments.pl?sid=1885890&cid=34358134","fromName":"Luke Kenneth Casson Leighton","fromEmail":"luke.leighton@gmail.com","sentAt":"2010-11-27T19:06:13Z","receivedAt":"2010-11-27T19:06:13Z","isPatch":false,"sender":{"key":"luke.leighton@gmail.com","avatar":null},"body":"On Sat, Nov 27, 2010 at 6:36 PM, Ævar Arnfjörð Bjarmason\n wrote:\n> On Sat, Nov 27, 2010 at 18:33, Luke Kenneth Casson Leighton\n> wrote:\n>\n>> I believe it is important that git-over-p2p be given a higher\n>> priority than it is at present.\n>\n> We give \"priority\" to the stuff people submit patches for. There isn't\n> a git-over-p2p because nobody is working on it, would you like to work\n> on it?\n\n i was / have been. unfortunately it's unfunded work, and, thanks to\nreceiving less money than i require for living and for feeding my\nfamily i now face a court hearing on 16th december (3 weeks time)\nwhich has the aim and ultimate goal of making us homeless. i\ntherefore respectfull request that you please don't try to make me\nfeel like i _should_ be working on this in order to \"earn your\nrespect\" ( that goes for you too, johnathon ).\n\n> I'm also not convinced that this is actually needed. It's trivial to\n> set up a p2p-like network by just emulating a darknet by manually\n> adding remotes & fetching/pushing.\n\n ah.. but that's the single-use case for a single shared repository.\ngit \"as-is\" the fetch/push over http is limited to a single\nrepository. i'm thinking more in terms of a global network, with\nsearches for checksummed objects \"accidentally\" potentially coming\nfrom wildly different repositories.\n\n the work i did a few months ago showed that just \"dropping\" git on\ntop of e.g. the bittorrent protocol as-is, files/dirs mapped to git\nobjects, wouldn't cut the mustard, because the bloody bittorrent\nprotocol \"chunks\" system treats the entire fileset as an ordered but\ncontiguous \"stream\" of data (metadata in the actual .torrent file says\nhow long each of the files are and in what order they are, and this is\nutilised at the receiving end to reconstruct the files after the\ndatablocks have been received. it means that if you want just one\nsingle file, even of 2 bytes in length, you might have to pull 2\nblocks of e.g. 256k and grab the byte and the end of the 1st and the\nbyte at the beginning of the 2nd! yukk...)\n\nbut i believe that a system where each git object was given its own\n.torrent file _would_ work, and you utilise DHT or other mechanism\n(perhaps even a torrent containing the metadata!) to get the\nstructural information. the success of this model depends on everyone\nbeing connected to everyone (a global network), in order to be able to\ndo a DHT search for the node(s) that are seeding that git object's\ntorrent file.\n\n the \"global network\" concept has the advantage of being far more\nrobust in the face of DNS take-overs, as you actually just need to\nknow _one_ DNS name out of potentially an unlimited number of DNS\nnames (or even an IP address or set of IP addresses) and you're joined\nto other peers.\n\n> That's not viable for the stuff that usually gets distributed on p2p\n> networks, but it sure is viable when you have people working on the\n> same codebase.\n\n codebase-in-a-git, wiki-in-a-git, website-in-a-git,\nbackups-of-filesystem-in-a-git (joey hess, one of the most amazing\ndebian developers around, has done all these things :)\n\n> Which would be the use case for a git-over-p2p client,\n> right?\n\nyes it would... but i'm thinking well beyond that :)\n\nat first glance the concept of git-over-darknet for example looks\ngreat. however it doesn't satisfy what i believe to be one of the key\nstrategic requirements to be \"many-pushers to many-pullers real-time\nresilient\". lose access to the darknet, or even if the git server\ndies, what do you do?? if it's a darknet, you've got serious\nproblems: you can't find out who it was!\n\ngit-over-p2p on the other hand _would_ be [m-p-t-m-p-r-t-r], because\nonce the git-objects have been pulled by one client, they can be\npulled from that client by other clients too. it's this feature of\np2p distribution that is so distinct from the way in which git is\nnormally described as \"a distributed revision control system\", and\nmakes it a bugger to explain what the fuss is all about over this\ngit-p2p stuff :)\n\n l.\n"},{"id":"156740","messageId":"20101127192808.GA2750@burratino","threadId":"25855","inReplyTo":"AANLkTima6meFsovFS-15X7CMTD53n=kkvueKrOeN4Yd4@mail.gmail.com","subject":"Re: http://tech.slashdot.org/comments.pl?sid=1885890&cid=34358134","fromName":"Jonathan Nieder","fromEmail":"jrnieder@gmail.com","sentAt":"2010-11-27T19:28:08Z","receivedAt":"2010-11-27T19:28:08Z","isPatch":false,"sender":{"key":"jrnieder@gmail.com","avatar":"https://avatars.githubusercontent.com/u/281595?v=4"},"body":"Luke Kenneth Casson Leighton wrote:\n\n> i\n> therefore respectfull request that you please don't try to make me\n> feel like i _should_ be working on this in order to \"earn your\n> respect\" ( that goes for you too, johnathon ).\n\nJust for clarity: I never meant to imply such a thing. Just that\nmessages like the one you just sent have quite an off-topic feeling\nand that this mailing list is better for design advice, usage advice,\nbug reporting, patch review, and similar things. Just my own\nopinion.\n\nKind regards,\nJonathan\n"},{"id":"156741","messageId":"AANLkTi=aCRGNtKxrPLH81H8_NvpBNOmJ-0MHgRms2a3T@mail.gmail.com","threadId":"25855","inReplyTo":"AANLkTima6meFsovFS-15X7CMTD53n=kkvueKrOeN4Yd4@mail.gmail.com","subject":"Re: http://tech.slashdot.org/comments.pl?sid=1885890&cid=34358134","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2010-11-27T20:19:00Z","receivedAt":"2010-11-27T20:19:00Z","isPatch":false,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"On Sat, Nov 27, 2010 at 20:06, Luke Kenneth Casson Leighton\n wrote:\n> On Sat, Nov 27, 2010 at 6:36 PM, Ævar Arnfjörð Bjarmason\n> wrote:\n>> On Sat, Nov 27, 2010 at 18:33, Luke Kenneth Casson Leighton\n>> wrote:\n>>\n>>> I believe it is important that git-over-p2p be given a higher\n>>> priority than it is at present.\n>>\n>> We give \"priority\" to the stuff people submit patches for. There isn't\n>> a git-over-p2p because nobody is working on it, would you like to work\n>> on it?\n>\n>  i was / have been.  unfortunately it's unfunded work, and, thanks to\n> receiving less money than i require for living and for feeding my\n> family i now face a court hearing on 16th december (3 weeks time)\n> which has the aim and ultimate goal of making us homeless.  i\n> therefore respectfull request that you please don't try to make me\n> feel like i _should_ be working on this in order to \"earn your\n> respect\" ( that goes for you too, johnathon ).\n\nI don't mean to discourage you. But as Jonathan points out this list\nis much better suited for submitting patches and discussing technical\ndetails.\n\nI'd be happy to review your patches, I'm very interested in getting\nsomething like this myself. If you've been working on something I'd\nlove to see it.\n\nAnd I hope you'll do all-right in your personal life.\n\n>> I'm also not convinced that this is actually needed. It's trivial to\n>> set up a p2p-like network by just emulating a darknet by manually\n>> adding remotes & fetching/pushing.\n>\n>  ah.. but that's the single-use case for a single shared repository.\n> git \"as-is\" the fetch/push over http is limited to a single\n> repository.  i'm thinking more in terms of a global network, with\n> searches for checksummed objects \"accidentally\" potentially coming\n> from wildly different repositories.\n\nRight, but is there an actual use case for people who are developing\ncode to use something like git over p2p? Maybe I'm just being\nunimaginative, but I can't see a case where people are working on the\nsame project and can't find a way to push/pull from each other using\nthe existing methods. Especially since it's easy to sign up for free\nGit hosting and use something like Tor to pull/push from there. Or to\nset up your own git HTTP server on a Tor *.onion server.\n\nBut of course using the Git data format for non-source-code would be\nvery interesting, and much better for something like git-p2p.\n\nAnyway, sorry about the brief reply. I'm not really knowledgable about\nthis, and didn't fully grok/read your reply (going out).\n"},{"id":"156792","messageId":"1291025571.4262.21.camel@wpalmer.simply-domain","threadId":"25855","inReplyTo":"AANLkTi=aCRGNtKxrPLH81H8_NvpBNOmJ-0MHgRms2a3T@mail.gmail.com","subject":"Re: http://tech.slashdot.org/comments.pl?sid=1885890&cid=34358134","fromName":"Will Palmer","fromEmail":"wmpalmer@gmail.com","sentAt":"2010-11-29T10:12:51Z","receivedAt":"2010-11-29T10:12:51Z","isPatch":false,"sender":{"key":"wmpalmer@gmail.com","avatar":"https://avatars.githubusercontent.com/u/357044?v=4"},"body":"On Sat, 2010-11-27 at 21:19 +0100, Ævar Arnfjörð Bjarmason wrote:\n> Right, but is there an actual use case for people who are developing\n> code to use something like git over p2p? Maybe I'm just being\n> unimaginative, but I can't see a case where people are working on the\n> same project and can't find a way to push/pull from each other using\n> the existing methods. Especially since it's easy to sign up for free\n> Git hosting and use something like Tor to pull/push from there. Or to\n> set up your own git HTTP server on a Tor *.onion server.\n\nTo me, the use-case wouldn't be because I /can't/ use existing methods,\nit's because I /don't want to/ use existing methods :)\np2p tends to imply:\n - Resume-able downloads\n - Downloads from multiple sources at the same time\n - Transparently using an alternative source if one becomes unavailable\n - Load-balancing\n - referencing/linking based on \"what it is\" instead of \"where it is\"\n\nThe reason this stuff isn't a big itch for anybody is that these\nproblems tend to come about when cloning, which doesn't really happen\nthat often. When there's a new release, fetching the latest\nrelease-tag /might/ bring about the same needs, though in my experience\n\"there's a new release\" generally means nothing to me, since I check if\nthere's a new release by running \"git remote update\" and looking for new\ntags, merging with my local patches if I see a relevant one.\n\nI'm not convinced that any of this is something which is git's job, so\nmuch as it sounds like something which could benefit from another\ntransport layer that git can optionally tie into. Sure, these could be\ndeveloped together (p2p with git in mind), but adding a giant p2p\nsection to the git codebase sounds like bloat.\n\nI want my version control software to use p2p concepts for efficiency. I\ndon't want my version control software to be a p2p client any more than\nI want my text-editor to be a mail client.\n\n(In case it's not clear, I am in favour of p2p (\"gittorrent\")-like\nfunctionality being in git)\n"},{"id":"156796","messageId":"AANLkTiksxy5xJiw+T37KTaCV0s6OU7KbQhgQSo=tQi7_@mail.gmail.com","threadId":"25855","inReplyTo":"1291025571.4262.21.camel@wpalmer.simply-domain","subject":"Re: http://tech.slashdot.org/comments.pl?sid=1885890&cid=34358134","fromName":"Nguyen Thai Ngoc Duy","fromEmail":"pclouds@gmail.com","sentAt":"2010-11-29T10:35:26Z","receivedAt":"2010-11-29T10:35:26Z","isPatch":false,"sender":{"key":"pclouds@gmail.com","avatar":"https://avatars.githubusercontent.com/u/720?v=4"},"body":"On Mon, Nov 29, 2010 at 5:12 PM, Will Palmer wrote:\n> On Sat, 2010-11-27 at 21:19 +0100, Ævar Arnfjörð Bjarmason wrote:\n>> Right, but is there an actual use case for people who are developing\n>> code to use something like git over p2p? Maybe I'm just being\n>> unimaginative, but I can't see a case where people are working on the\n>> same project and can't find a way to push/pull from each other using\n>> the existing methods. Especially since it's easy to sign up for free\n>> Git hosting and use something like Tor to pull/push from there. Or to\n>> set up your own git HTTP server on a Tor *.onion server.\n>\n> To me, the use-case wouldn't be because I /can't/ use existing methods,\n> it's because I /don't want to/ use existing methods :)\n> p2p tends to imply:\n>  ..\n>  - Downloads from multiple sources at the same time\n\nI would add \"automatically\" and it's a real case for me. I work on the\nsame project on different machines (each of them has different\nresources that I need from time to time). So I clone to all the\nmachines. If I make a fix in one machine, other machines should\nautomatically fetch it. It's like a private p2p network with a single\nuser. If I add another machine to the network, all nodes should be\naware of it, without me doing \"git remote add\" on each node.\n-- \nDuy\n"},{"id":"156830","messageId":"4CF3FEB0.9040806@eaglescrag.net","threadId":"25855","inReplyTo":"1291025571.4262.21.camel@wpalmer.simply-domain","subject":"Re: http://tech.slashdot.org/comments.pl?sid=1885890&cid=34358134","fromName":"J.H.","fromEmail":"warthog19@eaglescrag.net","sentAt":"2010-11-29T19:27:44Z","receivedAt":"2010-11-29T19:27:44Z","isPatch":false,"sender":{"key":"warthog19@eaglescrag.net","avatar":null},"body":"> I want my version control software to use p2p concepts for efficiency. I\n> don't want my version control software to be a p2p client any more than\n> I want my text-editor to be a mail client.\n\nKeep in mind that adding p2p concepts to something doesn't make it more\nefficient, in fact in most cases it makes it dramatically *LESS* efficient.\n\ngit-torrent like concepts have come up in the past, and I keep pointing\nout how and why they likely won't be useful. The biggest reason: there\nis no advantage for a client to stay in the cloud once they have their\ndata. You can force this, sure, but clones are seldom and rare to begin\nwith (as you mentioned) so there won't be a very large cloud to pull\nfrom to start with. With that in mind, I really see no advantage to p2p\ninside of the git core at all. It adds a lot of complexity for little gain.\n\nNow you do mention things that would be useful:\n\n- Ability to resume a clone that you only have a partial download for\n(maybe just pack files?)\n- Ability to include something like a 'meta-link' like list of\nrepositories to check for data (inferred from the multiple download\nlocations)\n\nThere are things we can learn from p2p, but I don't think adding it to\ngit is actually useful.\n\nJust my $0.02 though.\n\n- John 'Warthog9' Hawley\n"},{"id":"156893","messageId":"1291110230.11984.21.camel@wpalmer.simply-domain","threadId":"25855","inReplyTo":"4CF3FEB0.9040806@eaglescrag.net","subject":"Re: http://tech.slashdot.org/comments.pl?sid=1885890&cid=34358134","fromName":"Will Palmer","fromEmail":"wmpalmer@gmail.com","sentAt":"2010-11-30T09:43:50Z","receivedAt":"2010-11-30T09:43:50Z","isPatch":false,"sender":{"key":"wmpalmer@gmail.com","avatar":"https://avatars.githubusercontent.com/u/357044?v=4"},"body":"On Mon, 2010-11-29 at 11:27 -0800, J.H. wrote:\n> > I want my version control software to use p2p concepts for efficiency. I\n> > don't want my version control software to be a p2p client any more than\n> > I want my text-editor to be a mail client.\n> \n> Keep in mind that adding p2p concepts to something doesn't make it more\n> efficient, in fact in most cases it makes it dramatically *LESS* efficient.\n\nAs a simple use-case:\nEveryone comes into an office in the morning and runs \"git remote\nupdate\". This potentially causes a lot of traffic between the office and\ntheir offsite central repository. If this were a p2p scenario, the\ntransfer from the offsite could potentially happen only once.\n\nThat counts as \"more efficient\", to me.\n\n> \n> git-torrent like concepts have come up in the past, and I keep pointing\n> out how and why they likely won't be useful. The biggest reason: there\n> is no advantage for a client to stay in the cloud once they have their\n> data. \n\nThis is true of bittorrent as well: People stay in the cloud for\naltruistic reasons.\n\nYou're thinking p2p in terms of \"every peer serves data, and keeps\nserving data. The network is more robust over time\". I'm thinking p2p in\nterms of \"Every peer has the ability to serve data. Adding a server is\nas trivial as adding a client.\"\n\nThe greatest advantage I can think of is \"no existing server needs to\nagree to the addition of a new server\", or at least \"the addition of an\nexisting server is accepted by convention, no questions asked\"\n\n> ..... You can force this, sure, but clones are seldom and rare to begin\n> with (as you mentioned) so there won't be a very large cloud to pull\n> from to start with. With that in mind, I really see no advantage to p2p\n> inside of the git core at all. It adds a lot of complexity for little gain.\n\nI agree that git itself is not a good place to explore p2p concepts. I\nassume it would be much more useful to develop an independent p2p layer\nand allow git to somehow use that.\nAnd while there won't be \"a large cloud\", it's almost a guarantee that\nthere would be more than the /one/ server currently available during\nclones or long fetches.\n\n> Now you do mention things that would be useful:\n> \n> - Ability to resume a clone that you only have a partial download for\n> (maybe just pack files?)\n> - Ability to include something like a 'meta-link' like list of\n> repositories to check for data (inferred from the multiple download\n> locations)\n> \n> There are things we can learn from p2p, but I don't think adding it to\n> git is actually useful.\n> \n> Just my $0.02 though.\n> \n> - John 'Warthog9' Hawley\n> \n\nThe biggest hurdle, I assume, would be \"get git to talk to more than one\nsource of data at once\", even if one needs to set those up manually. If\nI understand correctly, packfiles are generated in a way which would not\nnecessarily be consistent between clones.\n"},{"id":"156894","messageId":"20101130110836.2549aa78@jk.gs","threadId":"25855","inReplyTo":"1291110230.11984.21.camel@wpalmer.simply-domain","subject":"Re: http://tech.slashdot.org/comments.pl?sid=1885890&cid=34358134","fromName":"Jan Krüger","fromEmail":"jk@jk.gs","sentAt":"2010-11-30T10:08:36Z","receivedAt":"2010-11-30T10:08:36Z","isPatch":false,"sender":{"key":"jk@jk.gs","avatar":"https://avatars.githubusercontent.com/u/1774?v=4"},"body":"--- Will Palmer wrote:\n\n> As a simple use-case:\n> Everyone comes into an office in the morning and runs \"git remote\n> update\". This potentially causes a lot of traffic between the office\n> and their offsite central repository. If this were a p2p scenario, the\n> transfer from the offsite could potentially happen only once.\n\nThe same savings can be achieved by:\n\n- Hosting the repository locally in the office;\n- Automatically updating the 'official' location whenever something is\n pushed to the local repository;\n- Having all developers use the office repository as their remote.\n\nTakes about five minutes to set up.\n\n(That does not invalidate your overall arguments, of course, but I\ndon't have time to address them in much detail.)\n\n-Jan\n"}]}