{"thread":{"id":"7493","subject":"Gittorent .. avahi ?","startedAt":"2007-04-01T17:59:43Z","lastAt":"2007-04-04T07:57:58Z","messageCount":8,"participants":["Srijak Rijal","Sam Vilain","Shawn O. Pearce","Jonas Fonseca","Jakub Narebski"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"38470","messageId":"9d916d5b0704011059g31394159kfdd5e01ba81915c2@mail.gmail.com","threadId":"7493","inReplyTo":null,"subject":"Gittorent .. avahi ?","fromName":"Srijak Rijal","fromEmail":"srijak@gmail.com","sentAt":"2007-04-01T17:59:43Z","receivedAt":"2007-04-01T17:59:43Z","isPatch":false,"sender":{"key":"srijak@gmail.com","avatar":null},"body":"Hi all,\nI hope this is the right forum for this :).\n I hope to be able to implement the gittorrent client/ tracker during\nGoogleSoc (if chosen :) ). I have been reading up on various git docs\nand listserv messages to figure out what features I want in it in\naddition to the basics. I have been concentrating especially on how to\nminimise tracker load.\nI was thinking about enabling avahi in the clients so that they can\nfind peers without putting that much extra load on the tracker.\nA small caveat is that avahi timeout is around 50ms, so this sort of\ndynamic peer discovery would probably turn out to work effectively\nonly among clients in the same LAN. However, best case scenario is\nclients in the same LAN would have only a couple of clients talking to\nthe tracker and spreading the data(as well as peer list etc) in their\nLAN.\n\nTo me, this feature seems be worth implementing if enough git users\nare in the same LAN trying to get at a certain git repository.\nWhat do you guys think ?\n\n-Srijak\n"},{"id":"38476","messageId":"46106AD3.6000204@vilain.net","threadId":"7493","inReplyTo":"9d916d5b0704011059g31394159kfdd5e01ba81915c2@mail.gmail.com","subject":"Re: Gittorent .. avahi ?","fromName":"Sam Vilain","fromEmail":"sam@vilain.net","sentAt":"2007-04-02T02:30:43Z","receivedAt":"2007-04-02T02:30:43Z","isPatch":false,"sender":{"key":"sam@vilain.net","avatar":"https://gravatar.com/avatar/8fc840ca854dbf6f7065b4335e3b934951c1dca3b11db688e95e471901f8f4a8?d=mp&s=160"},"body":"Srijak Rijal wrote:\n> Hi all,\n> I hope this is the right forum for this :).\n>  I hope to be able to implement the gittorrent client/ tracker during\n> GoogleSoc (if chosen :) ). I have been reading up on various git docs\n> and listserv messages to figure out what features I want in it in\n> addition to the basics. I have been concentrating especially on how to\n> minimise tracker load.\n>   \n\nNote that a minimal tracker can always return the same static file,\nlisting known stable peers; the P2P protocol is capable of discovering\nnew peers.\n\nAlso, unlike BitTorrent, the protocol is capable of negotiating the\ntracker update frequency.  A tracker under load would just start\nadvertising addresses for longer, and returning server busy responses\nfor peers that contact them too frequently.  See the \"valid\" key in the\nRequest, and the \"expires\" key in the Response.\n\nI think in BitTorrent there is an *over*-emphasis on making the tracker\nload light, because people want to be able to run them on the most\ndirt-cheap hosting account they can find, because they might get shut\ndown and don't want a huge outlay.  But with GitTorrent, a large number\nof the tracker providers are going to be software houses, and people\nalready running mirroring services - and so the increased load will be\nmore than justified by the reduced overall load.\n\n> I was thinking about enabling avahi in the clients so that they can\n> find peers without putting that much extra load on the tracker.\n> A small caveat is that avahi timeout is around 50ms, so this sort of\n> dynamic peer discovery would probably turn out to work effectively\n> only among clients in the same LAN. However, best case scenario is\n> clients in the same LAN would have only a couple of clients talking to\n> the tracker and spreading the data(as well as peer list etc) in their\n> LAN.\n>\n> To me, this feature seems be worth implementing if enough git users\n> are in the same LAN trying to get at a certain git repository.\n> What do you guys think ?\n>   \n\nLooks like avahi is only intended for the local network anyway, going by\nthe home page and Wikipedia article.\n\nI guess the use cases for this include programming houses and hackathons.\n\nAt a hackathon, you might scan the network for a list of projects, and\njump into one of the swarms.  Then you hack on the code.  Then to\n\"commit\", you set up and advertise another tracker that has your list of\nheads, but refers to the other repo as an \"alternative\" (see\nMetainfo/repo/alternatives).  The other people's avahi-enhanced\ngittorrent client could see that a new related project has been\nadvertised, and fetch new commits to \"remotes/\" references automatically.\n\nSVN fans would probably then configure it to automatically merge in new\nremote references to their current working tree, but that's not\nimportant right now :-).\n\nOne current TO-DO item in the specification to make this use case\nscalable is to allow connections to deal with multiple (possibly\nrelated) repositories at the same time.  Otherwise, with 6 \"committers\",\nyou've got 6 P2P swarms.  But this can happen with the next incremental\nversion of the specification.\n\nAt a programming house, setting up a tracker to commit wouldn't be as\nfrequent, as you expect people to have the appropriate access to\ncommit.  It would still be useful as a live directory of projects - but\nis it significantly more useful than a simple gitweb?\n\nAnyway, the old warning about scope creep applies to this idea - working\nfrom the bottom up is a lot better for getting things done than thinking\nbig.  Just look at me, I think big all the time and never get anything\ndone ;-)  There will be a lot of interesting technologies enabled by\ngittorrent.\n\nSam.\n"},{"id":"38492","messageId":"20070403053145.GF15922@spearce.org","threadId":"7493","inReplyTo":"46106AD3.6000204@vilain.net","subject":"Re: Gittorent .. avahi ?","fromName":"Shawn O. Pearce","fromEmail":"spearce@spearce.org","sentAt":"2007-04-03T05:31:45Z","receivedAt":"2007-04-03T05:31:45Z","isPatch":false,"sender":{"key":"spearce@spearce.org","avatar":"https://avatars.githubusercontent.com/u/34844?v=4"},"body":"Sam Vilain <sam@vilain.net> wrote:\n...\n> I think in BitTorrent there is an *over*-emphasis on making the tracker\n> load light, because people want to be able to run them on the most\n> dirt-cheap hosting account they can find, because they might get shut\n> down and don't want a huge outlay.  But with GitTorrent, a large number\n> of the tracker providers are going to be software houses, and people\n> already running mirroring services - and so the increased load will be\n> more than justified by the reduced overall load.\n...\n\nYes, I agree entirely.  Almost anyone who is going to be running a\ntracker for GitTorrent would already today be running the anonymous\nGit daemon.  These users (ahm... kernel.org) would like to reduce\nload on their servers if clients can get the data from elsewhere.\nHosting a tracker is still load, but should be a *much* lower load.\n\nNote that kernel.org's entire load appears to be disk IO (as stated\nby hpa on many occasions).  A little bit of extra CPU and network\nbandwidth (for tracker maintenance) to cut disk IO in half (due to\nless object directory accesses) would probably be major improvement\nfor them, and would still cut their overall network bandwidth as\nclients would get packs from other nodes in the P2P system.\n\n> Anyway, the old warning about scope creep applies to this idea - working\n> from the bottom up is a lot better for getting things done than thinking\n> big.  Just look at me, I think big all the time and never get anything\n> done ;-)  There will be a lot of interesting technologies enabled by\n> gittorrent.\n\nI agree.  The biggest risk to any GSoC project is not the student,\nor the community, or the technology... but its scope creep.\nOf course this is true of *any* project.\n\nGit evolved.  Linux evolved.  Apache evolved.  None of them set\nout on day one with a huge list of \"must have\" features that they\ncoded from scratch.  Every one of those projects organically grew\nas new features were desired and old bugs fixed.[*1*]\n\n\n[*1*] Yes, I know, Linus did set out when writing both Linux and\n      Git to create something that did X... but X was a pretty\n      small concept.  Git started as a \"track Linus' state thing\",\n      not as a full SCM.  ;-)\n\n-- \nShawn.\n"},{"id":"38515","messageId":"2c6b72b30704031125o4335aeefg88ccee163fa27142@mail.gmail.com","threadId":"7493","inReplyTo":"9d916d5b0704011059g31394159kfdd5e01ba81915c2@mail.gmail.com","subject":"Re: Gittorent .. avahi ?","fromName":"Jonas Fonseca","fromEmail":"jonas.fonseca@gmail.com","sentAt":"2007-04-03T18:25:26Z","receivedAt":"2007-04-03T18:25:26Z","isPatch":false,"sender":{"key":"jonas.fonseca@gmail.com","avatar":"https://gravatar.com/avatar/9b7fa23cce50269e5d164312b6ac5ae818a180f837b38f28f3bdf689dd7f96cd?d=mp&s=160"},"body":"Hello Srijak,\n\nOn 4/1/07, Srijak Rijal <srijak@gmail.com> wrote:\n>  I hope to be able to implement the gittorrent client/ tracker during\n> GoogleSoc (if chosen :) ). I have been reading up on various git docs\n> and listserv messages to figure out what features I want in it in\n> addition to the basics. I have been concentrating especially on how to\n> minimise tracker load.\n\nWhile the tracker is a very interesting part of GitTorrent, because it\nhas a lot of responsibility for 'advertising' new changes, I think the\nclient, in itself, would be a hard and big enough project. And as Sam\nsaid the tracker can be reduced to a simple static file that is\nregularly updated by a cron job or something like that. On the other\nhand, having a tracker might ease the task of implementing a client.\nHmm, chickens and eggs... ;)\n\nI made a report about GitTorrent from which you might be able to\nget some info from or at least some overall motivations and\nconsiderations. However, it is for the much simpler and stupid\n'initial' proposal, before Sam stepped in and made especially the\ncore of the peer wire protocol more scalable. So 'caveat emptor'\n(this especially goes for the more Git related parts).\n\nYou can find it at:\n\n  http://jonas.nitro.dk/tmp/foo/gittorrent.html/node38.html\n\nand the whole report at\n\n  http://jonas.nitro.dk/tmp/foo/gittorrent.pdf\n\nI really hope your project gets approved.\n-- \nJonas Fonseca (fonseca@diku.dk)\n"},{"id":"38523","messageId":"4612AEA6.3040706@gmail.com","threadId":"7493","inReplyTo":"2c6b72b30704031125o4335aeefg88ccee163fa27142@mail.gmail.com","subject":"Re: Gittorent .. avahi ?","fromName":"Srijak Rijal","fromEmail":"srijak@gmail.com","sentAt":"2007-04-03T19:44:38Z","receivedAt":"2007-04-03T19:44:38Z","isPatch":false,"sender":{"key":"srijak@gmail.com","avatar":null},"body":"Hi Jonas,\n>\n> While the tracker is a very interesting part of GitTorrent, because it\n> has a lot of responsibility for 'advertising' new changes, I think the\n> client, in itself, would be a hard and big enough project. And as Sam\n> said the tracker can be reduced to a simple static file that is\n> regularly updated by a cron job or something like that.\n\nYup. For now,  I plan to adhere closely to the spec and get the initial \nversion out :).\nI agree with Shawn in that letting the tool organically evolve to\ninclude new features and bug fixes as required is the best way to go.\nThe avahi idea is a product of me having time to think about the\nproject, but not being able to code :).\n\n> I made a report about GitTorrent from which you might be able to\n> get some info from or at least some overall motivations and\n> considerations. However, it is for the much simpler and stupid\n> 'initial' proposal, before Sam stepped in and made especially the\n> core of the peer wire protocol more scalable. So 'caveat emptor'\n> (this especially goes for the more Git related parts).\n>\n> You can find it at:\n>\n>  http://jonas.nitro.dk/tmp/foo/gittorrent.html/node38.html\n>\n> and the whole report at\n>\n>  http://jonas.nitro.dk/tmp/foo/gittorrent.pdf\n>\nGreat :). Thanks for the link. I think the report will be helpful.  I\nintend to look it over as soon as I get time. School in kinda hectic ATM.\n\n> I really hope your project gets approved.\nThanks, me too :),\n\n\n\n\nSrijak.\n"},{"id":"38577","messageId":"euurc0$js8$1@sea.gmane.org","threadId":"7493","inReplyTo":"9d916d5b0704011059g31394159kfdd5e01ba81915c2@mail.gmail.com","subject":"Re: Gittorent .. avahi ?","fromName":"Jakub Narebski","fromEmail":"jnareb@gmail.com","sentAt":"2007-04-04T00:30:05Z","receivedAt":"2007-04-04T00:30:05Z","isPatch":false,"sender":{"key":"jnareb@gmail.com","avatar":"https://avatars.githubusercontent.com/u/2706?v=4"},"body":"[Cc: Srijak Rijal <srijak@gmail.com>, git@vger.kernel.org]\n\nSrijak Rijal wrote:\n\n> I hope to be able to implement the gittorrent client/ tracker during\n> GoogleSoc (if chosen :) ).\n\nI hope that you have not missed deadline.\n\n> I was thinking about enabling avahi in the clients so that they can\n> find peers without putting that much extra load on the tracker.\n\nAvahi is Zeroconf implementation... wouldn't it be better to think\nabout implementing Kademlia?\n\n-- \nJakub Narebski\nWarsaw, Poland\nShadeHawk on #git\n"},{"id":"38578","messageId":"4612F823.2060503@gmail.com","threadId":"7493","inReplyTo":"euurc0$js8$1@sea.gmane.org","subject":"Re: Gittorent .. avahi ?","fromName":"Srijak Rijal","fromEmail":"srijak@gmail.com","sentAt":"2007-04-04T00:58:11Z","receivedAt":"2007-04-04T00:58:11Z","isPatch":false,"sender":{"key":"srijak@gmail.com","avatar":null},"body":"Hi Jakub,\n> I hope that you have not missed deadline.\n>\n>   \nI have already applied :)\n> Avahi is Zeroconf implementation... wouldn't it be better to think\n> about implementing Kademlia?\n>\n>   \nKademlia is something to look at. But not anytime soon :).\nThe current-\"est\" and working plan is to strictly follow the spec for\nfunctionality.\nThen, according to community requirement/recommendation etc. features\nwill be added.\n\n\nSrijak.\n"},{"id":"38608","messageId":"200704040957.58725.jnareb@gmail.com","threadId":"7493","inReplyTo":"4612F823.2060503@gmail.com","subject":"Re: Gittorent .. avahi ?","fromName":"Jakub Narebski","fromEmail":"jnareb@gmail.com","sentAt":"2007-04-04T07:57:58Z","receivedAt":"2007-04-04T07:57:58Z","isPatch":false,"sender":{"key":"jnareb@gmail.com","avatar":"https://avatars.githubusercontent.com/u/2706?v=4"},"body":"Srijak Rijal wrote:\n> Hi Jakub,\n>> I hope that you have not missed deadline.\n>>\n> I have already applied :)\n\nI hope then that you would be accepted.\n\n-- \nJakub Narebski\nPoland\n"}]}