{"thread":{"id":"18184","subject":"[GSoC] Google Summer of Code 2009 - new ideas","startedAt":"2009-03-07T00:44:17Z","lastAt":"2009-03-10T00:49:35Z","messageCount":8,"participants":["Jakub Narebski","Jan Janak","Tay Ray Chuan","P Baker","Shawn O. Pearce"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"107258","messageId":"200903070144.17457.jnareb@gmail.com","threadId":"18184","inReplyTo":null,"subject":"[GSoC] Google Summer of Code 2009 - new ideas","fromName":"Jakub Narebski","fromEmail":"jnareb@gmail.com","sentAt":"2009-03-07T00:44:17Z","receivedAt":"2009-03-07T00:44:17Z","isPatch":false,"sender":{"key":"jnareb@gmail.com","avatar":"https://avatars.githubusercontent.com/u/2706?v=4"},"body":"Time to submit application as mentoring organization to\nGoogle Summer of Code 2009 is close:  March 9 -- March 13.\n\nI'd like to add a few ideas to SoC2009Ideas wiki page, but before I do \nthis I'd like to ask for comments.  (The proposals also lacks proposed \nmentor).\n\nI am wondering if it would be worth it to make a separate class between \n\"New to Git?\" easy tasks, and \"Larger Projects\" hard tasks...\n\nBTW. some of ideas didn't make it from SoC2008Ideas wiki page to current \nyear page, namely: \n * Apply sparse To Fix Errors\n * Lazy clone / remote alternates\n * Implement git-submodule using .gitlink file\n * Teach git-apply the 3-way merge fallback git-am knows\n * Better Emacs integration\nWas this ommision deliberate or accidental?\n\n\n-- >8 --\n\n= New To Git? New To Open Source Development? =\n\n== Packfile caching for git-daemon ==\n\nEven with delta reuse, enumerating objects to be present in packfile \ngenerates significant load on server for pack-generating protocols, \nsuch as git:// protocol used by git-daemon.  Many of requests result in \nthe same packfile to be generated and sent; examples include full \nclone, or fetch of all branches since last update.  It would make sense \nthen to save (cache) packfiles, and if possible avoid regenerating \npackfiles by sending them from cache.  (Possible extension would be to \nsend slightly larger pack than needed if one can reuse cached packfile \ninstead).\n\nThe goal is for git-daemon to cache packfiles, use cached packfiles if \npossible, and to manage packfile cache.  Note that one would need in \nthe final version some way to specify upper limit on packfile cache \nsize and some cache entry expire policy.\n\n'''Goal:''' Support for packfile cache in git-daemon,\n            benchmark server load\n'''Language:''' C\n\n== Single credentials ==\n\nCurrently if you don't save your username and password in plain-text \n`.netrc` file (for HTTP transport), or avoid need for interactive \ncredentials using public key / private key pair (for SSH), you need to \nrepeat credentials many times during single git-fetch or git-clone \ncommand.  The goal is to reuse existing connections if possible, so the \nwhole transaction occurs using single connection and single \ncredentials; if that is not possible cache credentials (in secure way) \nso user need to provide username and password at most once.\n\n'''Goal:''' git-fetch and git-clone over HTTPS and git:// \n            requiring providing username and password at most once\n'''Language:''' C (perhaps also shell script)\n\n\n= Larger Projects =\n\n== Directory renames ==\n\nGit deals quite well with renames when merging.  One of the corner cases \nis when one side renamed some directory, and other side created ''new \nfiles'' in the old-name directory.  Git currently creates new files in \nresurrected old-name directory, while it could create new files under \nnew-name directory instead.\n\nThere is a bit of controversy about this feature, as for example in    \nsome programming languages (e.g. Java) or in some project build tool  \ninfo it is not posible to simply move a file (or create new file in    \ndifferent directory) without changing file contents.  Some say that    \nis better to fail than to do wrongly clean merge.\n\n'''Goal:''' At minimum option enabling wholesame directory rename \ndetection.  Preferred to add dealing with directory renames also to \nmerge.  At last, one can try to implement \"git log --follow\" for \ndirectories.\n'''Language:''' C\n'''See:''' [http://thread.gmane.org/gmane.comp.version-control.git/99529 \n|RFC PATCH v2 0/2| Detection of directory renames] thread on git \nmailing list (via GMane)\n'''See also:'''\n * \n[http://thread.gmane.org/gmane.comp.version-control.git/80912/focus=81362 \nmerge renamed files/directories?] subthread on git mailing list\n * [http://thread.gmane.org/gmane.comp.version-control.git/108106 \nComments on \"Understanding Version Control\" by Eric S. Raymond] thread \ncontains some thoughts on wholesame directory rename detection\n * [http://blog.teksol.info/2008/01/16/directory-renames-under-git \nDirectory renames under Git] blog post notice the issue\n * [http://www.markshuttleworth.com/archives/123 Renaming is the killer \napp of distributed version control] blog post by Mak Shuttleworth \n(pro-Bazaar).\n\n-- \nJakub Narebski\nPoland\n"},{"id":"107262","messageId":"20090307010921.GA6120@x61s.janakj.ryngle.net","threadId":"18184","inReplyTo":"200903070144.17457.jnareb@gmail.com","subject":"Re: [GSoC] Google Summer of Code 2009 - new ideas","fromName":"Jan Janak","fromEmail":"jan@ryngle.com","sentAt":"2009-03-07T01:09:21Z","receivedAt":"2009-03-07T01:09:21Z","isPatch":false,"sender":{"key":"jan@ryngle.com","avatar":null},"body":"On 07-03 01:44, Jakub Narebski wrote:\n> Time to submit application as mentoring organization to\n> Google Summer of Code 2009 is close:  March 9 -- March 13.\n> \n> I'd like to add a few ideas to SoC2009Ideas wiki page, but before I do \n> this I'd like to ask for comments.  (The proposals also lacks proposed \n> mentor).\n> \n> I am wondering if it would be worth it to make a separate class between \n> \"New to Git?\" easy tasks, and \"Larger Projects\" hard tasks...\n> \n> BTW. some of ideas didn't make it from SoC2008Ideas wiki page to current \n> year page, namely: \n>  * Apply sparse To Fix Errors\n>  * Lazy clone / remote alternates\n>  * Implement git-submodule using .gitlink file\n>  * Teach git-apply the 3-way merge fallback git-am knows\n>  * Better Emacs integration\n\nThere are already two (IMHO good) emacs modes for git, magit and egg:\n\nhttp://zagadka.vm.bytemark.co.uk/magit/\nhttp://github.com/bogolisk/egg/tree/master\n\n  Jan.\n"},{"id":"107270","messageId":"be6fef0d0903061856s21fdb4c4q9d52957dade96e94@mail.gmail.com","threadId":"18184","inReplyTo":"200903070144.17457.jnareb@gmail.com","subject":"Re: [GSoC] Google Summer of Code 2009 - new ideas","fromName":"Tay Ray Chuan","fromEmail":"rctay89@gmail.com","sentAt":"2009-03-07T02:56:39Z","receivedAt":"2009-03-07T02:56:39Z","isPatch":false,"sender":{"key":"rctay89@gmail.com","avatar":"https://avatars.githubusercontent.com/u/61553?v=4"},"body":"Hi,\n\nOn 3/7/09, Jakub Narebski <jnareb@gmail.com> wrote:\n> == Single credentials ==\n>\n> Currently if you don't save your username and password in plain-text\n> `.netrc` file (for HTTP transport), or avoid need for interactive\n> credentials using public key / private key pair (for SSH), you need to\n> repeat credentials many times during single git-fetch or git-clone\n> command.  The goal is to reuse existing connections if possible, so the\n> whole transaction occurs using single connection and single\n> credentials; if that is not possible cache credentials (in secure way)\n> so user need to provide username and password at most once.\n>\n> '''Goal:''' git-fetch and git-clone over HTTPS and git://\n>             requiring providing username and password at most once\n> '''Language:''' C (perhaps also shell script)\n\nPerhaps you might want to look at this:\n\nhttp://marc.info/?l=git&m=123599968929476&w=4\n\nAt that time, I was thinking more of removing git's reliance on curl's\nmulti interface so that it could use older versions of libcurl. But,\non this point, Daniel convinced me otherwise. In fact, it doesn't make\nsense if you could have a up-to-date git, but not an up-to-date curl.\n\nI didn't really get a reply on my point of \"minimized credential\nprompting\", though, and I think this GSoC proposal kinda gives support\nto it.\n\n>From a learning standpoint, I don't think this project would be too\nchallenging, nor can it sustain for a whole summer -- the basic\nstrategy to allow non-curl multi usage (ie. single connections) would\nbe to \"fork\" the current http slot methods and make them\nnon-curl_multi, then finding and replacing instances of them\nthroughout the code base.\n\nI already have a patch series that does that, plus a --persistent\noption for push. I'm fairly sure that it takes place on a single\nconnection (I'm relying on my firewall log though I'm doubting it's\nreliability on this issue).\n\n-- \nCheers,\nRay Chuan\n"},{"id":"107331","messageId":"526944450903071159v3ddc92c5q9b3239f2aef97ac8@mail.gmail.com","threadId":"18184","inReplyTo":"200903070144.17457.jnareb@gmail.com","subject":"Re: [GSoC] Google Summer of Code 2009 - new ideas","fromName":"P Baker","fromEmail":"me@retrodict.com","sentAt":"2009-03-07T19:59:20Z","receivedAt":"2009-03-07T19:59:20Z","isPatch":false,"sender":{"key":"me@retrodict.com","avatar":null},"body":"I posted to this list serve a few days ago about one of the 2008 SoC\nideas. Are those ideas still plausible? Specifically, I'm interested\nin pursuing the git-submodule update. Is this off the drawing board?\n\nP Baker\n\nOn 3/6/09, Jakub Narebski <jnareb@gmail.com> wrote:\n> Time to submit application as mentoring organization to\n>  Google Summer of Code 2009 is close:  March 9 -- March 13.\n>\n>  I'd like to add a few ideas to SoC2009Ideas wiki page, but before I do\n>  this I'd like to ask for comments.  (The proposals also lacks proposed\n>  mentor).\n>\n>  I am wondering if it would be worth it to make a separate class between\n>  \"New to Git?\" easy tasks, and \"Larger Projects\" hard tasks...\n>\n>  BTW. some of ideas didn't make it from SoC2008Ideas wiki page to current\n>  year page, namely:\n>   * Apply sparse To Fix Errors\n>   * Lazy clone / remote alternates\n>   * Implement git-submodule using .gitlink file\n>   * Teach git-apply the 3-way merge fallback git-am knows\n>   * Better Emacs integration\n>  Was this ommision deliberate or accidental?\n>\n>\n>  -- >8 --\n>\n>  = New To Git? New To Open Source Development? =\n>\n>  == Packfile caching for git-daemon ==\n>\n>  Even with delta reuse, enumerating objects to be present in packfile\n>  generates significant load on server for pack-generating protocols,\n>  such as git:// protocol used by git-daemon.  Many of requests result in\n>  the same packfile to be generated and sent; examples include full\n>  clone, or fetch of all branches since last update.  It would make sense\n>  then to save (cache) packfiles, and if possible avoid regenerating\n>  packfiles by sending them from cache.  (Possible extension would be to\n>  send slightly larger pack than needed if one can reuse cached packfile\n>  instead).\n>\n>  The goal is for git-daemon to cache packfiles, use cached packfiles if\n>  possible, and to manage packfile cache.  Note that one would need in\n>  the final version some way to specify upper limit on packfile cache\n>  size and some cache entry expire policy.\n>\n>  '''Goal:''' Support for packfile cache in git-daemon,\n>             benchmark server load\n>  '''Language:''' C\n>\n>  == Single credentials ==\n>\n>  Currently if you don't save your username and password in plain-text\n>  `.netrc` file (for HTTP transport), or avoid need for interactive\n>  credentials using public key / private key pair (for SSH), you need to\n>  repeat credentials many times during single git-fetch or git-clone\n>  command.  The goal is to reuse existing connections if possible, so the\n>  whole transaction occurs using single connection and single\n>  credentials; if that is not possible cache credentials (in secure way)\n>  so user need to provide username and password at most once.\n>\n>  '''Goal:''' git-fetch and git-clone over HTTPS and git://\n>             requiring providing username and password at most once\n>  '''Language:''' C (perhaps also shell script)\n>\n>\n>  = Larger Projects =\n>\n>  == Directory renames ==\n>\n>  Git deals quite well with renames when merging.  One of the corner cases\n>  is when one side renamed some directory, and other side created ''new\n>  files'' in the old-name directory.  Git currently creates new files in\n>  resurrected old-name directory, while it could create new files under\n>  new-name directory instead.\n>\n>  There is a bit of controversy about this feature, as for example in\n>  some programming languages (e.g. Java) or in some project build tool\n>  info it is not posible to simply move a file (or create new file in\n>  different directory) without changing file contents.  Some say that\n>  is better to fail than to do wrongly clean merge.\n>\n>  '''Goal:''' At minimum option enabling wholesame directory rename\n>  detection.  Preferred to add dealing with directory renames also to\n>  merge.  At last, one can try to implement \"git log --follow\" for\n>  directories.\n>  '''Language:''' C\n>  '''See:''' [http://thread.gmane.org/gmane.comp.version-control.git/99529\n>  |RFC PATCH v2 0/2| Detection of directory renames] thread on git\n>  mailing list (via GMane)\n>  '''See also:'''\n>   *\n>  [http://thread.gmane.org/gmane.comp.version-control.git/80912/focus=81362\n>  merge renamed files/directories?] subthread on git mailing list\n>   * [http://thread.gmane.org/gmane.comp.version-control.git/108106\n>  Comments on \"Understanding Version Control\" by Eric S. Raymond] thread\n>  contains some thoughts on wholesame directory rename detection\n>   * [http://blog.teksol.info/2008/01/16/directory-renames-under-git\n>  Directory renames under Git] blog post notice the issue\n>   * [http://www.markshuttleworth.com/archives/123 Renaming is the killer\n>  app of distributed version control] blog post by Mak Shuttleworth\n>  (pro-Bazaar).\n>\n>  --\n>  Jakub Narebski\n>  Poland\n>\n> --\n>  To unsubscribe from this list: send the line \"unsubscribe git\" in\n>  the body of a message to majordomo@vger.kernel.org\n>  More majordomo info at  http://vger.kernel.org/majordomo-info.html\n>\n"},{"id":"107342","messageId":"20090307205547.GN16213@spearce.org","threadId":"18184","inReplyTo":"526944450903071159v3ddc92c5q9b3239f2aef97ac8@mail.gmail.com","subject":"Re: [GSoC] Google Summer of Code 2009 - new ideas","fromName":"Shawn O. Pearce","fromEmail":"spearce@spearce.org","sentAt":"2009-03-07T20:55:47Z","receivedAt":"2009-03-07T20:55:47Z","isPatch":false,"sender":{"key":"spearce@spearce.org","avatar":"https://avatars.githubusercontent.com/u/34844?v=4"},"body":"P Baker <me@retrodict.com> wrote:\n> I posted to this list serve a few days ago about one of the 2008 SoC\n> ideas. Are those ideas still plausible? Specifically, I'm interested\n> in pursuing the git-submodule update. Is this off the drawing board?\n\nIts not off the table just because of what someone else proposed\nas an ideas list for 2009.\n\nA GSoC project is what the student makes of it.  You propose\nsomething that you are interested in working on, that you think\nyou can complete in the time available within the program calendar,\nand that the larger Git community would like to see implemented.\n\nI think your post fell flat a few days ago with no replies because\nit just didn't seem to invite any response from people.  Remember,\nwe're all quite busy with our own projects and lives too, just\nlike you are, and we do Git hacking/emailing in our spare time.\nAn email needs to be somewhat compelling and invite a reply in\norder to get a reply...\n\n-- \nShawn.\n"},{"id":"107415","messageId":"200903090059.36699.jnareb@gmail.com","threadId":"18184","inReplyTo":"be6fef0d0903061856s21fdb4c4q9d52957dade96e94@mail.gmail.com","subject":"Re: [GSoC] Google Summer of Code 2009 - new ideas","fromName":"Jakub Narebski","fromEmail":"jnareb@gmail.com","sentAt":"2009-03-08T23:59:36Z","receivedAt":"2009-03-08T23:59:36Z","isPatch":false,"sender":{"key":"jnareb@gmail.com","avatar":"https://avatars.githubusercontent.com/u/2706?v=4"},"body":"On Sat, 7 Mar 2009, Tay Ray Chuan wrote:\n> On 3/7/09, Jakub Narebski <jnareb@gmail.com> wrote:\n\n> > == Single credentials ==\n> >\n> > Currently if you don't save your username and password in plain-text\n> > `.netrc` file (for HTTP transport), or avoid need for interactive\n> > credentials using public key / private key pair (for SSH), you need to\n> > repeat credentials many times during single git-fetch or git-clone\n> > command.  The goal is to reuse existing connections if possible, so the\n> > whole transaction occurs using single connection and single\n> > credentials; if that is not possible cache credentials (in secure way)\n> > so user need to provide username and password at most once.\n> >\n> > '''Goal:''' git-fetch and git-clone over HTTPS and git://\n> >             requiring providing username and password at most once\n> > '''Language:''' C (perhaps also shell script)\n> \n> Perhaps you might want to look at this:\n> \n> http://marc.info/?l=git&m=123599968929476&w=4\n\nThank you for the link.\n\n> At that time, I was thinking more of removing git's reliance on curl's\n> multi interface so that it could use older versions of libcurl. But,\n> on this point, Daniel convinced me otherwise. In fact, it doesn't make\n> sense if you could have a up-to-date git, but not an up-to-date curl.\n> \n> I didn't really get a reply on my point of \"minimized credential\n> prompting\", though, and I think this GSoC proposal kinda gives support\n> to it.\n> \n> From a learning standpoint, I don't think this project would be too\n> challenging, nor can it sustain for a whole summer -- the basic\n> strategy to allow non-curl multi usage (ie. single connections) would\n> be to \"fork\" the current http slot methods and make them\n> non-curl_multi, then finding and replacing instances of them\n> throughout the code base.\n\nI was thinking more about caching credentials by git rather than forcing\nto use single connection.  Additionally you are solving the problem for\nthe HTTP(S) transport; admittedly for SSH there is much better solution\nof using public/private keys, instead of asking for password.\n\nI guess you are right and \"minimized credential prompting\" (aka \"single\ncredentials\") is too small a project for Google Summer of Code...\nI won't add it to SoC2009Ideas page.\n \n> I already have a patch series that does that, plus a --persistent\n> option for push. I'm fairly sure that it takes place on a single\n> connection (I'm relying on my firewall log though I'm doubting it's\n> reliability on this issue).\n\n-- \nJakub Narebski\nPoland\n"},{"id":"107423","messageId":"200903090218.27132.jnareb@gmail.com","threadId":"18184","inReplyTo":"20090309115026.obsvt34miowwcw8w@webmail.fussycoder.id.au","subject":"Re: [GSoC] Google Summer of Code 2009 - new ideas","fromName":"Jakub Narebski","fromEmail":"jnareb@gmail.com","sentAt":"2009-03-09T01:18:23Z","receivedAt":"2009-03-09T01:18:23Z","isPatch":false,"sender":{"key":"jnareb@gmail.com","avatar":"https://avatars.githubusercontent.com/u/2706?v=4"},"body":"Please do not trim CC list; at least do not remove git mailing list\nfrom addressees.\n\nOn Mon, 9 Mar 2009, thestar@fussycoder.id.au wrote:\n> Quoting Jakub Narebski <jnareb@gmail.com>:\n> \n> <snip>\n> > I was thinking more about caching credentials by git rather than forcing\n> > to use single connection.  Additionally you are solving the problem for\n> > the HTTP(S) transport; admittedly for SSH there is much better solution\n> > of using public/private keys, instead of asking for password.\n> \n> What about public/private keys that also have a password?  Isn't that  \n> the most recommended way to use ssh?\n\nYou use ssh-agent for that (via \"ssh-add <private key>\"), perhaps with\nwrapper around it for example keychain. The \"caching\" is done by SSH,\nno need to duplicate this in git.\n\n-- \nJakub Narebski\nPoland\n"},{"id":"107534","messageId":"20090310004935.GM11989@spearce.org","threadId":"18184","inReplyTo":"200903070144.17457.jnareb@gmail.com","subject":"Re: [GSoC] Google Summer of Code 2009 - new ideas","fromName":"Shawn O. Pearce","fromEmail":"spearce@spearce.org","sentAt":"2009-03-10T00:49:35Z","receivedAt":"2009-03-10T00:49:35Z","isPatch":false,"sender":{"key":"spearce@spearce.org","avatar":"https://avatars.githubusercontent.com/u/34844?v=4"},"body":"Jakub Narebski <jnareb@gmail.com> wrote:\n> I'd like to add a few ideas to SoC2009Ideas wiki page, but before I do \n> this I'd like to ask for comments.  (The proposals also lacks proposed \n> mentor).\n> \n> I am wondering if it would be worth it to make a separate class between \n> \"New to Git?\" easy tasks, and \"Larger Projects\" hard tasks...\n\nDone, there is now a \"Medium\" category.  Folks should start to\nrepaint the bikeshed if they disagre with my current choices\nin colors.\n \n> BTW. some of ideas didn't make it from SoC2008Ideas wiki page to current \n> year page, namely: \n>  * Apply sparse To Fix Errors\n>  * Lazy clone / remote alternates\n>  * Implement git-submodule using .gitlink file\n>  * Teach git-apply the 3-way merge fallback git-am knows\n>  * Better Emacs integration\n> Was this ommision deliberate or accidental?\n\nAccidental.  Most have been added back.  Of note I did not add back\nthe emacs integration as we have multiple emacs packages.\n \n-- \nShawn.\n"}]}