{"thread":{"id":"29985","subject":"GSoC Application [ Parallelism + Git.pm ]","startedAt":"2012-03-18T07:36:17Z","lastAt":"2012-03-22T13:33:03Z","messageCount":7,"participants":["Subho Banerjee","Jakub Narebski","Thomas Rast","Nguyen Thai Ngoc Duy"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"187199","messageId":"CAB3zAY1cbjmJvPQo2i38Vxv=1CTzcusvtBTPQehi30cYsDe1Zw@mail.gmail.com","threadId":"29985","inReplyTo":null,"subject":"GSoC Application [ Parallelism + Git.pm ]","fromName":"Subho Banerjee","fromEmail":"subs.zero@gmail.com","sentAt":"2012-03-18T07:36:17Z","receivedAt":"2012-03-18T07:36:17Z","isPatch":false,"sender":{"key":"subs.zero@gmail.com","avatar":"https://gravatar.com/avatar/6159910e05d7650fc7da0a77c233711a014ea449cc5b30f0f065d0017b3357dd?d=mp&s=160"},"body":"Hello,\n\nI am a fourth year undergraduate student of Computer Science and\nEngineering at LNMIIT, India. I am extremely comfortable coding in\nC/C++ and perl. I am really interested in taking up the Google Summer\nof Code program for the \"git\" project. I have been a git user for a\ncouple of years now, though I was introduced to github very recently.\nThough I have never tried tinkering around with the git source code, I\nam a fast learner and I would love to contribute towards making git\nbetter.\n\nI had a look at the \"Ideas\" page on the GSoC website and I really\nliked two particular project ideas, in which I believe I can\ncontribute to a larger extent.\n\n* Improving parallelism in various commands\n* Modernizing and expanding Git.pm\n\n\nPreviously, in the last two summers, I was an intern at the European\nOrganization for Nuclear Research(CERN). here I worked on mainly two\nprojects(though none of them used git for version control), whose\nexperience I believe can be used directly in the tasks I mentioned\nabove. Firstly, I was working on a HPC monte-carlo simulations and how\nto make them faster using pthreads, openmp and cuda. Secondly, I was\nworking on a grid middleware solution written completely in perl.\n\n>From what I understand of these tasks --\n\n* In the first one, which wants to parallelize certain commands in\ngit, I believe the major challenges will be to actually find a large\nlist of commands which can be parallelized.In addition to the commands\nmentioned in the Ideas page, only other place I currently think of\nexploiting parallelism is in traversing the commit tree when one is\ncloning a repository. I would really like it if someone can suggest\nmore places where this sort of parallelism might be usable, so that I\ncould use that to make a more complete application. I believe one of\nthe major difficulties I will initially face, is my unfamiliarity with\nthe code. This makes finding these commands which might have better\nperformance with parallelism a little difficult.\n\n* For the second one, which aims at improving the Git perl module. I\ntried looking around for this one on the net. I was a little confused\nsince I could not make out which module this was on CPAN. Is this one\nof the Git::* modules or is it all of them. Because the the\nfunctionality of the Git::Config and Git::Commit as mentioned in the\nIdeas page seems to be there in the Git::Repository module on CPAN.\nCould some one please clarify this.\n\nI would really appreciate any ideas or advice for making my\napplication for GSoC 2012 better.\n\n\nCheers,\nSubho.\n"},{"id":"187208","messageId":"m37gyhlvkz.fsf@localhost.localdomain","threadId":"29985","inReplyTo":"CAB3zAY1cbjmJvPQo2i38Vxv=1CTzcusvtBTPQehi30cYsDe1Zw@mail.gmail.com","subject":"Re: GSoC Application [ Parallelism + Git.pm ]","fromName":"Jakub Narebski","fromEmail":"jnareb@gmail.com","sentAt":"2012-03-18T16:31:17Z","receivedAt":"2012-03-18T16:31:17Z","isPatch":false,"sender":{"key":"jnareb@gmail.com","avatar":"https://avatars.githubusercontent.com/u/2706?v=4"},"body":"Subho Banerjee writes:\n\n[...]\n> I had a look at the \"Ideas\" page on the GSoC website and I really\n> liked two particular project ideas, in which I believe I can\n> contribute to a larger extent.\n> \n> * Improving parallelism in various commands\n> * Modernizing and expanding Git.pm\n\n[...]\n> From what I understand of these tasks --\n[...]\n> * For the second one, which aims at improving the Git perl module. I\n> tried looking around for this one on the net. I was a little confused\n> since I could not make out which module this was on CPAN. Is this one\n> of the Git::* modules or is it all of them. Because the the\n> functionality of the Git::Config and Git::Commit as mentioned in the\n> Ideas page seems to be there in the Git::Repository module on CPAN.\n> Could some one please clarify this.\n\nThe \"Modernizing and expanding Git.pm\" project refers to the Git\nmodule in git sources[1], that is used by git commands implemented in\nPerl like git-svn, git-send-email, and interactive part of git-add.\n\nIt is not on CPAN (though if you feel like it putting it on CPAN might\nbe part of this project, but it must be \"dual-lived\").\n\n[1]: http://repo.or.cz/w/git.git/blob/HEAD:/perl/Git.pm\n http://git.kernel.org/?p=git/git.git;a=blob;hb=HEAD;f=perl/Git.pm\n https://github.com/git/git/blob/master/perl/Git.pm\n\nYou can of course take inspiration and code (if it is with compatibile\nlicense) from various Git::* modules on CPAN to implement the\n\"expanding\" part of this project.\n\nNote that Git.pm must remain extremly portable, which includes\nActivePerl on MS Windows (msysGit or Cygwin). Use of non-core modules\n(for 5.8.0) should be also probably limited.\n \n> I would really appreciate any ideas or advice for making my\n> application for GSoC 2012 better.\n\nHTH\n-- \nJakub Narebski\n"},{"id":"187392","messageId":"871uomt929.fsf@thomas.inf.ethz.ch","threadId":"29985","inReplyTo":"CAB3zAY1cbjmJvPQo2i38Vxv=1CTzcusvtBTPQehi30cYsDe1Zw@mail.gmail.com","subject":"Re: GSoC Application [ Parallelism + Git.pm ]","fromName":"Thomas Rast","fromEmail":"trast@student.ethz.ch","sentAt":"2012-03-21T12:47:26Z","receivedAt":"2012-03-21T12:47:26Z","isPatch":false,"sender":{"key":"tr@thomasrast.ch","avatar":"https://avatars.githubusercontent.com/u/153510?v=4"},"body":"Subho Banerjee writes:\n\n> * In the first one, which wants to parallelize certain commands in\n> git, I believe the major challenges will be to actually find a large\n> list of commands which can be parallelized.In addition to the commands\n> mentioned in the Ideas page, only other place I currently think of\n> exploiting parallelism is in traversing the commit tree when one is\n> cloning a repository. I would really like it if someone can suggest\n> more places where this sort of parallelism might be usable, so that I\n> could use that to make a more complete application. I believe one of\n> the major difficulties I will initially face, is my unfamiliarity with\n> the code. This makes finding these commands which might have better\n> performance with parallelism a little difficult.\n\nPlease read my reply to Felipe in the other thread:\n\n http://thread.gmane.org/gmane.comp.version-control.git/193352/focus=193574\n\nas I'd have to repeat myself.\n\n-- \nThomas Rast\ntrast@{inf,student}.ethz.ch\n"},{"id":"187395","messageId":"CAB3zAY319iV5ogaAeumo5bXC5vkvq=x_-6E689+fN8yZ6jtg1w@mail.gmail.com","threadId":"29985","inReplyTo":"871uomt929.fsf@thomas.inf.ethz.ch","subject":"Re: GSoC Application [ Parallelism + Git.pm ]","fromName":"Subho Banerjee","fromEmail":"subs.zero@gmail.com","sentAt":"2012-03-21T13:53:06Z","receivedAt":"2012-03-21T13:53:06Z","isPatch":false,"sender":{"key":"subs.zero@gmail.com","avatar":"https://gravatar.com/avatar/6159910e05d7650fc7da0a77c233711a014ea449cc5b30f0f065d0017b3357dd?d=mp&s=160"},"body":"Hello,\nThanks for the links. Those really helped.\nThis threading that will have to be done, will this have to platform\nindependent, for example will it be used in windows without cygwin?\n\nCheers,\nSubho.\n\nOn Wed, Mar 21, 2012 at 6:17 PM, Thomas Rast wrote:\n>\n> Subho Banerjee writes:\n>\n> > * In the first one, which wants to parallelize certain commands in\n> > git, I believe  the major challenges will be to actually find a large\n> > list of commands which can be parallelized.In addition to the commands\n> > mentioned in the Ideas page, only other place I currently think of\n> > exploiting parallelism is in traversing the commit tree when one is\n> > cloning a repository. I would really like it if someone can suggest\n> > more places where this sort of parallelism might be usable, so that I\n> > could use that to make a more complete application. I believe one of\n> > the major difficulties I will initially face, is my unfamiliarity with\n> > the code. This makes finding these commands which might have better\n> > performance with parallelism a little difficult.\n>\n> Please read my reply to Felipe in the other thread:\n>\n>  http://thread.gmane.org/gmane.comp.version-control.git/193352/focus=193574\n>\n> as I'd have to repeat myself.\n>\n> --\n> Thomas Rast\n> trast@{inf,student}.ethz.ch\n"},{"id":"187396","messageId":"CACsJy8AQEKfVKSOCRhGWb1FNT+onno5sB3QCuEdmpj4j2OWBGQ@mail.gmail.com","threadId":"29985","inReplyTo":"CAB3zAY319iV5ogaAeumo5bXC5vkvq=x_-6E689+fN8yZ6jtg1w@mail.gmail.com","subject":"Re: GSoC Application [ Parallelism + Git.pm ]","fromName":"Nguyen Thai Ngoc Duy","fromEmail":"pclouds@gmail.com","sentAt":"2012-03-21T14:10:03Z","receivedAt":"2012-03-21T14:10:03Z","isPatch":false,"sender":{"key":"pclouds@gmail.com","avatar":"https://avatars.githubusercontent.com/u/720?v=4"},"body":"On Wed, Mar 21, 2012 at 8:53 PM, Subho Banerjee wrote:\n> Hello,\n> Thanks for the links. Those really helped.\n> This threading that will have to be done, will this have to platform\n> independent, for example will it be used in windows without cygwin?\n\nYes. We simulate a subset of pthreads API for Windows in\ncompat/win32/pthread.[ch]. Stick to those functions and you are fine\n(or add some more, of course).\n-- \nDuy\n"},{"id":"187416","messageId":"CAB3zAY0atsiXkBDxaRkC+WGzQUSOYgTnRZmOXdqKiXRw_DKkjQ@mail.gmail.com","threadId":"29985","inReplyTo":"m37gyhlvkz.fsf@localhost.localdomain","subject":"Re: GSoC Application [ Parallelism + Git.pm ]","fromName":"Subho Banerjee","fromEmail":"subs.zero@gmail.com","sentAt":"2012-03-21T18:52:30Z","receivedAt":"2012-03-21T18:52:30Z","isPatch":false,"sender":{"key":"subs.zero@gmail.com","avatar":"https://gravatar.com/avatar/6159910e05d7650fc7da0a77c233711a014ea449cc5b30f0f065d0017b3357dd?d=mp&s=160"},"body":"Hello Jakub,\nI had some time to look through the perl module in the Git sources and\nI wanted to summarize the changes that need to be made -\n\n[Primary Task]\n[1] Move exception handling from Error::Simple to Try::Tiny and Exception::Class\n\n[Additional]\n[2] A Git::Config module that parses the .gitconfig file and the\n.git/config file in each repository(Is it one of these files or both?)\n[3] Parsing Tree and Commit objects and then traverse the tree\nstructure in Perl through a Git::Commit module.\n[4] Cleaning-up improving the API.\nIn general move towards an module that can access and change data in\nthe configuration and commit status using Perl instead of the fork and\nIPC being used now.\n\nIs this what you expect as a part of the GSoC work? Could you please\ntell me if I am missing something.\n\nCheers,\nSubho.\n\nOn Sun, Mar 18, 2012 at 10:01 PM, Jakub Narebski wrote:\n>\n> Subho Banerjee writes:\n>\n> [...]\n> > I had a look at the \"Ideas\" page on the GSoC website and I really\n> > liked two particular project ideas, in which I believe I can\n> > contribute to a larger extent.\n> >\n> > * Improving parallelism in various commands\n> > * Modernizing and expanding Git.pm\n>\n> [...]\n> > From what I understand of these tasks --\n> [...]\n> > * For the second one, which aims at improving the Git perl module. I\n> > tried looking around for this one on the net. I was a little confused\n> > since I could not make out which module this was on CPAN. Is this one\n> > of the Git::* modules or is it all of them. Because the the\n> > functionality of the Git::Config and Git::Commit as mentioned in the\n> > Ideas page seems to be there in the Git::Repository module on CPAN.\n> > Could some one please clarify this.\n>\n> The \"Modernizing and expanding Git.pm\" project refers to the Git\n> module in git sources[1], that is used by git commands implemented in\n> Perl like git-svn, git-send-email, and interactive part of git-add.\n>\n> It is not on CPAN (though if you feel like it putting it on CPAN might\n> be part of this project, but it must be \"dual-lived\").\n>\n> [1]: http://repo.or.cz/w/git.git/blob/HEAD:/perl/Git.pm\n>     http://git.kernel.org/?p=git/git.git;a=blob;hb=HEAD;f=perl/Git.pm\n>     https://github.com/git/git/blob/master/perl/Git.pm\n>\n> You can of course take inspiration and code (if it is with compatibile\n> license) from various Git::* modules on CPAN to implement the\n> \"expanding\" part of this project.\n>\n> Note that Git.pm must remain extremly portable, which includes\n> ActivePerl on MS Windows (msysGit or Cygwin).  Use of non-core modules\n> (for 5.8.0) should be also probably limited.\n>\n> > I would really appreciate any ideas or advice for making my\n> > application for GSoC 2012 better.\n>\n> HTH\n> --\n> Jakub Narebski\n>\n"},{"id":"187466","messageId":"201203221433.03616.jnareb@gmail.com","threadId":"29985","inReplyTo":"CAB3zAY0atsiXkBDxaRkC+WGzQUSOYgTnRZmOXdqKiXRw_DKkjQ@mail.gmail.com","subject":"Re: GSoC Application [ Parallelism + Git.pm ]","fromName":"Jakub Narebski","fromEmail":"jnareb@gmail.com","sentAt":"2012-03-22T13:33:03Z","receivedAt":"2012-03-22T13:33:03Z","isPatch":false,"sender":{"key":"jnareb@gmail.com","avatar":"https://avatars.githubusercontent.com/u/2706?v=4"},"body":"On Wed, 21 Mar 2012, Subho Banerjee wrote:\n\n> I had some time to look through the perl module in the Git sources and\n> I wanted to summarize the changes that need to be made -\n> \n> [Primary Task]\n> [1] Move exception handling from Error::Simple to Try::Tiny and Exception::Class\n\nWell, that is not as much \"primary task\" as \"minimal scope\"...\n\nWe have also decide if we want to keep compatibility layer, including\ngit_cmd_try. Git commands in Perl can be rewriten to using Try::Tiny\ndirectly, but I wonder if there are out of tree Perl scripts and modules\nthat use Git module, and would be broken by not preserving backward\ncompatibility.\n\nAnother issue of note is whether we want to have all non-core Git.pm\nprerequisites included like private-Error.pm (though I think in 'inc/'\nor something).\n\n> [Additional]\n> [2] A Git::Config module that parses the .gitconfig file and the\n> .git/config file in each repository (Is it one of these files or both?)\n\nI don't think parsing git config file format in Perl is a good idea.\nWhat I had in mind was to use `git config -l -z` output, reading all\nconfiguration at once with a single git command. \n\nNote that this would require converting to specific types, like e.g.\nturning true values of git config (e.g. string \"true\") into Perl\ntrue (1). The --int, --bool, --bool-or-int and --path should be easy;\nthe problem could be with --get-colorbool and --get-color.\n\nOptionally (if possible) make it so Git / Git::Repo object uses either\none \"git config --get \" for each $git->config() request, or\ncached values from single \"git config -l -z\" via Git::Config, depending\non constructor options ('lazy_config' => 1). Then we could fall back\nto one command for one access for unknown types...\n\n> [3] Parsing Tree and Commit objects and then traverse the tree\n> structure in Perl through a Git::Commit module.\n\n'commit', 'tag' and 'tree', and parsing and formatting ident fields.\nNote that there is a difference between 'raw' output format (\"git cat-file\"),\nand e.g. \"git ls-tree\" for 'tree' objects.\n\nIf by 'traverse the tree structure in Perl through a Git::Commit module.'\nyou mean traversing DAG of revisions, then I think it is out of scope.\n\nAnd there is also matter of parsing diff output (raw/tree, numstat,\npatchset).\n\n> [4] Cleaning-up improving the API.\n> In general move towards an module that can access and change data in\n> the configuration and commit status using Perl instead of the fork and\n> IPC being used now.\n\nYes, so that most if not all operations can be done on the level of Git.pm\nmethods or subroutines, and not having to invoke git commands and parse\ntheir output by hand.\n\nThis probably needs to be done by examining what git commands in Perl\nneed.\n\n> Is this what you expect as a part of the GSoC work? Could you please\n> tell me if I am missing something.\n\nAnother task could be CPAN-ification of Git.pm. This _could_ include\ncreating a separate repository for Git.pm (with all thats entailed),\nand subtree-mergeing it into git.git like git-gui and gitk are. \n\nYet another task could be making all git commands in Perl use Git.pm\n(well, at least those actively maintained).\n\nPerhaps also cleaning those Perl::Critic warnings that make sense.\n-- \nJakub Narebski\nPoland\n"}]}