GSoC Application [ Parallelism + Git.pm ]

Hello,

I am a fourth year undergraduate student of Computer Science and Engineering at LNMIIT, India. I am extremely comfortable coding in C/C++ and perl. I am really interested in taking up the Google Summer of Code program for the "git" project. I have been a git user for a couple of years now, though I was introduced to github very recently. Though I have never tried tinkering around with the git source code, I am a fast learner and I would love to contribute towards making git better.

I had a look at the "Ideas" page on the GSoC website and I really liked two particular project ideas, in which I believe I can contribute to a larger extent.

* Improving parallelism in various commands
* Modernizing and expanding Git.pm

Previously, in the last two summers, I was an intern at the European Organization for Nuclear Research(CERN). here I worked on mainly two projects(though none of them used git for version control), whose experience I believe can be used directly in the tasks I mentioned above. Firstly, I was working on a HPC monte-carlo simulations and how to make them faster using pthreads, openmp and cuda. Secondly, I was working on a grid middleware solution written completely in perl.

* In the first one, which wants to parallelize certain commands in
git, I believe  the major challenges will be to actually find a large
list of commands which can be parallelized.In addition to the commands
mentioned in the Ideas page, only other place I currently think of
exploiting parallelism is in traversing the commit tree when one is
cloning a repository. I would really like it if someone can suggest
more places where this sort of parallelism might be usable, so that I
could use that to make a more complete application. I believe one of
the major difficulties I will initially face, is my unfamiliarity with
the code. This makes finding these commands which might have better
performance with parallelism a little difficult.

* For the second one, which aims at improving the Git perl module. I
tried looking around for this one on the net. I was a little confused
since I could not make out which module this was on CPAN. Is this one
of the Git::* modules or is it all of them. Because the the
functionality of the Git::Config and Git::Commit as mentioned in the
Ideas page seems to be there in the Git::Repository module on CPAN.
Could some one please clarify this.

I would really appreciate any ideas or advice for making my application for GSoC 2012 better.

Cheers, Subho.

Subho Banerjee <subs.zero@gmail.com> writes:

[...]

[...]

[...]

The "Modernizing and expanding Git.pm" project refers to the Git module in git sources[1], that is used by git commands implemented in Perl like git-svn, git-send-email, and interactive part of git-add.

It is not on CPAN (though if you feel like it putting it on CPAN might be part of this project, but it must be "dual-lived").

[1]: http://repo.or.cz/w/git.git/blob/HEAD:/perl/Git.pm
     http://git.kernel.org/?p=git/git.git;a=blob;hb=HEAD;f=perl/Git.pm
     https://github.com/git/git/blob/master/perl/Git.pm

You can of course take inspiration and code (if it is with compatibile license) from various Git::* modules on CPAN to implement the "expanding" part of this project.

Note that Git.pm must remain extremly portable, which includes
ActivePerl on MS Windows (msysGit or Cygwin).  Use of non-core modules
(for 5.8.0) should be also probably limited.
 

HTH

Hello Jakub, I had some time to look through the perl module in the Git sources and I wanted to summarize the changes that need to be made -

[Primary Task] [1] Move exception handling from Error::Simple to Try::Tiny and Exception::Class

[Additional] [2] A Git::Config module that parses the .gitconfig file and the .git/config file in each repository(Is it one of these files or both?) [3] Parsing Tree and Commit objects and then traverse the tree structure in Perl through a Git::Commit module. [4] Cleaning-up improving the API. In general move towards an module that can access and change data in the configuration and commit status using Perl instead of the fork and IPC being used now.

Is this what you expect as a part of the GSoC work? Could you please tell me if I am missing something.

Cheers, Subho.

On Sun, Mar 18, 2012 at 10:01 PM, Jakub Narebski <jnareb@gmail.com> wrote:

On Wed, 21 Mar 2012, Subho Banerjee wrote:

Well, that is not as much "primary task" as "minimal scope"...

We have also decide if we want to keep compatibility layer, including git_cmd_try. Git commands in Perl can be rewriten to using Try::Tiny directly, but I wonder if there are out of tree Perl scripts and modules that use Git module, and would be broken by not preserving backward compatibility.

Another issue of note is whether we want to have all non-core Git.pm prerequisites included like private-Error.pm (though I think in 'inc/' or something).

I don't think parsing git config file format in Perl is a good idea. What I had in mind was to use `git config -l -z` output, reading all configuration at once with a single git command.

Note that this would require converting to specific types, like e.g. turning true values of git config (e.g. string "true") into Perl true (1). The --int, --bool, --bool-or-int and --path should be easy; the problem could be with --get-colorbool and --get-color.

Optionally (if possible) make it so Git / Git::Repo object uses either one "git config --get <key>" for each $git->config(<key>) request, or cached values from single "git config -l -z" via Git::Config, depending on constructor options ('lazy_config' => 1). Then we could fall back to one command for one access for unknown types...

'commit', 'tag' and 'tree', and parsing and formatting ident fields. Note that there is a difference between 'raw' output format ("git cat-file"), and e.g. "git ls-tree" for 'tree' objects.

If by 'traverse the tree structure in Perl through a Git::Commit module.' you mean traversing DAG of revisions, then I think it is out of scope.

And there is also matter of parsing diff output (raw/tree, numstat, patchset).

Yes, so that most if not all operations can be done on the level of Git.pm methods or subroutines, and not having to invoke git commands and parse their output by hand.

This probably needs to be done by examining what git commands in Perl need.

Another task could be CPAN-ification of Git.pm. This _could_ include creating a separate repository for Git.pm (with all thats entailed), and subtree-mergeing it into git.git like git-gui and gitk are.

Yet another task could be making all git commands in Perl use Git.pm (well, at least those actively maintained).

Perhaps also cleaning those Perl::Critic warnings that make sense.

Subho Banerjee <subs.zero@gmail.com> writes:

Please read my reply to Felipe in the other thread:

  http://thread.gmane.org/gmane.comp.version-control.git/193352/focus=193574

as I'd have to repeat myself.

Hello, Thanks for the links. Those really helped. This threading that will have to be done, will this have to platform independent, for example will it be used in windows without cygwin?

Cheers, Subho.

On Wed, Mar 21, 2012 at 6:17 PM, Thomas Rast <trast@student.ethz.ch> wrote:

On Wed, Mar 21, 2012 at 8:53 PM, Subho Banerjee <subs.zero@gmail.com> wrote:

Yes. We simulate a subset of pthreads API for Windows in compat/win32/pthread.[ch]. Stick to those functions and you are fine (or add some more, of course).