libgit2 status

What is the status of libgit2 WRT the overall git project? I recall that there was some discussion of basing bits of git on libgit2 once it matures.

I ask because I'm starting a project to improve the abysmal speed of git-subtree split. It's unbearably slow at the moment and as far as I can puzzle out it's due almost entirely to repeated subshell invocations to run git commands.

I was planning on doing some experiments rewriting bits of git-subtree using libgit2 but I want to make sure that that isn't wasted work. It appears to be exactly what I need to code bits of git-subtree natively.

Thoughts?

                        -Dave

On 08/24/2012 04:02 PM, greened@obbligato.org wrote:

libgit2 is now maintained by Vicent Marti, who was once a gsoc student. He's employed by github and seems to spend most of his time working on libgit2.

Politically, I'm not sure how keen the git community is on handing over control to the core stuff of git to a commercial entity, but it doesn't seem to be a dying project, so I'd say go ahead and do it.

On Sat, Aug 25, 2012 at 2:56 AM, Andreas Ericsson <ae@op5.se> wrote:

The development of libgit2 happens 100% in the open. I don't know what "commercial entity" are you talking about, but there are several companies and independent contributors working on the Library at the moment.

The 25/08/12, Vicent Marti wrote:

Right but as far as I'm aware of Junio had reserves about libgit2 integration into git due to issues making repositories broken. Though, having libgit2 as git core would make libgit2 the the-facto standard which would a *very* big plus.

Also, I guess that integration into git would mean more developers contibuting for libgit2. Currently, issues seems to be a blocker for integration. So, libgit2 might appear to be a marginal/risky alternative for a long time which is sad.

[ I'm somewhat in the same situation of OP. ]

Nicolas Sebrecht <nicolas.s.dev@gmx.fr> writes:

The comment I saw about that was that at one point libgit2 had produced broken trees; which is true, the algorithm for the almost-alphanumeric sorting was slightly broken. This was fixed quite some time ago, which he also mentioned in the same message.

If you wait for it to be perfect, it's never going to happen. If your application would benefit, port it to libgit2 and report the issues you find. That's the only way we can know of the odd edge cases and improvements that we should make.

Note that the GitHub apps for Mac and Windows both use the Library to perform parts of their job. Their new backend for the website is also (going to be) based on libgit2.

I am also working on a project for a client involving the Library for importing data and the only problem we've had is that we discovered an edge case regarding symlinks and an assumption that one of the bindings made wrt diffs, which is getting fixed.

   cmn

I know julio notes about libgit2. Anyway the rpm5 mantainer had decided to integrate libgit2 recently. Jfi.

Regards

2012/8/25, Nicolas Sebrecht <nicolas.s.dev@gmx.fr>:

Andreas Ericsson <ae@op5.se> writes:

I do not think commercial-ness of any entity comes into the picture.

The only three things that matter are license compatibility (I think libgit2 licensed under GPLv2 + linkage exception is doing just fine in that department), maturity and quality of it (it is in early development phase), and the openness of the development process (it could do better by finding ways to better interact with the mainstream git development discussion that happens here in the longer term).

And the last one should really be a "longer term" item. It is more important for its codebase to get mature and robust, and that can only happen by various projects and products (e.g. GitHub for Mac) using it to improve it. I do not think "subtree" (or anything in contrib/ for that matter) is part of "the core stuff of git", and do not see a problem; such a move may help both subtree and libgit2.

Over a much longer timeperiod, I wouldn't be surprised if some "core stuff" gets reimplemented on top of libgit2 and distributed as part of the git-core.

There will be substantial integration and logistics hassles ahead of us before that can happen, though. E.g. we could point at libgit2 as our submodule, but that is not the only way to make git depend on libgit2; it could just be a Build-Depends like we depend on libz. Looking at the build dependency of libgit2 itself, I do not think tighter integration of the libgit2 itself into the git-core is not likely to happen very soon, and also is not necessarily a good thing to do.

Junio C Hamano <gitster@pobox.com> writes:

Obviously I meant "I think it is not likely to happen and is not necessaryly a good thing". Dumb double-negatives.

Junio C Hamano <gitster@pobox.com> writes:

I am hoping to move git-subtree into core once it performs a little better and I've fixed a couple of bugs. Will basing it on libgit2 delay that process significantly? Six months delay is no problem. 2 years would be problematic.

I would be happy to be a guinea pig for libgit2 in order to improve it, but I don't want to significantly impact git-subtree's move to core. I'll have to figure out the right balance there given feedback.

                         -Dave

<dag@cray.com> writes:

I expect it will take some time for libgit2 to allow our Makefile to start saying "LDFLAGS += -libgit2"; it will need to become as stable and widespread as other libraries we depend on, e.g. -lz and -lcurl.

Junio C Hamano <gitster@pobox.com> writes:

Well that's a chicken-and-egg problem, isn't it. How will a library become widespread unless something uses it?

Would it be enough to have libgit2 as an installable package in the major distributions?

                                 -Dave

<dag@cray.com> writes:

That something will not be the git core itself. Otherwise we will lose a stable reference implementation to catch its bugs.

Junio C Hamano <gitster@pobox.com> writes:

Well, the whole question here is whether git-subtree can become part of core if it is based on libgit2. It boils down to what you mean by "widespread," I guess. Does "widespread" mean "available as a package in major distributions," "installed by default in major distributions" or something else?

                                 -Dave

The 27/08/12, Junio C Hamano wrote:

This is exactly what I'm most afraid about. I tend to think it's a chicken-and-egg problem, too.

Do you expect one big merge of a very stable libgit2 at some point? Because, asking others to implement widely used tools on top of libgit2 in order to have it as stable/tested as curl is not going to happen, IMHO.

Otherwise, what about going with this optionnal "LDFLAGS += -libgit2" ASAP with good disclaimer that it's only intended for development and testing purpose? Then, git-core could slowly rely on functions of libgit2, one after the other.

Nicolas Sebrecht <nicolas.s.dev@gmx.fr> writes:

I don't think there's any need to merge libgit2 into the git project source. As a library, it should be perfectly usable as a project of its own, just like libcurl and libz.

This makes a lot of sense to me.

                            -David

<dag@cray.com> writes:

As I already said in my earlier message in $gmane/204305, I wouldn't be surprised if some "core stuff" gets reimplemented on top of libgit2 and distributed as part of the git-core. But at this moment, I see that just as a blip of possibility far in the future.

It would most likely start slowly, by adding lg-client/cat-file.c that is linked with libgit2 when "make USE_LIBGIT2=YesPlease" was asked for, and we compile the tried-and-true builtin/cat-file.c otherwise ("cat-file" may actually not the most trivial first step, though, but I think readers get the idea).

Even after most if not all commands have counterparts reimplemented on libgit2, I do not think we can afford to drop any of the original for a long time. For that to happen, at the very least:

 - libgit2 must be available in major distros so that people can say
   "[yum|apt-get] install libgit2-dev"; and

 - the version of it packaged for major distros are recent and
   stable enough, so that we never have to say "distro X ships with
   libgit2 that is too old, so people on that distro have to compile
   it from the source."

which is the quality we expect from libraries we would depend on, like -lz, -lcurl, etc.

It is OK if we have to conditionally compile out some of our code in the periphery when libgit2 that is available on the platform is too old (we allow it for -lcurl already).

But all of the above assumes one thing: the reimplementated result does not suck ;-) which is a large unknown.

Just an addendum, the libfication would come from within the core git developers, i.e, they start making the git library inside git itself. The git project would provide the API, not an external project.

With some structure like:

include/git.h src/git.c

...

whatever.

And doing the way Jeff did with argv_array for example and with rigid style guide defining how new APIs should be written. Just my 2 cents. Don't take it to seriously.

Thiago Farina wrote:

Junio- is it reasonable to expect the directory-restructuring by 2.0?

Ram

Ramkumar Ramachandra <artagnon@gmail.com> writes:

I actually hate "include/git.h vs src/git.c"; you have distinction between .c and .h already.

Junio C Hamano <gitster@pobox.com> writes:

+1

                          -David

Junio C Hamano <gitster@pobox.com> writes:

Having said that, I do not mind moving codeblocks around to make some files purely library-ish while others purely commands.

Ideally, if you run

    $ nm --defined-only -g builtin/frotz.o

you should see nothing but "T cmd_frotz" (there are exceptions, most notably, what the commands in the "log" family do are so close with each other that builtin/log.o can define cmd_* for all of them).

    $ nm --defined-only -og builtin/*.o  | grep -v 'T cmd_'

a handful of offenders. If these functions with external linkage are truly useful across subcommands, we should move them to a more library-ish location.

It may require splitting the bits that are too closely tied to the external interface they are implementing from these functions, generalizing only the core-ish logic from them, and moving them to a more library-ish file as a preparatory step. Such a library-ish file may be created inside lib/ subdirectory from the get-go.

Until that kind of code restructure happens, I do not see much sense in just moving files, e.g. renaming revision.c to src/revision.c or lib/revision.c or somesuch.

On Fri, Oct 19, 2012 at 5:13 PM, Junio C Hamano <gitster@pobox.com> wrote:

Which distinction are you talking about? This is not an issue of header file versus source file, but a public header file to be included by external projects versus internal header files that are intended to be included only by git itself and that will probably live under src/ directory.

On 10/19/2012 10:13 PM, Junio C Hamano wrote:

Agreed. The way libgit2 does it is to have "src/tag.[ch]", which are for internal use, and then "src/include/tag.h" which is the published version that others can use to write code against the tag library. src/tag.h always includes src/include/tag.h, so no code needs to be duplicated, but internal parts of the library can still use lower- level stuff if it wants to. It's a good compromise when creating a library from application code and there were no opaque types from the start.