[RFC] Common library for Git GUIs

Hello Folks,

Sorry for a long mail. I am CC'ing people who wrote some git guis; I hope it does not offend you.

Looking at the current situation with Git GUIs, I thought it might be useful to create a generic library that would make it easier to develop git guis (especially plugins to various tools) and to add a new features to many of them with less effort. What do you people think about such idea?

Unfortunately in current situation no gui really supports all I would need to get my colleagues at work to accept git (they are somewhat obsessed with plugin to Visual Studio and explorer and generally avoiding command line). I started working on VS plugin some time ago, but I feel like a bit more reuse would be in order.

The proposed library should contain:

 - Models for the most common data GUIs want to display

   All modern GUI libraries (Qt, Gtk, WinForms, Cocoa) have some kind of
   generic interface for obtaining list, textbox and perhaps editbox data.
   Morover, the interfaces are pretty similar, so it should possible to
   create one interface and adapt it for the remaining toolkits.

   Data we want to export like this are:

    - Tree of all files in the work area with their status
      Seeing status (unchanged/modified/staged/new/...) of individual files
      in Visual Studio is the most wanted features of my colleagues at $work.
      The feature is applicable for Windows shell extension (Tortoise Git),
      similar KDE and Gnome extensions and all IDE bindings.

    - Tree of files in given revision
      For browsing other revisions than the checked out one.

    - History *tree*
      Starting gitk as separate process is suboptimal, because versions from
      the tree can't be easily selected for operations (checkout, merge, ...).

    - Blob data
      For looking at other revisions than the checked out one.

    - Commit properties
      For showing details of revisions in the history view.

    - Diffs
      Obviously...

    - Annotations
      Again, should be integrated with the rest of gui, so selecting line in
      annotation can open the revision in diff view, select it in history
      etc.

    - Configuration
      Many tools (eg. gui designers) feature a tree view of all properties of
      some object (property grid) with editable values and short
      descriptions. It's not as nice gui designed for individual options, but
      can provide good-enough easy-to-write way to set all valid options from
      the gui.

    Quite likely you can think of more data. Having generic implementation
    would save some coding when implementing the various plugins and have the
    benefit that additional view could be easily added to other guis, when
    it's implemented for one, because the interfaces will be the same.

 - Menu and action definitions for the common operations

   At least Gtk and Qt (don't know about others) have a concept of Action,
   which combines function for executing some operation with icon, name and
   enabled status of that action. These are than used for general menu
   descriptions.

   A generic library could provide arrays of such actions, that would define
   generic context menus for many things. For file, all Guis probably want to
   be able to add, revert, diff and annotate it. For revisions, all Guis
   probably want to check it out, merge it or rebase on it. Ditto for branch,
   which can additionally be pushed and updated from remote etc. Most of
   these operations take no or one additional parameter or few defined types
   (revision, branch, file). So if we extend the Action concept a bit, adding
   support for new simple operations (stash, submodule operations, etc.)
   quite easy for the Guis.

   Also Git Gui supports defining such operations that run specified shell
   commands. Qgit has similar, slightly more limited feature. Defining it in
   reusable library would make allow once defined command show in all guis
   a user uses.

What it should use:

 - It should probably be in C++ or C, with bindings for at least Perl,
   Python, Ruby, C#(CLR) and Java. The bindings can be done either with Swig,
   or using some base library that already has them.

   I think Java or CLR, while more portable, would not be appropriate because
   there is no standard way to combine them with other languages like Perl,
   Python and Ruby and those languages are still superior for the UI
   programming itself. I somewhat prefer C++, because polymorphism and some
   template tools would be useful here, but I am open to arguments.

 - It needs a portable runtime supporting at least:
    - Starting processes. We need to run git and shell commands. Even when
      libgit2 is implemented and this library ported to it, ability to run
      external commands should remain and libgit2 is not usable yet anyway.
      Should include ability to properly quote and dequote command lines in
      Windows.
    - Event loop. We should be able to process git output asynchronously.
    - Threads. Not all guis will be able to integrate the event loop in their
      own, so we need ability to run the event loop in a thread and run
      callbacks there (which can than wake the gui main loop).
    - Filesystem access.
   Also it would be very nice to have:
    - File alteration monitor. It would be very nice to notice changes in the
      work area automatically and both Linux and Windows have, although
      greatly incompatible, ways to get notifications for that, so it would
      be nice to be able to use that.
    - Bindings for languages. We can use Swig, but it has e.g. no support for
      callbacks, so having portable runtime with already existing bindings
      that support this would be an advantage.

Portable runtime options:

 So what do you people think would be best? I see several options:

 - QtCore

   Qt seems to be the most popular library among Git GUI writers and since
   version 4.5 will be LGPL, so it will be allowed to link with anything.
   It is also probably the most portable one. On the downside, it's rather
   large and it's language bindings are a bit worse (the garbage collector
   integration was a bit bad last time I looked).

 - Glib

   This is C based, so the core could be in plain C. It is also quite modular
   and has very good support for bindings to various languages. On the
   downside it's a bit less portable and less used among the existing guis.
   C would mean more work, but we could probably save some of it by using
   gob2 (g object builder)

 - STL + Boost

   I don't have experience with it, though I read some of the documentation.
   It should be sensibly portable. I know it has python bindings, the rest
   would probably have to be dealt with using swig.

 - POSIX + Msys on Windows

   I guess it would technically be usable, but I think it would be rather lot
   of additional work. It would probably be quite lightweight, though.

 - Apache portable runtime

   I have no experience with this one.

Hi,

I don't know if it is possible to elect a toolkit. Each toolkit is good.

Concerning the design of the library, I think it is better to split the library into different layers. The part of the e-world I know concern Gtk/Gnome, so I will use it for my example.

IMHO, it should be interesting to build a library containing only GObject'ification of Git and some wrappers/helpers to construct these objects. For example, some objects to represent authors, commits, branches, remotes and so on. Coupled to these base-types, this library should provide solutions to construct these base-types (wrappers around Git commands and/or internal files). This library can be named libgit-glib for example. Such library can help developement of current UI (giggle, gitg, anjuta-git plugin, and probably others I don't know).

Then, on top of this library, we can imagine another one providing high-level widgets. But it seems harder to identify common widgets between different GUI.

In order to justify my idea, take a look at Qt. They started with a large library, merging low-level with widgets. And then, they split it in order to allow access to low-level features only.

I'm really interested in such project. So, if someone knows such project, or create such a project, drop me a line, please.

Regards,

On Tue, Feb 17, 2009 at 13:55:04 +0100, Guilhem Bonnefille wrote:

Unfortunately this is quite important question. The selected foundation needs to be very portable and fairly lightweight. Heavy dependency would discourage gui implementors from using it.

Note, that I am *not* talking about GUI toolkit! The layer I am proposing should be gui toolkit agnostic or mostly so. Consider a model-view-controller design pattern. The goal is to provide generic model and controller parts in one layer and adaptors to model interfaces (like QAbstractItemModel and GtkTreeModel) in a set of thin layers for each gui toolkit.

Another set of thin layers could wrap them for various dynamic languages -- python, perl, ruby, C# (CLR) and Java, either using binding infrastructure of the selected toolkit, or using Swig.

Yes -- that's my idea. Basically it would be one layer for calling to git program behind the scenes, parsing and caching the results and presenting them in a form of genericish API.

Such API must not be too generic -- that would be just pointless wrapping one interface in another. The API would be strongly tied to the GUI features -- e.g. an object holding all data a dialog for preparing commit needs. But it must not be tied to any particular GUI, because that is usually restricted by the environment one wants to integrate git with.

Actually it is quite possible. There is a rather restricted set of
primitives:
 - button (also toolbar button and simple menu item)
 - choice (toggle button, check button, radio button group, combo box)
 - entry (entry, label (read-only entry))
   - number entry (spinbutton, slider, scrollbar)
   - text edit (for longer texts)
 - multi-column tree

For each of these primitives, you can create a generic interface:
 - button:

   This is a method to be executed on the button activation. It might be
   accompanied by name of the action, it's icon and a flag whether it should
   be enabled.

   Gtk has GtkAction and Qt has QAction, that do just this. We can easily
   create simple object with the same properties and adapt it to both of
   these and any similar interface used by any other toolkit.

 - choice:

   This is a variable with current value, that can be read and written and an
   array of valid choices. Some kind of signal must be emited when the value
   is changed, either by gui or by underlying application logic.

   I don't think there is a generic interface for this in either Gtk or Qt,
   but a generic one can be written and connections to the relevant widgets
   would than be generic for each toolkit.

 - entry:

   This is just a variable with current value, that can be read and written.
   Again, I don't think there is generic interface in Gtk or Qt, but one can
   be written and the connections would than be generic.

 - number entry:

   This is again a variable with current value, this time accompanied by
   minimum, maximum, precision and possibly page size. Adaptation can be done
   generically again.

   I am not sure we actually need this. The only I can think of is some
   numerical configuration options (like gc.reflogexpire) and that might not
   be worth the hassle. Definitely for the earlier versions.

 - text entry:

   Gtk has GtkTextBuffer, but Qt does not seem to have anything similar and
   to tell the truth I don't think we need either. We simply provide the
   initial text and get fed the final from the widget.
   
   Simple interface would be like for entry, where we provide the intial text
   and get fed the final one.

   Advanced interface needs to additionally support handling selection, so we
   can support things like selecting hunks in a diff.

 - multi-column tree:

   This is the most complex thing. It provides a tree of items, each of them
   consisting of several columns. Each column may additionally be editable,
   so it may need to be necessary to create one of the simple interfaces for
   it. Selection handling, at least single-line, would also be needed.

   For Gtk this would implement the GtkTreeModel and for Qt the
   QAbstractItemModel. All modern widget toolkits (Gtk, Qt, Windows.Forms,
   Cocoa, ...) have some kind of interface like that and they will be similar
   enough so that we can create some and than generically implement the rest
   in terms of that.

This is where I'd stop, since everything above that is very much specific to the selected toolkit. However, if layer like this is done right, it can really simplify creating the GUI itself quite a lot.

I am not aware of it existing. I was playing with an idea of refactoring some of the existing GUIs to get a generic library out of it. Mainly because I am not really satisfied with any of them and I thought such refactoring would simplify adding new features.

Hi,

On Feb 16, 2009, at 9:24 PM, Jan Hudec wrote:

It should be either C++ or C. If you want git devvers to work on it too, you'll probably want to go with C.

I think JGit is pretty far along for someone who wants to create a Java GUI.

I'd say bindings are pretty easy to create yourself.

None of these, if you want any GUI's to use it. Noone is going to create a Gtk / Cocoa / Windows app that depends on Qt. Nobody wants to use Boost in any situation and Glib, while being smaller than the rest, is also difficult as it isn't shipped with many OS's, for example OS X.

I think lightweight is the way to go. If you go for C++, you can also use the STL.

But, isn't this time spent better on getting libgit2 off the ground?

On Tue, Feb 17, 2009 at 20:08:23 +0000, Pieter de Bie wrote:

I don't think the really core devs need to work on this -- their time is best spent on the core. And many of the existing guis are in C++. For me, it depends on the user portable runtime more than anything else.

The advantage of swing is, that one definition with a few typedefs would generate Python, Perl, Ruby, CLR, Java and a few more bindings. GObject would need more language-specific work, but would nicely solve integration into the garbage collector. You know, I want to save as much work as possible.

I fully agree that nobody will want to depend on Qt -- QtCore is now a separate library, but the sources are not shipped separately AFAIK, so it'd be a pain. I would not think the case is as strong against boost and glib, though. People would either be getting binaries, in which case we can just bundle whatever dependency along, or building it and than one extra source tree (that can also be bundled for convenience) is not so much pain.

STL does not have any support for threads, event loop nor signals. Though
thinking about it, we may not actually need them.
 - we only need threads if our event loop can't be integrated into gui's one
   and the gui can start our in thread itself -- it's not too much code.
 - we only need file descriptors in the event loop and it needs to be
   integratable into the gui's one anyway.
 - simple callback is quite likely good enough for us -- the gui will need
   multiple callbacks, but it will need to connect in it's own signal system
   anyway.
So the shell invocation remains and that's little enough we can cut&paste
that from glib.

No, because what I have in mind is orthogonal to libgit2. libgit2 is supposed to be generic API for git, while I am proposing a specifically gui-oriented interface, which should implement all logic of a gui except opening dialogs and the widgets themselves, allowing the guis built on top of it to be totally dumb. Actually part of my idea is to create something, that can be later ported to libgit2 and immediately benefit many git interfaces.

Hi,

On Tue, 17 Feb 2009, Jan Hudec wrote:

Thank you for choosing what I should work on.

Ciao, Dscho

On Wed, Feb 18, 2009 at 00:45:33 +0100, Johannes Schindelin wrote:

Sorry. I didn't mean to tell you what should be doing.

What would we mere mortals do without people like you working on git core, thoug?

I think TortoiseGit need C\C++ git library, which should be also used by git itself. Otherwise, it is difficult sync with git.

2009/2/18 Jan Hudec <bulb@ucw.cz>:

On Wed, Feb 18, 2009 at 09:38:42 +0800, Frank Li wrote:

I don't mean to reimplement a single bit of what is implemented in git itself. I want to factor out some stuff that is above git, only useful for _graphical_ user interfaces.

Ah!! Sorry i missed that detail in the orig long mail :( I still don't think it making things simpler. What you are proposing is yet one more abstraction. But me thinks cli abstraction is enough.

Abstractions at interface levels are usefull, abstractions at functional level (gui's vs clis) are complex and don't solv anything. BAIN

I don't think lack of library is holding this back, all the functionality is exposed through cli and i find it perfectly fine interface. ( on teamgit side its lack of devtime :( thats holding it back) On the other hand a generic Qt/Gtk lib to interface with cli's would be nice. In any event i do not plan to switch to a library. Mainly because there is no value add (barring performance). And of-course, i already worked so hard to interface with the cli :D.

BTW take a look at git extensions for windows, it should provide both, win ui and VS plugin based on msysgit

I think an effort to convert git into libgit+cli is already underway, unless i missed something very obvious and you are talking about/for the same effort.

Anyway, Sorry folks no interest here.

BAIN

On Mon, Feb 16, 2009 at 22:24, Jan Hudec <bulb@ucw.cz> wrote:

Hi Jan,

   nice to hear you again.

 I fail to see the reason to do it given that git GUIs are already
quite mature and stable tools, feature addition at this stage is only
very limited and for only small things. Big features are already in
from a long time, as example (I speak about QGit but the same applies
to the others):

Already in. Modified files are bolded.

What qgit misses is new files (but this could be easily added if needed because alre already known to qgit) and staged files (but I dont understund the useful of this info).

Already in.

Already in.

Already in.

Already in.

Already in.  Obviously :-)

Already in.

To what should this feature apply? Some examples please?

Already in. Plus customizable actions/macros.

I never felt the need for a garbage collector at all. I strongly prefer to spend time to manually fix the (very few) memory leaks that slept in. Qt class model already does object housekeeping for you at deterministic and well known times (when parent object is deleted so are all corresponding children).

Thanks Marco

On Tue, Feb 17, 2009 at 08:37:13 +0100, Marco Costalba wrote:

Unfortunately I would not say that. There are many, many GUIs, but for me each of them fails in something. Qgit does not have pull/push and per-hunk staging (which git gui does), while git gui does not integrate with gitk (to eg. select revision to merge or cherry-pick from the version tree). Most others don't have version tree etc.

Currently we finally dropped ConfusingCase at work, but my boss does not want to switch to git until it has good shell extension and visual studio plugin. Git Extensions are starting to look pretty good, but does not have the file status at all yet and in current TortoiseGit it does not work yet (some icons are overlaid, but they are not correct). When that works, that's windows, but what about KDE, Gnome and OS X...

So the motivation is to create something, that could factor out the features from the existing GUIs to make it easier for other GUIs to adopt. So all plugins can adopt version tree easily etc. Also the goal is to make writing new GUIs -- meaning mainly plugins to more IDEs and other tools -- easier.

In a stand-alone GUI not big, but in a plugin for an IDE it is nice to see in the list of files in a project which ones are modified etc. Also all other version control plugins do that, so it's what people expect.

Yes - in Qgit. However, if I wanted to e.g. write a plugin for e.g. KDevelop, I would have to re-implement it all by direct calls to git program, because the implementation inside existing guis is not much reusable. Which is why I propose to create some reusable code for such tasks.

Often I need to configure some options in ~/.gitconfig and .git/config, but the GUIs generally only allow to set very few basic ones. I had to resort to editing ~/.gitconfig to turn off the cursed core.autocrlf, because git gui does not have that setting. And there are many more settings like that that users may want to tweak and something to guide them would be highly appreciated especially by the command-line-fearing windooze users.

So my idea is to provide some kind of "property sheet" -- a treeview with all the sections and options with editable values with proper constraints (so boolean and enum values could be entered with drop-down menus) and descriptions. If a generic library read the list of options from some data file and provided the values in a QAbstractItemModel, than any Qt gui could simply design the widgets (table + info pane) and instantiate that and have the feature. And since Gtk, Windows.Forms and other toolkits have similar interfaces, it is possible to provide an interface, that is easily adapted to any of them. If done right, that would simplify providing that feature and when new option would be defined in git, there would be just one place that would need to know about it to make it settable with all the guis.

I know that Qgit has it. Git Gui has it too -- done in an incompatible way, so if you define it in Qgit, you won't have it in Git Gui and vice versa. Common implementation would improve the user experience in this case.

My complaint is not about Qt in C++ -- where the ownership model is reasonable -- but about it's bindings for other languages. In python programmer expects all objects he has access to to be valid, but Qt objects will be destroyed with their parents and accessing a reference to them afterwards will cause the python interpreter to segfault. The programmer can avoid that by being careful not to leave such references around, but it's a thing python programmer is not supposed to care about. That's why I say the bindings are a bit worse (comparing to e.g. Glib/Gtk, which gets this part right). It can be fixed by better bindings -- it is just that the bindings when I last saw them didn't handle it.

On Tue, Feb 17, 2009 at 20:28, Jan Hudec <bulb@ucw.cz> wrote:

This is nice. Thanks for the idea, I will implement that in qgit when I find a bit of time :-)

Regarding your proposal I really wish you good luck and especially "have fun!!!"

For me it is like to trash out 95% of the stuff and start again from zero because of the last 5% is missing (but I can add it anyway with much smaller effort and time spent).

I understand the main reason, as per any GPL project, is having fun and good time coding and exchanging ideas with peers, but I really lack time and I am now moving to different interests. So, thank you very much, but I think I'll stick with qgit.

Best Marco

On  0, Marco Costalba <mcostalba@gmail.com> wrote:

One thing, though, that I think everyone can agree on is that getting a common lib for git guis is really step #2.

Step #1 would be to work towards libgit, since it is something that everyone would benefit from regardless of language or toolkit. I know that Shawn and co. have been working towards this goal. Keep it in C, keep it simple, and keep git stupid. A solid C core is portable and easy to wrap for Python, Ruby, Perl, etc.

BTW I recall that one of the first questions in this thread was "what toolkit" with proposed choices of QtCore, glib, POSIX+Msys, etc.

Just my $.02 -- I feel that the POSIX + MSys combination is the most viable solution since it has already been proven by the hard work of the msysgit team. It is also the environment which is most familiar to core git developers and thus there is much benefit to staying within that world. It's also the same choice made by Shawn in his libgit efforts.

The fact that git's output is identical regardless of platform (for instance, git ls-files always uses "/" as its path delimiter, even on windows) is really what has made creating portable git guis possible.

Git to me is like a familiar and happy land that I know I can escape to even if I have the misfortune of being stuck on a windows machine. Thus, a system like msys that bends over backwards trying to make windows into something unix-like feels like the right way to go if you ask me. I'm *not* a windows user, though =)