[VOTE] git versus mercurial

No, no, I'm not the one calling for a vote. You old-timers here will know the name Matt Dillon, who is leading the dragonflybsd project (www.dragonflybsd.org).

Matt is the one who is calling for the vote in his thread "Vote for your source control system" in the dragonfly.kernel group, accessible via nntp://nntp.dragonflybsd.org.

I've already cast my vote for git, which I confess is not very honest because I've never even tried mercurial. I would truly be grateful to anyone here who knows both git and mercurial who could contribute verifiable facts to that debate.

Thanks.

[Cc: gmane.comp.version-control.git,
     gmane.comp.version-control.mercurial.general]

walt <w41ter@gmail.com> writes:

I also used only Git, but I have read a bit about Mercurial; however the information I have on Mercurial might be out of date.

Below I have tried to compare Git with Mercurial, pointing the most important (to me) facts:

1. Documentation and ease of use. 

   Mercurial is supposedly better documented and easier to use; I
   think this descends from the early days of Git, where it was not
   very user friendly. IMHO Git has much improved since.  Mercurial
   had 'hgbook' from the beginning; Git User's Manual is more recent.

   I also think that ease of use is just different learning curve.
   I am also biased and I think that Mercurial starts easy, but it has
   more difficult things (like e.g. merging, multiple branches in
   single repository etc.) harder than it Git.

   I'll admit that Mercurial UI is better designed; Git UI was not as
   much designed as it evolved from 'stupid content tracker' to
   full-featured SCM, therefore there are a few oddities (for example
   git-revert might do not what you expect, if you are accustomed to
   Subversion UI).

2. Implementation, portability, bindings and extending

   Mercurial is implemented in Python, with core written in C for
   better performance.  It has a plugin system, and additional extra
   features implemented through extensions.  I don't know what is the
   status of bindings (or implementations) in other programming
   languages; but it has some API for use in extensions at least.

   Git is implemented as mixture of C, shell scripts, Perl and Tcl/Tk
   (for GUI).  The main way of extending it is by scripting around
   powerfull set of low level tools, called 'plumbing', meant to be
   used in scripts.  JGit is for example _reimplementation_ of Git in
   Java.

3. Repository design and performance.

   Git is designed around idea of content adressed object database;
   objects are adressed by their content (by SHA-1 of their type and
   content).  Commits for example have information about author and
   comitter, pointer to zero or more parent commits, and pointer to
   tree object (representing top directory in project).  Branches
   and tags are just pointers to DAG (Direct Acyclic Graph) of
   commits; current branch is denoted by HEAD pointer to branch.
   There is explicit staging area called 'index', used for conflict
   resolution dureing merges, and which can be used to make commit in
   stages, allow uncomitted changes in tree (in working directory).
   Git design supports very well multiple branches in single
   repository, and tracking changes in multiple remote repositories
   each with multiple branches.

   Mercurial, from what I have read of its documentation, and from the
   few discussion on #revctrl IRC channel, and from what I understand
   is based on changes stored per file, with information about files
   and their versions stored in manifest (flat) file, and with changes
   described in changelog-like file (changerev).  One of limitations
   of "record" database (as opposed to Git's object database) is that
   commits can have zero (root commit), one or two (merge commits)
   parents only.  There is apparent in design that Mercurial was
   developed with single branch per repository paradigm in mind.
   Local branches from what I understand are marked in CVS-like
   fashion using tags.  Tags are implemented as either local to
   repository and untransferable, or as .hgtags versioned file with
   special case treatment.  (But I'm obviously biased here).

   Git and Mercurial have similar performance, although it is thought
   that due to design Mercurla has faster patch applying and is
   optimized for cold cache case, while Git has faster merging and is
   optimized for warm cache case.

   Mercurial may have (or had) problems with larger binary files, from
   what I have heard.

3. Advanced features, interfaces and tools

   I don't know much about Mercurial beside basic usage, what I
   remember from 'hgbook', but I think that most if not all advanced
   Git features are available either in Mercurial core, or as
   Mercurial extensions (plugins).

   For example both Git and Mercurial have bisect command for finding
   bug by searching (possibly nonlinear) history for commit which
   introduced bug, ForestExtension is rough equivalent of git
   submodules (or third party git-externals), there is Transplain
   extension for git-rebase, etc.

   Mercurial has hgserve which can function as both web repository
   browser and as anonymous server; in Git they are split between
   git-daemon for anonymous repository access, and gitweb (or other
   web interfaces: cgit, git-php, ViewGit, Gitorious,..) for web
   interface.

   NOTE: In Git repacking and garbage collecting is explicit
   (although can be automated with "git gc --auto", and some of it
   happens automatically); Git use _rename detection_ rather than
   _rename tracking_, which has its advantages and disadvantages.

They are many articles comparing Mercurial and Git; if they are blog
posts please read the comments too.  Among them are:
* "Git vs. Mercurial: Please Relax" (Git is MacGyver, Mercurial
   is James Bond)
  http://importantshock.wordpress.com/2008/08/07/git-vs-mercurial/
* "The Differences Between Mercurial and Git"
  http://www.rockstarprogrammer.org/post/2008/apr/06/differences-between-mercurial-and-git/
* "Git vs. Mercurial"
  http://blog.experimentalworks.net/archives/38-Git-vs.-Mercurial.html
* "Git versus Mercurial..."
  http://codeheadsystems.wordpress.com/2008/05/10/git-versus-mercurial/
* "Git vs Mercurial"
  http://www.simplicidade.org/notes/archives/2007/12/git_vs_mercuria_1.html

Note however that the comparison at "Better SCM Initiative" has some
wrong information about Git: see
* "Git at Better SCM Initiative comparison of VCS (long)"
  http://thread.gmane.org/gmane.comp.version-control.git/95809/focus=97253

On 26 Oct 2008 15:15:57 +0100, Jakub Narebski <jnareb@gmail.com> wrote:

Also, there is http://book.git-scm.com/ that is similar to hgbook, I think.

Thanks for the comprarision!

On 10/26/08, Maxim Vuets <maxim.vuets@gmail.com> wrote:

I have been using Mercurial for about two years and am very comfortable with it. Here are some cons and pros

Mercurial PROS:
* Easier and more consistent UI. Newbie friendly.
* Better documentation. IMHO, hgbook is by far better than
http://book.git-scm.com/
* Windows support (personally, I do not care)

Mercurial CONS:
* Less potential than git. Once Ted Tso even said that "git has more
legs than mercurial", see
http://thunk.org/tytso/blog/2007/03/24/git-and-hg/
* Hg is strictly an SCM system while GIT is a content addressable file
system that can be used in other ways, hence the name Global
Information Tracker (GIT)
* Recently, Hg development seems to have somewhat slowed down. To
simply put it, there is not enough room in the world for several
similar SCM systems. With git's pace and momentum the other SCMs
including Hg are fighting an uphill battle.

Just my two cents. --Leo--

I'm not sure if Mercurial mailing list is not subscribe only. Git isn't.

On Sun, 26 Oct 2008, Leo Razoumov wrote:

I think that _might_ be example of "Worse is better" scenario, with Git having UI which evolved rather than was designed, and therefore less consistent.

Also if you are limiting to what is described in main chapters of 'hgbook', namely one branch (one "fork") per repository paradigm everything is simpler.

And probably better than "Git User's Manual". There are lot of various git-related documentation: "Git Magic", "Git for Computer Scientists", "Git from bottoms up"...

And I think it is not important for DragonflyBSD.

Besides git _has_ MS Windows support, in the form of Cygwin and in the form of msysGit project. It is still not as full as Linux support (for example git-svn comes to mind), but it is not bad. Well, Mercurial has TortiouseHg, while Git-Cheetah is in very early stages...

I agree, and I think it is at least partially because of Git having cleaner design, even if you have to understand more terms at first.

Errr... I think you are mislead by tongue-in-cheek backronym, which was created in the beginning, when git had very weak porcelain (i.e. SCM UI).

The competing _distributed_ version control systems left seems to be Bazaar-NG (Ubuntu), Mercurial (OpenSolaris, Mozilla), Git (Linux kernel, Freedesktop.org, Ruby on Rails people). There are many IDEs, many editors, many web browsers; there is Linux and there are *BSD; I hope that Mercurial would continue to be developed, and not vanish in obscurity like Arch and clones...

Am Sonntag 26 Oktober 2008 19:55:09 schrieb Jakub Narebski:

Before we get tangled in this train of thought: 

I created a head-to-head code_swarm of Mercurial and Git and it clearly shows that Mercurial development didn't slow down.

The code_swarm isn't a fancy one with music and annotations, but I think you'll directly see for yourself what I mean:

- http://www.rakjar.de/shared_codeswarm/hg-vs-git-short.avi

red is git, blue is Mercurial.

It is a result of my shared_codeswarm project with which you can create code_swarms from more than one repository automatically - and update them incrementally, creating new code_swarms of only the new commits in the repositories:

- http://www.rakjar.de/shared_codeswarm/project_activity_battle_swarm.html

Best wishes, Arne

-- My stuff: http://draketo.de - stories, songs, poems, programs and stuff :) -- Infinite Hands: http://infinite-hands.draketo.de - singing a part of the history of free software. -- Ein Würfel System: http://1w6.org - einfach saubere (Rollenspiel-) Regeln.

-- PGP/GnuPG: http://draketo.de/inhalt/ich/pubkey.txt

On 10/26/08, Arne Babenhauserheide <arne_bab@web.de> wrote:

I am not familiar with code swarms, sorry. My impressions are subjective are thoroughly un-scientific:-) (1) Judging by the activity of mailing lists git community is several times larger and more active in terms of actual submitted patches. (2) Hg forest extension is still not in the tree with outdated and incorrect documentation in the wiki. For me it was biggest reason to migrate from Hg to git.

--Leo--

Am Montag 27 Oktober 2008 05:15:11 schrieb Leo Razoumov:

That's always the case with code_swarms. 

They only show the commit activity: How often how many files where changed. 

They aren't a fair comparision but a damn unfair battle relying strongly on development style, programming language (influences the style) and such.

What you can see very clearly in them is how activity patterns _change_. 

And the Mercurial activity doesn't slow down. 

Instead in the beginning you can see them pacing each other, git always the bigger activity. There was a moment in may this year when git activity had receded to the point where it was equal to Mercurials activity, but it recovered from that.

An artifact in Mercurial is that it took an almost two week break in July this year, but apart from that development always rolled on, and in august the commits where coming fast again.

The smaller activity can for example be a result of a development style where changes are thouroughly discussed before they get implemented.

This is something which didn't change. Git had higher activity from the start, yet Mercurials actual code paced it well and was faster at some things.

Git still has higher activity, but that can simply stem from Mercurial being almost completely done in Python which need less code to do the same work.

Why didn't you instead update the documentation in the wiki? 

I don't use the forest extension, so I can't judge whether it is fit for inclusion in the tree.

But I wrote the group extension and learned that way that writing Mercurial extensions is far easier than I thought. And different from the shell, Python code is platform independent.

Best wishes, Arne

-- My stuff: http://draketo.de - stories, songs, poems, programs and stuff :) -- Infinite Hands: http://infinite-hands.draketo.de - singing a part of the history of free software. -- Ein Würfel System: http://1w6.org - einfach saubere (Rollenspiel-) Regeln.

-- PGP/GnuPG: http://draketo.de/inhalt/ich/pubkey.txt

Hello,

On Mon, Oct 27, 2008 at 9:45 AM, Leo Razoumov <slonik.az@gmail.com> wrote:

There are a few extensions that ought to be part of main hg, I have proposed rdiff and am still waiting to hear. Since hg depends a lot on extensions to extend it (good design), commnly used or useful extensions must be made part of mainstream at a steady pace. Otherwise, new users who are not aware of existence of various extensions will start comparing main hg with git. Since git has most of the features in the core part, hence the comparisons will be not apple to apple.

-dhruva

Am Sonntag 26 Oktober 2008 19:55:09 schrieb Jakub Narebski:

What do you mean by "cleaner design"? 

From what I see (and in my definition of "design"), Mercurial is designed as VCS with very clear and clean design, which even keeps things like streaming disk access in mind.

Also, looking at git, git users still have to garbage collect regularly, which shows to me that the design wasn't really cleaner.

As an example: If I want some revision in hg, my repository just reads the files in the store, jumps to the latest snapshots, adds the changes after these and has the data.

In git is has to check all changesets which affect the file. 

If you read the hgbook, you'll find one especially nice comment: 

"Unlike many revision control systems, the concepts upon which Mercurial is 
built are simple enough that it’s easy to understand how the software really 
works. Knowing this certainly isn’t necessary, but I find it useful to have a 
“mental model” of what’s going on."
- http://hgbook.red-bean.com/hgbookch4.html

I really like that, and in my opinion it is a great compliment to hg, for two reasons:

1) Hg is easy to understand
2) You don't have to understand it to use it

And both are indications of a good design, the first of the core, the second of the UI.

Best wishes, Arne

-- My stuff: http://draketo.de - stories, songs, poems, programs and stuff :) -- Infinite Hands: http://infinite-hands.draketo.de - singing a part of the history of free software. -- Ein Würfel System: http://1w6.org - einfach saubere (Rollenspiel-) Regeln.

-- PGP/GnuPG: http://draketo.de/inhalt/ich/pubkey.txt

On Mon, 27 Oct 2008, Arne Babenhauserheide wrote:

Clean _underlying_ design. Git has very nice underlying model of graph (DAG) of commits (revisions), and branches and tags as pointers to this graph.

I have read description of Mercurial's repository format, and it is not very clear in my opinion. File changesets, bound using manifest, bound using changerev / changelog.

Mercurial relies on transactions and O_TRUNC support, while Git relies on atomic write and on updating data then updating reference to data.

I don't quite understand comment about streaming disk access...

Well, they have to a lot less than they used to, and there is "git gc --auto" that can be put in crontab safely.

Explicit garbage collection was a design _decision_, not a sign of not clear design. We can argue if it was good or bad decision, but one should consider the following issues:

 * Rolling back last commit to correct it, or equivalently amending
   last commit (for example because we forgot some last minute change,
   or forgot to signoff a commit), or backing out of changes to the
   last commit in Mercurial relies on transactions (and locking) and
   correct O_TRUNC, while in Git it leaves dangling objects to be
   garbage collected later.

 * Mercurial relies on transaction support. Git relies on atomic write
   support and on the fact that objects are immutable; those that are
   not needed are garbage collected later. Beside IIRC some of ways of
   implementing transaction in databases leads to garbage collecting.

 * Explicit packing and having two repository "formats": loose and
   packed is a bit of historical reason: at the beginning there was
   only loose format. Pack format was IIRC invented for network
   transport, and was used for on disk storage (the same format!) for
   better I/O patterns[1]. Having packs as 'rewrite to pack' instead
   of 'append to pack' allows to prefer recency order, which result in
   faster access as objects from newer commits are earlier in delta
   chain and reduction in size in usual case of size growing with time
   as recency order allows to use delete deltas. Also _choosing_ base
   object allows further reduce size, especially in presence of
   nonlinear history.

 * From what I understand Mercurial by default uses packed format for
   branches and tags; Git uses "loose" format for recent branches
   (meaning one file per branch), while packing older references.
   Using loose affects performance (and size) only for insane number of
   references, and only for some operations like listing all references,
   while using packed format is IMHO a bit error prone when updating.

 * Git has reflogs which are pruned (expired) during garbage collecting
   to not grow them without bounds; AFAIK Mercurial doesn't have
   equivalent of this feature.

   (Reflogs store _local_ history of branch tip, noting commits, 
   fetches, merges, rewinding branch, switching branches, etc._

[1] You wrote about "streaming disk access". Git relies (for reading) on good mmap implementation.

If you want to show some revision in Git, meaning commit message and diff in patch format (result of "git show"), Git just reads the commit, outputs commit message, reads parent, reads trees and performs diff.

If you want to checkout to specific revision, Git just reads commit,
reads tree, and writes this tree (via index) to working area.
 

I don't understand you here... if I understand correctly above, then you are wrong about Git.

Because it is simple... and less feature rich, c.f. multiple local branches in single repository.

You don't have to understand details of Git design (pack format, index, stages, refs,...) to use it either.

Well, Git is built around concept of DAG of commits and branches as references to it. Without it you can use Git, but it is hard. But if you understand it, you can understand easily most advanced Git features.

I agree that Mercurial UI is better; as usually in "Worse is Better" case... :-)

Am Montag 27 Oktober 2008 02:52:22 schrieb Jakub Narebski:

This grows very simple if you keep common filesystem layout in mind. 

inodes and datanodes (the files in the store), organized in directories which keep many files (manifests) bound in changesets which keep additional data.

For most operations Mercurial just relies on appending support. 

If you tell a disk "give me files a, b, c, d, e, f (of the whole abc)", it is faster then if you tell it "give me files a k p q s t", because the filesystem can easier optimize that call.

That's why for example Mercurial avoids hashing filenames. 

relying on crontab which might not be available in all systems (I only use GNU/Linux, but what about friends of mine who have to use Windows?)

As far as I know the only problem woth O_TRUNC was that it sadly had bugs in Linux.

But Mercurial normally works on standard filesystems, so this isn't the case for normal operations.

You culd say, though, that git implements a very simple transaction model: Keep all old data until it gets purged explicitely.

So having multiple packs is equivalent to the automatic snapshot system in Mercurial which doesn't need user interaction.

As far as I know, Mercurial got that "using packed format" right from the beginning.

As far as I know Mercurial only tracks the state of the working directory, so it doesn't track your whole local history.

But others can better tell you more about that in greater detail. 

Might be that I remember incorrectly about what git does. 

Are its commits "the whole changed file" or "the diff of the changes"? 

If the latter, it needs to walk back all commits to the snapshot revision to get the file data.

One story I experienced with that: 

My amd64 GNU/Linux box suffers from performance problems when it gets high levels of disk activity (something about the filesystem layer doesn't play well with amd64 - reported by others, too).

When I pulled a the Linux kernel repository with git half a year ago, my disk started klicking and the whole computer slowed down to a crawl.

When I pulled the same repository data from a Mercurial repository, the computer kept running smooth, the disk stayed silent and happily wrote the data.

Mercurial felt smooth, while git felt damn clumsy (though not slow). 

That works quite well. People just don't use it very often, because the workflow of having multiple repositories is easier with hg.

I remember that to have been incorrect about half a year ago, when I stumbled over many problems in git whenever I tried to do something a bit nonstandard.

It took me hours (and in the end asking a friend) to find out about 

"git checkout ."

just to get back my deleted files. 

The answer I got when I asked why it's done that way was "this is because of the inner workings of git. You should know them if you use it".

What do you mean with that? 

Best wishes, Arne

-- My stuff: http://draketo.de - stories, songs, poems, programs and stuff :) -- Infinite Hands: http://infinite-hands.draketo.de - singing a part of the history of free software. -- Ein Würfel System: http://1w6.org - einfach saubere (Rollenspiel-) Regeln.

-- PGP/GnuPG: http://draketo.de/inhalt/ich/pubkey.txt

On Mon, 27 Oct 2008, Arne Babenhauserheide wrote:

Well, for you it might be simple, for others it is binding things together with duct tape and spit. (I am exaggerating here).

What Mercurial repository design did not get correctly, in my opinion
is its handling of tags and [named] branches.
 

I would expect _good_ filesystem to be able to optimize this call as well. As I said it looks like Mercurial and Git are optimized for different cases: Git relies on filesystem for caching, and optimizes for warm cache performance.

First, git does not hash filenames. The hash is contents, not of name.
You can say it stores objects (in loose format) in hash-based filenames.
 
Second, in packed repository you don't have to ask filesystem to "give
me files a, k, p, q, s, t (of the whole abc)"; you ask filesystem to
mmap a _single_ pack file (well, almost, there is also pack index to
mmap).

Yes, that means that you should periodically repack for better performance... but currently git tries to use packed format as much as possible, keeping packs from network if they are not too small, repacking if creating large number of objects, etc.

So they would have to either periodically repack by hand, or use some crontab equivalent on MS Windows.

But that doesn't matter in the context of this discussion, which is DragonflyBSD; worse or better support for MS Windows doesn't matter here, does it?

Mercurial implements transactions as a way to keeping operations atomic. So I don't know what you mean by "normally works on standard filesystem" here.

Git just uses different way to keep operations atomic, different way of implementing transactions.

I'm not sure if I should have mentioned transactions in databases here. Oh, well... Note however that there are advanced way of doing transactions in relational databases which lead to dangling things to be purged when transaction is interrupted. But this is not to the point...

Snapshot system doesn't change the fact that Mercurial (from what I understand) implements forward deltas, from older version to never version, and not from newer version to older.

Also from what I remember Mercurial didn't implement deltification right from the start; it had problems with nonlinear history (it used delta from last version appended, not from the parent version).

And probably requires transactions and locks for that. Git simply uses atomic write solution for atomic update of references.

Reflogs are very useful, and are natural extension of simple rollback last transaction Mercurial has (which Git had equivalent from the very beginning in the form of ORIG_HEAD). They allow for example for you go back to the state before incorrect rewinding a branch, or before applying series of patches from email, etc.

Git is snapshot based SCM, although 'behind the scenes' it uses deltas in the pack format. So to get file data at given revision (i.e. to do something like "git show <revision>:<filename>") it needs to access <revision>, access its tree, and access contents of a file (blob).

Behind the scenes, at a lower level, Git does necessary delta resolving. Delta chains in packs have limited length (as they have in Mercurial).

The answer usually is: did you have this repository packed? I admit that it might be considered one of disadvantages of git, this having to do garbage collection from time to time... just like in C ;-)

Workflow of having multiple repositories, or one branch per repository, is IMHO as simple in Git as in Mercurial, and as in Bazaar-NG.

Well, understanding "git checkout ." doesn't require understanding inner workings of git. Your friend was incorrect here. I'll agree though that it is a bit of quirk in UI[1] (but I use usually "git reset --hard" to reset to last committed state).

[1] Having git-checkout behave very differently with and without pathname parameter, and overloading of git-checkout.

Just Google for "Worse is Better". But what I actually mean that Git feature set and UI has evolved from very bare-bones plumbing, adding features and UI _as needed_, instead of being designed according to what designer thought it was needed.

For example in http://gitster.livejournal.com/9970.html Junio C Hamano (git maintainer) writes:

  By the time the basic structure as we currently know has stabilized,
  we had help from literally dozens of contributors to add many things
  on top of the very original version:
  [...]
  
  * We did not envision that multiple branches in a single repository
    would turn out to be such a useful way to work, and did not have
    support for switching branches.
  [...]
  
  It still is amazing that all of these were done without having to go
  back to the drawing board.  It shows how sound the initial conceptual
  design was.

P.S. See "Innovations in git", http://gitster.livejournal.com/16077.html

For the record: transactions are applicable to all kinds of databases, not only relational ones.

  Leslie

Am Montag 27 Oktober 2008 10:41:53 schrieb Jakub Narebski:

The problem is by which knowledge the filesystem should optimize this call when it is storing the files in the first place.

It only matters, if some developers are forced to work on WIndows machines at times.

I just meant "databases are a bit off topic" :) 

That's what I wanted to express. 

Doesn't atomic write also need locks, though on a lower level (to ensure atomicity)?

So both do snapshots - they seem more and more similar to me :) 

I cloned from the official repositories. 

I hope Linus had his repository packed :) 

Damn - one more way how I could have archieved what I wanted... one more way I didn't find.

And that's how it feels to me. 

A great testing ground, but it developed too many stumbling blocks which keep me from trying things.

When I now use git, I only do the most basic operations: clone, pull, push, add, commit, checkout. When anything else arises, I check if it is worth the risk of having to read up for hours - and since that wasn't the case for the last few months, I then just ignore the problem or ask someone else if he can fix it.

As a contrast, when I encounter a problem with Mercurial, I simply check the commands for a moment and then try to solve it, and normally I have what I wanted within seconds to minutes.

Git instead bit me once too often. 

I know that this isn't something which hits everyone and that it is subjective, but since it hit me, I'm wary of git, because in my view it isn't something for the majority of people - and you almost always have someone from that "majority" in your project.

I don't want people getting afraid of solving their own problems, so I avoid systems which bite so often, that they create fear (for example of losing much time on something which should be a side issue).

All in all it's a UI issue - while the git UI bit me quite often, the Mercurial UI just works.

Best wishes, Arne

-- My stuff: http://draketo.de - stories, songs, poems, programs and stuff :) -- Infinite Hands: http://infinite-hands.draketo.de - singing a part of the history of free software. -- Ein Würfel System: http://1w6.org - einfach saubere (Rollenspiel-) Regeln.

-- PGP/GnuPG: http://draketo.de/inhalt/ich/pubkey.txt

On Mon, 27 Oct 2008, Arne Babenhauserheide wrote:

Well, that is a question for filesystem designer, and VFS designer...

What I want to emphasize that perhaps Mercurial is optimized for "streaming access", but fully packed Git repository requires only single (well, up to details) mmap, which I think is even better.

DragonFly BSD developers? I think they would work on DragonFly BSD (eating one's own dogfood and all that...).

Sidenote: I don't know if DragonFly BSD is more like Linux kernel, or
as Linux distribution. It would be in my opinion good idea to ask
similar projects about the impressions about SCM they use (Linux kernel,
Android, ALT Linux distribution, Debian (build tools etc.), CRUX Linux
distribution, Exherbo, grml, Source Mage GNU/Linux for impressions
on their Git usage; OpenSolaris, Conary, Heretix, Linux HA, perhaps
Mozilla for impressions on their Mercurial usage; 

IIRC ALSA moved from Mercurial to Git, so they could be of help there.

[...]

No, you can use create then rename to final place trick, making use
of the fact (assumption) that renames are atomic. And Git first write
data, then write references which are used to access this data (this
relies on pruning dangling objects).
 
I'm not saying that git does not use locks at all, because it does,
for example to edit config file, or update branch and its reflog as
atomic operation. But it needs locking in very few places.

There are differences: Mercurial from what I understand uses forward deltas (from older to never) while Git prefers recency order; delta chains in Git doesn't need to form single line, but can be forest of delta chains; Git searches for good delta basis from large range of objects (see pack.window); there is pack index which allow for random access as if objects were in loose format (resolving deltas behind the scenes).

I also don't know how Mercurial deals with binary files; in Git pack format uses binary delta from LibXDiff by Davide Libenzi (File Differential Library), heavy modified.

Well, that also depends on _when_ did you try this. In older versions of Git pack file got from network (git:// and ssh:// protocols) was exploded into loose objects; now is kept if it is large enough, only expanding it to make it thick, self contained pack file.

Unless you used http:// protocol, which I think kept packs as they were, and as dumb protocol (along ftp:// and rsync://) depends on remote repository being well packed.

Well, there is a difference between "git checkout ." and "git reset --hard", but it does not matter here.

By the way, the design of Git allowed to add lately new feature: "git checkout --merge <file>..." to recreate conflicted merge in specified paths. For example if you completely borked merge resolution, and want to start from scratch.

Well, as shown in "Worse is better", evolved design wins (Lisp machines versus Unix) :-)

Understanding Git "mental model" certainly helps.

[...]

But _that_ might be because you are used to Mercurial UI, isn't it?

On Mon, Oct 27, 2008 at 2:52 AM, Jakub Narebski <jnareb@gmail.com> wrote:

Git and Mercurial are very close from that point of view. Mercurial explicitely disallow octopus merges (and we don't think there's a good reason to allow them, although I agree with Linus, they look very nice in gitk ;) ). And we don't have "branches as pointer" in core, but the bookmark extension does that. Appart from that I think the underlying format are interchangeable, someone could use the git format with the hg ui, or use revlogs (the basic format of mercurial) like packs.

The only special thing about revlogs is the linkrev stuff, it's a pointer to the first revision that introduced an object, so we can easily find what to send in our network protocol (we don't have to read the manifest, ie the "tree" of objects"). linkrev can be useful to speedup "hg log" too.

just do a s/// with the git terminology: filelog -> blob manifest -> tree changelog -> commit object

regards,

Benoit

Dnia poniedziałek 27. października 2008 10:29, Benoit Boissinot napisał:

From what I see Mercurial disallows octopus merges (merges with more than two parents) because of its rigid-record database repository design, while Git is more like object database. Fixed width records of VMS vs delimited records of Unix... There is simply place on zero, one or two parents (two parent fields, which can be null) in Mercurial changerev format.

By the way flexibility of Git design allowed to add 'encoding' header to commit message (if commits message is encoded not in utf-8) after the fact, without affecting older repository data, and playing well with old git installations which do not understand this header.

I disagree. Mercurial implementation of tags is strange, and from what I remember and from discussion on #revctrl implementation of local named branches is also strange (CVS-like). They are IMHO not well designed.

Also the 'hidden' branches after fetching from remote repository (hg pull) but before merging (hg update) are IMHO worse design than explicit remote-tracking branches in Git, especially in presence of multiple [named] branches in repositories.

I don't think so. The 'content addressed filesystem' idea of Git is quite pervasive along Git implementation and Git thoughtflows.

At first I thought: what a nice idea... but then I realized that in distributed environment there is no way to define "first revision that introduced an object". Take for example the following history (independent introduction):

  .---.---.---.---x---.---.---.
           \
            --x---.---.

where both 'x' have the same version of an object. The top branch appeared first in current repository, but the bottom branch had 'x' with earlier timestamp (earlier authordate).

Git just relies on the fact that traversing revision is a part of it that is heavily optimized and really fast. Git very much by design doesn't store any backlinks in repository object database.

True. But as I see it they are bound in reverse order in Mercurial: deltas are stored in filelog, filelogs are bound together in manifest, manifest are bound using changelog, while in Git commit object references tree (and parents), trees references blobs, and blob store content of a file. But that might be just my impression.

.......................................................................

By the way, going back to the matter of choosing version control system
for DragonflyBSD; some time ago I have written post
 * "Mercurial's only true "plugin" extension: inotify... 
    and can it be done in Git?"
   http://thread.gmane.org/gmane.comp.version-control.git/76661
   (current answer: it is possible using 'assume unchanged' bit)
about how nearly every Mercurial extension has equivalent functionality
in Git. 

But what about the reverse, about the following features and issues in Mercurial:

 * Merging in presence of criss-cross merges[1], and in presence of
   file renames, i.e what merge-recursive does in Git.

 * git-rerere, reusing recorded resolution of conflicted merges.
   Resolving the same merge happens often if you use topic branches
   and trial merging.

 * git-grep that allows you to "and" the match criteria together,
   and also pick a file (not a line) that matches all the criteria;
   and of course allow searching given revision and not only working
   directory.

 * pickaxe search (git log -S) which contrary to blame/annotate
   allow to find commit which _deleted_ given fragment.

 * easy management of multiple repositories you fetch from with 
   remote-tracking branches and git-remote command.

 * blame that follows block-of-line movement (it was invented by Linus
   as a vision long time ago, but it took very long time to materialize).

 * a way to review merge resolution, something that is done in git
   by using combined diff format

 * git-stash, allowing to stash away changes to go back to them later;
   it allows to stash away even partially resolved merge conflict
   (merge resolution in progress).

 * git-filter-branch (based on cg-admin-rewrite-hist), which allow
   to rewrite history for example to remove file which should never
   be added to version control (for example because of copyright
   or license).

References:
===========
[1] http://revctrl.org/CrissCrossMerge
    BTW I wonder why reverting spam is made so hard on revctrl.org wiki
  

2008/10/27 Jakub Narebski <jnareb@gmail.com>

Jakub,

Do you know if git supports the equivalent of hg bundle? Thanks.

vk

On Mon, 27 Oct 2008, 0000 vk wrote:

[...]

The equivalent of "hg bundle" (http://www.selenic.com/mercurial/hg.1.html#bundle) would be "git bundle" (http://git.or.cz/man/git-bundle). I think git-bundle was inspired by Mercurial feature, just like fast-import format and bisect went in other direction.

P.S. Could you _please_ quote only relevant fragments of email you are replying to, especially if it is so long?

Jakub Narebski wrote:

I think you missed the most convincing argument _for_ explicit garbage collection.

By allowing explicit repository packing, git allows you to delay a cpu intensive operation til later, when time doesn't matter, like at the end of the day right before I go home. It also allows more cpu intensive delta/compression algorithms to be used.

By contrast, mercurial performs deltafication and compression on each commit. So, acceptable commit speed must be weighed against the complexity of the deltafication/compression algorithm and file format.

-brandon

On Mon, 27 Oct 2008, Brandon Casey wrote:

On the one hand one can use different compression for loose (immediate) and packed (in a free time) objects.

On the other access from "smart" client (git://, ssh://, future "smart" HTTP server) results in creating a pack, so we cannot allow for too tight pack compression, or to be more exact too much CPU load taken.

The ability to vary 'quality' of pack compression is very useful to distinguish between very loosely packed fast-import pack, and tightly repacked in free time.

On Mon, 27 Oct 2008, Jakub Narebski wrote:

Don't forget that, in those cases, the created pack for streaming is copying chunks of data from existing packs for most objects, effectively reusing for free the work that has previously been done to tightly compress those packs.

Nicolas

On Sun, Oct 26, 2008 at 4:15 PM, Jakub Narebski <jnareb@gmail.com> wrote:

<snip/>

The fact that hg is changeset based means that certain operations are slower, like checkout a specific commit. In hg my bet is you would need to gather a bunch of changesets while in git the operation is done in a single step.

It also means that bare clones are not possible in hg, or at least very complicated.

Note: I'm not sure if what I'm claiming is correct.

I'm not sure if Mercurial mailing list is not subscribe only. Git isn't.

On Sun, 26 Sep 2008, Felipe Contreras wrote:

Actually from what I have read Mercurial stores current version
(snapshot) from time to time, so time to resolve specific commit is
limited.  Also if you have packed your Git repository (good idea not
only to limit size, but also for performance (I/O performance)), then
resolving specific commit also might require some delta resolution
(by default delta chain length is limited to 50, see pack.depth).
 

I think it is things like .hgtags which make bare clones (without working directory) to be hard or even impossible in Mercurial.

Hmmm...

On Sun, Oct 26, 2008 at 9:07 PM, Jakub Narebski <jnareb@gmail.com> wrote:

Ah, ok, good to know.

Oops, I meant shallow clones (git clone --depth=1).

walt wrote:

The official vote was 19 to 19, plus one for perforce and one for svn. Matt has proposed a primary git repository and a mirror in hg, and that's being debated now.

I've already learned a lot from following this topic in both lists and it seems this is a topic of great interest to many, so I'll continue reading in both places.

Thanks!

Hi,

On Tue, 28 Oct 2008, walt wrote:

While many may say that that is a half-baked solution, I actually like it. Mercurial and Git are pretty similar in their concept (if not in how the data is actually stored).

Note that with git fast-export and hg fast-import, it should be relatively simple to convert from one data format to the other, even incrementally.

And for the other direction, you could use hg fast-export from the fast-export.git repository (I am working on a better one at the moment, too, so that incremental fast-export would be possible, too).

Ciao, Dscho

Johannes Schindelin:

That touches on something that I have been thinking about for a while.

How difficult are the storage formats? Would it be possible, in a reasonable amount of work, to add support for the Mercurial protocol and format in "git clone", so that I could clone a Mercurial repository and work on it with Git, and then possibly use "git push" to possibly push the result back to Mercurial?

It seems to me that use of DVCS is polarising between Git, Mercurial and Bzr. It would be nice to have easy interoperability between the systems, at least as far as can be covered by the lowest common denominator of what they support. I would love to be able to use Git to clone a Bzr repository that I need to be able to access, since bzr is just different enough from Git to be annoying. Same goes for Mercurial. And I am sure that users of the other tools feel the same.

Would it be possible to design a common transfer format that could be implemented by all three (and that would be a little smarter than fast-export/fast-import)?

Hi,

On Tue, 28 Oct 2008, Peter Krefting wrote:

There was talk about imitating Mercurial's wire protocol in order to have an efficient HTTP server. Shawn is working on that front;

We discussed it briefly, and there might be some cute ways to copy it: since we are not append-only, we have to download the pack index first (which is not downloaded ATM, as we generate it from the downloaded pack while verifying it). With that index, we can determine which parts we need in order to regenerate the pack; it would still be pretty stupid when there are a lot of branches and we are really only interested in one of them.

But I doubt that it will be possible to use the wire protocol to pull/push between different DVCSes. I _strongly_ doubt that the SHA-1s in the Mercurial repositories could _ever_ be reused in Git mirrors of them, as our data format (on which the hash depends) is different.

Wasn't bzr touting it as one of their major features that they could have foreign-scm remotes? If I remembered that correctly, that might be the route you want to take.

Ciao, Dscho

Johannes Schindelin <Johannes.Schindelin@gmx.de> writes:

Yes, see http://bazaar-vcs.org/BzrForeignBranches . That can be compared to git-svn for git, except that one uses the exact same command set to interact with the remotes (i.e. you "bzr push" to an svn repository, while you would "git svn dcommit" with git).

There are read-only implementations of Git and Mercurial foreign branches. AFAIK, unfortunately, they're more proof of concepts than real "production ready" plugins.

On Tue, 28 Oct 2008, Peter Krefting wrote:

The git protocol is intimately tied to its repository storage format, making any interoperability at the protocol level really hard. It is probably easier to perform the clone/push operations with native tools and do the interoperability dance locally between repositories, possibly with some wrappers hiding all the details. In the end you could still be doing a "git push" but the native tool is best for handling transfer protocols. Yes, there is git-cvsserver outperforming a real CVS server, but that's another story.

Nicolas

On 28 okt 2008, at 15:41, Peter Krefting wrote:

What would you want that the fast-export/imports are lacking? I think they are excellent tools to build some integration on.

You might want to look at my git-bzr script (http://github.com/pieter/git-bzr/tree/master ), which I created so I could use bazaar branches easily in git. It allows you to fetch a bzr branch, merge it in with your local work, and then push it out again:

	git bzr add bzr-branch ~/bazaar/something
	git bzr fetch bzr-branch
	git merge bzr/bzr-branch
	git bzr push bzr-branch

It's quite crude, but it's also only 100 lines or so and does what I need. It should also be simple enough to adapt it to also incorporate hg. When I needed the bzr integration, I looked into hg as well, but there wasn't a hg fast-import yet. If I understand dscho correctly, that exists now, so it should be easy enough to integrate that as well.

- Pieter

On Tue, Oct 28, 2008 at 04:33:54PM +0100, Pieter de Bie <pdebie@ai.rug.nl> wrote:

That's new to me. Theodore Ts'o once mentioned on this list that there is a "hg fast-export" but actually he just referred to "there is a git2hg conversion tool in hg's contrib dir" and it has nothing with fast-import.

There is an other reference to hg fast-import:

http://www.nabble.com/cvs2git-2.1-causes-git-fast-import-to-exit-with-an-error-td16049922.html

but I found no code so far.

To sum up, I'm not so sure about a working hg fast-import is available at the moment.

On Tue, Oct 28, 2008 at 08:12:34PM +0100, Miklos Vajna <vmiklos@frugalware.org> wrote:

I wrote too fast.

There is a minimal implementation here:

http://hg.opensource.lshift.net/hg-fastimport/

(I haven't tried it yet myself.)

On Tue, Oct 28, 2008 at 08:12:34PM +0100, Miklos Vajna wrote:

The code I was referring to was called hg-fast-export, which is part of the "fast export" tools that front-end into git fast-import. The git repository can be found here:

	http://repo.or.cz/w/fast-export.git
	git://repo.or.cz/fast-export.git

I ended up using a very customized version of that script to convert the hg e2fsprogs repository to git.

In the past I've looked at the possibility of creating a bi-directional, incremental gateway between hg and git repositories. The main thing which makes this difficult is that hg stores tags in-band inside the change-controlled .hgtags file. This means that if you cut a release, tag it, and then create a commit to further modify the repository, the new commit is descended from the tag commit, whereas in git, the tag is a "bookmark" --- perhaps signed via GPG, but not part of the revision history.

I think the git method is much more sane, but what it means is that topologically, the commit tree for git and hg can never be identical. It also means that if you add a tag to a git tree after making several commits on that branch, how you reflect that in the hg repository is highly problematic. Do you rewrite the branch? Do you add the tag later on, disturbing the parent-child relationship of later commits? How do you keep track of when a tag hg repository topology if you are trying to maintain a bidirectional mapping between commits?

It's not impossible, but it makes it much more difficult, since in the hg world, tag commits can be inserted between arbitrary commits. This also means that if you want to create a bidrectional gateway between hg and git, it has to be a single gateway so it can keep track of this state information. If you try to have multiple gateways they would need to synchronize on when a tag entered the hg universe, and with what commit ID (and what timestamp).

						- Ted

On Tue, Oct 28, 2008 at 05:31:44PM -0400, Theodore Tso <tytso@mit.edu> wrote:

My bad, I did not quote the article properly, so here is what I meant:

http://article.gmane.org/gmane.comp.version-control.git/42298

and I just wanted to say that this one does not use fast-import, so it's not really a "hg-fast-import" (it's not something that can parse the output of git-fast-export).

* Theodore Tso:

Couldn't you just keep the .hgtags file and have everyone interested in the tags use special scripts?

(Admittedly, I'm horribly totally by Git's behavior in this area. I haven't figured out yet under what circumstances tags are pushed and pulled, so I'm not totally opposed to the Mercurial model. 8-/)

On Sat, Nov 1, 2008 at 9:06 AM, Florian Weimer <fw@deneb.enyo.de> wrote:

Tag are pushed when you tell it so, and pulled if they are pointing to the history you have, or you can ask to fetch them explicitly.

HTH, Santi

Florian Weimer <fw@deneb.enyo.de> writes:

I think you don't understand the issue here.

First, I think that we all agree here that by definition tags should named reference (for example 'v1.0' or '1.0') to some immutable snapshot of a state of repository, so for example when somebody says 'v1.0' everybody knows what it is. In Git tags are immutable (you can't checkout a tag, you can only checkout state pointed by tag) external pointers to commits in the DAG (graph) of revisions.

Global tags (tags used to mark releases like 'v1.0') have to _not versioned_ and _transferable_. Transferable (global) because we want to know for example what 'v1.0' version was in each clone / each repository. Non-versioned because we want to have the same set of tags independent on what branch we are (when we are on 'master', we want to be able to know about 'v1.0.1' which is on 'maint'), and what revision we have checked out (for example during bisection, we want to be able to compare to 'v1.0' even if we have checked out revision which is earlier than 'v1.0').

Do you agree that global tags should be both non-versioned and trasferable?

Now Mercurial has chosen to use in-tree '.hgtags' file to have global tags transferable. Never mind the fact that it had to treat this file in special way to have it non-versioned (as opposed to for example .*ignore file, which should be both transferable and versioned); the fact that in-tree file is used means that tag is visible to outside (transferable) only after you commit changes in .hgtags file.

In Git tags are external to object database; they reside in refs/tags/* namespace. They are of course non-versioned, as not being in-tree. In default configuration however (from what I understand) if you transfer (get) some tagged commit, you also get a tag that points to transferred commit. You don't need to create "PROJECT 1.0" (or "Tagged v1.0") commit to make tag visible to outside.

In short, if you want to have bi-directional gateway between Mercurial
and Git, Git has to be limited:
 * no octopus merges (with more than two parents)
 * always create 'tagging commits' (bump version number for example)
   for tagging purposes on Mercurial side.

* Jakub Narebski:

Probably yes.

Yes, I do. In case of Git, I've got trouble with understanding how to actually implement the "transferable" part with Git. The Mercurial way is easier to understand, but it means that tags may need some sort of "tag at this revision" qualifier to disambiguate, which is rather problematic.

Oops, thought this file was versioned.  Things like

  <http://tycoon.hpl.hp.com/changeset/932:931d181e9f58/.hgtags>

suggest it was at some point.

* Florian Weimer:

Okay, checked it--.hgtags files are in fact versioned. So my suggestion to preserve the .hgtags file on the Git side remains.

On Sat, 1 Nov 2008, Florian Weimer wrote:

In first page of git-fetch(1):

     When <refspec> stores the fetched result in  tracking  branches,  the  tags
     that  point  at  these branches are automatically followed. This is done by
     first fetching from the remote using  the  given  <refspec>s,  and  if  the
     repository has objects that are pointed by remote tags that it does not yet
     have, then fetch those missing tags. If the other end has tags  that  point
     at branches you are not interested in, you will not get them.

You push tags explicitly (using "tag <tagname>" refspec, or specifying
--tags, --all or --mirror, or setting it up in a config) in git.
 

It is half-versioned... :-X From what I remember of discussion on #revctrl and reading hgbok it is treated in special way: if you switch branches or rewind to some older version, .hgtags is always checked out the most recent version.

I don't know how Mercurial deals with situation where one reposityr added one tag, and other repository add other...

On Sat, Nov 01, 2008 at 11:44:21AM +0100, Florian Weimer wrote:

.hgtags is stored as a versioned file in Mercurial. That's one of the problems, and it leads to no shortage of headaches.

Some of the problems are discussed here:

   http://www.selenic.com/mercurial/wiki/index.cgi/Tag

Specifically, to quote the Mercurial Wiki:

    The fact that tags identify changesets and are also parts of
    changesets has some potentially confusing implications:

    * The changeset that a tag refers to is always older than the
      changeset that commits the tag itself.

    * Updating a working dir to a particular tag will take that
      directory back to a point before the tag itself existed.

    * Cloning a repo to a particular tag will give you a new repo that
      does not have that tag.

In addition, Mercurial has to play interesting special case games in a repository which has multiple heads (in git terms, "branches"). When looking up a tag, it has to take the union of all of the .hgtags files at the tips of each of the branches. So if you have a large number of heads in your Mercurial repository, tags access doesn't scale well at all. Mercurial is very much optimized for having a single or at best a few heads/branches in a repository.

As I mentioned earlier this makes it much more difficult to do a bidrectional hg/git gateway, since the two DAG's are not toplogically equivalent, and in fact, *when* you add a tag to a particular commit/revision can make a distinct difference to the shape of the DAG. If you tag right after creating the revision in question, then there is a tag commit right after the revision. If you commit a few dozen changes into the repository, and *then* tag a point many commits earlier, then the tag commit will be tacked onto the head of whatever branch you happened to be on.

In fact, in the worst case, if you accidentally tag a revision on the "maint" branch while you happened to be on the "devel" branch, the tag for the commit in the "maint" branch will be attached to the "devel" branch, and while it will work just fine for *you*, someone who only fetches the "maint" branch might never see the tag that you placed on the maint branch --- unless they happen to also pull the devel branch.

What fun, eh?

					- Ted

On Sat, 1 Nov 2008, Theodore Tso wrote:

I told people this was insane long long ago, and I thought the hg people had learnt to use local tags. They act sanely, as far as I know (ie they act the same way git tags do).

Of course, the problem with hg local tags is that hg apparently has no sane way to _propagate_ such local tag-space information from one repository to another. But that's purely a problem with hg itself. I don't know why that hasn't gotten fixed.

			Linus

On Sat, Nov 01, 2008 at 10:51:51AM -0700, Linus Torvalds wrote:

Yeah, well, hg calls them _local_ tags, and so people consider that by design, they aren't supposed to be propagated outside of the local repository. As I recall, hg doesn't support GPG signing local tags, for the same reason.

							- Ted

Hi!

Pieter de Bie:

Speed. I am cloning from a rather overloaded server on the other side of the globe (the US). If I were to go the fast-export/import route, I fear it would take even longer.

Looks very interesting. *adding to my things-to-examine list*

Johannes Schindelin <Johannes.Schindelin@gmx.de> wrote:

FWIW at the Google Summer of Code Mentor Summit this past weekend we had a "Git vs. Hg" talk with both Git and Hg represented by contributors to each project.

Slides are online here:

  http://docs.google.com/Presentation?id=dcfz2dg9_0hqqz3dsr

On Wed, 29 Oct 2008, Shawn O. Pearce wrote:

But how do I save the presentation? I do not seem to be able to do it. I would like to view it off-line.

Thanks,

-- Boyd Gerber <gerberb@zenez.com> ZENEZ 1042 East Fort Union #135, Midvale Utah 84047

Hi,

On Wed, 29 Oct 2008, Boyd Lynn Gerber wrote:

Just an idea: print to PDF? There is a link "print slides" on the lower right.

Ciao, Dscho

On Wed, 29 Oct 2008, Johannes Schindelin wrote:

That worked. I had to make my window full screen to see the option. It did not show-up in my normal window.

Thanks,

-- Boyd Gerber <gerberb@zenez.com> ZENEZ 1042 East Fort Union #135, Midvale Utah 84047

walt <w41ter@gmail.com> writes:

Boy, whoever has the popcorn concession must be making a fortune!

-Miles

Emanuele Aina schrieb:

Yes, it's the same: a DAG with hashes. But there are limitations due to the implementation (and not the design). Just as a bad and completely useless example (don't start to argue, I know it's nothing someone would like to have): you cannot force mercurial to merge two revisions and create a merge commit if one is the others ancestor,which is possible in git with git --no-ff. In addition they differ in some other ways: Mercurial doesn't have an index to stage commits, which is something that git has and allows very powerful features (such as git add -i, etc).

On Mon, 27 Oct 2008, Arne Babenhauserheide wrote:

I think Mercurial UI is more compatibile with Subversion UI than Git; additionally Mercurial uses separate, different names to avoid ambiguities: examples are 'hg rollback' and 'hg backout' for commands which various SCM name reset and revert, sometimes referring to one and sometimes to the other.

AFAIK Git mostly follows BitKeeper UI; it is quite natural as it was meant as replacement for BitKeeper for Linux kernel development.

[...]

There are advantages and disadvantages to each method.

Git was developed in true Unix tools style, as described in TAOUP i.e. "The Art of UNIX Programming" by ESR (http://catb.org/esr/writings/taoup/html/) In "Programming Pearls" by Jon Bentley there is in one of chapters description of prototyping 'spell' using ready UNIX tools, pipelines, and a few of custom tools, to examine the needs of 'spell'.

One of the consequences of this type of design is that there isn't (yet?) something like git library; this is caused by the fact that tools (plumbing) that make up git are designed to run once, and let operating system take care of freeing resources.

On the other hand clean and clear design of git repository made it possible to create native Git (re)implementation in Java: the JGit project. (Well, Git is GPLv2 and JGit/Egit is BSD 3-clause/EPL, so reimplementation might have been needed for licensing purposes anyway. Well, it is not _full_ implementation yet...

Besides is writing plugin in Python for Mercurial that much easier than writing new command or stuff in C for Git? Well, perhaps it is, as only recently there began to appear API documentation, and there were created utility mini-libraries like strbuf, or string_list, or parseopt.

Also often third-party projects or stuff in contrib gets incorporated into git proper; sometimes stuff is moved from git core to contrib, if there is no maintainer however (git-svnimport, git-p4import). Git repository has many roots: one from git core, one with gitk (graphical history viewer), one from what was git mail tools, one with gitweb (web interface), and one with git-gui (graphical commit tool).

The extending via plugins idea used by Mercurial, so succesfull IMHO for Firefox, and I think quite successfull for ikiwiki for example, is not without drawbacks, especially that there is no plugins clearinghouse, and plugins are required for what is considered pretty required functionality, like bisect before hg 1.0.

See also blog post on vcscompare: "Plugins In Version Control" http://vcscompare.blogspot.com/2008/06/plugins-in-version-cotnrol.html

Am Montag 27 Oktober 2008 19:01:48 schrieb Jakub Narebski:

It is a great choice for an SCM intended for kernel development where everyone knows bitkeeper, but might not be optimal for other tasks.

Yes, for two main reasons: 

1) Writing Python is much easier and quicker than writing C, especially when 
you can just experiment with the Python interpreter (or better still: with 
ipython). No memory management hassle, no strange compiler bugs, no stray 
pointers. Just plain writing what you want to do. But if you need C speed, you 
can still include parts written in C - where you really need it. For all other 
cases you have more readable and far more compact code. 

2) You can easily access every core function, and you can replace every 
command. 
You don't have to invent a "git foolog" command. Instead you can just adapt 
the regular log to do a foolog which people can use via "hg log". 

Maybe I should include the extensions in the codeswarm to have matching repositories.

But they can just be included in the core distibution once they become central enough.

It's a way of allowing people to add functionality they miss without forcing them to mess with core code instantly.

I just read it, and it didn't convince me, because in my opinion a VCS has certain core functions which it should provide to everyone, and other functionality which only certain people need.

For example I normally don't need rebasing, mercurial queues, transplanting and similar. Why should my tool include the commands by default?

The defaults should be the most common way to use the tool, so people can easily learn it.

Advanced stuff can be added with extensions. 

And because the most used plugins are distributed with Mercurial, I can also activate them when I don't control the Mercurial installation - and should something be missing which I need, I can just download and activate a plugin without having to compile anything, since they are simply Python modules.

Just set [extensions] foo = /blah/foo.py

in ~/.hgrc and the foo plugin is active. 

Gits missing plugin system might just be a reason, why its usability still suffers from many problems: They have to do everything for everyone all the time, so the chances are high, that they won't do it really good for anyone (but the main git coders).

Best wishes, Arne

-- My stuff: http://draketo.de - stories, songs, poems, programs and stuff :) -- Infinite Hands: http://infinite-hands.draketo.de - singing a part of the history of free software. -- Ein Würfel System: http://1w6.org - einfach saubere (Rollenspiel-) Regeln.

-- PGP/GnuPG: http://draketo.de/inhalt/ich/pubkey.txt

On Mon, Oct 27, 2008 at 09:48:50PM +0100, Arne Babenhauserheide <arne_bab@web.de> wrote:

You compare Python to C here, but did you realize that in git you can write your git command in any language you want? Of course it's recommended to do it in C/shell/perl if you want to get it included in git.git, but that's just a decision.

IIRC the main reason git aliases can't overwrite git commands is because that would break scripts relying on the output of existing git commands. Given that I install such an extension, won't my script break?

From a user's point of view, I think external git commands and such hg plugins are equal. The user instally the "foo" extension/command/plugin/whatever and gets the "git/hg foo" command.

Same for git, as long as it's written in a scripting language; you should include git-foo in PATH then you can use git foo.

Am Montag 27 Oktober 2008 22:07:16 schrieb Miklos Vajna:

I was asked explicitely about the difference of writing a Mercurial extension and of writing some addition for git in C, so I answered that.

Since that "script" will likely be an extension which will use the core function instead of the UI command, it won't break.

Stuff which does command line parsing can naturally break when I change the output. But it can also directly use the advanced features.

[snip]

Means git can provide additional commands and only has the limitation that I can't overwrite the basic commands, right?

But we're slowly moving off topic, aside from "OK, git also has extensions - they are called external commands".

Best wishes, Arne

-- My stuff: http://draketo.de - stories, songs, poems, programs and stuff :) -- Infinite Hands: http://infinite-hands.draketo.de - singing a part of the history of free software. -- Ein Würfel System: http://1w6.org - einfach saubere (Rollenspiel-) Regeln.

-- PGP/GnuPG: http://draketo.de/inhalt/ich/pubkey.txt

On Mon, Oct 27, 2008 at 10:30:44PM +0100, Arne Babenhauserheide <arne_bab@web.de> wrote:

Yes. And in general the API is the output of the plumbing commands, not the API of libgit which is totally unstable.

I think we're offtopic since the dragonfly list is not in cc. :)

Arne Babenhauserheide wrote:

But then you're back with a single language, taking valuable freedom away from the addon author. How many perl gurus have skipped writing stuff for hg because it's a "python-or-bust" thing?

And please don't give me that rubbish of "but Python is obviously better than C". Which one's true (if any) depends only on how you define "better".

Am Dienstag 28 Oktober 2008 18:48:28 schrieb Andreas Ericsson:

Not really. 

Extension authors just have to take care to keep their output compatible. 

You can do command line parsing just like with git, but additionally you can change the workings of the basic commands, but then you have to take care to keep the output compatible.

For example when I wrote the group extension, I made sure that the log only gives grouped output, when it is explicitely asked to do so, either via --group or via the grouped_log=True setting in .hgrc.

How many Python people decided to write an extension for hg, because it can very nicely be accessed via Python?

(and which one of these has the higher effect? :) )

Best wishes, Arne

-- My stuff: http://draketo.de - stories, songs, poems, programs and stuff :) -- Infinite Hands: http://infinite-hands.draketo.de - singing a part of the history of free software. -- Ein Würfel System: http://1w6.org - einfach saubere (Rollenspiel-) Regeln.

-- PGP/GnuPG: http://draketo.de/inhalt/ich/pubkey.txt

On Tue, Oct 28, 2008 at 08:11:36PM +0100, Arne Babenhauserheide wrote:

And wouldn't they contribute at all, if there would be hg bindings for other programming languages, would they?

Best, Gábor

Are really programming languages that important? If I find a tool which is useful and has features I need, and while using it I find some drawback, and I am able to find how could I work on it, I can pick up enough of the language it is written in. I am not a "guru" if I can't.

Main showstoppers in this area are not the programming languages, but lack of documentation, miserable APIs, or lack of process. Or ... the fact that the tool is so good that it does not really need extensions.

On Thu, Nov 6, 2008 at 5:25 PM, Marcin Kasperski <Marcin.Kasperski@softax.com.pl> wrote:

Considering the success of some of them, having the worst possible syntax and semantics, I believe it does not matter much.

I couldn't agree more. Sorry for such a useless reply, but I felt some emotion reading "common sense".

Cheers.

Andreas> And please don't give me that rubbish of "but Python is obviously Andreas> better than C". Which one's true (if any) depends only on how you Andreas> define "better".

But any way you define it, Perl is "better" than either of those!

:-)

Dnia poniedziałek 27. października 2008 21:48, Arne Babenhauserheide napisała:

In Git you can write in C, using (underdocumented) git API, or you can
script (in shell script, in Perl, in Python, in Ruby) using git plumbing
commands which are meant for scripting (or bindings in appropriate
language).
 
Most new features, like git-remote tool to manage interaction with
multiple remote repositories, each of which can have multiple branches,
start as a shell or Perl script, and when they and their UI mature they
get converted into C (made into builtin). Builtinification is done not
only for performance, but also for portability (think Perl support on
MS Windows).

So in Mercurial you can write in Python, or you can write in C; in Git you can write in any scripting language (e.g. shell script, Perl, Tcl/Tk), or you can write in C... yes, I know it is oversimplification...

Well, if I remember correctly if you drop git-foo in EXEC_PATH, then you would be able to call it as "git foo". So adding commands is easy.

Git provides a few entry points which can be used to extend functionality. They are: hooks system; gitattributes to define custom merge, diff and clean/smudge (checkout) drivers per file (pathname); custom merge strategies; EXTRENAL_DIFF and EXTERNAL_GREP.

I'm not sure if other messing with core functions is a good idea to have such ability accessible.

Having some extensions blessed to be included with core program (like
ikiwiki with goodstuff, and similar to Git with contrib/ section)
solves some problems of relying on extensions for basic functionality.
I for example consider bisect and patch+mail workflow tools to be basic,
while patch queue management (well, patch management in general) to be
something that can be built on top of SCM, like StGit, Guilt, TopGit
for Git, or mq (Mercurial Queues) for Mercurial.
 

The problem with extensions IMVVVHO is that they don't require to follow strict "inclusion in core" standards, which means that there is no pressure to add them to core... which for example leads to including bisect in core only since hg v1.0, "because it is available as extension".

Requiring to use large amount of extensions to having required
functionality is also one of bad consequences of extension based
development, although having default set of extensions that can be
turned on via some metaextension / metapackage (like ikiwiki's
goodstuff) reduces impact of this.
 
Extensions / modules / plugins are good in projects with high
barriers of development, like Mozilla / Firefox, GNU Emacs, etc.
but I am not sure if it doesn't make for slower core development...

Well, Git doesn't have plugin system, but is easily scriptable...

And, at least according to annual Git User's Survey results (on git wiki) many people create their custom scripts and scriplets to make their work with SCM easy...