Windows support

How serious are you guys about Windows support? I'm talking fully-functional port, not Cygwin. I did a lot of searching for a new SCM to switch to (from Perforce). And Git is my #1 choice. I love it's internals design and it's expressive power. I've also tested git-p4 and it has worked like a charm with my depot (with few tweaks that I may contribute later). But I do all my work on Windows so I need Git-For-Windows-Done-Right :-). The current mingw port is not there yet.

Transition to the new SCM must happen now, so basically I have 2 choices:
1. Survive for a few months with the current CygWin port of Git
knowing that Windows support is coming
2. Use another SCM (#2 is Mercurial, #3 is Monotone)

I'd realy love to do #1, but I need to know how long do I have to wait.

- Dmitry

Hi,

On Wed, 25 Jul 2007, Dmitry Kakurin wrote:

Okay, let's talk business:

Pay me decently, and you will have to wait for a few weeks.

Ciao, Dscho

Johannes Schindelin wrote:

I propose that you set up a fundable:

http://fundable.com/

This is a system where anyone can contribute money to the project, but not have to pay unless the required amount of money has been contributed in total.

Figure out your price, describe what you will deliver, and announce it.

Regards, Asger Ottar Alstrup

Hi,

On Thu, 2 Aug 2007, Asger Ottar Alstrup wrote:

I am not that much interested in money, really. But I do want to get something back in return for my efforts. And no, this does not include whining and complaints.

It includes useful bug reports. It includes a willingness to keep working with me until the bugs are fleshed out. It possibly includes making (and maintaining!) an installer.

At the moment, I am happy to say that Git works for me. Even on Windows. I have no problems with the command line, and both Cygwin and MinGW Git do their job reliably and joyfully here.

But there might come a time when I get into a position much like Shawn, when I have to work with Aunt and Uncle Tillies, who are not as clueful and intelligent^W^W^Wused to the command line as I am.

So I want to exploit Open Source, meaning that I give _you_ something, and _you_ give me something back. And that might very well be just a suggestion "make the commit button stick out; I did not find it, since there are so many buttons". Or a nice comic "how to use git". Also a beer is appreciated. Or whatever.

My complaint "let's talk business" was some (frustrated) way to get the attention of people who do _not_ give something back, and are astonished that just complaining will not get them anywhere.

FWIW I just applied for the project "Git on MSys" on SourceForge. Let's see how this will work out.

Oh, and I will _not_ do such a thing as think about what you might want, and then proclaim what would be my price for it. You _know_ what you want, so why don't you just tell me, as a detailed list?

Ciao, Dscho

Dmitry Kakurin wrote:

Well, it's really a matter of someone stepping up and doing the work. Much (nearly all?) of the core git team never touches Windows, so they both have no selfish motivation to get it working well and no way to test their changes even if they decide to take it up for the greater good.

As has been pointed out, there are a lot of people coming to the list and asking for Windows support, but precious few actually contributing any code. If everyone who asked for Windows support had been willing to fix one Windows-related issue, git's Windows support would be stellar by now. I'm as guilty as anyone of asking for stuff without doing it myself, so I say this as an observation, not an accusation!

There is a port that uses MinGW instead of Cygwin, FYI. It is still perhaps not as native-Windows-like as one might prefer, but it should be less alien than Cygwin, anyway.

-Steve

On 7/25/07, Steven Grimm <koreth@midwinter.com> wrote:

This actually answers my question (if it's true). If core team is not interested in supporting Windows then I cannot trust this system with my source code :-(.

My concerns are (mostly):
* lack of (or insufficient) testing for Windows platform
* possibly lower code quality of Windows port, since core devs don't
touch it and don't care
* possible troubles with support if issues arise
* Windows port could become abandoned if those few brave people, who
work on it right now will leave

In short, all kinds of issues associated with software not being a first class citizen :-).

- Dmitry

Dmitry Kakurin <dmitry.kakurin@gmail.com> wrote:

It more or less is. Those of us that are most active as Git developers don't really use Windows as our core development platform. Well, that is not entirely true. Day-job forces Windows on me, because its the Most Secure Operating System Evar!. :-) I run Cygwin there so I have a sane user interface, and build Git under Cygwin rather than MSYS because I just expect the UNIX-like environment that Cygwin gives me.

Why Cygwin? Because I have to use Windows, but I'd rather use Linux. No, Linux isn't permitted. And Solaris/x86 is only allowed on "servers". I have yet to find a way to classify my desktop as a server. :-|

git-gui is fairly well supported under Cygwin, as I use it a lot in my day-job. As do a lot of my coworkers. Which actually gives me a pretty good testing ground; ~20 people all beating on git-gui all day long is a pretty sizable testing group. I actually wonder some days if git-gui is better tested on Cygwin than it is on Linux.

But as has been stated on this thread, Cygwin isn't native Windows.
 

We do care. Its just not our primary focus. Dscho, Junio, Daniel Barkalow, Johannes Sixt, myself, even Linus have all contributed patches to git that help make it run better on Windows, or make it easier to port there. But none of us are running out and dedicating our lives to making Git the best software to ever run on that platform. There's other things more important to us.

That's always a concern. Heck, day-job invested untold fortunes in a product we purchased from a large commerical vendor. Runs only on Windows. Vendor just up and decided to no longer support the product anymore and has left us hanging out to dry. Did I mention that the product is also closed source and less stable than Git is on Windows?

So no matter what you use, if the developers leave, you are stuck. But one thing I *really* love about Git is how simple the data structures are and how easy it is to read the repository. Its under 500 lines of C code to unpack a working directory. More if you want something that's blazing fast and always reliable, but if you just want to get the data out its quite simple.

Its also fully open source.  GPL'd even.  So there's never the
issue that your vendor runs away and prevents you from taking on
development yourself, or just fixing those minor issues that you
really need to have fixed.
 

On Jul 26, 2007, at 5:15 AM, Shawn O. Pearce wrote:

So apparently you're working in a reasonably sized group of people all testing git on cygwin. I'd be completely satisfied if git ran rock solid on cygwin.

I found the following list of warnings about cygwin in the wiki entry WindowsInstall [1]. Some points look quite scary to me.

What is your real-world experience? Are the warning still valid? Must I really fear to break cygwin if I press Ctrl-C?

Do I really need to reboot regularly? I don't think this is an option. Nowadays our Windows boxes run for months, too. I can't seriously tell people that they need to regularly reboot if they want to use git.

Here's the list, copied from http://git.or.cz/gitwiki/WindowsInstall

    * Use git on local NTFS disks -- Network drives disks don't  
support the filesystem semantics GIT needs; for interoperability  
purposes you can store bare repositories on FAT32 disks.
    * Be careful with the case in filenames. Similarly, avoid special  
chars in filenames.
    * Run git gc early and often. There are slowdowns with many  
unpacked objects. Be careful to not create very big packfiles (bigger  
than 2 Gb).
    * Avoid using ActiveState Perl if possible. Ask in the  
MailingLists if you must.
    * Try to avoid interrupting (Ctrl-C) processes - it breaks cygwin.
    * Consider setting core.fileMode to false (git repo-config  
core.fileMode false) if file modes are frequently the only  
differences detected by Git. Many Windows applications make the  
execute bit be set in Cygwin when they save a file. Besides Cygwin  
detects file mode by stupid combination of content analysis, file  
name extension and moon phase.
    * Insert "set CYGWIN=tty binmode" after the first line of C: 
\cygwin\cygwin.bat, so you can use Ctrl-z in cygwin's bash to suspend  
a program.
    * Windows usually writes end-of-line as CRLF, while Unix/POSIX  
writes LF. This can cause a variety of problems. There are current  
efforts to address this.
    * Setup binary mode for cygwin (there is an option in cygwin's  
setup program), otherwise Cygwin mangles everything read and written  
(Git repos have binary files in control structures).
    * Avoid big repos.
    * Avoid big blobs (very big files. Basically anything larger than  
10Mb is too big).
    * Avoid big trees (directories with many files in them).
    * Avoid deep hierarchies.
    * Reboot regularly (memory fragmentation)
    * Defragment often (filesystems fragmentation)

	Steffen

Steffen Prohaska <prohaska@zib.de> wrote:

It is just as rock solid as it is on Mac OS X (my real Git development system) and Solaris. If your Cygwin DLL does not support pread() you do need to compile with NO_PREAD=1, but that's a minor issue. Personally I build Git on Cygwin with pread() and mmap() enabled and it runs fine. Of course we only use it on local drives, and only on NTFS drives.

As has been stated already, Git's checksums work nicely to make
sure data hasn't been corrupted.  I've seen one user have trouble
with checksums failing in his repository.  Usually he recopies it
from another user and picks up where he left off.  Twice I've seen
his packfile corrupted and Git caught the corruption.  We seriously
suspect some bad blocks on his drive.  But budget says we cannot
replace the disk for another 4 years.
 

Its not that Cygwin breaks. Its that sometimes pressing Ctrl-C doesn't actually stop the process, e.g. the signal isn't sent or just gets ignored. So its annoying because you can't abort things as readily as you might on a good UNIX. Sometimes you get weird stack traces from a Git process when you Ctrl-C it. But I also see this same garbage out of a native Windows JVM when I Ctrl-C it from a Cygwin bash. Its just general Cygwin-ism or something.

Despite those failures I've never seen that stack dump actually
corrupt Git data.  Think about it.  Git needs to be safe on any
platform, even if the running Git program terminates unexpectedly
in the middle of an operation, such as because of an OOM from the
kernel, or an angry admin `kill -9`ing it.  So this little stack
spew is just annoying more than anything.
 

I *never* reboot my Windows system at day-job. Except when our local adminstration staff shoves some Microsoft uber-patch down onto our systems and that patch forces us to reboot the machine. So I never reboot for Cygwin (or Git's) sake. Ever.

Still true. Network drives have some issues as the SMB protocol doesn't support everything nicely. NTFS locally is fine. FAT32 has issues with mmap() not being well supported. If you must use FAT32 compile Git with NO_MMAP. Which is the default on Cygwin, as a lot of people still use FAT32.

This is true. Git doesn't like getting file names with case only differences on such a platform. E.g. just today I wanted to do the following:

  git mv foo.c Foo.c

but had to instead do:

  git mv foo.c CRAP && git mv CRAP Foo.c

because the former won't work on a filesystem that ignores case. I have the same problem on my Mac OS X HFS+ volume as it also ignores case.

Both of these are still true. git-gui on Windows suggests a repack if you have ~256 loose objects, on UNIX platforms it suggest a repack at ~2048 loose objects. The problem is really just a performance issue, the more files we have to open to access data the slower things go. The loose objects tend to be the really recent stuff (that's why they aren't packed yet) and the really recent stuff tends to be what is accessed most.

Opening 200 files takes time on Windows. Its just a limitation of the OS apparently. And its a fundamental property of the Git object store that we always write to loose objects first, as its fast and easy to make safe.

Another aside to this is `git grep --cached` or `git grep ... TREE` is *always* faster for me then grepping the working directory. The first two will return nearly instantly (tiny lag) while the working directory grep will take days. On Linux and Mac OS X the exact reverse is true, the working directory grep is usually faster if the disk cache is hot.

Yea, we've had some issues with that. This comes from one particular user (Alex Riesen) who uses Cygwin but for strange reasons is not allowed to use the Cygwin perl and instead must only use the ActiveState Perl. We've had some issues in the past with our Perl scripts running on that perl port. Alex has fixed many of them, but some may still be lurking.

Already talked about above.

We currently default core.fileMode to false on Cygwin, for this very reason. We used to not do that. We got smarter and realized that although Cygwin itself (and all Cygwin tools) will properly handle the executable bit on NTFS native Windows tools (e.g. Eclipse) won't. Users use the native Windows tools, then blame Git. So we disable it.

Oooooh. I did not know this tip. I still just cuss at Cygwin anytime I want to suspend a job and cannot.

See the crlf feature in gitattributes. You can now have Git create working tree files in CRLF format, but check them into the object database with only LF.

I think binary mode is the default now on Cygwin. It used to not be. Because of this problem.

Yea, sort of. I'm using about 180M (fully packed as best as I can make Git do) and its fine. I don't know what a definition of "big" is.

This I can't speak to. All of my blobs are source code, so they are small-ish.

Probably true. Most of my trees are reasonably well distributed (they aren't that big). I think my largest is 900 files in the same directory.

I/use/java/programs/on/windows/and/much/of/my/source/is/in/that/format. :-) I don't really have issues with deep trees, and I have some pretty darn deep source code trees.

Don't see that.

Yes! Very much so. The packfiles are the first things to fragment, and what with all of the small files that Git creates, especially with frequent branch switching, and then the small object files that my build system creates, my drive is almost always completely fragmented.

Which reminds me, I need to defrag again...

You can turn off case-insensitivity in the Windows kernel, by using RegEdit, and setting the following registry key to 0:

HKLM\SYSTEM\CurrentControlSet\Control\Session Manager\kernel\obcaseinsensitive

I haven't tried it, but it should help your case above. Just keep in mind that you can then check in files which your coworkers can't checkout :-)

PS: You'll have to reboot for it to take effect, and don't blame _me_ 
if it doesn't reboot successfully! ;-)

Wrote this reply privately earlier; forwarding to the list at Dmitry's suggestion (though it's rendered slightly less relevant by his clarifications)...

Dmitry Kakurin wrote:

I certainly understand the conclusion, but I'm not sure I would share it. Unless you have reason to believe there's something in particular about the Windows environment that would cause git to lose data in circumstances where it wouldn't do so under UNIX-ish systems, it seems like your data should be perfectly safe.

In the year-and-a-bit I've been lurking on the git mailing list and making occasional contributions to the code, git has never lost any data for anyone to my knowledge. Its design is extremely paranoid in that regard, and the paranoia is not really anything platform-dependent. It's stuff like, never overwrite files in place (always write a new file then, once it's written successfully, get rid of the old one if needed). Or, as importantly, keep SHA1 hashes of *everything* and double-check them often. Those approaches are just as valid on Windows as on any other OS. The SHA1 hashes in particular are pretty unimpeachable, IMO; the times people have thought their git repositories have gotten corrupted, it has always turned out to be underlying filesystem or disk corruption that git's SHA1 checking has caught.

If there are data loss bugs in git (and of course it's possible, even if none have been reported to my knowledge) IMO they're vastly more likely to be generic than platform-specific.

One nice thing about git is you don't have to take its word for your data integrity. You can, without a whole lot of effort, dump out every file in the repository and verify that it is what git says it is.

Anyway, I guess my feeling would be, if I were going to choose to not use git on Windows it would be because of smoothness of the experience, lack of integration with Windows tools, difficult installation process, or stuff like that. Data integrity would not even cross my mind as a downside of git.

-Steve

Dmitry Kakurin wrote:

Pfft, that'll teach me to reply after only skimming the original message. Oops. But the main part of my reply is still valid, I think.

-Steve

On 7/25/07, Dmitry Kakurin <dmitry.kakurin@gmail.com> wrote:

What features is mingw port missing?

FYI, I'm working on getting rid of msys requirement from mingw port. I can't tell you how long it would take though. Could be one month or two.

Hi,

On Wed, 25 Jul 2007, Nguyen Thai Ngoc Duy wrote:

Is there a repo out there?

Ciao, Dscho

On 7/25/07, Johannes Schindelin <Johannes.Schindelin@gmx.de> wrote:

http://repo.or.cz/w/git/pclouds.git?a=shortlog;h=gitbox

There are some patches on mob I have not merged to gitbox branch yet.

Hi,

On Wed, 25 Jul 2007, Nguyen Thai Ngoc Duy wrote:

Thanks, Dscho

On 7/25/07, Nguyen Thai Ngoc Duy <pclouds@gmail.com> wrote:

beautiful piece of work, IMHO. I really like the fact you managed some busybox applets to actually work without msys. Really cool idea!

it seems you're not very far off. I believe you intend to replace in git-commit "#!c:/msys/bin/sh" with something like "#!c:/gitbox/bin/gitbox ash", right ?

On 7/26/07, Christian MICHON <christian.michon@gmail.com> wrote:

No. I tweaked try_shell_exec() to use gitbox shell if the interpreter is "sh".

On 7/26/07, Nguyen Thai Ngoc Duy <pclouds@gmail.com> wrote:

interesting. have you tried inserting busybox vi inside git-box ?

I can commit using "git commit -a -m ok", but then I get this kind of error message (and ash dies, I go back to xp/cmd prompt)

mv: cannot rename '.git/next-index4540': File exists C:/gitbox/bin/git-commit: exit: line 658: Illegal number: -1

nice piece of work. it could really be tiny once fixed.

I suggest to replace most git-* by git-box/ash shell wrappers, calling the git.exe binary directly.

On 7/26/07, Christian MICHON <christian.michon@gmail.com> wrote:

Not yet. That would be fun but I rather focus on ash in the moment. You could set EDITOR to notepad or something else.

Baah.. something goes wrong again.

Um.. that "Illegal number" might be an ash bug. Intended to fix it but then forgot :(

Cool. Will do.

Nguyen Thai Ngoc Duy wrote:

The problem is likely that rename() of MSCVRT.DLL is implemented in terms of MoveFile(), which can't move over an existing file. A wrapper is needed that uses MoveFileEx() instead.

We have the same problem also in git-apply: One of the tests fails for this reason.

-- Hannes

On 7/25/07, Nguyen Thai Ngoc Duy <pclouds@gmail.com> wrote:

Well, 'git commit' from a regular cmd prompt does not work. IMHO, That's a pretty serious omission :-).

"Dmitry Kakurin" <dmitry.kakurin@gmail.com> writes:

I was under the impression that is only because you do not have MSYS installed. Doesn't Windows people have automated way to pull and install other packages on the dependency?

Junio C Hamano <gitster@pobox.com> wrote:

Package management? On Windows? Surely you aren't talking about that silly "Add/Remove Programs" control panel that just destroys your machine.

I do know the best way to uninstall dependencies on Windows is to boot a live Linux CD and `mkfs /dev/hda1`. But as far as installing things go you pray that your vendor has packaged everything you need in their shiny click-through installer thing, and if they haven't, you cry wolf and switch to another vendor.

"Shawn O. Pearce" <spearce@spearce.org> writes:

If that is the case, "Git for Windows" probably should package MSYS as part of it, I would think, to match the expectation of the users there. I know two Johannes'es and Han-Wen spent quite a lot of effort on Windows port and packaging, but perhaps that little (well, I should not be judging if that is a little or huge, as I do not do Windows) finishing touch would make Windows users much happier?

Hi,

On Wed, 25 Jul 2007, Junio C Hamano wrote:

Windows users are only happy when they can bug developers.

Seriously again, the biggest problem with Han-Wen's installer was that it insists on cross-compiling _all_ the packages. This makes it easy for Han-Wen to upgrade packages and compile the thing on Linux in one go. However, it never worked with bash, and I could not fix it: I can read Python, but not _that_ Python.

So my plan was to wrap everything needed from an existing MinGW/MSYS installation, with a minimal installer (NullSoft or whatever) to setup the exec dir, perl lib path etc...

However, my time is scarce, and it does not exactly help that all I can expect from those who should be thankful is even more complaining.

I mean, I understand Linus' point. I don't even expect a Windows user to compile C. It's long time since I was silly enough to believe that. But just wrapping it up in an installer, and a little testing, seems to be too much to ask. When I don't need the darned thing to begin with.

Ciao, Dscho

Johannes Schindelin wrote:

The problem is not really the python. If you supply me with a shell script that will x-compile bash, I'll hapily code the python spec. IMO the real problem is that bash is a unix shell (tied to unix internals) and therefore, compiling it for something as horrid as windows is far from trivial.

fwiw, I briefly tried compiling msys, but I couldn't even find its sources, so I quickly gave up.

A second option is that someone supplies me with an unpacked, installed tree of msys' bash shell. I can easily package that up along with the rest of the installer, if it doesnt' require further trickery (setting registry entries, etc.)

Hi,

[Funny, you quoted me, but culled _me_ from the Cc: list]

On Wed, 25 Jul 2007, Han-Wen Nienhuys wrote:

For me, it is. Probably you know by now that I am not really a fan of Python, mainly because people can write unelegant code which looks elegant.

Will do.

Did you succeed in adding perl? It is not that important, because I plan to make git-gui the main user interface with this installer. But Junio keeps adding Perl scripts (ATM add -i and remote) that I have to convert later...

Ciao, Dscho

2007/7/25, Johannes Schindelin <Johannes.Schindelin@gmx.de>:

It's because gmane does not do SMTP

God forbid no. Perl is enormous, and I shudder at the thought of making all those modules compile, or even worse, writing actual perl code.

I don't see what this is good for. I would suggest to making a clear decision of what are recommended languages, and move everything else to contrib/ .. Currently, C and bash seem the most reasonable choice, but you could decide for perl, but then the consequence should be that the bash scripts are translated into perl. Having both bash and perl serves no purpose, and will lead to duplication of library code to interact with the git binary.

Han-Wen Nienhuys <hanwenn@gmail.com> wrote:

What git-gui is good for? Its a GUI. For people who perfer to use mice and push buttons over keys and a command prompt. A large number of people in this world (many of them on Windows) like these things. Me, I'm more command line than I am GUI, yet I develop git-gui. So I find myself using it a lot, just so I can eat my own dogfood.

Or do you mean Dscho's other point about rewriting tools into C?

Sure, but there's some stuff that shell is good at, and other stuff that Perl is good at. Forcing everything into one mold while we prototype new features is really limiting.

But both are slower on fork challenged systems than using native C. Look at git-fetch for example; my ~400+ branch repository is taking upwards of 5 minutes to run a no-argument, no-changes git-fetch in. All sh and fork overhead.

2007/7/26, Shawn O. Pearce <spearce@spearce.org>:

The last one. The windows installers actually includes a copy of tcl/tk so you can run gitk on windows. .

I'm not contradicting that, but merely suggesting that they go into contrib/ and are not recommended as standard git commands, and don't need to be packaged for windows.

Han-Wen Nienhuys wrote:

They can be not packaged for windows, but for example git-send-email (which is written in Perl) is IMHO important enough to be in git proper and not delegated to contrib/; but it is packaged in separate RPM, git-email. Same with git-svn package...

On Thu, 26 Jul 2007, Shawn O. Pearce wrote:

Well, that really doesn't make much sense from the Linux POV. Bash, perl and C are all well supported languages, each with its own set of strengths. The tools that are being written are true parts of git - not optional contributed bolt-ons.

Admittedly it would probably increase the motivation to make everything built in if only C programs were allowed outside of contrib - but git would probably have not got where it is in that case.

I have a repo with ~9000 refs - it's what motivated me to start rewriting fetch in C ...

times for fetch to decide there were no changes (on Linux, local XFS disk):

pure-shell: ~45 mins shell + C (fetch--tool): ~30 secs pure C: ~0.5 secs

(the C version isn't the current version that Daniel has, but rather an older incomplete version that I had got far enough to do that much - so it may actually be slightly slower now ...)

On 7/26/07, Han-Wen Nienhuys <hanwenn@gmail.com> wrote:

microperl [1] maybe? I haven't tried it yet.

[1] http://www.foo.be/docs/tpj/issues/vol5_3/tpj0503-0003.html

On 7/26/07, Nguyen Thai Ngoc Duy <pclouds@gmail.com> wrote:

it won't work. I tried that few months back.

plus the fact you'll still need perl modules.

I just had a look at your gitbox gitweb. Did you really manage to get busybox-1.6.1 to work with mingw ?

On 7/26/07, Christian MICHON <christian.michon@gmail.com> wrote:

Most of tools (that are included) work fine. Ash almost works. It can run git status, git commit, git clone.. and most of test cases. There are still some missing pieces and bugs to hunt down though.

Hi,

On Thu, 26 Jul 2007, Nguyen Thai Ngoc Duy wrote:

Thank you for working on this!

However, I am not completely convinced that having a builtin shell is all that useful. I for one would like to have MinGW busybox _separate_ from git...

Yes, you could not use the nice "ln -s busybox ash" idiom, since Windows lacks symlinks, but you could still say "busybox ash" with a relatively small, single executable.

Ciao, Dscho

On 7/26/07, Johannes Schindelin <Johannes.Schindelin@gmx.de> wrote:

I make MinGW busybox part of git for some reasons:

 - Making a full MinGW busybox would take lots of time. I don't need
busybox for Windows. What I need is a shell and enough POSIX utilities
to run git shell scripts without any dependencies. Windows users
(including myself when I have to use Windows) hate dependencies.
 - I don't want MinGW busybox to be used outside of git (if it is
installed separated from git), there are cygwin and msys already. I
don't want to compete them. And I don't like conflicts (not sure
though) because you have multiple UNIX emulations on the same system.
 - Making ash part of git has an advantage that you could tune the
shell to fit git. Earlier you had to replace find/sort with
/usr/bin/find and /usr/bin/sort in git scripts to avoid Windows
alternatives. I don't like that. If you have control over the shell,
you could make it ignore whatever program out there and use your own
ones. This one is not a strong point though.
 - MinGW busybox (or gitbox as I call it now) utilizes compat/mingw.c
and other stuff like run-command.c... Making it separate (as source
code) duplicates code for nothing.
 - If you meant separating from git.exe binary, not from source code,
then it's ok.

Hi,

On Thu, 26 Jul 2007, Nguyen Thai Ngoc Duy wrote:

I think that if you succeed to compile ash on MinGW, the rest is easy.

But you'd be my hero.

Installing Cygwin is often overkill if all I need is just a tiny shell with just enough POSIX tools to run my scripts.

Installing MinGW is painful. Not because of MinGW, but because there is no single installer for all I want. You need to install MinGW, MSYS, MSYS DTK, iconv, bash (because the default is to old), etc. etc.

With busybox it would be busybox.exe.

I doubt that this is useful. We do want to support the other systems as well, so we have to kinda stick with the available workarounds.

It is not duplication.  It is forking.  Which is a good thing.

Yes. Although I see your point in making it a builtin "git-ash" that can be called without an extra fork(), using beginthread instead.

Ciao, Dscho

On 7/26/07, Johannes Schindelin <Johannes.Schindelin@gmx.de> wrote:

No it's not. With a couple of ifdefs you can compile it fine. Then there goes fork(), fcntl(F_DUPFD), /dev/*, job/signal handling... Fortunately Git does not use lots of features. It only needs /dev/null (and /dev/zero for tests), SIGEXIT and no job usage.. That cuts down the effort porting ash.

Can't say I don't love to ;-) It's just that I don't have enough time. Once project "busybox for Windows" is born, people may scream for features. Even if they don't, there are still bunch of work because I have only ported a small number of tools. With Git as its sole customer, I can beg Git contributors to limit POSIX tools that they use.

But if someone steps up for the project, I'm all for it.

I see your point. Although you can install git and have git-box to play with (oh spread git! lolz) Just keep in mind you don't have all POSIX tools.

I still don't see why duplicating compat/*, git-compat-util.h, run-command.[ch]... and keeping fixing bugs in two places is a good thing.

"Nguyen Thai Ngoc Duy" <pclouds@gmail.com> writes:

And here I was tempted to multithread builtin-update-index.c: it is actually quite natural to let one process scan directories non-recursively, stat the files, sort them on a per-directory grain and feed a sorted pseudo-index into a pipeline (recursing to scanning whenever hitting a directory), then let another process/thread do a merge-pass of pseudo-index and real index, immediately writing the output to a new index-to-be. When this is finished and another process invalidated the old index already, reuse the index-to-be as pseudo-index and merge it with the new-index-which-got-in-ahead-of-me.

Would be a fun exercise in particular when merely using (block-buffered!) pipes, and could presumably make a difference on multiprocessor-capable machines.

Anyway, just something that had been spinning in my head. The "streaming merge" idea has the advantage of keeping memory usage low pretty much independently of project size: project memory is pretty much determined by the reader pass since it has to read in a complete directory level before it can sort it and output the next element, and it has to retain the not-yet-output elements of the ancestry.

And it is nice to have some potential for parallel processing. But if it is a lethal stumbling block for Windows... It is conceivable to do the same job instead of with pipes and files with buffers and just switch manually between the directory scanning and merging phases. But it would be less fun.

On 7/26/07, David Kastrup <dak@gnu.org> wrote:

(snip)

If you are going to do it. I suggest to base on official mingw branch. I haven't looked at builtin-update-index.c (hey, I'm all doing sh scripts these days) so no comments here.

"Nguyen Thai Ngoc Duy" <pclouds@gmail.com> writes:

Why would I do that? I am not using Windows. Since Windows NT was flaunting threads big style years before Linux, I should not think this implementation approach really to be a major porting hurdle, at least if one indeed uses pipes for the IPC.

It should actually be doable with a clone system call or pthread_create, whatever is easier to translate into Windows semantics. The latter probably is more portable nowadays.

Hi,

On Thu, 26 Jul 2007, Nguyen Thai Ngoc Duy wrote:

Actually, it would pretty easy to set up a tracking script with Git, I guess. But I can look into that once you finished your gitbox. Thanks for doing it BTW...

Ciao, Dscho

Nguyen Thai Ngoc Duy wrote:

(..snip..)

Hi Duy,

*drool* This was an extremely good idea! Thank you so much for working on it! I'll be sure to play around with it, and see if there's any way I can help out. Guess I finally have to get that MinGW compile environment set up then :-)

Btw, are you compiling with MinGW on Windows, or cross-compiling on Linux?

Neat neat neat!

-- .marius

On 7/26/07, Marius Storm-Olsen <marius@trolltech.com> wrote:

I cross-compile all the time (and test it on Windows when I have one, on the buggy Wine when not). I'd absolutely appreciate bug reports regarding building it on Windows ;-)

On 7/26/07, Nguyen Thai Ngoc Duy <pclouds@gmail.com> wrote:

earlier on, when I reported a successful compilation and few problems while committing, it was on XP. I'll be offline for the next 2 weeks, but I can dedicate some time to your porting.

I second also what Dscho said. You'd be my hero too if porting bbox over XP. Imagine "bbox + tcc + make + git"...

:)

torsdag 26 juli 2007 skrev Johannes Schindelin:

Will windows developers need to get Linux just in order to do a two line fix, or will the build process work on Windows too (provided the developer gets a number of extra packages).

-- robin

Robin Rosenberg wrote:

The build process works on Windows, too. That's how I do it. README.MinGW lists the packages that you need.

-- Hannes

Hi,

On Wed, 25 Jul 2007, Dmitry Kakurin wrote:

Not true.

Ciao, Dscho

On 7/25/07, Johannes Schindelin <Johannes.Schindelin@gmx.de> wrote:

Well, repro is very simple:
* Follow instructions on Git Wiki and install Git from
http://lilypond.org/git/binaries/mingw/
* run git commit:
E:\Git\usr\bin>git commit
git: 'commit' is not a git-command

I've tried on both Win XP and Vista machines.

- Dmitry

Johannes Schindelin <Johannes.Schindelin@gmx.de> wrote:

Use git-gui.  ;-)  It doesn't need a shell to make commits.

It currently uses the shell for fetch and for merge. I'm fixing merge this week. I'm hoping Daniel's native C fetch will fix the fetch.

Hi,

On Thu, 26 Jul 2007, Shawn O. Pearce wrote:

Of course. Ever since I saw git-gui, I was convinced that _this_ is the tool Windows users should use.

Ciao, Dscho

On 7/26/07, Johannes Schindelin <Johannes.Schindelin@gmx.de> wrote:

QGit might be a good alternative too, especially because QT4 is available for Windows.

Henning

On Thursday 2007 July 26, Henning Rogge wrote:

I have a colleague using qgit4 on Windows with git on cygwin. Works very well; and qgit being native rather than tcl/tk+cygwin makes it a lot faster.

I've also seen him using gitweb for browsing around my repository.

We've never lost data, and have had significantly more success using git itself rather than git-cvsserver, which was a constant struggle.

Andy

On Jul 25, 2007, at 12:35 PM, Dmitry Kakurin wrote:

What's wrong with the Cygwin port?

Is it just that windows developer hate cygwin because it's to complex to install or is there any severe limitation? functionality? stability? performance?

I'm personally only working on Windows if force to, but people are asking me the same question that you have. Does git seriously and fully support Windows?

	Steffen

Cygwin tries to make Windows look like unix (from a command-line POV), so it very much runs against the grain of "real" windows programs.

Plus, it's a pain to install and invoke, and doesn't deal nicely with real windows paths (it maps the windows filesystem to a unix-y style single root path structure),

Steffen Prohaska wrote:

Disclaimer: http://www.peralex.com/disclaimer.html

Hi,

On Wed, 25 Jul 2007, Noel Grandin wrote:

Okay, just because you insist, I will introduce a crash, so that it does not look too much like Unix. Maybe I will do this as an alternate hang/crash.

Ciao, Dscho

Johannes Schindelin <Johannes.Schindelin@gmx.de> writes:

You forgot an obligatory smiley.  That is not amusing.

On 7/25/07, Steffen Prohaska <prohaska@zib.de> wrote:

I actually have no problems with cygwin and find it works pretty well with git repositories. Starting the xserver to run git-gui is pretty annoying though. Windows-based development teams are going to expect easy access to those kinds of tooling. Otherwise, the champion will be pushing a type of workflow change that would hinder adoption anyway and leave a sour taste for a long time.

In addition, performance is atrocious.  In my particular case I have
an older P4 running F7 and a newer machine running Windows and cygwin.
 On a pserver based cvsimport of a large, enterprise project, Linux
was able to generate the full history in 4 hours, cygwin took 3 and a
half days.  When I sync up every now and then, typical times for
windows are 25 minutes and Linux is around 4.  That should give you an
idea of what kind of multiplier we are talking about.

I don't know if the performance problems are cygwin or not. More knowledgeable people might be able to answer, it's just what I'm observing right now. It could be more fundamental to the types of access being performed en masse on inode-based versus NTFS systems.

Stephen Cuppett <cuppett <at> gmail.com> writes:

I have the version of git that came with cygwin, and I never have to run an X server to run git-gui or gitk.

Personally, I can't imagine running git without cygwin. Course, I want my desktop to feel as much like unix as possible. My experience with git under cygwin has been excellent. My only gripe has to do with CRLF. The repository has everything checked in with dos line endings, I'd like to check everything out with unix line endings, and then check it back in with dos line endings. I hate seeing the ^M's everywhere.

Granted, the performance isn't equal to git running on a real unix, but compared to working with SVN under win32, I would say it performs quite well.

Hi Russ,

Try playing with the core.autocrlf config option.

Cheers, Emil.

This e-mail, and any associated attachments have been classified as: -------------------------------------------------------------------- [x] Public [ ] Freescale Semiconductor Internal Use Only [ ] Freescale Semiconductor Confidential Proprietary

-----Original Message-----
From: git-owner@vger.kernel.org [mailto:git-owner@vger.kernel.org] On
Behalf Of Russ Dill
Sent: Wednesday, July 25, 2007 12:56 PM
To: git@vger.kernel.org
Subject: Re: Windows support

Stephen Cuppett <cuppett <at> gmail.com> writes:

I have the version of git that came with cygwin, and I never have to run an X server to run git-gui or gitk.

Personally, I can't imagine running git without cygwin. Course, I want my desktop to feel as much like unix as possible. My experience with git under cygwin has been excellent. My only gripe has to do with CRLF. The repository has everything checked in with dos line endings, I'd like to check everything out with unix line endings, and then check it back in with dos line endings. I hate seeing the ^M's everywhere.

Granted, the performance isn't equal to git running on a real unix, but compared to working with SVN under win32, I would say it performs quite well.

Medve Emilian-EMMEDVE1 <Emilian.Medve <at> freescale.com> writes:

It seems to do the exact opposite of what I would like. My repository is imported from SVN with git-svn and all the text files have dos line endings. I would like to checkout with unix line endings, and checkin with dos line endings.

On Wed, 25 Jul 2007, Stephen Cuppett wrote:

I think cygwin may add some overhead, but people should really realize that Linux is quite often an order of magnitude faster (or more) than other systems on some very basic operations.

That's especially true for filesystem operations. We really are just that good.

Really simple things like stat/open/read/write/close are just damn fast on Linux. To the point where you really do notice it when you compare to other systems. If something takes hours on Linux, and it's very filesystem-intensive, I'm not at all surprised that it might take days on Windows.

(OS X is probably better than Windows when it comes to filesystem ops, but their memory management absolutely sucks, and I can pretty much guarantee that their filesystem operation latency doesn't hold a candle to Linux, so while I'd expect git to perform "pretty well" on OS X, it's still going to be slower than on Linux)

Linux really *can* be that much faster. You may not see it as much on some other loads, where most of the load is about normal user code, and system call performance is likely to be just a small fraction, but for git, most of what it does is filesystem interactions (I used to think that SHA1's would be noticeable - they're not, and while zlib overhead *can* be noticeable, it usually isn't a big deal except for some very specific cases).

But I bet that git ends up being faster on Windows than many other SCM's are (on Windows). Going native will help, and avoiding things like shell scripting will help a *lot*, but it's still always going to be slower on Windows than it is on Linux. And that is not about anything else than the fact that Linux simply kicks *ss on filesystem ops.

So for doing things like big imports, you might well want to do them on Linux. But that doesn't mean that git will suck on Windows for normal operations.

(It will just not be so *blazingly* fast, ie things like "git status" will generally not be instantaneous).

			Linus

El 25/7/2007, a las 20:43, Linus Torvalds escribió:

Would be very interesting to see some "scientific" benchmarks of Git performance on the different platforms.

Anyone got an Intel Mac with Windows and Linux installed on it as well?

Cheers, Wincent

Linus Torvalds said the following on 25.07.2007 20:43:

(..snip..)

Let me present some numbers: My repository is ~680MB, and 19323 tracked files, in 2264 directories. When in a compiled state the total is 27540 files, in 4885 directories.

When file system cache is warm, I get the following times:
 Native: dir /B /S               1.077s
         dir /S                  1.707s (shows time, size/type)
 MSys:   ls -f1AUR              34.608s
         find . -type f          6.718s
         git diff (empty diff)   1.18s
         git status              5.5s
and when the system cach is cold:
         git status             54.55s

Maybe you guys have other git commands which are also/more interesting to look at/benchmark?

Windows people should also be aware that it's possible to tweak the amount of memory which the OS uses for the file cache. On XP you can change it _roughly_ in System Properties panel (Right-click on My Computer), then Advanced - Performance Settings - Advanced - Memory Usage: Normal setting is "Programs" for non-servers Windows systems, while you can select "System cache" make the OS allocate more memory for the system caches. I've tried both, and the setting doesn't really affect the file operations much when the cache is warm, but it probably affects how long the cache stays warm.

Also note that you can use Sysinternal's CacheSet (free), to
manipulate the working-set parameters of the system file cache.
You'll find that here:
    http://www.microsoft.com/technet/sysinternals/FileAndDisk/CacheSet.mspx

Stephen Cuppett <cuppett@gmail.com> wrote:

I've never even tried to run it that way. I know the Cygwin packages have two different Tcl/Tk binaries: one that is actually a hacked up native Tcl/Tk (which uses native Win32 graphics) and one that is a straightup recompile of the UNIX Tcl/Tk (which uses X11 only). I only have the native Win32 Tcl/Tk installed, and thus run native graphics.

An advantage of the native Tcl/Tk is just that, its native. A downside is it is native. It doesn't use the Cygwin APIs for exec, it directly goes through CreateProcess(). It doesn't use the Cygwin APIs for file IO, it goes directly through the native Win32 stuff. Which means it sometimes has a difficult time with Cygwin paths. There are a few places in git-gui where I run `cygpath -w` just to handle this.

I've also recently tested git-gui under the ActiveState Tcl distribution. It ran, but for reasons unknown to me (I didn't research it yet) the .git/info/exclude rules didn't apply when git-gui spawned a `git ls-files --others` helper process.

I have considered doing a starkit of git-gui, msys based git executables/dlls, and the ActiveState Tcl engine. That should give users a single git-gui.exe that they can just download and launch, no installation required. Haven't started it yet, partly because I haven't finished removing the need for shell scripts to support git-gui. I'm almost there, especially with Daniel Barkalow's work on a native C fetch.

I agree. They also want this thing called "explorer integration" or something like that. I've never had good luck with cra^H^H^Hpackages that install themselves into the Windows explorer UI. I would probably never develop such a thing myself.

As Linus described its NTFS/Windows that is horrid here, and not really Cygwin. Linux is just fast. Almost all modern UNIXes are. At least when compared to the Windows kernel running on a crappy 4500 RPM IDE drive that is also hampered with a virus scanner that wants to scan 12 GiB of data every 30 minutes, even when the volume has only 8 GiB of files on it. ;-)

Git is fast. Its faster than SVN on the same hardware. Even under Windows. About the only thing that I think is "slow" is two areas:

  * status.  Doing lstat() on 8,000+ files takes a little while.
  Hooking into the OS' native file monitoring facility and having
  that tell us which files are stat-dirty would reduce the need
  for these massive lstat() runs.

  * for-each-ref.  Opening 400 ref files to read their current SHA-1
  values is not fast on Windows.  More aggressively packing refs
  (at least on Windows) may actually be worthwhile.  Another option
  might be to have the same process that is watching the OS' file
  monitoring interface cache the ref values, so we can get to them
  via say shared memory or a pipe instead of file IO.

Neither feature is really necessary on a good UNIX however, as most kernel development teams have just made sure their system has good file IO throughput. Oh, and they don't have to run virus scanners that get higher IO priority than everything else.

On Wed, 25 Jul 2007, Dmitry Kakurin wrote:

If the issue is the shell scripts, replacing those with C code is coming along nicely; there's a big section (fetch and everything it uses) which is ready to go after 1.5.3 comes out, and I believe a number of the other core parts are being taken care of in the same sort of time frame by the GSoC people.

I've also been working on reducing the fork/exec usage, if that's the issue, but I'm not sure how much of that is left to do or how long it will take.

(Personally, I don't touch windows at all, but I have been working on fixing things that seem to be problems for porting to windows, which may be relevant)

	-Daniel
*This .sig left intentionally blank*