Windows symlinks

It looks like one of the bigger (biggest?) hurdles for git adoption at my company is going to be handling symlinks on Windows. We may be able to sidestep the issue entirely by having developers run Linux in a virtual machine (or better yet, Windows in a VM) because we deploy to Linux and thus can make a very strong argument that developers should develop on Linux, too. That run into an image problem where people might start blaming git for needing to have an extra operating system around just for revision control.

We currently use Clearcase snapshot views on Windows and it has some kind of magic to be able to deal with symlinks. I haven't actually used the Windows version of Clearcase so I don't know how it does it, but my understanding is that it handles them with copies, and I guess it somehow knows to update all the copies when a change is made.

Has anyone thought about a way for git to handle symlinks? Vista seems to have added native symlinks, but you need have elevated privilege to create them. NTFS junction points seem helpful for older versions of Windows, but don't work for anything except directories, and seem to be dangerous to use with tools that do recursive deletes. Neither junction points nor native symlinks sound like great options.

Cygwin's clever symlink trick seems to work pretty well in practice. I'm not exactly sure what it's doing, but it seems to create a shortcut that it's own programs understand. Some other non-Cygwin programs seem to understand them, too, but Java does not which is a big problem for me.

I just don't see any good solutions here. Does anyone have any better ideas?

Thanks, Patrick

On 6/24/08, Patrick.Higgins@cexp.com <Patrick.Higgins@cexp.com> wrote:

It seems somewhat odd that your developers are relying on an operating system feature that doesn't even exist. I might suggest one of:

- add an abstraction layer to your Java applet that understands
cygwin-style symlinks

- actually just check in the file multiple times; git will store it
efficiently anyway

- improve your Makefile to automatically copy the files and/or create
symlinks at build time rather than storing the symlinks in revision
control.

My understanding of Clearcase is that it actually loads a virtual filesystem driver in Windows and does all kinds of insane magic - slowly and inefficiently, too - behind the scenes. Something tells me git (and every other VCS :)) really doesn't want to go there.

Have fun,

Avery

On Tue, Jun 24, 2008 at 7:18 PM, Avery Pennarun <apenwarr@gmail.com> wrote:

Just FYI, this is only the case if you're using so-called dynamic views, which require the MVFS driver, as you say. The OP said he was using snapshot views -- these don't use MVFS and instead just checkout to the OS's native filesystem.

j.

Patrick Higgins <Patrick.Higgins@cexp.com> writes:

If only MS Windows supported other filesystems which have symlinks...

First, I think that both "git on Windows" solutions, namely Cygwin and msysGit port, don't use symlinks either in installed programs, nor in repository layout.

Second, the problem there can be _only_ if your repository contains (or contained) symlinks, and then it is your own damn fault. I don't know how Cygwin, or msysGit deals with symlinks in a wirking directory, but you can work around symlinks (although in a bit unwieldy way) by using `core.symlinks' configuration variable; see git-config(1):

 core.symlinks::
        If false, symbolic links are checked out as small plain files
        that contain the link text. git-update-index(1) and git-add(1)
        will not change the recorded type to regular file. Useful on
        filesystems like FAT that do not support symbolic links. True
        by default.

Agreed. The first thing we started working on was getting symlinks out of our repositories. Unfortunately, we chose to use them because we are using broken development tools that don't support proper modularity. We found the best way to get the IDE's compilation to work was to symlink shared code into each project. It's a nasty solution, but it was the only one that worked. Clearcase made this solution possible. As powerful and flexible as git is, it would have left us out in the cold in this situation.

msysGit just created a file with the target of the symlink as it's contents, but its tools didn't know how to make use of it. I think Cygwin did the same thing, but ls, cat, and all the other Cygwin tools were able to make use of it. I tried to run the Java compiler over a symlinked file and it didn't work, though.

Copying files at build time is an interesting idea, but since the point was to get an IDE's build system to work, I don't I'll have the level of control I would need to make this work.

On 6/24/08, Patrick.Higgins@cexp.com <Patrick.Higgins@cexp.com> wrote:

Perhaps git-submodule would do what you're looking for.

Avery

We might be able to get by with them, but submodules appear to be significantly more complex than symlinks, and we sometimes symlink just a file or two if that's all we need. Splitting up submodules to that level of granularity would be hard to manage.

My understanding of the submodule workflow is:

Projects A, B, and C (we actually have about 17 of these) all share common code in project Common. Then, each of A, B, and C adds Common as a submodule. While working on project A, the need arises to modify Common, so the developer changes it there, commits, pushes the change to Common, then commits and pushes the change to A. To update B and C, they would have to change to each of those projects, run a git pull, then git submodule update, and compile and test.

Is that correct? If so, it's a lot more work than letting a symlink propagate the change to all the other projects. Of course, since Windows doesn't have symlinks...

Patrick.Higgins@cexp.com wrote:

That is correct, but you only need to do the change if the projects B, C, D, n... actually requires the new feature/bugfix introduced in Common. Otherwise you can just leave it as-is.

Besides, when an important bugfix is added to Common, you probably want to cut new releases from B, C, D, n... anyway, so it still means doing stuff to those repositories.

onsdagen den 25 juni 2008 00.42.49 skrev Patrick.Higgins@cexp.com:

Right, they could use only Linux and not have an extra OS for email..

It is one of the few sane things clearcase does. msysgit's approach is just unusable. I've not come up with the patches yet. Clearcase makes copies instead of the symlinks and updates them on checkout/checkin and update. Git could do one step better by updating copies when index is updated, avoiding an unnecessary commit (which is pretty annoying in CC). Invalid symlinks, i.e. pointing at non-existent objects or outside then VOB (repository in Git parlance, don't show up at all in snapshot views. Git could do the same thereby making the workdir usable when it contains and depends on symlinks.

Obviously one could make this a switchable feature and enable it when the file system supports it. As for junction points i'd say no too. What little testing I did on symlinks in vista they seem capable of most sane uses.

No, copies. If you have a patch I can test it and then implement it for jgit, which doesn't have symlinks either.

-- robin