Re: Not going beyond symbolic links

Linus Torvalds <torvalds@linux-foundation.org> writes:

While I admit that I have managed a large directory split across partitions grafted via symlinks in pre-git days myself, ever since you started "tracking" symbolic links with 8ae0a8c (git and symlinks as tracked content, 2005-05-05), you have pretty much been committed to "track" symbolic links.

This goes even before that commit. The readdir() loop done in show-files.c with 8695c8b (Add "show-files" command to show the list of managed (or non-managed) files., 2005-04-11) does not dereference symbolic links pointing at a directory elsewhere, which we still have as read_directory() in dir.c without much change in the basic structure.

I would give some leeway to other people who made comments in this thread, who may not be so familiar with the low-level git codebase, but I have to say that if you claim that dereferencing symbolic links in the middle is a feature, you are not being completely honest, you haven't thought through the issues, and/or you simply forgot the details. I'd suspect most likely it is the last one ;-).

The thing is, the "feature" is not very well supported, even without the fixes from last night. If you have a symlink "sym" that points at "dir" that has "file" in it, and if neither "sym" nor "dir/file" are tracked, you can "git add sym/file" to add it (I called it a bug).

However:

 (1) starting from the same condition, "git add ." does _not_ add it (you
     get the symbolic link "sym" added to your index instead, as well as
     "dir/file");

 (2) after you add "sym/file" through the bug, if you say "git add .", it
     will be removed from the index and you will instead have "sym" and
     "dir/file" (with an ancient git before 1.5.0, you will get "unable to
     add sym" error instead).

 (3) after you add "sym/file", "git diff" will immediately notice that you
     have removed it (this is a fairly recent fix; 1.5.4.X doesn't notice
     it).

You cannot have it as a reliably usable feature without a major surgery, and this is fundamental. You simply cannot have it both ways without telling git which symlink is "tracked" and which are only there for storage sizing.

If you seriously want to claim that we support such a feature, you would at least need to:

 (0) have a way for the user to say, "the project tree may have a
     directory D, but I do not want to check it out as a directory because
     my partition is too small.  Whenever you need to create a directory
     there and hang a tree underneath, instead create a symlink that
     points at /export/large/D instead".  Most likely this information
     would go to .git/config;

 (1) whereever we run "create leading directories", we notice and honor
     the above configuration (mostly entry.c::create_directories() called
     from entry.c::checkout_entry());

 (2) whenever we need to check out a file to path D, instead of
     recursively remove everything under it, we remove the symlink and
     deposit the file there (mostly entry.c::remove_subtree() and
     unpack-trees.c::verify_absent());

 (3) whenever we traverse working tree using readdir(), notice that the
     symbolic link we are looking at is the funny "pointing elsewhere but
     this is really a directory" specified in (0) and recurse into the
     directory pointed by it (dir.c::read_directory_recursive() but there
     may be others)

 (4) and we teach has_symlink_leading_path() to special case such a path
     you configured in (0).

I personally do not think adding these to support such a "feature" is such a high priority, and I do not think it is honest to claim we support such a feature without doing any of the above. The current reality is that our symlink support is still broken in corner cases, and being able to easily add "sym/path" via "git add" and "git update-index --add" is one of them.