getting rid of extra directories

Hi folks,

I've been using the latest Cogito. When I seek back and forth in a git repo, directories I've created tend to persist backward in time, cluttering up earlier versions.

By the discussions on this list, I'm pretty sure there's a way to stop this, but I don't know clearly what it is, and I'm worried about trying things that might corrupt my repository. Could someone spell it out for me please?

Be well, Zack

Zack Brown <zbrown@tumblerings.org> writes:

I do not know exactly what Cogito "seek back and forth" does, but updating to git-core "master" branch may help.

Specifically, this commit intends to remove empty directory from the working tree, where previously checked out tree object had files/directories inside while newly checked out tree does not.

    commit 340e4f88c083b0692e6554b1c2c27fd43c7cc8d3
    Author: Junio C Hamano <junkio@cox.net>
    Date:   Mon Oct 10 17:34:08 2005 -0700

        Remove empty directories after read-tree -u.

        This fixes everybody's favorite gripe that switching branch
        with 'git checkout' leaves empty directories.

        Signed-off-by: Junio C Hamano <junkio@cox.net>

On Sat, Oct 15, 2005 at 11:00:52AM -0700, Junio C Hamano wrote:

It means to "cg-seek" back to an earlier state of the repo, and forward to a more recent state of the repo. "seeking back and forth" means doing a lot of "cg-seek" commands. Sorry that wasn't clear.

I already use that branch.

Here's some commands that reproduce the problem:

$ mkdir tmpproj $ cd tmpproj $ cg-init

give the initial commit message and save

$ ls > a $ cg-add a $ cg-commit

give the commit message and save

$ cg-tag nodirs $ mkdir d $ mv a d/a $ cg-rm a $ cg-add d/a $ cg-commit

give the commit message and save

$ cg-seek nodirs

now I would expect to see the original directory, with no 'd' subdirectory, and just the 'a' file sitting there.

$ ls -F a d/ $

See the problem? The 'd' directory did not exist in this version of the repository, but it's there anyway.

Be well, Zack

Zack Brown <zbrown@tumblerings.org> writes:

I usually do not use Cogito, so I updated to the latest and tried it myself. No wonder.

cg-seek does not use "git-read-tree -m -u $old $new", which was updated by that commit I mentioned to remove the empty directories. It does "git-read-tree -m $new" and does the removal part manually by reading from diff-tree $old $new. I suspect the code was there before the two-tree read-tree was invented.

A similar sequence as you did, in pure git, would remove empty directory d/ when switching branches between.

    $ mkdir junkproj
    $ cd junkproj
    $ git-init-db
    $ ls >a
    $ git-add a
    $ git-commit -a -m 'Add a'
    $ git-checkout -b adddir
    $ mkdir d
    $ mv a d/a
    $ git-add d
    $ git-commit -a -m 'Move a to d/a'

    $ git-checkout master

I suspect cg-Xlib::tree_timewarp, which currently does "git-read-tree -m $branch" followed by diff-tree piped to xargs, can be taught to use "git-read-tree -m -u $base $branch" (and lose the git-checkout-index -f -a immediately after that while we are at it), but I do not do Porcelain, so...

Dear diary, on Sun, Oct 16, 2005 at 12:34:29AM CEST, I got a letter where Junio C Hamano <junkio@cox.net> told me that...

This is problematic, since with our current way of tree timewarping, restoring local changes to files which got deleted will make patch error out, etc.

I wanted to make tree_timewarp do three-way merge, but didn't figure a good way to do it. First, what do I want - a two-way merge between two trees, which will however respect (not die on) local changes in the working tree.

One approach is to take the working tree, construct a tree object from it and then run the regular three-way merge on that. But that is a waste of time and pollutes the database with nonsensical objects. The advantage is that I can reuse the per-file merge resolution script without any changes.

However, this seems to be the only way to do this right now. What do you think about the disadvantages - should we care? A possible alternative I can imagine is to make a two-way merge mode which somehow records the conflicts with index back in the index file instead of committing suicide. Possibly the hex(-1) sha1 might be used for the relevant index items, meaning "the current working copy". The per-file merge resolution scripts would have to be tweaked for this then, but it shouldn't be that hard; what I found hard was actually getting my head 'round the weird git-read-tree code recording the stages in the index file. ;-)

Opinions?

Petr Baudis <pasky@suse.cz> writes:

For a large tree, constructing a full tree just to merge a couple of potentially conflicting paths might turn out be a waste of time, but I suspect that is something you could optimize later. git-am 3-way fallback logic constructs a sparse temporary tree and uses it for running regular 3-way, and you may be able to do something similar. Right now git-am takes an e-mail patch and when it runs all the way it creates a commit, but the core logic that applies patch and falls back to 3-way could be separated out for your application. Then you could: (1) 'git diff HEAD' before tree-warp to preserve the user changes, (2) clean up the working tree to match HEAD, (3) warp to the other tree, and (4) fed the user change perserved to git-am core logic. That might give you something near optimum.

About "nonsensical" objects, I am not so sure how nonsensical those objects you would record from the working tree (intermediate state) are. If you find two trees match on a path that has changed in the working tree, there won't be any conflict on that paths so it is like the user did an extra git-update-index on that path when no merge or warp is involved, from object database pollution POV; I do not personally think that is such a bad thing.

Dear diary, on Thu, Nov 03, 2005 at 09:02:10AM CET, I got a letter where Junio C Hamano <junkio@cox.net> told me that...

I'm confused now - if the tree is sparse, how does git-read-tree -m know that the missing files from the other tree are not deletes?

At this point I get nervous since there is a potential for data loss when something goes wrong at this point (e.g. system crash). I'm not saying there are no places like that in Cogito now, but I generally try to avoid them whenever possible.

But if they don't match, you'll get superfluous blobs for the original index versions which are almost certain to go away.

Petr Baudis <pasky@suse.cz> writes:

The git-am 3-way fallback works like this:

    - the patch did not apply to our current head cleanly.

    - however, we have all the original blobs, recorded on
      'index xxxxxxx..xxxxxxx' line of the patch.

    - construct a temporary tree T0 that has only paths affected
      by that patch, with the original blobs.

    - apply the patch to T0 to construct another temporary tree
      T1.

    - 3-way merge our current head and T1, pretending as if T0
      is our common ancestor.  I.e. "read-tree -m -u T0 HEAD T1"

The paths that did not change by the patch (in your case, the paths without local modification before tree-warp) are not present in either T0 or T1, so the 3-way merge in the last step would see that our current head added those paths while T0->T1 transition kept their absense intact --- which results in added-in-one-side-only.

I think the problem you are solving would be solved by what I suggested in the earlier message, but probably always doing that may not be the optimum solution.

    - the working tree (and index) is derived from HEAD, but may
      not match HEAD (local modifications).

    - you want to tree-warp to T1, doing an equivalent of
      "read-tree -m -u HEAD T1" but preserving the local
      modifications.

You would want a merge between INDEX and T1 using HEAD as the common ancestor (INDEX is a tree that your user would get if all the local modification in the working tree were registered with update-index and then git-write-tree were run).

	 INDEX
        /
    HEAD
        \
         T1

Two-tree form of read-tree -m -u should work as long as differences between HEAD and INDEX do not interfere with HEAD->T1 transition. So you probably would first want to try "read-tree -m -u HEAD T1", and use the 3-way fallback I described earlier when it fails?

Another way of putting it is that I would not particularly worry about those dangling blobs and trees -- that's what we have 'git prune' for.

Petr Baudis <pasky@suse.cz> writes:

I remember that we used to have --emu23 flag to read-tree. It was yanked out when we did multi-merge-base conversion, because nobody was using it, but I wonder if that is close to what you are looking for.

$ git whatchanged -S--emu23 read-tree.c

tells me that 32192e6622d78347448cfc0572827d6e64e0de28 commit completed it and ee6566e8d70da682ac4926dd8a67ac821b2c1743 commit removed it.