What's cooking in git.git (topics)

Here are the topics that have been cooking. Commits prefixed with '-' are only in 'pu' while commits prefixed with '+' are in 'next'. The topics list the commits in reverse chronological order.

* jc/send-pack-pipeline (Fri Dec 29 12:14:30 2006 -0800) 2 commits
 + Documentation: illustrate send-pack pipeline.
 + send-pack: fix pipeline.

There was a longstanding bug that was exposed only by accident when used with Shawn's sliding mmap changes (sp/mmap), and these are to fix it. I'll merge this to 'master' before v1.5.0-rc1.

* sp/mmap (Sat Dec 30 22:13:43 2006 -0500) 25 commits
 + Update packedGit config option documentation.
 + mmap: set FD_CLOEXEC for file descriptors we keep open for mmap()
 + pack-objects: fix use of use_pack().
 + Fix random segfaults in pack-objects.
 + Cleanup read_cache_from error handling.
 + Replace mmap with xmmap, better handling MAP_FAILED.
 + Release pack windows before reporting out of memory.
 + Default core.packdGitWindowSize to 1 MiB if NO_MMAP.
 + Test suite for sliding window mmap implementation.
 + Create pack_report() as a debugging aid.
 + Support unmapping windows on 'temporary' packfiles.
 + Improve error message when packfile mmap fails.
 + Ensure core.packedGitWindowSize cannot be less than 2 pages.
 + Load core configuration in git-verify-pack.
 + Fully activate the sliding window pack access.
 + Unmap individual windows rather than entire files.
 + Document why header parsing won't exceed a window.
 + Loop over pack_windows when inflating/accessing data.
 + Replace use_packed_git with window cursors.
 + Refactor how we open pack files to prepare for multiple windows.
 + Create read_or_die utility routine.
 + Use off_t for index and pack file lengths.
 + Refactor packed_git to prepare for sliding mmap windows.
 + Introduce new config option for mmap limit.
 + Replace unpack_entry_gently with unpack_entry.

This is Shawn's sliding mmap series to allow smaller virtual memory footprint to access larger packfiles. I started using this series in production tonight. Although the size of the series is somewhat intimidating, they are sane changes and I think it may be worth considering for 'master'. This does not change the user experience majorly as has almost no UI elements, so it could go in either before or after v1.5.0.

* jc/3way (Wed Nov 29 18:53:13 2006 -0800) 1 commit
 + git-merge: preserve and merge local changes when doing fast
   forward

A few people wanted to have this in v1.5.0, but I am a bit reluctant to do so --- I think the behaviour of its failure mode is rather nasty, even though it tries to help the user by dropping the local diff in .git/LOCAL_DIFF file.

* sp/merge (Sun Dec 31 00:02:13 2006 -0500) 6 commits
 - Refresh the index before starting merge-recursive.
 - Improve merge performance by avoiding in-index merges.
 - Avoid git-fetch in `git-pull .` when possible.
 + Use merge-recursive in git-am -3.
 + Allow merging bare trees in merge-recursive.
 + Move better_branch_name above get_ref in merge-recursive.

I'm reasonably happy with the earlier three of this series but not really about the latter, and I've already described why.

* jc/diff (Mon Dec 25 01:08:50 2006 -0800) 2 commits
 - test-para: combined diff between HEAD, index and working tree.
 - para-walk: walk n trees, index and working tree in parallel
* jc/explain (Mon Dec 4 19:35:04 2006 -0800) 1 commit
 - git-explain
* jc/web (Wed Nov 8 14:54:09 2006 -0800) 1 commit
 - gitweb: steal loadavg throttle from kernel.org
* jc/pickaxe (Sun Nov 5 11:52:43 2006 -0800) 1 commit
 - blame: --show-stats for easier optimization work.
* jc/diff-apply-patch (Fri Sep 22 16:17:58 2006 -0700) 1 commit
 + git-diff/git-apply: make diff output a bit friendlier to GNU patch
   (part 2)

These are not for 'master' for now.

Junio C Hamano writes:
 > * jc/3way (Wed Nov 29 18:53:13 2006 -0800) 1 commit
 >  + git-merge: preserve and merge local changes when doing fast
 >    forward
 > 
 > A few people wanted to have this in v1.5.0, but I am a bit
 > reluctant to do so --- I think the behaviour of its failure mode
 > is rather nasty, even though it tries to help the user by
 > dropping the local diff in .git/LOCAL_DIFF file.

It happened to me (not really an experienced user) for the first time on the last pull. Though unexpected it wasn't an unpleasant experience. The failure mode doesn't feel smooth, but after thinking about possible improvements I couldn't come up with anything better. It already states explicitly what it's doing including telling the user about LOCAL_DIFF. The only thing I wasn't sure about was how to get rid of the unmerged index entries after I fixed the conflicts without adding the local change to the index. In the end I did an update-index/reset combination, but perhaps repeating the list of files with conflicting local changes after the merge and cleaning the index instead of leaving unmerged entries in the index would be better.

I think (because of LOCAL_DIFF) it is safe to have this in 1.5 even in its current incarnation.

Hi,

On Sun, 31 Dec 2006, Junio C Hamano wrote:

I am not urging the inclusion in 1.5.0, but how about storing not the diff, but the _state_ in .git/UNCOMMITTED_TREE? This would work with all the git tools without any problem, and git-gc would eventually clean up older intermediate stages.

Ciao, Dscho

Junio C Hamano <junkio@cox.net> wrote:

I know your testing is saying this series may be slightly faster than
the prior code in 'master', and at least on Linux can be so without
using a lot of virtual address space as mmap() is so fast there.
(Nice job kernel people!)  Anyway, I haven't done any performance
testing on this myself.  It would be nice to know more about how
it behaves.
 

I know you've stated a number of good reasons against "Avoid git-fetch in `git-pull .` when possible.". One way I've thought of trying to resolve that would be to verify the arguments to `git pull .` are refs only (and not SHA1 expressions) which makes the behavior the same as `git pull .` and *might* still save time, as the fetch can still get bypassed.

The only objection I know of to "Improve merge performance by avoiding in-index merges." was that it needs more testing to ensure its doing the same thing as before, which basically means we hold it out until post v1.5.0. Since it is only a performance improvement I think that's reasonable.