What's cooking in git.git (topics)

Here are the topics that have been cooking. Commits prefixed with '-' are only in 'pu' while commits prefixed with '+' are in 'next'. The topics list the commits in reverse chronological order.

* jc/fsck-reflog (Fri Dec 22 23:42:30 2006 -0800) 9 commits
 + reflog expire: do not punt on tags that point at non commits.
 + reflog expire: prune commits that are not incomplete
 + Don't crash during repack of a reflog with pruned commits.
 + git reflog expire
 + Move in_merge_bases() to commit.c
 + reflog: fix warning message.
 + Teach git-repack to preserve objects referred to by reflog
   entries.
 + Protect commits recorded in reflog from pruning.
 + add for_each_reflog_ent() iterator

I'd like to push this out before we go -rc1, since the reflogs are now enabled by default, and otherwise would grow unbounded.

* jc/utf8 (Mon Dec 25 11:48:35 2006 -0800) 3 commits
 - Teach log family --encoding
 - i18n.logToUTF8: convert commit log message to UTF-8
 - Move encoding conversion routine out of mailinfo to utf8.c

This allows repositories to have commit messages in "wrong" encodings, and converts them to UTF-8 upon "git log" output. I think it is much nicer solution than insisting the commit log message to always be in UTF-8, especially with automated conversion when making a commit. If the conversion is botched (or involves non-reversible conversion) at commit time, the information is lost forever, but even if the conversion at the output time fails or loses information, the log message can be recovered in the original encoding, and conversion can later be fixed/improved.

This is not a must-have for v1.5.0, but I feel it would be better to push it out as it can introduce slight backward incompatibility to people's scripts, if they are expecting to get messages in original encodings. If we are going to break them, we would be better off breaking them at a major revision boundary than later.

* sp/mmap (Sun Dec 24 00:47:23 2006 -0500) 20 commits
 - Replace mmap with xmmap, better handling MAP_FAILED.
 - Release pack windows before reporting out of memory.
 - Default core.packdGitWindowSize to 1 MiB if NO_MMAP.
 - Test suite for sliding window mmap implementation.
 - Create pack_report() as a debugging aid.
 - Support unmapping windows on 'temporary' packfiles.
 - Improve error message when packfile mmap fails.
 - Ensure core.packedGitWindowSize cannot be less than 2 pages.
 - Load core configuration in git-verify-pack.
 - Fully activate the sliding window pack access.
 - Unmap individual windows rather than entire files.
 - Document why header parsing won't exceed a window.
 - Loop over pack_windows when inflating/accessing data.
 - Replace use_packed_git with window cursors.
 - Refactor how we open pack files to prepare for multiple windows.
 - Create read_or_die utility routine.
 - Use off_t for index and pack file lengths.
 - Refactor packed_git to prepare for sliding mmap windows.
 - Introduce new config option for mmap limit.
 - Replace unpack_entry_gently with unpack_entry.

I wanted to have this in 'next' but it appears that this makes git-push with a non-trivial amount of data to fail. v1.5.0 does not have to wait for this because this should not change any UI.

All the rest are post v1.5.0; some of them will never be merged.

* jc/3way (Wed Nov 29 18:53:13 2006 -0800) 1 commit
 + git-merge: preserve and merge local changes when doing fast
   forward
* js/shallow (Fri Nov 24 16:00:13 2006 +0100) 15 commits
 + fetch-pack: Do not fetch tags for shallow clones.
 + get_shallow_commits: Avoid memory leak if a commit has been
   reached already.
 + git-fetch: Reset shallow_depth before auto-following tags.
 + upload-pack: Check for NOT_SHALLOW flag before sending a shallow
   to the client.
 + fetch-pack: Properly remove the shallow file when it becomes
   empty.
 + shallow clone: unparse and reparse an unshallowed commit
 + Why didn't we mark want_obj as ~UNINTERESTING in the old code?
 + Why does it mean we do not have to register shallow if we have
   one?
 + We should make sure that the protocol is still extensible.
 + add tests for shallow stuff
 + Shallow clone: do not ignore shallowness when following tags
 + allow deepening of a shallow repository
 + allow cloning a repository "shallowly"
 + support fetching into a shallow repository
 + upload-pack: no longer call rev-list

* jc/diff-apply-patch (Fri Sep 22 16:17:58 2006 -0700) 1 commit
 + git-diff/git-apply: make diff output a bit friendlier to GNU patch
   (part 2)

* jc/diff (Mon Dec 25 01:08:50 2006 -0800) 2 commits
 - test-para: combined diff between HEAD, index and working tree.
 - para-walk: walk n trees, index and working tree in parallel
* jc/explain (Mon Dec 4 19:35:04 2006 -0800) 1 commit
 - git-explain
* jn/web (Sat Dec 16 17:12:55 2006 +0100) 1 commit
 - gitweb: Add some mod_perl specific support
* jc/web (Wed Nov 8 14:54:09 2006 -0800) 1 commit
 - gitweb: steal loadavg throttle from kernel.org
* jc/pickaxe (Sun Nov 5 11:52:43 2006 -0800) 1 commit
 - blame: --show-stats for easier optimization work.

Junio C Hamano <junkio@cox.net> wrote:

Really? Not good. Do you have some sort of test case that has caused this? I'll try to reproduce it here on my own.

Shawn Pearce <spearce@spearce.org> wrote:

I've pushed all of git.git into an empty repository, and then pushed another 1639 objects on top of that, without any errors. So I can't seem to reproduce the problem here in my Mac OS X system.

BTW, I do agree that sp/mmap should not be in v1.5.0.

Junio C Hamano wrote:

I'd still like the preserving reflogged data during pruning and repacking to be optional (default to on). But failing that I'd like to have option to "reflog expire" to remove only specific (pattern match, prefix match?) entries, for example to remove all the "commit --amend" and StGIT work, but leaving rebases, resets, merges and other stuff.

I'm about to send improved series of patches, few first ready to be applied, the rest for review.

Jakub Narebski <jnareb@gmail.com> writes:

What does that really _mean_?

Some time after reflog was introduced, you started talking about it on #git channel saying "you could refer to master@{yesterday} to get it back, but only unless you pruned".

I think that behaviour was a bug, which was there only because reflog was bolted onto the plumbing as an afterthought. If the data reflog keeps is integral part of git (and in v1.5.0 we are making it so by turning it on by default), you should not have to say "but only unless" part, which indicates loss of useful information.

So you either keep objects that reflog entries need, or you discard reflog entries that point at them when you discard certain objects you do not need anymore. Removing objects without removing reflog entries that become stale because of missing objects takes us back to the "bolted on, not really integrated" situation. It makes reflog "sometimes work but only sometimes", and most likely it does not work when the user most wants it to.

With the current code in 'next', you can discard unwanted reflog entries using "reflog expire", and then do prune/repack, and can be assured that the objects that remaining reflog entries want are still there (modulo bugs -- at least that is the intent). You could update prune/repack so that they do not look at reflog data, and have them also remove reflog entries that have become stale after they do their primary things (either pruning or repacking), as part of the same invocation. I will not code that for you because I do not think it is an improvement over what we have now (for one thing, you cannot prune selectively, as I already said).

I would say "Patches welcome"; this is not from my sarcastic side but because I suspect there might be a remote chance that it might turn out to be useful in some situation. It is just that my suspicion is not strong enough to tempt me to do so myself, because I think that even with such an elaborate selection mechanism in place, what the users would do in practice would boil down to what --expire-unreachable does.