[RFH] hackday and GSoC topic suggestions

This Saturday I'm going to be attending a Git hackday held by Bloomberg in New York. The participants will be eager C coders who have experience using git, but not contributing to it. As somebody who has read The Mythical Man Month, I don't expect huge productivity, but I'm hoping to do some bug triage and fixes, and maybe get some people involved who might join the community. I'm hoping some of the folks will have features _they_ want to work on, and I can help get them started. But I'd like to have a list of potential projects to direct people towards as a backup.

On a similar note, the GSoC application deadline is Feb 14th. I am happy to be admin again and put together the application, but we will need an idea page. I'll set up a page to collect them, but in the meantime, please dump any ideas/discussion in this thread.

Below is a list of features / bugs that I am taking to the hackday. The bug list was collected by grepping the mailing list for items without responses. They haven't been triaged at all, so I'm sure some of them are "not a bug" or "won't fix". But the triage process is part of what we'll be doing on Saturday.

I don't think the list below and the potential GSoC list really have any overlap, as the project scales are completely different. But I'd be happy to take suggestions for either.

--- features:

 - negative refspecs
   http://thread.gmane.org/gmane.comp.version-control.git/240997/focus=241019

 - optionally remove tempfiles on failed pack-objects (especially when
   we get ENOSPC)
   http://article.gmane.org/gmane.comp.version-control.git/241466

 - previewing "git pull"
   http://article.gmane.org/gmane.comp.version-control.git/236732

bugs:

 - branch.*.merge interpreted too strictly by tracking logic
   http://article.gmane.org/gmane.comp.version-control.git/241582

 - relative core.worktree is resolved from symlink and not its target
   http://article.gmane.org/gmane.comp.version-control.git/241519

 - Branch rename breaks local downstream branches
   http://article.gmane.org/gmane.comp.version-control.git/241228

 - git clone on out-of-space device causes incorrect errors
   http://article.gmane.org/gmane.comp.version-control.git/241206

 - commit-msg hook and merges
   http://article.gmane.org/gmane.comp.version-control.git/241203

 - inconsistent include behaviour for core.sharedRepository
   http://article.gmane.org/gmane.comp.version-control.git/241277

 - Rebase options '--whitespace=fix' and '--keep-empty' are incompatible
   http://article.gmane.org/gmane.comp.version-control.git/238055

 - git stash doesn't use --index as default
   http://article.gmane.org/gmane.comp.version-control.git/235892

 - git describe --contains --abbrev=0 <sha1> doesn't work as expected
   http://article.gmane.org/gmane.comp.version-control.git/236707

 - using git commit-tree with "-F -" adds trailing newlines
   http://article.gmane.org/gmane.comp.version-control.git/236583

 - Pull and fetch don't honor `--progress` flag
   http://thread.gmane.org/gmane.comp.version-control.git/236257/focus=236262

 - Unexpected outputs of git pull on stdout v.s. stderr
   http://article.gmane.org/gmane.comp.version-control.git/235716

 - MERGE_HEAD lost with git checkout master
   http://article.gmane.org/gmane.comp.version-control.git/233806

 - git stash doesn't always save work dir as-is: bug?
   http://article.gmane.org/gmane.comp.version-control.git/234153

 - Well-past commit dates unsupported
   http://article.gmane.org/gmane.comp.version-control.git/236827

 - we do not handle integer overflow in commit/author timestamps
   [no reference, but something I have noticed; we should probably
   return a sentinel "0" rather than a random overflow value, and
   we should probably detect and warn in git-fsck]

 - git stash does not work when directory is replaced by a symlink to itself
   http://article.gmane.org/gmane.comp.version-control.git/236798

 - rebase not recovering gracefully from repack error
   http://article.gmane.org/gmane.comp.version-control.git/234261

 - (broken ?) output of "git diff --color-word"
   http://article.gmane.org/gmane.comp.version-control.git/237370

 - git filter-branch does not make tree replacements permanent
   http://article.gmane.org/gmane.comp.version-control.git/220931

 - 'git show' gives duplicate errors for ambiguous args
   http://article.gmane.org/gmane.comp.version-control.git/205023

 - (git commit --patch --message $MESSAGE) disallows hunk editing
   http://article.gmane.org/gmane.comp.version-control.git/208213

 - filter-branch --parent-filter in bare repository
   http://article.gmane.org/gmane.comp.version-control.git/203619

insanely hard bugs:

 - shallow clones over http
   http://article.gmane.org/gmane.comp.version-control.git/223682

On Wed, Feb 5, 2014 at 11:57 PM, Jeff King <peff@peff.net> wrote:

Matthieu Moy created this nice page some time ago in the wiki:

https://git.wiki.kernel.org/index.php/SmallProjectsIdeas

and I think there is good stuff in it.

I would be interested in mentoring a GSoC student working on the "git bisect fix/unfixed" feature. Some of Matthieu's students worked on it a few years ago but didn't finish.

Thanks, Christian.

Christian Couder <christian.couder@gmail.com> writes:

Interestingly, I used the feature in real-life last week, and had to think upside-down to type the right "good"/"bad" keywords ;-).

Right. There was still quite some work to do, but this is most likely too small for a GSoC project. But that could be a part of it. I'm not sure how google welcomes GSoC projects made of multiple small tasks, but my experience with students is that it's much better than a single (too) big task, and I think that was the general feeling on this list when we discussed it last year.

On Thu, Feb 06, 2014 at 10:51:54AM +0100, Matthieu Moy wrote:

I think Google leaves it up to us to decide. I'd be OK with a project made of multiple small tasks, as I think it would be an interesting experiment. I'd rather not do all of them like that, though. And bonus points if they are on a theme that will let the student use the ramp-up time from one for another.

-Peff

On Thu, Feb 13, 2014 at 9:50 AM, Jeff King <peff@peff.net> wrote:

Yeah, a student working on the "git bisect fix/unfixed" feature, could fix git bisect testing too many merge bases, and if there is still time work on moving more code from shell to C.

Christian Couder <christian.couder@gmail.com> writes:

In the context of programming tasks, "if there is still time" is a prime candidate for the successful application of branch prediction.

Jeff King <peff@peff.net> writes:

Sorry for being so procrastinative :-(

I suggest we make it a rule that old projects cannot be proposed from year to year.

On Sat, Feb 08, 2014 at 07:43:40PM +0100, Thomas Rast wrote:

I would very much agree with that. There are projects that I have posted multiple years, and nobody ever selects them. I should take that as a hint that they are not appealing to students for whatever reason, and are probably just clogging up the ideas list.

I'd make an exception for somebody who wants to substantially rework the proposal in an attempt to make it more appealing, or who has a reason why it is being re-proposed (e.g., we had student interest, but we ended up not selecting it due to running out of slots or something). But those should be the exception.

-Peff

Jeff King <peff@peff.net> writes:

I have the list below at

  https://github.com/trast/git/wiki/Todo-items

(started at git-merge last year). I did a quick triage, but don't take my word for it. Perhaps it's not too late for your hackathon yet ;-)

Some items may have been fixed by other people when I wasn't looking, so first check if the problem still exists. The ones I knew are done I moved to the bottom.

Easy:

* Add -p 'e' when it fails to apply should offer an obvious way of
  starting from the original hunk (not the broken one) or both

* filter-branch should apply {tree,index}-filter to HEAD and complain if
  it's not invariant (cf. BFG)

* git-config should follow symlinks when rewriting the file, so as to
  avoid replacing a symlink at ~/.gitconfig with the actual contents

* git-send-email should complain when given an option --cc= (i.e., --cc
  with empty argument) instead of silently not using any address
  there. It's easy to accidentally cause this in combination with
  $()-expanding a script that searches for the address by name when
  there are no matches. Note that the cccmd feature should not error if
  it doesn't come up with anything!

Medium difficulty:  
  
* LC_CTYPE=C to work around an old issue in glibc that has since been
  fixed (ask jrn). We should autodetect if we still need the fix, and
  otherwise avoid it; it causes trouble with localized messages

* rebase -i --exec should be able to work elsewhere so you can continue
  doing useful work

* git-send-email could build smarter reference lists if it had
  strategies to fetch the in-reply-to email. One obvious, and very
  useful for git and linux, strategy to do so would be to append the
  References header of the mail obtained from
  http://mid.gmane.org/$in_reply_to (follow the 403 and then append /raw
  to get the headers).

* git-svn should have a diff subcommand, to match svn's output for
  feeding into tools that need the precise format with 'Index:' lines

* Resurrect the move of config var descriptions onto the separate pages
  for commands

  
Hard or unknown difficulty:
 
* Add -p 's' followed by 'e' fails to apply even a no-op edit if the
  context is small

* git-bisect appears to test a linear number of merge bases, thus
  ruining its usually log(n) complexity, in cases involving 'git bisect
  good origin/next', 'git bisect bad origin/pu', in git.git

* I have a script git-pie which runs 'perl -i -pe' as follows:

        #!/bin/sh
        
        cmd=$1
        shift
        
        git ls-files -z "$@" |
        xargs -0 perl -i -pe "$cmd"

  The prompt display (__git_ps1) in the prompt immediately after using
  'git pie ...' always shows that there are no uncommitted changes, even
  if that is not correct. Hitting RET again yields a correct status.

Already underway or done (only here so people don't say "but you dropped this from the wiki list"):

* In-core merge: a clean merge should be able to work fully in memory,
  esp. without any worktree

* For the issue of "what did the evil merge do", should try to flatten
  out the conflicted merge (i.e. mechanically re-merge, and scrap the
  conflicted state in favor of conflict hunks in stage 0) and diff that
  against the merge result. if that works out, store the tree as a note
  somewhere for caching; git-notes should be able to handle that

  * For the last point: need a way of storing trees as
    notes. Technically the notes infrastructure does not require things
    to be blobs, but in practice all the frontends we have error out, or
    do worse things, when handed a tree as a note.

* ./t0000-foo.sh --valgrind should be able to: a) run only a single
  specific test under valgrind, using the normal git for the rest; and
  b) parallelize over (a) so as to speed up a complete ./t0000-foo.sh
  --valgrind run

On Sun, Feb 9, 2014 at 2:03 AM, Thomas Rast <tr@thomasrast.ch> wrote:

If it's too easy, you can add a command to change diff display settings (--color-words, context size...)

Duy Nguyen <pclouds@gmail.com> writes:

If you mean for GSoC, that violates the "previously suggested" rule -- I had it as a project proposal for at least two years and nobody was ever interested.

Jeff King <peff@peff.net> writes:

If you have a branch B that builds on A, if you are renaming A to C, you may want B to automatically set to build on C in some cases, and in other cases your renaming A is done explicitly in order to severe the tie between A and B and setting the latter to build on C can be a bad thing---after all, the user's intention may be to create a branch A starting at some commit immediately after this rename so that B will keep building on that updated A.

So I am not sure if this is a bug.

I tend to think "git stash" was designed to work this way from day one.

According to the initial commit that introduced this option, it deliberately calls strbuf_complete_line() to make sure we do not leave an incomplete line at the end.

Many of the other items in your "bugs" section seem to be worth fixing.

Thanks.

Junio C Hamano <gitster@pobox.com> writes:

Having said that, the current behaviour of leaving B half-configured to build on a missing branch is undesirable. If we were to change this so that any branch B that used to build on branch A being renamed to build on the branch under the new name C, the user may have to do an extra "--set-upstream-to A" on B after recreating A if this was done to save away the current state of A to C and then keep building B on an updated A, so we may have to give _some_ clue what we are doing behind their back when we rename, e.g.

	$ git branch -m A C
        warning: branch B is set to build on C now.

or something.

On Tue, Feb 11, 2014 at 11:38:09AM -0800, Junio C Hamano wrote:

Yeah, I agree. I think most of the time people would want the dependency to move with the branch, and the key is being clear about it so the user can override. I don't think there is a problem with overriding with `--set-upstream-to` after the fact, rather than giving a special option to the move operation.

There was a team working on this at the hackday, and they seemed to make reasonable progress, but I haven't seen the final output yet. Most teams did not finish their projects, or if they did, I didn't get a chance to coach them through the patch submission process. I'll hopefully be revisiting those in the next week or two as they finish them up offline.

My goal is to get them all posting to the list themselves, but I fear I may have to pick up the pieces in some cases.

-Peff

On Tue, Feb 11, 2014 at 10:57:22AM -0800, Junio C Hamano wrote:

Right. The thing that bothers me is the data loss of:

  echo 1 >foo
  git add foo
  echo 2 >foo
  git stash
  git stash pop

The content in "1" is stashed, but gets lost on the apply, and our pop drops the stash.

Yeah, I think this one is a non-bug. I hadn't read it carefully but the "trailing newlines" made me think it was adding multiple extra blank lines. But it looks like it is just the normal strbuf_complete_line, and that is the right thing to be doing (it has _never_ been encouraged to have a commit message that does not have a newline at all, and I think stripspace has enforced that from day one).

As I mentioned elsewhere, I've yet to see the full results of the hackday. Hopefully soon. :)

-Peff