[Q] Mark files for later commit?

Hi folks!

I just hit into an situation where a certain file contains a human written changelog, which is modified (not prepended) for each single commit in the project, is being committed along with the actual patch.

This approach makes it very difficult to backport the commits, since it will always try patching the changelog file as well.

The changelog is only supposed to be committed at new releases, but is kept maintained continuously so stuff won't be forgotten.

So this makes me wonder, is there a way to mark certain files for being committed later on? Which does not automatically get added to the staging area (on "git commit -a" or "git add ." and so on) unless it's specifically mentioned by "git add"?

We've discussed making it generated automatically, but that's not as easy as it may sound.

How would you like a git feature like described above, marking files for later inclusion?

Thanks! -- Daniel

Daniel Nyström <daniel.nystrom@timeterminal.se> writes:

That does not sound like a feature but merely a source of confusion.

So far, "commit -a", "add", "add ." etc have _all_ been a way to tell git to add the current state of the content to the index. What is the point of making it more complex by letting the user say "I am telling you to add everything in the working tree by explicitly saying 'git add .', but I do not really mean it"?

In the meantime, some misguided souls might suggest assume-unchanged, but that is not guaranteed to work for this purpose, so ignore them. This is because assume-unchanged is a promise you make git that you will _not_ change the working tree files, and that promise implies a permission for git to use blob object recorded in the index that corresponds to such a path instead of reading from the working tree files while doing certain operations (such as "git diff") if it is more convenient.

Junio C Hamano venit, vidit, dixit 29.03.2011 08:49:

While I've used assume-unchanged before, it really is a great foot-gun (to shoot yourself into...) to be used only when circumstances force you.

In your case, if you really can't use shortlog or the like, a "light" way of generating a changelog might be using notes. Attach changelog notes to each commit (maybe in refs/notes/changelog rather than the standard). Then, when you need to create the changelog between vX and vY, you can do

git log --pretty="%N%n" vX..vY

(and go wild on the pretty format, of course). Note that you can share notes when you set up refspecs etc in a push/pull based workflow. (I'm not sure how well we support merging/pulling notes refs yet.) This does not work well in a patch/e-mail-based workflow.

Michael

On Tuesday 29 March 2011, Michael J Gruber wrote:

Since v1.7.4, you can use "git notes merge" for merging notes ref. It's not yet integrated with "pull", though, so typically you want to fetch it to a different ref name, and then manually "git notes merge" that into the original notes ref.

E.g. if you want to sync "refs/notes/foo" with remote "bob", you can "git fetch bob refs/notes/foo:refs/notes/bobs_foo", and then merge them with "git notes --ref=foo merge bobs_foo".

...although there have been mailing list discussions on getting notes added to "git format-patch" output (after the "---"), and then picked up again by "git am".

...Johan

Johan Herland venit, vidit, dixit 29.03.2011 09:38:

Yes, but what I meant with "well" was: If I merge regular branches, do the notes get merged automatically? And the answer is "no", and as you explain above, "git notes merge" copes with that (and with merging notes for the same commit).

Longing for that refs/ layout overhaul...

So, here my "not well" was even overly positive ;)

Michael

Junio C Hamano <gitster@pobox.com> writes:

Let's step back a bit, as I think "mark them not to be added" is a classic XY problem.

First let's concentrate on this part: "a certain file contains a human written changelog, which is modified (not prepended) for each single commit in the project, is being committed along with the actual patch."

Doesn't that sound quite similar to our release notes?

So far, I've been updating the file after topics graduate to the 'master' branch, but there is no fundamental reason not to make it a responsibility of the patch contributors to add an entry to the release notes when they make their changes to the system. Then when the topics graduate, the entry that describes the topic will be automatically merged into the release notes of the 'master' branch.

What will happen if we made that the convention of our project? For one thing, I don't have to spend time on coming up with a concise and clear description of the topics (the contributor that worked on the topic should be better equipped to describe it than I am). People may make their own entries in the single release notes file in various places (by convention, the entries are sorted by the name of the command the change affects, except for the ones that have system-wide effects that are described at the top), and merging them might produce many conflicts, though.

But what choice would I have? Obviously I shouldn't be punting on the merges, taking one side of the release notes file as the merge result using "-s ours" or other silly magic and discarding the valuable input from contributors. I _do_ want all the release notes entries from all topics. Merging such a conflict-heavy file and keeping it maintained properly is an important part of the job (iow, "suck it up").

I suspect that using the "union" merge attribute may be a possibility in this case; even though the release notes is not an "append only" file, it is an "insert only" file, and it is a very good candidate for the union merge driver.

Now let's look at this part: "The changelog is only supposed to be committed at new releases, but is kept maintained continuously so stuff won't be forgotten."

You are willing to leave your changes uncommitted in the working tree, and not committing it until the release (perhaps using your Y solution that lets you mark it not to be committed without --force option), is perfectly Ok by you. You are not getting any accident protection benefit from your SCM until the next release point and the intermediate changes between releases you make to this file are not version controlled.

If that is the case, then probably you shouldn't be even updating the "changelog" file constantly to begin with. Instead, how about creating a separate "changelog+" file that is not tracked, and keep its contents maintained continuously so stuff won't be forgotten? When the time to release comes, you can "mv changelog changelog", and commit it.

That would still require you to find a relevant entry in changelog+ and modify the true changelog file if you cherry pick a particular change to an older maintenance track and make a release out of it. Keeping the entries in the changelog, and cherry picking the changes together with the associated entries will probably reduce that "remembering" mental burden, while it may require trivial conflict resolutions in the file that can be helped with the union merge driver.

Just a few cents.

Den 29 mars 2011 09:51 skrev Junio C Hamano <gitster@pobox.com>:

If we ignore this particular case, how would a "git hold <file>..." feature do? -- $ git status # On branch master # Changed but not updated: # (use "git add <file>..." to update what will be committed) # (use "git checkout -- <file>..." to discard changes in working directory) # # modified: package/qt/qt.mk # # Changed but on hold: # (use "git add --holds <file>..." to update what will be committed) # # modified: CHANGES # no changes added to commit (use "git add" and/or "git commit -a") -- Would there be other use-cases?

Daniel Nyström venit, vidit, dixit 29.03.2011 11:18:

As Junio pointed out, your "hold file" (ChangeLog+) is really not versioned (tracked) at all, so it has no place in the worktree. Otherwise you'll have constant nagging during the release cycle one way or the other, if you want git to remind you of the files on hold. If you don't need the reminder, git does not need to know about the file. Simply store it somewhere else (such as in .git/description if you don't use that, or under .git/info/).

Michael

Den 29 mars 2011 14:01 skrev Michael J Gruber <git@drmicha.warpmail.net>:

Good points, indeed. But untracked files will be tracked on "git add ." which files on hold won't.

But I agree, the "hold" feature requires a better motivation and more actual use cases. I thought it may catch some interests on the list.