inexact rename detection warning on "git log -p"

Matthew Walster· Apr 13, 2015, 10:47 UTC · lore

Out of idle curiosity, I cloned git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git and chose to generate some statistics:

$ find -not -iname '.git' | wc
  52154   52154 1820305

That's a few files...

$ git log | wc 8359290 37279781 334525133

Hmmm, well that's not too useful, let's see how many commits there are:

$ git log | grep ^commit | wc
 507296 1014592 24350208

and merges:

$ git log | grep ^Merge: | wc
  36019  111146  854397

I wonder what the total size of all changes are:

$ git log --patch | wc warning: inexact rename detection was skipped due to too many files. warning: you may want to set your diff.renameLimit variable to at least 779 and retry the command. 93295818 366207219 3072072294

My bug report is in reference to the warning: It's not clear whether this warning just "warns" the user (hence the warning rather than an error) or whether it actually had an fault and didn't complete the operation successfully. It's also not clear whether the detection was aborted halfway through, or whether the work it has so far done has been discarded and is therefore idempotent if I *lowered* the renameLimit value (as opposed to raising it to fix the warning).

Could the wording be changed to indicate that the operation continued without the rename detection? My preferred wording would be:

warning: inexact rename detection was skipped due to too many files. warning: you may want to set your diff.renameLimit variable to at least 779 and retry the command. warning: continuing processing without inexact rename detection

I'm a native English speaker and even I found the wording a little confusing -- although people used to gcc will know warning is informational and error is abortive, not all git users are aware of the distinction.

Additionally, is the renameLimit something that could potentially be something dynamic (i.e. it inspects how much free RAM is available and increases the buffer if appropriate) or is that a conscious decision not to give the process "free reign"?

Cheers,

Matthew Walster

Michael J Gruber· Apr 13, 2015, 15:11 UTC · re: Matthew Walster · lore

Re: inexact rename detection warning on "git log -p"

Matthew Walster venit, vidit, dixit 13.04.2015 12:47:

Show 6 quoted lines

> Out of idle curiosity, I cloned 
> git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git and chose 
> to generate some statistics:
> 
> $ find -not -iname '.git' | wc
>   52154   52154 1820305

git ls-files | wc -l

> That's a few files...
> 
> $ git log | wc
> 8359290 37279781 334525133

??

> Hmmm, well that's not too useful, let's see how many commits there are:
> 
> $ git log | grep ^commit | wc
>  507296 1014592 24350208

git rev-list --count HEAD

> and merges:
> 
> $ git log | grep ^Merge: | wc
>   36019  111146  854397

git rev-list --count --merges HEAD

I wonder why people are so found of wc.

> I wonder what the total size of all changes are:
> 
> $ git log --patch | wc

That counts quite a bit more than just patch lines: Boiler plate, log messages, diff headers.

Show 24 quoted lines

> warning: inexact rename detection was skipped due to too many files.
> warning: you may want to set your diff.renameLimit variable to at least 779 
> and retry the command.
> 93295818 366207219 3072072294
> 
> My bug report is in reference to the warning: It's not clear whether this 
> warning just "warns" the user (hence the warning rather than an error) or 
> whether it actually had an fault and didn't complete the operation 
> successfully. It's also not clear whether the detection was aborted halfway 
> through, or whether the work it has so far done has been discarded and is 
> therefore idempotent if I *lowered* the renameLimit value (as opposed to 
> raising it to fix the warning).
> 
> Could the wording be changed to indicate that the operation continued 
> without the rename detection? My preferred wording would be:
> 
> warning: inexact rename detection was skipped due to too many files.
> warning: you may want to set your diff.renameLimit variable to at least 779 
> and retry the command.
> warning: continuing processing without inexact rename detection
> 
> I'm a native English speaker and even I found the wording a little confusing 
> -- although people used to gcc will know warning is informational and error 
> is abortive, not all git users are aware of the distinction.

"was skipped" and "without foo" sound pretty similar to me. But what's worse: "continue processing without foo" creates the impression that all further processing is done without "foo", which is a bit misleading: in fact, foo was skipped for at least one diff. So I think the original is closer to the truth.

> Additionally, is the renameLimit something that could potentially be 
> something dynamic (i.e. it inspects how much free RAM is available and 
> increases the buffer if appropriate) or is that a conscious decision not to 
> give the process "free reign"?

I think it's more about time than memory. Note that this is quadratic in the number of files.

If you want to take a shot at an automatic setting: grepping for "rename_limit" (especially "needed_rename_limit" and "diff_rename_limit_default") will give you some pointers.

Michael