git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: renormalize histroy with smudge/clean-filter

From
Elijah Newren <newren@gmail.com>
Date
Feb 8, 2025, 21:43 UTC
Message-ID
<CABPp-BGwZ029Y8Kfr2kkGiUDZ613kxS81JXzk36V85=77KcYfA@mail.gmail.com>
In-Reply-To
<ba65ce17-8768-4d60-aec6-badd12930b81@gmail.com>
Hi Phillip,
On Sat, Feb 8, 2025 at 3:15 AM Phillip Wood <phillip.wood123@gmail.com> wrote:
Show 37 quoted lines
>
> Hi Elijah and Josef
>
> On 08/02/2025 00:23, Elijah Newren wrote:
> > On Fri, Feb 7, 2025 at 12:34 PM Josef Wolf <jw@raven.inka.de> wrote:
> >> On Fri, Feb 07, 2025 at 06:01:43AM -0800, Elijah Newren wrote:
> >>> On Fri, Feb 7, 2025 at 3:13 AM Chris Torek <chris.torek@gmail.com> wrote:
> >>
> > I also see I didn't look closely enough at Phillip's
> > suggestion, which was:
> >
> >     git rebase --root -x 'git add --renormalize . && { git diff --quiet
> > --cached || git commit --amend --no-edit; }'
> >
> > which will work if you do a lot of manual work to resolve line ending
> > difference conflicts.  Since the git add at each step will modify the
> > files on which the next commit is based, that causes the application
> > of the subsequent commit to conflict,
>
> Indeed, I'd missed that (like you I've not actually used any
> smudge/clean filters)
>
> > and you probably will have
> > difficulty seeing those conflicts since they tend to just be line
> > ending differences.  But, mixing that with Brian's suggestion, you
> > get:
> >
> >    git rebase --root -X renormalize -x 'git add --renormalize . && {
> > git diff --quiet --cached || git commit --amend --no-edit; }'
> >
> > which should probably work if you have a linear history
>
> I've tried that out with a small modification in the script below which
> seems to work. The modification is to add "--attr-source=$(git rev-parse
> HEAD)" between "git" and "rebase" so that git always has a
> .gitattributes file to read when rebasing commits that were made before
> that file was added.

Ooh, nice catch. If folks had an appropriate .gitattributes file in place in older versions of history, they probably wouldn't have gotten into the mess.

> I wonder if we should add something about
> renormalizing a repository to the FAQ based on your footnote.
and perhaps your helpful example?  (although it does assume linear history)  :-)
Show 40 quoted lines
>  > [1] The renormalize option to the merge machinery ensures that new
>  > blobs produced by the merge have normalized content, and avoid
>  > conflicts when the only differences between files are normalization
>  > ones.  This option does not ensure that new trees only reference new
>  > content nor that they only reference normalized content; _any_
>  > pre-existing blobs in the repository are fair game for new trees to
>  > reference.  As per the manual: "renormalize...This runs a virtual
>  > check-out and check-in of all three stages of a file when resolving a
>  > three-way merge..."  So, the existing behavior of the renormalize
>  > option to rebase/cherry-pick/merge is correct.  It may not be what you
>  > want, but I don't think cherry-picking/rebasing/merging with the
>  > renormalize option is the right tool for this job.
>  >
>
> Best Wishes
>
> Phillip
>
> --- >8 ---
> #!/bin/sh
> set -e
> d="$(mktemp -d)"
> cd "$d"
> git init
> echo "The   quick  brown" >file
> git add file
> git commit -m line-1
> echo "fox  jumps    over" >>file
> git commit -a -m line-2
> echo "the      lazy   dog" >>file
> git commit -a -m line-3
> echo "file filter=space" >.gitattributes
> git config filter.space.clean "sed -e 's/  */ /g'"
> git config filter.space.smudge cat
> git add .gitattributes
> git commit -a -m 'add .gitattributes'
> git reset --hard HEAD
> git --attr-source=$(git rev-parse HEAD) rebase --root -X renormalize \
>      -x 'git add --renormalize . && { git diff --cached --quiet || git
> commit --amend --no-edit; }'

So, I'm slightly surprised here. Does the --attr-source specified to the outer git become an environment variable or something for the inner git-add invocation? How does the git add subprocess know about it?

...<does some searches ending with>...

$ git grep -5 GIT_ATTR_SOURCE -- git.c git.c- } else if (!strcmp(cmd, "--attr-source")) { git.c- if (*argc < 2) { git.c- fprintf(stderr, _("no attribute source given for --attr-source\n" )); git.c- usage(git_usage_string); git.c- } git.c: setenv(GIT_ATTR_SOURCE_ENVIRONMENT, (*argv)[1], 1); git.c- if (envchanged) git.c- *envchanged = 1; git.c- (*argv)++; git.c- (*argc)--; git.c- } else if (skip_prefix(cmd, "--attr-source=", &cmd)) { git.c- set_git_attr_source(cmd); git.c: setenv(GIT_ATTR_SOURCE_ENVIRONMENT, cmd, 1); git.c- if (envchanged) git.c- *envchanged = 1; git.c- } else if (!strcmp(cmd, "--no-advice")) { git.c- setenv(GIT_ADVICE_ENVIRONMENT, "0", 1); git.c- if (envchanged)

ahah, so it is passed via environment variable to the subprocess.
Anyway, nice catch.
Previous: Josef WolfNext: Josef Wolf
Message 18 of 44 in “renormalize histroy with smudge/clean-filter”
  1. Josef WolfFeb 5, 2025
  2. brian m. carlsonFeb 5, 2025
  3. Josef WolfFeb 5, 2025
  4. brian m. carlsonFeb 6, 2025
  5. Elijah NewrenFeb 6, 2025
  6. Josef WolfFeb 6, 2025
  7. Josef WolfFeb 6, 2025
  8. Chris TorekFeb 7, 2025
  9. Josef WolfFeb 7, 2025
  10. Torsten BögershausenFeb 7, 2025
  11. Chris TorekFeb 7, 2025
  12. Chris TorekFeb 7, 2025
  13. Elijah NewrenFeb 7, 2025
  14. Josef WolfFeb 7, 2025
  15. Elijah NewrenFeb 8, 2025
  16. Phillip WoodFeb 8, 2025
  17. Josef WolfFeb 8, 2025
  18. Elijah NewrenFeb 8, 2025
  19. Josef WolfFeb 8, 2025
  20. D. Ben KnobleFeb 9, 2025
  21. Josef WolfFeb 9, 2025
  22. Elijah NewrenFeb 9, 2025
  23. Josef WolfFeb 9, 2025
  24. D. Ben KnobleFeb 10, 2025
  25. Josef WolfFeb 8, 2025
  26. Elijah NewrenFeb 8, 2025
  27. Josef WolfFeb 9, 2025
  28. Torsten BögershausenFeb 9, 2025
  29. Josef WolfFeb 9, 2025
  30. Josef WolfFeb 9, 2025
  31. Josef WolfFeb 9, 2025
  32. Josef WolfFeb 7, 2025
  33. Junio C HamanoFeb 7, 2025
  34. Phillip WoodFeb 6, 2025
  35. Elijah NewrenFeb 6, 2025
  36. Junio C HamanoFeb 6, 2025
  37. Josef WolfFeb 11, 2025
  38. Torsten BögershausenFeb 12, 2025
  39. Josef WolfFeb 12, 2025
  40. Collisions while cloning (was: Re: renormalize histroy with smudge/clean-filter, again)Josef Wolf, Feb 13, 2025
  41. Torsten BögershausenFeb 13, 2025
  42. Josef WolfFeb 14, 2025
  43. brian m. carlsonFeb 14, 2025
  44. Josef WolfFeb 14, 2025

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.