git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: [PATCH 2/2] Add keyword unexpansion support to convert.c

From
Nicolas Pitre <nico@cam.org>
Date
Apr 18, 2007, 02:39 UTC
Message-ID
<alpine.LFD.0.98.0704172154160.4504@xanadu.home>
In-Reply-To
<7vy7kqlj5r.fsf@assigned-by-dhcp.cox.net>
On Tue, 17 Apr 2007, Junio C Hamano wrote:
Show 18 quoted lines
> Nicolas Pitre <nico@cam.org> writes:
> 
> >> I would like to, however this doesn't currently integrate
> >> well with git. I've been told in the past that once
> >> .gitattributes is in place then the hooks for the crlf stuff
> >> can be generalized to allow for calls out to custom code to
> >> do this sort of thing.
> >
> > And I agree that this is a perfectly sensible thing to do.  The facility 
> > should be there for you to apply any kind of transformation with 
> > external tools on data going in or out from Git.  There are good and bad 
> > things you can do with such a facility, but at least it becomes your 
> > responsibility to screw^H^H^H^Hfilter your data and not something that 
> > is enforced by Git itself.
> 
> You have to be careful, though.  Depending on what kind of
> transformation you implement with the external tools, you would
> end up having to slow down everything we would do.
So what?  

We provide a rope with proper caveat emptor. Up to others to hang themselves with it if they so desire. It is not our problem anymore.

Show 16 quoted lines
> It boils down to this statement from Andy:
> 
>     ..., keywords (in other VCSs, and so why not in git) are
>     only updated when a file is checked out.  There is no need
>     to touch every file.  It's actually beneficial, because the
>     keyword in the file is the state of the file at the time it
>     was checked in - which is actually more useful than updating
>     it to the latest commit every time.
> 
>     That means you're only ever expanding in a file that your
>     changing anyway - so it's effectively free.  git-checkout
>     would still be immediate and instantaneous.
> 
> Back up a bit and think what "when a file is checked out" means.
> His argument assumes the current behaviour of not checking out
> when the underlying blob objects before munging are the same.

And I think that should remain. If someone really wants full transformation aka keyword expansion or whatever then he'd just need to force a full read-tree after switching branch.

Show 6 quoted lines
> But with keyword expansion and fancier "external tools" whose
> semantics are not well defined (iow, defined to be "do whatever
> they please"), does it still make sense to consider two blobs
> that appear in totally different context "the same" and omit
> checking out (and causing the external tools hook not getting
> run)?

I think so. At least by default. And if we really want to be kind we could provide a special attribute just for disabling such optimization i.e. to force a checkout of everything marked with such an attribute everytime. But we might as well wait to see if someone actually ask about that.

But for many other cases having such a facility would be just nice especially for those cases that don't depend on the branch/commit but only on the content itself. For example it was pointed out that Open Office documents are gzipped XML files. In that case it would be extremely advantageous to have the ability to specify an input filter as "gzip -d" and an output filter as "gzip -c" so Git has a chance to actually perform some kind of delta compression.

I'm sure there might be other type of filters for situation we've not thought about yet, and that we might not want to carry as a builtin feature. Who knows, maybe someone might want to port Git to the System/360 and will need an EBCDIC filter on checked out text files. Or maybe a byte swapping filter for some kind of binary files when on a system with a different endianness.

> I already pointed out to Andy that the branch name the
> file was taken from, if it were to take part of the keyword
> expansion, would come out incorrectly in his printed svg
> drawing.
Tough.
Show 10 quoted lines
> If you want somebody's earlier example of "giving a file with
> embedded keyword to somebody, who modifies and sends the result
> back in full, now you would want to incorporate the change by
> identifying the origin" to work, you would want "$Source$" (I am
> looking at CVS documentation, "Keyword substitution/Keyword
> List") to identify where that file came from (after all, a
> source tree could have duplicated files) so that you can tell
> which file the update is about, and this keyword would expand
> differently depending on where in the project tree the blob
> appears.

Like we don't record renames, I don't think we should record such thing in checked out files either. Using a search for the closest match (like we do for rename detection) is probably a better avenue than trusting that the ID embedded in the file wasn't messed with. Linus already provided a small script that would do that with pretty good results.

Show 17 quoted lines
> It is not just the checkout codepath.  We omit diffs when we
> know from SHA-1 that the blobs are the same before decoration.
> We even omit diffs when we know from SHA-1 that two trees are
> the same without taking possible decorations that can be applied
> differently to the blobs they contain into account.  Earlier,
> Andy said he wanted to grep for the expanded text if he is
> grepping in the working tree, and I think that makes sense, but
> that means git-grep cannot do the same "borrow from working tree
> when expanding from blob object is more expensive" optimization
> we have for diff.  We also need to disable that optimization
> from the diff, regardless of what the correct semantics for
> grepping in working trees should be.
> 
> I suspect that you would have to play safe and say "when
> external tools are involved, we need to disable the existing
> content SHA-1 based optimization for all paths that ask for
> them" to keep your sanity.

Maybe. If that is what's really needed then so be it. People who really want to do strange things will have the flexibility to do so, but they'll have to pay the price in loss of performance.

Nicolas
Previous: Junio C HamanoNext: Junio C Hamano
Message 16 of 66 in “Add keyword unexpansion support to convert.c”
  1. 2/2 Add keyword unexpansion support to convert.cAndy Parkins, Apr 17, 2007
  2. Junio C HamanoApr 17, 2007
  3. Andy ParkinsApr 17, 2007
  4. Linus TorvaldsApr 17, 2007
  5. Andy ParkinsApr 17, 2007
  6. Linus TorvaldsApr 17, 2007
  7. Andy ParkinsApr 17, 2007
  8. Nicolas PitreApr 17, 2007
  9. David LangApr 17, 2007
  10. Nicolas PitreApr 17, 2007
  11. David LangApr 17, 2007
  12. Nicolas PitreApr 17, 2007
  13. David LangApr 17, 2007
  14. Andy ParkinsApr 17, 2007
  15. Junio C HamanoApr 17, 2007
  16. Nicolas PitreApr 18, 2007
  17. Junio C HamanoApr 18, 2007
  18. Nicolas PitreApr 18, 2007
  19. Johannes SchindelinApr 18, 2007
  20. Nicolas PitreApr 18, 2007
  21. Johannes SchindelinApr 19, 2007
  22. David LangApr 21, 2007
  23. Junio C HamanoApr 21, 2007
  24. Nicolas PitreApr 21, 2007
  25. David LangApr 21, 2007
  26. Rogan DawesApr 18, 2007
  27. Linus TorvaldsApr 18, 2007
  28. Nicolas PitreApr 18, 2007
  29. Rogan DawesApr 18, 2007
  30. Nicolas PitreApr 18, 2007
  31. Rogan DawesApr 18, 2007
  32. Alon ZivApr 18, 2007
  33. Linus TorvaldsApr 17, 2007
  34. Andy ParkinsApr 17, 2007
  35. Add keyword collapse support to convert.cAndy Parkins, Apr 17, 2007
  36. Linus TorvaldsApr 17, 2007
  37. Linus TorvaldsApr 17, 2007
  38. Johannes SchindelinApr 18, 2007
  39. Nikolai WeibullApr 20, 2007
  40. Martin LanghoffApr 17, 2007
  41. Junio C HamanoApr 17, 2007
  42. Jakub NarebskiApr 20, 2007
  43. David LangApr 21, 2007
  44. Linus TorvaldsApr 17, 2007
  45. Johannes SixtApr 17, 2007
  46. Linus TorvaldsApr 17, 2007
  47. Andy ParkinsApr 17, 2007
  48. Rogan DawesApr 17, 2007
  49. Linus TorvaldsApr 17, 2007
  50. Rogan DawesApr 17, 2007
  51. Robin H. JohnsonApr 17, 2007
  52. Junio C HamanoApr 18, 2007
  53. J. Bruce FieldsApr 18, 2007
  54. Linus TorvaldsApr 18, 2007
  55. Junio C HamanoApr 18, 2007
  56. Linus TorvaldsApr 18, 2007
  57. Robin H. JohnsonApr 18, 2007
  58. Junio C HamanoApr 18, 2007
  59. Junio C HamanoApr 18, 2007
  60. Robin H. JohnsonApr 18, 2007
  61. Daniel BarkalowApr 18, 2007
  62. Johannes SchindelinApr 18, 2007
  63. Martin LanghoffApr 18, 2007
  64. David KågedalApr 18, 2007
  65. Robin H. JohnsonApr 18, 2007
  66. Matthieu MoyApr 17, 2007

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.