git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: metastore

From
Daniel Barkalow <barkalow@iabervon.org>
Date
Sep 16, 2007, 22:02 UTC
Message-ID
<Pine.LNX.4.64.0709161715090.5298@iabervon.org>
In-Reply-To
<Pine.LNX.4.64.0709161346150.24221@asgard.lang.hm>
On Sun, 16 Sep 2007, david@lang.hm wrote:
Show 40 quoted lines
> On Sun, 16 Sep 2007, Daniel Barkalow wrote:
> 
> > > I however think your idea to have extra "permission information
> > > file" is very interesting.  What would be more palatable, than
> > > mucking with the core level git, would be to have an external
> > > command that takes two tree object names that tells it what the
> > > old and new trees our work tree is switching between, and have
> > > that command to:
> > >
> > >  - inspect the diff-tree output to find out what were checked
> > >    out and might need their permission information tweaked;
> > >
> > >  - inspect the differences between the "permission information
> > >    file" in these trees to find out what were _not_ checked out,
> > >    but still need their permission information tweaked.
> > >
> > >  - tweak whatever external information you are interested in
> > >    expressing in your "permission information file" in the work
> > >    tree for the paths it discovered in the above two steps.
> > >    This step may involve actions specific to projects and call
> > >    hook scripts with <path, info from "permission information
> > >    file" for that path> tuples to carry out the actual tweaking.
> >
> > Why not have the command also responsible for creating the files that need
> > to be created (calling back into git to read their contents)? That way,
> > there's no window where they've been created without their metadata, and
> > there's more that the core git doesn't have to worry about.
> 
> my initial thoughts were to have git do all it's normal work and hook into git
> at the point where it's writing the file out (where today it chooses between
> writing the data to a file on disk, pipeing to stdout, or pipeing to a pager)
> by adding the option to pipe into a different program that would deal with the
> permission stuff. this program would only have to write the file and set the
> permissions, it wouldn't have to know anything about git other then where to
> find the permissions it needs to know.
> 
> it sounds like you are suggesting that the hook be much earlier in the
> process, and instead of one copy of git running and calling many copies of the
> writing program, you would have one copy of the writing program that would
> call many copies of git.

A lot of the git commands are actually currently shell scripts that call back to git, so that's not too different. The reason to have a single copy of the writing program is that it would be able to get the whole set of differences that need to be handled, and first pick out the metadata file, process it to figure out the writing instructions once, figure out the changes in the writing instructions, and figure out the changes in the content, and decide what to do.

Show 11 quoted lines
> > I could see the program getting the index, the target tree, and the
> > directory to put files in, and being told to do the whole 2-way merge
> > (except, perhaps, updating the index to match the tree, which git could do
> > afterwards). As far as git would be concerned, it would mostly be like a
> > bare repository.
> 
> if this functionality does shift to earlier in the process, how much of the
> git logic needs to be duplicated in this program?
> 
> if this program needs to do the merge, won't it have to duplicate the merge
> logic, including the .gitattributes checking for custom merge calls?

This is two-way merge, not three-way merge. The basic concept is that you're in state A, and you want to be in state B. Rather than writing out all of state B, you write out all of state B that's different from state A. Think of taking a diff of two big trees and then applying it as a patch, instead of copying the new tree onto the old tree; the benefit is that stuff that doesn't change doesn't get rewritten, and the diff is blazingly fast, given how we store our information.

3-way merge will be handled by git, and not in a live /etc directory anyway (that is, you'd want to fix up the metadata files as plain text files, not as metadata bits on a checked out directory; otherwise, you'll be trying to put conflict markers in mode bits, and that's clearly not what you want).

Show 21 quoted lines
> I have been thinking primarily in terms of doing a complete checkout,
> overwriting all files, and secondarily how do do a checkout of just a few
> files, but again where all files selected overwrite the existing files.
> 
> I wasn't thinking of the fact that git optimizes the checkout and avoids
> writing a file that didn't change.
> 
> this changes things slightly
> 
> prior to this I was thinking that the permission file needed to be handled
> differently becouse writing it out needed to avoid doing any circular
> refrences where you would need to check the contents of it to write it out.
> 
> it now appears as if what really needs to happen is that if the permission
> file changes a different program needs to be called when it's written out then
> when the other files are written out. by itself this isn't hard as
> .gitattributes can have a special entry for this filename and that entry can
> specify a different program, and that program fixes all the permissions
> (and/or detects that they can't be fixed due to user/filesystem limits,
> records the error, checks if the repository is set appropriately, and screams
> to the user if it isn't)

While we're at it, you probably don't even want to write the permission file to the live filesystem. It's just one more thing that could leak information, and changes to the permissions of files that you record by committing the live filesystem would presumably be done by changing the permissions of files in the filesystem, not by changing the text file.

(Of course, you could check out the same commits as ordinary source, with developer-owned 644 files and a 644 "permissions" file, and there you'd have the permissions file appear in the work tree, and you could edit it and check it in in a totally mundane way.)

Show 8 quoted lines
> it would be a nice optimization to this permission checkout for it to compare
> the old and the new permissions so that it only tries to change the
> permissions where it needs to, but is that really nessasary? the program can
> look at the permissions of the existing files to see what they are and decide
> if it needs to change them (this would tromp on local changes that aren't
> checked in. how big of a problem is this?) my initial reaction is that having
> to know the two commits and do the comparison between them is adding a lot of
> logic and git interaction that I'd rather avoid if I could.

You probably want to be able to keep local uncommitted changes. People like to be able to have things slightly different in their particular deployment from the way things are in the repository, for stuff that only applies to one system and isn't "how it should be".

	-Daniel
*This .sig left intentionally blank*
Previous: david@lang.hmNext: david@lang.hm
Message 28 of 72 in “Re: Track /etc directory using Git”
  1. Thomas Harning Jr.Sep 14, 2007
  2. Nicolas VilzSep 14, 2007
  3. metastore (was: Track /etc directory using Git)martin f krafft, Sep 15, 2007
  4. Johannes SchindelinSep 15, 2007
  5. David KastrupSep 15, 2007
  6. martin f krafftSep 15, 2007
  7. Grzegorz KulewskiSep 15, 2007
  8. Johannes SchindelinSep 15, 2007
  9. Randal L. SchwartzSep 15, 2007
  10. david@lang.hmSep 16, 2007
  11. Randal L. SchwartzSep 16, 2007
  12. david@lang.hmSep 16, 2007
  13. Francis MoreauSep 17, 2007
  14. Randal L. SchwartzSep 17, 2007
  15. Daniel BarkalowSep 15, 2007
  16. Johannes SchindelinSep 15, 2007
  17. david@lang.hmSep 16, 2007
  18. Johannes SchindelinSep 16, 2007
  19. david@lang.hmSep 16, 2007
  20. Junio C HamanoSep 16, 2007
  21. David KastrupSep 16, 2007
  22. david@lang.hmSep 16, 2007
  23. Daniel BarkalowSep 16, 2007
  24. david@lang.hmSep 16, 2007
  25. Junio C HamanoSep 16, 2007
  26. Daniel BarkalowSep 16, 2007
  27. david@lang.hmSep 16, 2007
  28. Daniel BarkalowSep 16, 2007
  29. david@lang.hmSep 16, 2007
  30. martin f krafftSep 17, 2007
  31. david@lang.hmSep 17, 2007
  32. Josh EnglandSep 17, 2007
  33. david@lang.hmSep 16, 2007
  34. Junio C HamanoSep 16, 2007
  35. david@lang.hmSep 16, 2007
  36. Junio C HamanoSep 17, 2007
  37. david@lang.hmSep 17, 2007
  38. Junio C HamanoSep 17, 2007
  39. david@lang.hmSep 17, 2007
  40. Junio C HamanoSep 17, 2007
  41. Daniel BarkalowSep 17, 2007
  42. Junio C HamanoSep 17, 2007
  43. Jan HudecSep 16, 2007
  44. david@lang.hmSep 16, 2007
  45. martin f krafftSep 16, 2007
  46. Jan HudecSep 16, 2007
  47. david@lang.hmSep 16, 2007
  48. martin f krafftSep 17, 2007
  49. david@lang.hmSep 16, 2007
  50. martin f krafftSep 16, 2007
  51. David HärdemanSep 19, 2007
  52. martin f krafftOct 2, 2007
  53. David HärdemanOct 2, 2007
  54. David KastrupOct 2, 2007
  55. david@lang.hmOct 2, 2007
  56. martin f krafftOct 2, 2007
  57. david@lang.hmOct 2, 2007
  58. martin f krafftOct 2, 2007
  59. david@lang.hmOct 2, 2007
  60. martin f krafftOct 2, 2007
  61. David HärdemanOct 2, 2007
  62. martin f krafftOct 2, 2007
  63. Julian PhillipsOct 2, 2007
  64. david@lang.hmOct 3, 2007
  65. Johannes SchindelinOct 3, 2007
  66. Daniel BarkalowOct 2, 2007
  67. Pierre HabouzitSep 15, 2007
  68. martin f krafftSep 15, 2007
  69. Pierre HabouzitSep 15, 2007
  70. martin f krafftSep 15, 2007
  71. martin f krafftSep 15, 2007
  72. David KastrupSep 15, 2007

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.