git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: [PATCH v2 7/8] verify_path(): disallow symlinks in .gitattributes and .gitignore

From
Jonathan Nieder <jrnieder@gmail.com>
Date
Oct 27, 2020, 23:43 UTC
Message-ID
<20201027234309.GA1298045@google.com>
In-Reply-To
<20201027075853.GH3005508@coredump.intra.peff.net>
Jeff King wrote:
> On Mon, Oct 26, 2020 at 08:35:18PM -0700, Jonathan Nieder wrote:
Show 15 quoted lines
>> Observations:
>>
>> - since some widely used repositories have .gitignore symlinks, I
>>   think we can't forbid it in fsck, alas
>
> I am a little puzzled here. You said you had the fsck checks for the
> last year, so why did they just come up now? I guess nobody sets
> transfer.fsckObjects, and because you were testing only with clients,
> your server implementation didn't reject pushes?
>
> I agree it's annoying for them if they fail fsck, but it's not entirely
> a show-stopper. There are config options for fine-tuning what you're
> willing to enforce or ignore. But they of course are also annoying to
> use, because every receiver of a transfer needs to set them (on GitHub,
> for example, you have to email Support).

Yes. And to reiterate the point a little: the reason nobody sets transfer.fsckObjects is that we haven't made it easy to distinguish between "hard error, should never be overridden" checks (like BAD_PARENT_SHA1), "new tools shouldn't write these but they exist in important repos like perl.git and anything consuming Git repositories needs to cope with them" (like MISSING_SPACE_BEFORE_DATE from some commits' concatenated authors), and so on.

> So I won't be too devastated to remove the symlink checks, or possibly
> downgrade them to purely warnings (or "info"; the naming in fsck.c is
> confusing, because the transfer operations take even warnings as fatal.
> I suspect we could do with some cleanup there).

Downgrading the .gitignore check to warning sounds okay. .gitmodules would still want to be an error, of course.

Show 11 quoted lines
>> - it would be useful to be able to check whether these symlinks would
>>   not escape the worktree, for a more targeted check.  It might be
>>   nice to even respect these settings when they would not escape the
>>   worktree, but not necessarily
>
> I actually wrote a patch several years ago for checking symlinks (not
> just these ones, but _any_ symlinks in the repo, but of course it would
> be easy to limit it more). It's included at the end of this mail. It's
> been part of my daily build for many years, so I'm confident it doesn't
> crash or have other bad behavior. But it's possible the logic for what
> it catches is faulty.
[...]
> Here's the patch.
Nice!  I'll try to find time to experiment with this.
[...]
Show 11 quoted lines
>> - we could use a clearer error message than "invalid path".
>
> That part is tricky. The "invalid path" error comes from the caller of
> verify_path(), and we have no way to pass back an intelligent error
> there. We can call error() ourselves, of course. That adds an extra line
> of output, but it's rare enough for verify_path() to fail that it's
> likely OK. However, I would worry that some callers might be surprised
> by it producing output at all.
>
> An alternative is letting the caller pass in a strbuf that we fill out
> with an extra error string.

I think passing in a struct to govern error handling sounds nice. Alternatively, this might suggest that verify_path is not the right place for the user-facing check: it's useful because it's exhaustive, but it may be that we can also catch the same issue earlier and produce a nicer diagnostic.

Show 17 quoted lines
>> There's some room for improvement in "git checkout"'s error handling,
>> too --- I think my ideal would be if the operation would fail
>> entirely, with an advice message describing a checkout command that
>> would succeed (But how do I checkout another commit while excluding
>> some files? Should it suggest a sparse checkout?).
>
> I suspect it's too late for "fail entirely". We may have already written
> to the filesystem, and rolling back is difficult and error-prone. In
> general I'd expect to checkout what we can, produce errors for the rest,
> and let the user work from there with "git status".
>
> But I may be wrong. The problem is loading the value into the index, not
> writing it to the filesystem. So perhaps the relevant code paths load
> the index fully before writing out anything to the filesystem, and it's
> easy to rollback. But I imagine they are using unpack-trees' flag to
> update the filesystem, and I assume that checks out as it loads entries
> (but I didn't confirm).

Right, this would require taking two passes. I think we do already perform two passes (first deletions, then additions); I'll have to look a little more closely to see whether this is straightforward to do.

I do still like the idea of not respecting .gitignore and .gitattributes symlinks, by the way. What the experiences upthread tell me is just that users need better facilities for repairing existing repositories.

Thanks, Jonathan

Previous: Jeff KingNext: Junio C Hamano
Message 41 of 55 in “forbidding symlinked .gitattributes and .gitignore”
  1. 0/7 forbidding symlinked .gitattributes and .gitignoreJeff King, Oct 5, 2020
  2. 1/7 fsck_tree(): fix shadowed variableJeff King, Oct 5, 2020
  3. Jonathan NiederOct 5, 2020
  4. Jeff KingOct 5, 2020
  5. Jonathan NiederOct 5, 2020
  6. 2/7 fsck_tree(): wrap some long linesJeff King, Oct 5, 2020
  7. Jonathan NiederOct 5, 2020
  8. 3/7 t7415: rename to expand scopeJeff King, Oct 5, 2020
  9. Jonathan NiederOct 5, 2020
  10. Jeff KingOct 5, 2020
  11. Jonathan NiederOct 5, 2020
  12. Jeff KingOct 5, 2020
  13. 4/7 t7450: test verify_path() handling of gitmodulesJeff King, Oct 5, 2020
  14. Jonathan NiederOct 5, 2020
  15. Jeff KingOct 5, 2020
  16. Jonathan NiederOct 5, 2020
  17. 5/7 t0060: test obscured .gitattributes and .gitignore matchingJeff King, Oct 5, 2020
  18. Jonathan NiederOct 5, 2020
  19. Jeff KingOct 5, 2020
  20. Johannes SchindelinOct 5, 2020
  21. Jeff KingOct 6, 2020
  22. 6/7 verify_path(): disallow symlinks in .gitattributes and .gitignoreJeff King, Oct 5, 2020
  23. Jonathan NiederOct 5, 2020
  24. Jeff KingOct 5, 2020
  25. 7/7 fsck: complain when .gitattributes or .gitignore is a symlinkJeff King, Oct 5, 2020
  26. Jonathan NiederOct 5, 2020
  27. Jeff KingOct 5, 2020
  28. Jonathan NiederOct 5, 2020
  29. Jeff KingOct 5, 2020
  30. 0/8 forbidding symlinked .gitattributes and .gitignoreJeff King, Oct 5, 2020
  31. 1/8 fsck_tree(): fix shadowed variableJeff King, Oct 5, 2020
  32. 2/8 fsck_tree(): wrap some long linesJeff King, Oct 5, 2020
  33. 3/8 t7415: rename to expand scopeJeff King, Oct 5, 2020
  34. 4/8 t7450: test verify_path() handling of gitmodulesJeff King, Oct 5, 2020
  35. 5/8 t7450: test .gitmodules symlink matching against obscured namesJeff King, Oct 5, 2020
  36. 7/8 verify_path(): disallow symlinks in .gitattributes and .gitignoreJeff King, Oct 5, 2020
  37. Jonathan NiederOct 27, 2020
  38. Jeff KingOct 27, 2020
  39. Junio C HamanoOct 27, 2020
  40. Jeff KingOct 28, 2020
  41. Jonathan NiederOct 27, 2020
  42. Junio C HamanoOct 28, 2020
  43. 6/8 t0060: test obscured .gitattributes and .gitignore matchingJeff King, Oct 5, 2020
  44. 8/8 fsck: complain when .gitattributes or .gitignore is a symlinkJeff King, Oct 5, 2020
  45. Junio C HamanoOct 6, 2020
  46. Philip OakleyOct 20, 2020
  47. documentation symlink restrictions for .git* filesJeff King, Oct 23, 2020
  48. Jeff KingOct 23, 2020
  49. Philip OakleyOct 26, 2020
  50. Jeff KingOct 26, 2020
  51. Junio C HamanoOct 26, 2020
  52. Jeff KingOct 27, 2020
  53. Junio C HamanoOct 27, 2020
  54. Philip OakleyOct 27, 2020
  55. Junio C HamanoOct 28, 2020

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.