git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: [PATCH 2/6] Accept object data in the fsck_object() function

From
Jeff King <peff@peff.net>
Date
Aug 29, 2014, 23:10 UTC
Message-ID
<20140829231034.GD24834@peff.net>
In-Reply-To
<xmqqvbpc8hon.fsf@gitster.dls.corp.google.com>
On Thu, Aug 28, 2014 at 01:47:52PM -0700, Junio C Hamano wrote:
Show 23 quoted lines
> Johannes Schindelin <johannes.schindelin@gmx.de> writes:
> 
> > When fsck'ing an incoming pack, we need to fsck objects that cannot be
> > read via read_sha1_file() because they are not local yet (and might even
> > be rejected if transfer.fsckobjects is set to 'true').
> >
> > For commits, there is a hack in place: we basically cache commit
> > objects' buffers anyway, but the same is not true, say, for tag objects.
> >
> > By refactoring fsck_object() to take the object buffer and size as
> > optional arguments -- optional, because we still fall back to the
> > previous method to look at the cached commit objects if the caller
> > passes NULL -- we prepare the machinery for the upcoming handling of tag
> > objects.
> >
> > The assumption that such buffers are inherently NUL terminated is now
> > wrong, of course, hence we pass the size of the buffer so that we can
> > add a sanity check later, to prevent running past the end of the buffer.
> 
> A nice side effect may be that we can now check (and perhaps warn) a
> commit buffer with a NUL inside, perhaps?   I am not suggesting to
> add such a check to this series, but mentioning the possibilty here
> may have a merit.

I think that is a good check to add at some point. I demonstrated quite a while ago that you can get up to some mischief by "hiding" bytes after a NUL in the commit message. If we ever see feasible collision attacks against sha1, this is going to be an obvious vector for hiding random bytes to achieve the collisions.

The downside is that git's data model in theory promises to store arbitrary bytes in people's commit messages. IMHO we have gotten far enough from that in practice that I do not think anybody is seriously doing it (you could not really use most of the history inspection tools in a reasonable way; you would have to write a parallel set of tools that never shows log messages, and then extracts and does something separate with the commit content).

-Peff
Previous: Junio C HamanoNext: Jeff King
Message 6 of 68 in “Improve tag checking in fsck and with transfer.fsckobjects”
  1. 0/6 Improve tag checking in fsck and with transfer.fsckobjectsJohannes Schindelin, Aug 28, 2014
  2. 1/6 Refactor type_from_string() to avoid die()ing in case of errorsJohannes Schindelin, Aug 28, 2014
  3. Junio C HamanoAug 28, 2014
  4. 2/6 Accept object data in the fsck_object() functionJohannes Schindelin, Aug 28, 2014
  5. Junio C HamanoAug 28, 2014
  6. Jeff KingAug 29, 2014
  7. Jeff KingAug 29, 2014
  8. 3/6 Make sure fsck_commit_buffer() does not run out of the bufferJohannes Schindelin, Aug 28, 2014
  9. Junio C HamanoAug 28, 2014
  10. Jeff KingAug 29, 2014
  11. 4/6 fsck: check tag objects' headersJohannes Schindelin, Aug 28, 2014
  12. Junio C HamanoAug 28, 2014
  13. Junio C HamanoAug 28, 2014
  14. Jeff KingAug 29, 2014
  15. Junio C HamanoAug 31, 2014
  16. Jeff KingSep 3, 2014
  17. Junio C HamanoSep 3, 2014
  18. Jeff KingSep 4, 2014
  19. Jeff KingAug 29, 2014
  20. Junio C HamanoSep 2, 2014
  21. Jeff KingSep 3, 2014
  22. 5/6 Add regression tests for stricter tag fsck'ingJohannes Schindelin, Aug 28, 2014
  23. 6/6 Make sure that index-pack --strict fails upon invalid tag objectsJohannes Schindelin, Aug 28, 2014
  24. 0/6 Improve tag checking in fsck and with transfer.fsckobjectsJohannes Schindelin, Sep 10, 2014
  25. Johannes SchindelinSep 10, 2014
  26. Junio C HamanoSep 10, 2014
  27. Junio C HamanoSep 10, 2014
  28. Johannes SchindelinSep 11, 2014
  29. 0/6 Improve tag checking in fsck and with transfer.fsckobjectsJohannes Schindelin, Sep 11, 2014
  30. 1/6 Refactor type_from_string() to avoid die()ing in case of errorsJohannes Schindelin, Sep 11, 2014
  31. 2/6 Accept object data in the fsck_object() functionJohannes Schindelin, Sep 11, 2014
  32. 3/6 Make sure fsck_commit_buffer() does not run out of the bufferJohannes Schindelin, Sep 11, 2014
  33. 4/6 fsck: check tag objects' headersJohannes Schindelin, Sep 11, 2014
  34. 5/6 Add regression tests for stricter tag fsck'ingJohannes Schindelin, Sep 11, 2014
  35. 6/6 Make sure that index-pack --strict checks tag objectsJohannes Schindelin, Sep 11, 2014
  36. Junio C HamanoSep 11, 2014
  37. Junio C HamanoSep 11, 2014
  38. 0/3 hash-object --literallyJunio C Hamano, Sep 11, 2014
  39. 1/3 hash-object: reduce file-scope staticsJunio C Hamano, Sep 11, 2014
  40. 2/3 hash-object: pass 'write_object' as a flagJunio C Hamano, Sep 11, 2014
  41. 3/3 hash-object: add --literally optionJunio C Hamano, Sep 11, 2014
  42. Johannes SchindelinSep 12, 2014
  43. 0/6 Improve tag checking in fsck and with transfer.fsckobjectsJohannes Schindelin, Sep 12, 2014
  44. 1/6 Refactor type_from_string() to avoid die()ing in case of errorsJohannes Schindelin, Sep 12, 2014
  45. 2/6 Accept object data in the fsck_object() functionJohannes Schindelin, Sep 12, 2014
  46. 3/6 Make sure fsck_commit_buffer() does not run out of the bufferJohannes Schindelin, Sep 12, 2014
  47. 4/6 fsck: check tag objects' headersJohannes Schindelin, Sep 12, 2014
  48. 5/6 Add regression tests for stricter tag fsck'ingJohannes Schindelin, Sep 12, 2014
  49. 6/6 Make sure that index-pack --strict checks tag objectsJohannes Schindelin, Sep 12, 2014
  50. Junio C HamanoSep 12, 2014
  51. Johannes SchindelinSep 13, 2014
  52. 1/6 Refactor type_from_string() to avoid die()ing in case of errorsJohannes Schindelin, Sep 10, 2014
  53. 2/6 Accept object data in the fsck_object() functionJohannes Schindelin, Sep 10, 2014
  54. 3/6 Make sure fsck_commit_buffer() does not run out of the bufferJohannes Schindelin, Sep 10, 2014
  55. Junio C HamanoSep 10, 2014
  56. Johannes SchindelinSep 11, 2014
  57. Junio C HamanoSep 11, 2014
  58. Eric SunshineSep 10, 2014
  59. 4/6 fsck: check tag objects' headersJohannes Schindelin, Sep 10, 2014
  60. Junio C HamanoSep 10, 2014
  61. 5/6 Add regression tests for stricter tag fsck'ingJohannes Schindelin, Sep 10, 2014
  62. Junio C HamanoSep 10, 2014
  63. Johannes SchindelinSep 11, 2014
  64. 6/6 Make sure that index-pack --strict fails upon invalid tag objectsJohannes Schindelin, Sep 10, 2014
  65. Junio C HamanoSep 10, 2014
  66. Johannes SchindelinSep 11, 2014
  67. Junio C HamanoSep 11, 2014
  68. Johannes SchindelinSep 11, 2014

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.