git/list[1] front-page[2] threads[3] people[4] search[5] about
 

[PATCH 0/23] parsing and fsck cleanups

From
Jeff King <peff@peff.net>
Date
Oct 18, 2019, 04:41 UTC
Message-ID
<20191018044103.GA17625@sigill.intra.peff.net>
The thread starting at:
  https://public-inbox.org/git/xmqqo8zxnz0m.fsf@gitster-ct.c.googlers.com/

discusses some issues with our handling of corrupt objects, as well as some weirdness in fsck. This series is my attempt to clean it up. The number of patches is a little daunting, but the early ones are the most interesting. The latter half is part of a big refactor/cleanup that's mostly mechanical (and isn't strictly necessary; see below for discussion).

  [01/23]: parse_commit_buffer(): treat lookup_commit() failure as parse error
  [02/23]: parse_commit_buffer(): treat lookup_tree() failure as parse error
  [03/23]: parse_tag_buffer(): treat NULL tag pointer as parse error
  [04/23]: remember commit/tag parse failures
    These ones are tightening up our parser to report failures more
    consistently. The first one definitely fixes a demonstrable bug, and
    I suspect the rest of them are fixing hard-to-trigger but lurking
    segfaults.
  [05/23]: fsck: stop checking commit->tree value
  [06/23]: fsck: stop checking commit->parent counts
  [07/23]: fsck: stop checking tag->tagged
  [08/23]: fsck: require an actual buffer for non-blobs
    These ones clean up weirdness where fsck is dependent on the results
    of parse_commit(), etc, rather than just looking at the buffer we
    gave it. I don't think they're _hurting_ anything, but it certainly
    makes following the fsck logic more confusing.
  [09/23]: fsck: unify object-name code
    Cleanup that fixes a few minor bugs.
  [10/23]: fsck_describe_object(): build on our get_object_name() primitive
  [11/23]: fsck: use oids rather than objects for object_name API
  [12/23]: fsck: don't require full object structs for display functions
  [13/23]: fsck: only provide oid/type in fsck_error callback
  [14/23]: fsck: only require an oid for skiplist functions
  [15/23]: fsck: don't require an object struct for report()
  [16/23]: fsck: accept an oid instead of a "struct blob" for fsck_blob()
  [17/23]: fsck: drop blob struct from fsck_finish()
  [18/23]: fsck: don't require an object struct for fsck_ident()
  [19/23]: fsck: don't require an object struct in verify_headers()
  [20/23]: fsck: rename vague "oid" local variables
  [21/23]: fsck: accept an oid instead of a "struct tag" for fsck_tag()
  [22/23]: fsck: accept an oid instead of a "struct commit" for fsck_commit()
  [23/23]: fsck: accept an oid instead of a "struct tree" for fsck_tree()
    This a string of refactors that ends up with all of the
    type-specific fsck functions not getting an object struct at all.
    My goal there was two-fold:
       - it makes it harder to introduce weirdness like we saw in
	 patches 5-8.
       - it _could_ make things less awkward for callers like index-pack
	 which don't necessarily have object structs. And at the end, we
	 basically have an fsck_object() that doesn't need an object
	 struct. But index-pack still calls fsck_walk(), which does (and
	 which relies on the parsed values to traverse). It's not
	 entirely clear to me whether index-pack needs to be doing
	 fsck_walk() in the first place, or if it should be relying on
	 the usual connectivity check.
	 So I'm undecided whether this is worth taking on its own, or if
	 trying to avoid object structs in the fsck code is just a
	 fool's errand. I do think the result isn't too bad to look at,
	 though and there are some minor improvements along the way
	 (e.g., patch 17 is able to drop some awkwardness).
    Most of the patches are pretty mechanical. There are so many because
    I split it by call stack layer. If A calls B calls C, then I
    converted "C" away from "struct object" first, which enables
    converting "B", and so on.
 builtin/fsck.c                         | 126 ++++----
 commit-graph.c                         |   3 -
 commit.c                               |  33 ++-
 fsck.c                                 | 386 +++++++++++--------------
 fsck.h                                 |  39 ++-
 t/t1450-fsck.sh                        |   2 +-
 t/t5318-commit-graph.sh                |   2 +-
 t/t6102-rev-list-unexpected-objects.sh |   2 +-
 tag.c                                  |  21 +-
 9 files changed, 312 insertions(+), 302 deletions(-)
Next: Jeff King
Message 1 of 39 in “parsing and fsck cleanups”
  1. 0/23 parsing and fsck cleanupsJeff King, Oct 18, 2019
  2. 08/23 fsck: require an actual buffer for non-blobsJeff King, Oct 18, 2019
  3. 04/23 remember commit/tag parse failuresJeff King, Oct 18, 2019
  4. Junio C HamanoOct 24, 2019
  5. Jonathan TanOct 24, 2019
  6. Jeff KingOct 24, 2019
  7. 05/23 fsck: stop checking commit->tree valueJeff King, Oct 18, 2019
  8. Junio C HamanoOct 24, 2019
  9. 06/23 fsck: stop checking commit->parent countsJeff King, Oct 18, 2019
  10. 09/23 fsck: unify object-name codeJeff King, Oct 18, 2019
  11. Junio C HamanoOct 24, 2019
  12. Jeff KingOct 24, 2019
  13. Junio C HamanoOct 25, 2019
  14. Jeff KingOct 25, 2019
  15. 10/23 fsck_describe_object(): build on our get_object_name() primitiveJeff King, Oct 18, 2019
  16. Junio C HamanoOct 24, 2019
  17. 11/23 fsck: use oids rather than objects for object_name APIJeff King, Oct 18, 2019
  18. 07/23 fsck: stop checking tag->taggedJeff King, Oct 18, 2019
  19. 12/23 fsck: don't require object structs for display functionsJeff King, Oct 18, 2019
  20. 13/23 fsck: only provide oid/type in fsck_error callbackJeff King, Oct 18, 2019
  21. 14/23 fsck: only require an oid for skiplist functionsJeff King, Oct 18, 2019
  22. 15/23 fsck: don't require an object struct for report()Jeff King, Oct 18, 2019
  23. 16/23 fsck: accept an oid instead of a "struct blob" for fsck_blob()Jeff King, Oct 18, 2019
  24. 17/23 fsck: drop blob struct from fsck_finish()Jeff King, Oct 18, 2019
  25. 18/23 fsck: don't require an object struct for fsck_ident()Jeff King, Oct 18, 2019
  26. 19/23 fsck: don't require an object struct in verify_headers()Jeff King, Oct 18, 2019
  27. 20/23 fsck: rename vague "oid" local variablesJeff King, Oct 18, 2019
  28. 21/23 fsck: accept an oid instead of a "struct tag" for fsck_tag()Jeff King, Oct 18, 2019
  29. 22/23 fsck: accept an oid instead of a "struct commit" for fsck_commit()Jeff King, Oct 18, 2019
  30. 23/23 fsck: accept an oid instead of a "struct tree" for fsck_tree()Jeff King, Oct 18, 2019
  31. 01/23 parse_commit_buffer(): treat lookup_commit() failure as parse errorJeff King, Oct 18, 2019
  32. Junio C HamanoOct 24, 2019
  33. Jeff KingOct 24, 2019
  34. 02/23 parse_commit_buffer(): treat lookup_tree() failure as parse errorJeff King, Oct 18, 2019
  35. Jonathan TanOct 24, 2019
  36. Jeff KingOct 24, 2019
  37. 03/23 parse_tag_buffer(): treat NULL tag pointer as parse errorJeff King, Oct 18, 2019
  38. Jonathan TanOct 24, 2019
  39. Junio C HamanoOct 25, 2019

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.