git/list[1] front-page[2] threads[3] people[4] search[5] about
 

[PATCH 0/4] rev-list: introduce NUL-delimited output mode

From
Justin Tobler <jltobler@gmail.com>
Date
Mar 10, 2025, 19:28 UTC
Message-ID
<20250310192829.661692-1-jltobler@gmail.com>

When walking objects, git-rev-list(1) prints each object entry on a separate line in the form:

        <oid> LF

Some options, such as `--objects`, may print additional information about the object on the same line:

        <oid> SP [<path>] LF

In this mode, if the object path contains a newline it is truncated at the newline.

When the `--missing={print,print-info}` option is provided, information about any missing objects encountered during the object walk are also printed in the form:

        ?<oid> [SP <token>=<value>]... LF

where values containing LF or SP are printed in a token specific fashion so that the resulting encoded value does not contain either of these two problematic bytes. For example, missing object paths are quoted in the C style so they contain LF or SP.

To make machine parsing easier, this series introduces a NUL-delimited output mode for git-rev-list(1) via a `-z` option following a suggestion from Junio in a previous thread[1]. In this mode, instead of LF, each object is delimited with two NUL bytes and any object metadata is separated with a single NUL byte. Examples:

        <oid> NUL NUL
        <oid> [NUL <path>] NUL NUL
        ?<oid> [NUL <token>=<value>]... NUL NUL

In this mode, path and value info are printed as-is without any special encoding or truncation.

For now this series only adds support for use with the `--objects` and `--missing` options. Usage of `-z` with other options is rejected, so it can potentially be added in the future.

One idea I had, but did not implement in this version, was to also use the `<token>=<value>` format for regular non-missing object info while in the NUL-delimited mode. I could see this being a bit more flexible instead of relying strictly on order. Interested if anyone has thoughts on this. :)

This series is structured as follows:
        - Patches 1 and 2 do some minor preparatory refactors.
        - Patch 3 adds the `-z` option to git-rev-list(1) to print
          objects in a NUL-delimited fashion. Printed object paths with
          the `--objects` option are also handled.
        - Patch 4 teaches the `--missing` option how to print info in a
          NUL-delimited fashion.

Thanks for taking a look, -Justin

[1]: <xmqq5xlor0la.fsf@gitster.g>
Justin Tobler (4):
  rev-list: inline `show_object_with_name()` in `show_object()`
  rev-list: refactor early option parsing
  rev-list: support delimiting objects with NUL bytes
  rev-list: support NUL-delimited --missing option
 Documentation/rev-list-options.adoc | 26 +++++++++
 builtin/rev-list.c                  | 86 ++++++++++++++++++++++-------
 revision.c                          |  8 ---
 revision.h                          |  2 -
 t/t6000-rev-list-misc.sh            | 34 ++++++++++++
 t/t6022-rev-list-missing.sh         | 30 ++++++++++
 6 files changed, 155 insertions(+), 31 deletions(-)
base-commit: 87a0bdbf0f72b7561f3cd50636eee33dcb7dbcc3
-- 
2.49.0.rc2
Next: Justin Tobler
Message 1 of 60 in “rev-list: introduce NUL-delimited output mode”
  1. 0/4 rev-list: introduce NUL-delimited output modeJustin Tobler, Mar 10, 2025
  2. 1/4 rev-list: inline `show_object_with_name()` in `show_object()`Justin Tobler, Mar 10, 2025
  3. Junio C HamanoMar 10, 2025
  4. 2/4 rev-list: refactor early option parsingJustin Tobler, Mar 10, 2025
  5. Junio C HamanoMar 10, 2025
  6. Justin ToblerMar 12, 2025
  7. 3/4 rev-list: support delimiting objects with NUL bytesJustin Tobler, Mar 10, 2025
  8. Junio C HamanoMar 10, 2025
  9. Justin ToblerMar 12, 2025
  10. Patrick SteinhardtMar 12, 2025
  11. Justin ToblerMar 12, 2025
  12. 4/4 rev-list: support NUL-delimited --missing optionJustin Tobler, Mar 10, 2025
  13. Junio C HamanoMar 10, 2025
  14. Junio C HamanoMar 10, 2025
  15. Justin ToblerMar 11, 2025
  16. Justin ToblerMar 11, 2025
  17. Junio C HamanoMar 11, 2025
  18. Patrick SteinhardtMar 12, 2025
  19. Justin ToblerMar 12, 2025
  20. D. Ben KnobleMar 10, 2025
  21. Justin ToblerMar 11, 2025
  22. Jeff KingMar 11, 2025
  23. Patrick SteinhardtMar 12, 2025
  24. Junio C HamanoMar 12, 2025
  25. Patrick SteinhardtMar 13, 2025
  26. Justin ToblerMar 12, 2025
  27. Jeff KingMar 13, 2025
  28. Justin ToblerMar 13, 2025
  29. Jeff KingMar 14, 2025
  30. Junio C HamanoMar 14, 2025
  31. Jeff KingMar 14, 2025
  32. Justin ToblerMar 14, 2025
  33. Junio C HamanoMar 14, 2025
  34. Justin ToblerMar 19, 2025
  35. 0/6 rev-list: introduce NUL-delimited output modeJustin Tobler, Mar 13, 2025
  36. 1/6 rev-list: inline `show_object_with_name()` in `show_object()`Justin Tobler, Mar 13, 2025
  37. 2/6 rev-list: refactor early option parsingJustin Tobler, Mar 13, 2025
  38. 3/6 revision: support NUL-delimited --stdin modeJustin Tobler, Mar 13, 2025
  39. 4/6 rev-list: support delimiting objects with NUL bytesJustin Tobler, Mar 13, 2025
  40. Patrick SteinhardtMar 13, 2025
  41. Justin ToblerMar 13, 2025
  42. 5/6 rev-list: support NUL-delimited --boundary optionJustin Tobler, Mar 13, 2025
  43. 6/6 rev-list: support NUL-delimited --missing optionJustin Tobler, Mar 13, 2025
  44. Patrick SteinhardtMar 13, 2025
  45. Justin ToblerMar 13, 2025
  46. 0/6 rev-list: introduce NUL-delimited output modeJustin Tobler, Mar 13, 2025
  47. 1/6 rev-list: inline `show_object_with_name()` in `show_object()`Justin Tobler, Mar 13, 2025
  48. 2/6 rev-list: refactor early option parsingJustin Tobler, Mar 13, 2025
  49. 3/6 revision: support NUL-delimited --stdin modeJustin Tobler, Mar 13, 2025
  50. 4/6 rev-list: support delimiting objects with NUL bytesJustin Tobler, Mar 13, 2025
  51. Christian CouderMar 19, 2025
  52. Justin ToblerMar 19, 2025
  53. 5/6 rev-list: support NUL-delimited --boundary optionJustin Tobler, Mar 13, 2025
  54. 6/6 rev-list: support NUL-delimited --missing optionJustin Tobler, Mar 13, 2025
  55. 0/5 rev-list: introduce NUL-delimited output modeJustin Tobler, Mar 19, 2025
  56. 1/5 rev-list: inline `show_object_with_name()` in `show_object()`Justin Tobler, Mar 19, 2025
  57. 2/5 rev-list: refactor early option parsingJustin Tobler, Mar 19, 2025
  58. 3/5 rev-list: support delimiting objects with NUL bytesJustin Tobler, Mar 19, 2025
  59. 4/5 rev-list: support NUL-delimited --boundary optionJustin Tobler, Mar 19, 2025
  60. 5/5 rev-list: support NUL-delimited --missing optionJustin Tobler, Mar 19, 2025

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.