git/list[1] front-page[2] threads[3] people[4] search[5] about
 

strbuf API

From
Pierre Habouzit <madcoder@debian.org>
Date
Sep 2, 2007, 22:42 UTC
Message-ID
<20070902224213.GB431@artemis.corp>
  I read the recent thread Timo Sirainen raised about string APIs in
git. ANd I went read the strbuf.[hc] module.
  I believe that the choice made in that module are wrong and could be
made better. I actually use to work with a string buffer API (that
interested readers can look at on [0]), that work almost the same
(except for the eof flag, but it's trivial to keep), but have
two significant differences:
  First, and that's the most important one: the buffer is always NUL
terminated, after its official "len". That means, in terms of strbuf
API, that "alloc" is always greater or equal to len+1 to be able to
store the ending NUL[1]. The advantages are obvious: you can pass the
buffer to any legacy C string function without any fear of read
overflow.  strtol comes to mind, and atm, git has to explicitely use
strbuf_end to put that ending nul to be able to call legacy
applications. But once done, the NUL is accounted into the string (aka
it's in "len") which makes it a non C-string (I mean you cannot append
any more data in it anymore). So current implementations tries to
workaround an issue (the non systematical NUL-termination) but IMHO the
wrong way.
  The other shortcoming is that you cannot tell the buffer "Hey, it's
very likely that you'll end up being _that_ long. That's why, in some
parts of the code (see write_tar_entry in archive-tar.c e.g.) the
programmer actually messes with the buffer allocation, outside from the
strbuf module, which makes it well, useless. In my API, I have a
"buffer_ensure" call, that is supposed to do that: "please ensure that
this buffer still has _this_ amount of free and allocated space to put
more data".
  So my question is, do people think I raise a valid point, and would
patches that would refactor the strbuf module to have those functions,
and would fix the code that uses strbuf's to interact properly, be
accepted ?
  Also, the efficiency of the buffer module API I use has a lot to do
with the fact that copying functions, and length tests are inlined in
the .h, so that the compiler can optimize the ones it already tested 10
calls before. I'm not sure if this is frowned upon or if it makes sense.
  [0] http://git.madism.org/?p=pfixtools.git;a=blob;f=buffer.h;hb=HEAD
      http://git.madism.org/?p=pfixtools.git;a=blob;f=buffer.c;hb=HEAD
  [1] Of course, ensuring the NUL-termination has a cost, though it's
      often benign, and for performance-critical places where characters
      are copied one by one, it's always possible to use an "unsafe"
      addch (that would not maintain the invariant), and then call an
      equivalent of strbuf_end (that would not append a \0 like it does
      now, but just would fix the invariant that for any strbuf,
      buf->buf[buf->len] == '\0') explicitely. For places where the
      invariant generate negligible cost (like concatenating two paths
      parts with a middle '/' e.g.) then we gain safety without even
      having to think about it.
-- 
·O·  Pierre Habouzit
··O                                                madcoder@debian.org
OOO                                                http://www.madism.org
Next: Johan Herland
Message 1 of 90 in “strbuf API”
  1. Pierre HabouzitSep 2, 2007
  2. Johan HerlandSep 3, 2007
  3. Pierre HabouzitSep 3, 2007
  4. Miles BaderSep 4, 2007
  5. strbuf new semantics, let's give it a tryPierre Habouzit, Sep 4, 2007
  6. Rework strbuf API and semantics.Pierre Habouzit, Sep 4, 2007
  7. Johannes SchindelinSep 4, 2007
  8. Pierre HabouzitSep 4, 2007
  9. Andreas EricssonSep 4, 2007
  10. Rework strbuf API and semantics.Pierre Habouzit, Sep 4, 2007
  11. Johannes SchindelinSep 4, 2007
  12. Pierre HabouzitSep 4, 2007
  13. Wincent ColaiutaSep 4, 2007
  14. Simplify strbuf uses in fast-import.c using the proper functions.Pierre Habouzit, Sep 4, 2007
  15. René ScharfeSep 4, 2007
  16. René ScharfeSep 4, 2007
  17. Pierre HabouzitSep 5, 2007
  18. Junio C HamanoSep 5, 2007
  19. Pierre HabouzitSep 5, 2007
  20. Rework strbuf API and semantics.Pierre Habouzit, Sep 5, 2007
  21. Simplify strbuf uses in archive-tar.c using the proper functions.Pierre Habouzit, Sep 5, 2007
  22. Simplify strbuf uses in fast-import.c using the proper functions.Pierre Habouzit, Sep 5, 2007
  23. Use proper strbuf API, and also simplify cmd_data code.Pierre Habouzit, Sep 5, 2007
  24. Simplify write_tree using strbuf's.Pierre Habouzit, Sep 5, 2007
  25. Further strbuf re-engineering.Pierre Habouzit, Sep 5, 2007
  26. Eradicate yet-another-buffer implementation in buitin-rerere.cPierre Habouzit, Sep 5, 2007
  27. More strbuf uses in cache-tree.c.Pierre Habouzit, Sep 5, 2007
  28. Junio C HamanoSep 19, 2007
  29. Pierre HabouzitSep 5, 2007
  30. Junio C HamanoSep 19, 2007
  31. Pierre HabouzitSep 19, 2007
  32. Junio C HamanoSep 6, 2007
  33. Pierre HabouzitSep 6, 2007
  34. Junio C HamanoSep 6, 2007
  35. Pierre HabouzitSep 6, 2007
  36. Use proper strbuf API, and also simplify cmd_data code.Pierre Habouzit, Sep 4, 2007
  37. Miles BaderSep 5, 2007
  38. Add strbuf_fread, use it in fast-import.c.Pierre Habouzit, Sep 4, 2007
  39. Matthieu MoySep 3, 2007
  40. Pierre HabouzitSep 3, 2007
  41. Matthieu MoySep 3, 2007
  42. Junio C HamanoSep 3, 2007
  43. Pierre HabouzitSep 3, 2007
  44. Johannes SchindelinSep 3, 2007
  45. strbuf new API, take 2 for inclusionPierre Habouzit, Sep 6, 2007
  46. 1/7 Rework strbuf API and semantics.Pierre Habouzit, Sep 6, 2007
  47. 2/7 Simplify strbuf uses in archive-tar.c using the proper functions.Pierre Habouzit, Sep 6, 2007
  48. 3/7 Use proper strbuf API, and also simplify cmd_data code.Pierre Habouzit, Sep 6, 2007
  49. 4/7 Simplify write_tree using strbuf's.Pierre Habouzit, Sep 6, 2007
  50. 5/7 Further strbuf re-engineering.Pierre Habouzit, Sep 6, 2007
  51. 6/7 Eradicate yet-another-buffer implementation in buitin-rerere.cPierre Habouzit, Sep 6, 2007
  52. 7/7 More strbuf uses in cache-tree.c.Pierre Habouzit, Sep 6, 2007
  53. Johannes SchindelinSep 6, 2007
  54. Pierre HabouzitSep 6, 2007
  55. David KastrupSep 6, 2007
  56. Pierre HabouzitSep 6, 2007
  57. Junio C HamanoSep 7, 2007
  58. Pierre HabouzitSep 7, 2007
  59. Kristian HøgsbergSep 6, 2007
  60. Pierre HabouzitSep 6, 2007
  61. Kristian HøgsbergSep 6, 2007
  62. Pierre HabouzitSep 6, 2007
  63. René ScharfeSep 6, 2007
  64. Johannes SchindelinSep 6, 2007
  65. Jeff KingSep 6, 2007
  66. David KastrupSep 6, 2007
  67. Jeff KingSep 6, 2007
  68. David KastrupSep 6, 2007
  69. Jeff KingSep 6, 2007
  70. David KastrupSep 6, 2007
  71. Johannes SixtSep 6, 2007
  72. David KastrupSep 6, 2007
  73. Jeff KingSep 6, 2007
  74. Kristian HøgsbergSep 6, 2007
  75. Jeff KingSep 6, 2007
  76. Pierre HabouzitSep 6, 2007
  77. Jeff KingSep 6, 2007
  78. Pierre HabouzitSep 6, 2007
  79. Use strbufs in commit.c (pretty printing)Pierre Habouzit, Sep 8, 2007
  80. 1/3 Add strbuf_rtrim (to remove trailing spaces).Pierre Habouzit, Sep 8, 2007
  81. 2/3 Change semantics of interpolate to work like snprintf.Pierre Habouzit, Sep 8, 2007
  82. 3/3 Rework pretty_print_commit to use strbufs instead of custom buffers.Pierre Habouzit, Sep 8, 2007
  83. David KastrupSep 8, 2007
  84. Pierre HabouzitSep 8, 2007
  85. Pierre HabouzitSep 8, 2007
  86. René ScharfeSep 8, 2007
  87. Pierre HabouzitSep 8, 2007
  88. René ScharfeSep 8, 2007
  89. Pierre HabouzitSep 8, 2007
  90. Pierre HabouzitSep 8, 2007

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.