git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: [PATCH] pkt-line: do not chomp EOL for sideband progress info

From
Jiang Xin <worldhello.net@gmail.com>
Date
Sep 25, 2023, 00:25 UTC
Message-ID
<CANYiYbF+Xmk4rCNLMJe+i_CFafg8=QU5vbXWNUZbOVsDLTe5QQ@mail.gmail.com>
In-Reply-To
<20230920210832.2305886-1-jonathantanmy@google.com>
On Thu, Sep 21, 2023 at 5:08 AM Jonathan Tan <jonathantanmy@google.com> wrote:
Show 40 quoted lines
>
> Jiang Xin <worldhello.net@gmail.com> writes:
> > From: Jiang Xin <zhiyou.jx@alibaba-inc.com>
> >
> > In the protocol negotiation stage, we need to turn on the flag
> > "PACKET_READ_CHOMP_NEWLINE" to chomp EOL for each packet line from
> > client or server. But when receiving data and progress information
> > using sideband, we will turn off the flag "PACKET_READ_CHOMP_NEWLINE"
> > to prevent mangling EOLs from data and progress information.
> >
> > When both the server and the client support "sideband-all" capability,
> > we have a dilemma that EOLs in negotiation packets should be trimmed,
> > but EOLs in progress infomation should be leaved as is.
> >
> > Move the logic of chomping EOLs from "packet_read_with_status()" to
> > "packet_reader_read()" can resolve this dilemma.
> >
> > Signed-off-by: Jiang Xin <zhiyou.jx@alibaba-inc.com>
>
> I think the summary is that when we use the struct packet_reader with
> sideband and newline chomping, we want the chomping to occur only on
> sideband 1, but the current code also chomps on sidebands 2 and 3 (3
> is for fatal errors so it doesn't matter as much, but for 2, it really
> matters).
>
> This makes sense to fix.
>
> As for how this is fixed, one issue is that we now have 2 places in
> which newlines can be chomped (in packet_read_with_status() and with
> this patch, packet_reader_read()). The issue is that we need to check
> the sideband indicator before we chomp, and packet_read_with_status()
> only knows how to chomp. So we either teach packet_read_with_status()
> how to sideband, or tell packet_read_with_status() not to chomp and
> chomp it ourselves (like in this patch).
>
> Of the two, I would prefer it if packet_read_with_status() was taught
> how to sideband - as it is, packet_read_with_status() is used 3 times
> in pkt-line.c and 1 time in remote-curl.c, and 2 of those times (in
> pkt-line.c) are used with sideband. Doing this does not only solve the
> problem here, but reduces code duplication.

Yes, there are two places we can choose to fix. My first instinct is that changes on packet_reader_read will have less impact. I will new implementation in next reroll.

Show 22 quoted lines
> > @@ -624,12 +630,19 @@ enum packet_read_status packet_reader_read(struct packet_reader *reader)
> >                       break;
> >       }
> >
> > -     if (reader->status == PACKET_READ_NORMAL)
> > +     if (reader->status == PACKET_READ_NORMAL) {
> >               /* Skip the sideband designator if sideband is used */
> >               reader->line = reader->use_sideband ?
> >                       reader->buffer + 1 : reader->buffer;
> > -     else
> > +
> > +             if ((reader->options & PACKET_READ_CHOMP_NEWLINE) &&
> > +                 reader->buffer[reader->pktlen - 1] == '\n') {
> > +                     reader->buffer[reader->pktlen - 1] = 0;
> > +                     reader->pktlen--;
> > +             }
>
> When we reach here, we have skipped all sideband-2 pkt-lines, so
> unconditionally chomping it here is good. Might be better if there was
> also a check that use_sideband is set, just for symmetry with the code
> near the start of this function.
>

You find my bug. Without checking the use_sideband flag, two consecutive EOLwill be removed.

BTW, the new reroll is not coming as fast as I planned, because when I adding new test cases, I find another issue in pkt-line. I will fix these two issues in this series.

-- Jiang Xin

Previous: Jonathan TanNext: Jiang Xin
Message 4 of 19 in “pkt-line: do not chomp EOL for sideband progress info”
  1. pkt-line: do not chomp EOL for sideband progress infoJiang Xin, Sep 19, 2023
  2. Junio C HamanoSep 19, 2023
  3. Jonathan TanSep 20, 2023
  4. Jiang XinSep 25, 2023
  5. 1/3 test-pkt-line: add option parser for unpack-sidebandJiang Xin, Sep 25, 2023
  6. 2/3 pkt-line: memorize sideband fragment in readerJiang Xin, Sep 25, 2023
  7. 3/3 pkt-line: do not chomp newlines for sideband messagesJiang Xin, Sep 25, 2023
  8. Junio C HamanoSep 25, 2023
  9. Oswald BuddenhagenSep 26, 2023
  10. Jiang XinOct 4, 2023
  11. Junio C HamanoOct 4, 2023
  12. 0/3 Sideband demultiplexer fixesJiang Xin, Oct 4, 2023
  13. 2/3 pkt-line: memorize sideband fragment in readerJiang Xin, Oct 4, 2023
  14. 1/3 test-pkt-line: add option parser for unpack-sidebandJiang Xin, Oct 4, 2023
  15. 3/3 pkt-line: do not chomp newlines for sideband messagesJiang Xin, Oct 4, 2023
  16. 0/3 Sideband-all demultiplexer fixesJiang Xin, Dec 17, 2023
  17. 1/3 test-pkt-line: add option parser for unpack-sidebandJiang Xin, Dec 17, 2023
  18. 2/3 pkt-line: memorize sideband fragment in readerJiang Xin, Dec 17, 2023
  19. 3/3 pkt-line: do not chomp newlines for sideband messagesJiang Xin, Dec 17, 2023

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.