git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: [PATCH v4] pkt-line: allow writing of LARGE_PACKET_MAX buffers

From
Eric Sunshine <sunshine@sunshineco.com>
Date
Dec 10, 2014, 20:14 UTC
Message-ID
<CAPig+cSx2dw7DsZgiebHqZUG7DbnQtpgxKbi2883Z42VajeAJA@mail.gmail.com>
In-Reply-To
<20141210094702.GA8917@peff.net>
On Wed, Dec 10, 2014 at 4:47 AM, Jeff King <peff@peff.net> wrote:
Show 45 quoted lines
> Subject: pkt-line: allow writing of LARGE_PACKET_MAX buffers
>
> When we send out pkt-lines with refnames, we use a static
> 1000-byte buffer. This means that the maximum size of a ref
> over the git protocol is around 950 bytes (the exact size
> depends on the protocol line being written, but figure on a sha1
> plus some boilerplate).
>
> This is enough for any sane workflow, but occasionally odd
> things happen (e.g., a bug may create a ref "foo/foo/foo/..."
> accidentally).  With the current code, you cannot even use
> "push" to delete such a ref from a remote.
>
> Let's switch to using a strbuf, with a hard-limit of
> LARGE_PACKET_MAX (which is specified by the protocol).  This
> matches the size of the readers, as of 74543a0 (pkt-line:
> provide a LARGE_PACKET_MAX static buffer, 2013-02-20).
> Versions of git older than that will complain about our
> large packets, but it's really no worse than the current
> behavior. Right now the sender barfs with "impossibly long
> line" trying to send the packet, and afterwards the reader
> will barf with "protocol error: bad line length %d", which
> is arguably better anyway.
>
> Note that we're not really _solving_ the problem here, but
> just bumping the limits. In theory, the length of a ref is
> unbounded, and pkt-line can only represent sizes up to
> 65531 bytes. So we are just bumping the limit, not removing
> it.  But hopefully 64K should be enough for anyone.
>
> As a bonus, by using a strbuf for the formatting we can
> eliminate an unnecessary copy in format_buf_write.
>
> Signed-off-by: Jeff King <peff@peff.net>
> ---
> diff --git a/t/t5527-fetch-odd-refs.sh b/t/t5527-fetch-odd-refs.sh
> index edea9f9..85bcb2e 100755
> --- a/t/t5527-fetch-odd-refs.sh
> +++ b/t/t5527-fetch-odd-refs.sh
> @@ -26,4 +26,37 @@ test_expect_success 'suffix ref is ignored during fetch' '
>         test_cmp expect actual
>  '
>
> +test_expect_success 'try to create repo with absurdly long refname' '
> +       ref240=$_z40/$_z40/$_z40/$_z40/$_z40/$_z40
Maybe you want to keep the &&-chain intact here?
Show 35 quoted lines
> +       ref1440=$ref240/$ref240/$ref240/$ref240/$ref240/$ref240 &&
> +       git init long &&
> +       (
> +               cd long &&
> +               test_commit long &&
> +               test_commit master
> +       ) &&
> +       if git -C long update-ref refs/heads/$ref1440 long; then
> +               test_set_prereq LONG_REF
> +       else
> +               echo >&2 "long refs not supported"
> +       fi
> +'
> +
> +test_expect_success LONG_REF 'fetch handles extremely long refname' '
> +       git fetch long refs/heads/*:refs/remotes/long/* &&
> +       cat >expect <<-\EOF &&
> +       long
> +       master
> +       EOF
> +       git for-each-ref --format="%(subject)" refs/remotes/long >actual &&
> +       test_cmp expect actual
> +'
> +
> +test_expect_success LONG_REF 'push handles extremely long refname' '
> +       git push long :refs/heads/$ref1440 &&
> +       git -C long for-each-ref --format="%(subject)" refs/heads >actual &&
> +       echo master >expect &&
> +       test_cmp expect actual
> +'
> +
>  test_done
> --
> 2.2.0.454.g7eca6b7
>
Previous: Eric SunshineNext: Jeff King
Message 17 of 20 in “pkt-line: allow writing of LARGE_PACKET_MAX buffers”
  1. pkt-line: allow writing of LARGE_PACKET_MAX buffersJeff King, Dec 9, 2014
  2. Jeff KingDec 9, 2014
  3. Junio C HamanoDec 9, 2014
  4. Michael BlumeDec 10, 2014
  5. pkt-line: allow writing of LARGE_PACKET_MAX buffersJeff King, Dec 10, 2014
  6. Eric SunshineDec 10, 2014
  7. Eric SunshineDec 10, 2014
  8. Eric SunshineDec 10, 2014
  9. Jeff KingDec 10, 2014
  10. 0/3 convert read_packed_refs to use strbufJeff King, Dec 10, 2014
  11. 1/3 read_packed_refs: use a strbuf for reading linesJeff King, Dec 10, 2014
  12. 2/3 read_packed_refs: pass strbuf to parse_ref_lineJeff King, Dec 10, 2014
  13. 3/3 read_packed_refs: use skip_prefix instead of static arrayJeff King, Dec 10, 2014
  14. Junio C HamanoDec 10, 2014
  15. pkt-line: allow writing of LARGE_PACKET_MAX buffersJeff King, Dec 10, 2014
  16. Eric SunshineDec 10, 2014
  17. Eric SunshineDec 10, 2014
  18. Jeff KingDec 10, 2014
  19. Johannes SixtDec 9, 2014
  20. Jeff KingDec 9, 2014

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.