Re: [RFC/PATCH 1/2] bundle: allowing to read from an unseekable fd
- From
Phil Hord <phil.hord@gmail.com>
- Date
- Oct 14, 2011, 12:31 UTC
- Message-ID
- <CABURp0qnhNQ06pNHw_6_PkT2+WNPdQ1WzUHF8p6q3Hx3JkoC8A@mail.gmail.com>
- In-Reply-To
- <7vpqi034l0.fsf@alter.siamese.dyndns.org>
On Thu, Oct 13, 2011 at 6:32 PM, Junio C Hamano <gitster@pobox.com> wrote:
Show 33 quoted lines
> The current code opens a given file with fopen(), reads it until the end > of the header and runs ftell(), and reopens the same file with open() and > seeks to skip the header. This structure makes it hard to retarget the > code to read from input that is not seekable, such as a network socket. > > This patch by itself does not reach that goal yet, but I think it is a > right step in that direction. > > Signed-off-by: Junio C Hamano <gitster@pobox.com> > --- > > * It would be nice if we can avoid byte-by-byte reading from the file > descriptor by over-reading into the strbuf and pass the remainder to > the caller of read_bundle_header(), but I suspect that it would require > us to carry the "here is the remainder from the previous read" buffer > around throughout the transport layer. Parsing of the header wouldn't > be performance critical compared to the computation cost of actually > reading the rest of the bundle, hopefully, so... > > bundle.c | 99 ++++++++++++++++++++++++++++++++++++++++---------------------- > 1 files changed, 64 insertions(+), 35 deletions(-) > > diff --git a/bundle.c b/bundle.c > index f48fd7d..3aa715c 100644 > --- a/bundle.c > +++ b/bundle.c > @@ -23,49 +23,78 @@ static void add_to_ref_list(const unsigned char *sha1, const char *name, > list->nr++; > } > > -/* returns an fd */ > +/* Eventually this should go to strbuf.[ch] */ > +static int strbuf_readline_fd(struct strbuf *sb, int fd)
A size limiter would be useful here. Since it's readline, maybe the limit can even be hardcoded.
Without a limit, calling this with something stupid like "/dev/zero" will consume all memory and never return.
Phil