From: Phil Hord Date: Fri, 14 Oct 2011 12:31:25 GMT Subject: Re: [RFC/PATCH 1/2] bundle: allowing to read from an unseekable fd Message-ID: In-Reply-To: <7vpqi034l0.fsf@alter.siamese.dyndns.org> On Thu, Oct 13, 2011 at 6:32 PM, Junio C Hamano wrote: > The current code opens a given file with fopen(), reads it until the end > of the header and runs ftell(), and reopens the same file with open() and > seeks to skip the header. This structure makes it hard to retarget the > code to read from input that is not seekable, such as a network socket. > > This patch by itself does not reach that goal yet, but I think it is a > right step in that direction. > > Signed-off-by: Junio C Hamano > --- > >  * It would be nice if we can avoid byte-by-byte reading from the file >   descriptor by over-reading into the strbuf and pass the remainder to >   the caller of read_bundle_header(), but I suspect that it would require >   us to carry the "here is the remainder from the previous read" buffer >   around throughout the transport layer. Parsing of the header wouldn't >   be performance critical compared to the computation cost of actually >   reading the rest of the bundle, hopefully, so... > >  bundle.c |   99 ++++++++++++++++++++++++++++++++++++++++---------------------- >  1 files changed, 64 insertions(+), 35 deletions(-) > > diff --git a/bundle.c b/bundle.c > index f48fd7d..3aa715c 100644 > --- a/bundle.c > +++ b/bundle.c > @@ -23,49 +23,78 @@ static void add_to_ref_list(const unsigned char *sha1, const char *name, >        list->nr++; >  } > > -/* returns an fd */ > +/* Eventually this should go to strbuf.[ch] */ > +static int strbuf_readline_fd(struct strbuf *sb, int fd) A size limiter would be useful here. Since it's readline, maybe the limit can even be hardcoded. Without a limit, calling this with something stupid like "/dev/zero" will consume all memory and never return. Phil