Re: [PATCH 1/2] Function stripspace now gets a buffer instead file descriptors.
- From
- Carlos Rica <jasampler@gmail.com>
- Date
- Jul 12, 2007, 00:14 UTC
- Message-ID
- <1b46aba20707111714v63bf921dh15000e2629bcf260@mail.gmail.com>
- In-Reply-To
- <7vd4yy1svw.fsf@assigned-by-dhcp.cox.net>
2007/7/12, Junio C Hamano <gitster@pobox.com>:
Show 11 quoted lines
> Carlos Rica <jasampler@gmail.com> writes: > > @@ -28,52 +26,67 @@ static int cleanup(char *line, int len) > > * Remove empty lines from the beginning and end > > * and also trailing spaces from every line. > > * > > + * Note that the buffer will not be null-terminated. > > + * > > The name of the sentinel character '\0' is NUL, not null (which > is a different word, used to call a pointer that points > nowhere). The buffer will not be "NUL-terminated".
Thank you Junio, I will use it on the future.
Show 20 quoted lines
> > int cmd_stripspace(int argc, const char **argv, const char *prefix)
> > {
> > - stripspace(stdin, stdout, 0);
> > + char *buffer;
> > + unsigned long size;
> > +
> > + size = 1024;
> > + buffer = xmalloc(size);
> > + if (read_pipe(0, &buffer, &size))
> > + die("could not read the input");
>
> The command used to be capable of streaming and filtering a few
> hundred gigabytes of text on a machine with small address space,
> as it operated one line at a time, but now it cannot as it has
> to hold everything in core before starting.
>
> I do not think we miss that loss of capability too much, but I
> wonder if we can be a bit more clever about it, perhaps feeding
> a chunk at a time. Not a very strong request, but just
> wondering if it is an easy change.I did those changes because I was needing those tests that I had written before in order to develop the function. After that, we now can restore the previous function with file descriptors to make it capable of filter a few hundred gigabytes of text, provided that the text does not have long long lines on it.
Indeed, the implementation for composing a tag (header, cleaned message and optional signature) in "builtin-tag.c", now pass it to the function write_sha1_file as a buffer on memory, so it won't support sizes bigger than memory available on the system. Messages should not be so big, but I don't know how to limit those.