Re: [PATCH] Teach "git add" and friends to be paranoid
- From
Junio C Hamano <gitster@pobox.com>
- Date
- Feb 19, 2010, 17:52 UTC
- Message-ID
- <7v635tkta7.fsf@alter.siamese.dyndns.org>
- In-Reply-To
- <20100219082813.GB17952@dpotapov.dyndns.org>
Dmitry Potapov <dpotapov@gmail.com> writes:
> ... I think it is possible > to avoid the overhead of being on the safe side in a few common cases. > Here is a patch. I have not had time to test it, but changes appear to > trivial.
Yeah, these are obvious "paranoia not needed" cases.
How much "speed-up" are we talking about, though? Can we quantify? I personally think it is not even worth to quantify it but instead simply say "avoid unnecessary computation" without saying "speed up", though ;-)
Thanks.
Show 57 quoted lines
> -- >8 --
> From 3e53610a41c4aad458dff13135a73bb4944f456b Mon Sep 17 00:00:00 2001
> From: Dmitry Potapov <dpotapov@gmail.com>
> Date: Fri, 19 Feb 2010 11:00:51 +0300
> Subject: [PATCH] speed up "git add" by avoiding the paranoid mode
>
> While the paranoid mode preserve the git repository from corruption in the
> case when the added file is changed simultaneously with running "git add",
> it has some overhead. However, in a few common cases, it is possible to
> avoid this mode and still be on the safe side:
>
> 1. If mmap() is implemented as reading the whole file in memory.
>
> 2. If the whole file was read in memory as result of applying some filter.
>
> 3. If the added file is small, it is faster to use read() than mmap().
>
> Signed-off-by: Dmitry Potapov <dpotapov@gmail.com>
> ---
> sha1_file.c | 5 ++++-
> 1 files changed, 4 insertions(+), 1 deletions(-)
>
> diff --git a/sha1_file.c b/sha1_file.c
> index d8a7722..4efeb21 100644
> --- a/sha1_file.c
> +++ b/sha1_file.c
> @@ -2469,6 +2469,7 @@ static int index_mem(unsigned char *sha1, void *buf, size_t size,
> write_object ? safe_crlf : 0)) {
> buf = strbuf_detach(&nbuf, &size);
> re_allocated = 1;
> + paranoid = 0;
> }
> }
>
> @@ -2490,7 +2491,7 @@ int index_fd(unsigned char *sha1, int fd, struct stat *st, int write_object,
> size_t size = xsize_t(st->st_size);
>
> flag = write_object ? INDEX_MEM_WRITE_OBJECT : 0;
> - if (!S_ISREG(st->st_mode)) {
> + if (!S_ISREG(st->st_mode) || size < 262144) {
> struct strbuf sbuf = STRBUF_INIT;
> if (strbuf_read(&sbuf, fd, 4096) >= 0)
> ret = index_mem(sha1, sbuf.buf, sbuf.len,
> @@ -2500,7 +2501,9 @@ int index_fd(unsigned char *sha1, int fd, struct stat *st, int write_object,
> strbuf_release(&sbuf);
> } else if (size) {
> void *buf = xmmap(NULL, size, PROT_READ, MAP_PRIVATE, fd, 0);
> +#ifndef NO_MMAP
> flag |= INDEX_MEM_PARANOID;
> +#endif
> ret = index_mem(sha1, buf, size, type, path, flag);
> munmap(buf, size);
> } else
> --
> 1.7.0
>
> -- >8 --