{"thread":{"id":"15330","subject":"[PATCH] pack-objects: don't include missing preferred base objects","startedAt":"2008-09-02T14:22:20Z","lastAt":"2008-09-02T14:22:22Z","messageCount":3,"participants":["Nicolas Pitre"],"isPatch":true,"patchVersion":1,"patchTotal":null},"messages":[{"id":"89506","messageId":"1220365342-21308-1-git-send-email-nico@cam.org","threadId":"15330","inReplyTo":null,"subject":"[PATCH] sha1write: don't copy full sized buffers","fromName":"Nicolas Pitre","fromEmail":"nico@cam.org","sentAt":"2008-09-02T14:22:20Z","receivedAt":"2008-09-02T14:22:20Z","isPatch":true,"sender":{"key":"nico@fluxnic.net","avatar":"https://avatars.githubusercontent.com/u/702790?v=4"},"body":"No need to memcpy() source buffer data when we might just process the\ndata in place instead of accumulating it into a separate buffer.\nThis is the case when a whole buffer would have been copied, summed,\nwritten out and then discarded right away.\n\nAlso move the CRC32 processing within the loop so the data is more likely\nto remain in the L1 CPU cache between the CRC32 sum, SHA1 sum and the\nwrite call.\n\nSigned-off-by: Nicolas Pitre <nico@cam.org>\n---\n csum-file.c |   27 +++++++++++++++++----------\n 1 files changed, 17 insertions(+), 10 deletions(-)\n\ndiff --git a/csum-file.c b/csum-file.c\nindex 2838954..bb70c75 100644\n--- a/csum-file.c\n+++ b/csum-file.c\n@@ -11,10 +11,8 @@\n #include \"progress.h\"\n #include \"csum-file.h\"\n \n-static void sha1flush(struct sha1file *f, unsigned int count)\n+static void sha1flush(struct sha1file *f, void *buf, unsigned int count)\n {\n-\tvoid *buf = f->buffer;\n-\n \tfor (;;) {\n \t\tint ret = xwrite(f->fd, buf, count);\n \t\tif (ret > 0) {\n@@ -39,7 +37,7 @@ int sha1close(struct sha1file *f, unsigned char *result, unsigned int flags)\n \n \tif (offset) {\n \t\tSHA1_Update(&f->ctx, f->buffer, offset);\n-\t\tsha1flush(f, offset);\n+\t\tsha1flush(f, f->buffer, offset);\n \t\tf->offset = 0;\n \t}\n \tSHA1_Final(f->buffer, &f->ctx);\n@@ -47,7 +45,7 @@ int sha1close(struct sha1file *f, unsigned char *result, unsigned int flags)\n \t\thashcpy(result, f->buffer);\n \tif (flags & (CSUM_CLOSE | CSUM_FSYNC)) {\n \t\t/* write checksum and close fd */\n-\t\tsha1flush(f, 20);\n+\t\tsha1flush(f, f->buffer, 20);\n \t\tif (flags & CSUM_FSYNC)\n \t\t\tfsync_or_die(f->fd, f->name);\n \t\tif (close(f->fd))\n@@ -62,21 +60,30 @@ int sha1close(struct sha1file *f, unsigned char *result, unsigned int flags)\n \n int sha1write(struct sha1file *f, void *buf, unsigned int count)\n {\n-\tif (f->do_crc)\n-\t\tf->crc32 = crc32(f->crc32, buf, count);\n \twhile (count) {\n \t\tunsigned offset = f->offset;\n \t\tunsigned left = sizeof(f->buffer) - offset;\n \t\tunsigned nr = count > left ? left : count;\n+\t\tvoid *data;\n+\n+\t\tif (f->do_crc)\n+\t\t\tf->crc32 = crc32(f->crc32, buf, nr);\n+\n+\t\tif (nr == sizeof(f->buffer)) {\n+\t\t\t/* process full buffer directly without copy */\n+\t\t\tdata = buf;\n+\t\t} else {\n+\t\t\tmemcpy(f->buffer + offset, buf, nr);\n+\t\t\tdata = f->buffer;\n+\t\t}\n \n-\t\tmemcpy(f->buffer + offset, buf, nr);\n \t\tcount -= nr;\n \t\toffset += nr;\n \t\tbuf = (char *) buf + nr;\n \t\tleft -= nr;\n \t\tif (!left) {\n-\t\t\tSHA1_Update(&f->ctx, f->buffer, offset);\n-\t\t\tsha1flush(f, offset);\n+\t\t\tSHA1_Update(&f->ctx, data, offset);\n+\t\t\tsha1flush(f, data, offset);\n \t\t\toffset = 0;\n \t\t}\n \t\tf->offset = offset;\n-- \n1.6.0.1.276.g59b81\n"},{"id":"89505","messageId":"1220365342-21308-2-git-send-email-nico@cam.org","threadId":"15330","inReplyTo":"1220365342-21308-1-git-send-email-nico@cam.org","subject":"[PATCH] pack-objects: don't include missing preferred base objects","fromName":"Nicolas Pitre","fromEmail":"nico@cam.org","sentAt":"2008-09-02T14:22:21Z","receivedAt":"2008-09-02T14:22:21Z","isPatch":true,"sender":{"key":"nico@fluxnic.net","avatar":"https://avatars.githubusercontent.com/u/702790?v=4"},"body":"This improves commit 6d6f9cddbe a bit by simply not including missing\nbases in the list of objects to process at all.\n\nSigned-off-by: Nicolas Pitre <nico@cam.org>\n---\n builtin-pack-objects.c |    8 ++++++++\n 1 files changed, 8 insertions(+), 0 deletions(-)\n\ndiff --git a/builtin-pack-objects.c b/builtin-pack-objects.c\nindex ba2cf00..e16b7e8 100644\n--- a/builtin-pack-objects.c\n+++ b/builtin-pack-objects.c\n@@ -1725,6 +1725,14 @@ static void prepare_pack(int window, int depth)\n \t\t\tif (entry->type < 0)\n \t\t\t\tdie(\"unable to get type of object %s\",\n \t\t\t\t    sha1_to_hex(entry->idx.sha1));\n+\t\t} else {\n+\t\t\tif (entry->type < 0) {\n+\t\t\t\t/*\n+\t\t\t\t * This object is not found, but we\n+\t\t\t\t * don't have to include it anyway.\n+\t\t\t\t */\n+\t\t\t\tcontinue;\n+\t\t\t}\n \t\t}\n \n \t\tdelta_list[n++] = entry;\n-- \n1.6.0.1.276.g59b81\n"},{"id":"89507","messageId":"1220365342-21308-3-git-send-email-nico@cam.org","threadId":"15330","inReplyTo":"1220365342-21308-2-git-send-email-nico@cam.org","subject":"[PATCH] t5300: improve SHA1 collision test","fromName":"Nicolas Pitre","fromEmail":"nico@cam.org","sentAt":"2008-09-02T14:22:22Z","receivedAt":"2008-09-02T14:22:22Z","isPatch":true,"sender":{"key":"nico@fluxnic.net","avatar":"https://avatars.githubusercontent.com/u/702790?v=4"},"body":"Make sure the reason for the command failure is actually due to\nthe detection of SHA1 collision.\n\nSigned-off-by: Nicolas Pitre <nico@cam.org>\n---\n t/t5300-pack-object.sh |    3 ++-\n 1 files changed, 2 insertions(+), 1 deletions(-)\n\ndiff --git a/t/t5300-pack-object.sh b/t/t5300-pack-object.sh\nindex 83abe5f..8a8b0f1 100755\n--- a/t/t5300-pack-object.sh\n+++ b/t/t5300-pack-object.sh\n@@ -272,7 +272,8 @@ test_expect_success \\\n \n test_expect_success \\\n     'make sure index-pack detects the SHA1 collision' \\\n-    'test_must_fail git-index-pack -o bad.idx test-3.pack'\n+    'test_must_fail git-index-pack -o bad.idx test-3.pack 2>msg &&\n+     grep \"SHA1 COLLISION FOUND\" msg'\n \n test_expect_success \\\n     'honor pack.packSizeLimit' \\\n-- \n1.6.0.1.276.g59b81\n"}]}