{"thread":{"id":"28966","subject":"[PATCH] csum-file: introduce sha1file_checkpoint","startedAt":"2011-11-18T00:36:31Z","lastAt":"2011-11-18T00:36:31Z","messageCount":1,"participants":["Junio C Hamano"],"isPatch":true,"patchVersion":1,"patchTotal":null},"messages":[{"id":"179653","messageId":"7vy5vel10w.fsf@alter.siamese.dyndns.org","threadId":"28966","inReplyTo":null,"subject":"[PATCH] csum-file: introduce sha1file_checkpoint","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2011-11-18T00:36:31Z","receivedAt":"2011-11-18T00:36:31Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"It is useful to be able to rewind a check-summed file to a certain\nprevious state after writing data into it using sha1write() API. The\nfast-import command does this after streaming a blob data to the packfile\nbeing generated and then noticing that the same blob has already been\nwritten, and it does this with a private code truncate_pack() that is\ncommented as \"Yes, this is a layering violation\".\n\nIntroduce two API functions, sha1file_checkpoint(), that allows the caller\nto save a state of a sha1file, and then later revert it to the saved state.\nUse it to reimplement truncate_pack().\n\nSigned-off-by: Junio C Hamano <gitster@pobox.com>\n---\n\n * An extremely low-prio clean-up during the pre-release freeze period,\n   but this is necessary to advance the jc/stream-to-pack topic forward.\n\n csum-file.c   |   20 ++++++++++++++++++++\n csum-file.h   |    9 +++++++++\n fast-import.c |   25 ++++++++-----------------\n 3 files changed, 37 insertions(+), 17 deletions(-)\n\ndiff --git a/csum-file.c b/csum-file.c\nindex fc97d6e..53f5375 100644\n--- a/csum-file.c\n+++ b/csum-file.c\n@@ -158,6 +158,26 @@ struct sha1file *sha1fd_throughput(int fd, const char *name, struct progress *tp\n \treturn f;\n }\n \n+void sha1file_checkpoint(struct sha1file *f, struct sha1file_checkpoint *checkpoint)\n+{\n+\tsha1flush(f);\n+\tcheckpoint->offset = f->total;\n+\tcheckpoint->ctx = f->ctx;\n+}\n+\n+int sha1file_truncate(struct sha1file *f, struct sha1file_checkpoint *checkpoint)\n+{\n+\toff_t offset = checkpoint->offset;\n+\n+\tif (ftruncate(f->fd, offset) ||\n+\t    lseek(f->fd, offset, SEEK_SET) != offset)\n+\t\treturn -1;\n+\tf->total = offset;\n+\tf->ctx = checkpoint->ctx;\n+\tf->offset = 0; /* sha1flush() was called in checkpoint */\n+\treturn 0;\n+}\n+\n void crc32_begin(struct sha1file *f)\n {\n \tf->crc32 = crc32(0, NULL, 0);\ndiff --git a/csum-file.h b/csum-file.h\nindex 6a7967c..3b540bd 100644\n--- a/csum-file.h\n+++ b/csum-file.h\n@@ -17,6 +17,15 @@ struct sha1file {\n \tunsigned char buffer[8192];\n };\n \n+/* Checkpoint */\n+struct sha1file_checkpoint {\n+\toff_t offset;\n+\tgit_SHA_CTX ctx;\n+};\n+\n+extern void sha1file_checkpoint(struct sha1file *, struct sha1file_checkpoint *);\n+extern int sha1file_truncate(struct sha1file *, struct sha1file_checkpoint *);\n+\n /* sha1close flags */\n #define CSUM_CLOSE\t1\n #define CSUM_FSYNC\t2\ndiff --git a/fast-import.c b/fast-import.c\nindex 8d8ea3c..a8db41b 100644\n--- a/fast-import.c\n+++ b/fast-import.c\n@@ -1143,17 +1143,11 @@ static int store_object(\n \treturn 0;\n }\n \n-static void truncate_pack(off_t to, git_SHA_CTX *ctx)\n+static void truncate_pack(struct sha1file_checkpoint *checkpoint)\n {\n-\tif (ftruncate(pack_data->pack_fd, to)\n-\t || lseek(pack_data->pack_fd, to, SEEK_SET) != to)\n+\tif (sha1file_truncate(pack_file, checkpoint))\n \t\tdie_errno(\"cannot truncate pack to skip duplicate\");\n-\tpack_size = to;\n-\n-\t/* yes this is a layering violation */\n-\tpack_file->total = to;\n-\tpack_file->offset = 0;\n-\tpack_file->ctx = *ctx;\n+\tpack_size = checkpoint->offset;\n }\n \n static void stream_blob(uintmax_t len, unsigned char *sha1out, uintmax_t mark)\n@@ -1166,8 +1160,8 @@ static void stream_blob(uintmax_t len, unsigned char *sha1out, uintmax_t mark)\n \tunsigned long hdrlen;\n \toff_t offset;\n \tgit_SHA_CTX c;\n-\tgit_SHA_CTX pack_file_ctx;\n \tgit_zstream s;\n+\tstruct sha1file_checkpoint checkpoint;\n \tint status = Z_OK;\n \n \t/* Determine if we should auto-checkpoint. */\n@@ -1175,11 +1169,8 @@ static void stream_blob(uintmax_t len, unsigned char *sha1out, uintmax_t mark)\n \t\t|| (pack_size + 60 + len) < pack_size)\n \t\tcycle_packfile();\n \n-\toffset = pack_size;\n-\n-\t/* preserve the pack_file SHA1 ctx in case we have to truncate later */\n-\tsha1flush(pack_file);\n-\tpack_file_ctx = pack_file->ctx;\n+\tsha1file_checkpoint(pack_file, &checkpoint);\n+\toffset = checkpoint.offset;\n \n \thdrlen = snprintf((char *)out_buf, out_sz, \"blob %\" PRIuMAX, len) + 1;\n \tif (out_sz <= hdrlen)\n@@ -1245,14 +1236,14 @@ static void stream_blob(uintmax_t len, unsigned char *sha1out, uintmax_t mark)\n \n \tif (e->idx.offset) {\n \t\tduplicate_count_by_type[OBJ_BLOB]++;\n-\t\ttruncate_pack(offset, &pack_file_ctx);\n+\t\ttruncate_pack(&checkpoint);\n \n \t} else if (find_sha1_pack(sha1, packed_git)) {\n \t\te->type = OBJ_BLOB;\n \t\te->pack_id = MAX_PACK_ID;\n \t\te->idx.offset = 1; /* just not zero! */\n \t\tduplicate_count_by_type[OBJ_BLOB]++;\n-\t\ttruncate_pack(offset, &pack_file_ctx);\n+\t\ttruncate_pack(&checkpoint);\n \n \t} else {\n \t\te->depth = 0;\n-- \n1.7.8.rc3.111.g7d421\n"}]}