# [PATCH v2 00/12] object_id part 11 (the_hash_algo) 19 messages from 2018-02-01 to 2018-02-08. Participants: brian m. carlson, Stefan Beller, Junio C Hamano. Thread: https://gitlist.dev/t/47731 ## brian m. carlson, 2018-02-01 02:18 Subject: [PATCH v2 00/12] object_id part 11 (the_hash_algo) Message-ID: <20180201021848.533188-1-sandals@crustytoothpaste.net> URL: https://gitlist.dev/e/20180201021848.533188-1-sandals%40crustytoothpaste.net ``` This series includes various changes to adopt the use of the_hash_algo for abstracting hash algorithms away. Changes from v1: * Fix comments referring to SHA-1. * Switch hash function wrappers to take git_hash_ctx *. * Use a const int variable for a constant value. * Wrap an overly long line. brian m. carlson (12): hash: move SHA-1 macros to hash.h hash: create union for hash context allocation builtin/index-pack: improve hash function abstraction builtin/unpack-objects: switch uses of SHA-1 to the_hash_algo sha1_file: switch uses of SHA-1 to the_hash_algo fast-import: switch various uses of SHA-1 to the_hash_algo pack-check: convert various uses of SHA-1 to abstract forms pack-write: switch various SHA-1 values to abstract forms read-cache: abstract away uses of SHA-1 csum-file: rename sha1file to hashfile csum-file: abstract uses of SHA-1 bulk-checkin: abstract SHA-1 usage builtin/index-pack.c | 108 +++++++++++++++++++++++------------------------ builtin/pack-objects.c | 52 +++++++++++------------ builtin/unpack-objects.c | 18 ++++---- bulk-checkin.c | 28 ++++++------ cache.h | 25 ----------- csum-file.c | 46 ++++++++++---------- csum-file.h | 38 ++++++++--------- fast-import.c | 70 +++++++++++++++--------------- hash.h | 40 +++++++++++++++--- pack-bitmap-write.c | 30 ++++++------- pack-check.c | 32 +++++++------- pack-write.c | 77 ++++++++++++++++----------------- pack.h | 4 +- read-cache.c | 58 ++++++++++++------------- sha1_file.c | 72 +++++++++++++++---------------- 15 files changed, 351 insertions(+), 347 deletions(-) ``` ## brian m. carlson, 2018-02-01 02:18 Subject: [PATCH v2 01/12] hash: move SHA-1 macros to hash.h Message-ID: <20180201021848.533188-2-sandals@crustytoothpaste.net> URL: https://gitlist.dev/e/20180201021848.533188-2-sandals%40crustytoothpaste.net In-Reply-To: <20180201021848.533188-1-sandals@crustytoothpaste.net> ``` Most of the other code dealing with SHA-1 and other hashes is located in hash.h, which is in turn loaded by cache.h. Move the SHA-1 macros to hash.h as well, so we can use them in additional hash-related items in the future. Signed-off-by: brian m. carlson --- cache.h | 25 ------------------------- hash.h | 25 +++++++++++++++++++++++++ 2 files changed, 25 insertions(+), 25 deletions(-) diff --git a/cache.h b/cache.h index d8b975a571..bfde6f757a 100644 --- a/cache.h +++ b/cache.h @@ -16,31 +16,6 @@ #include "sha1-array.h" #include "repository.h" -#ifndef platform_SHA_CTX -/* - * platform's underlying implementation of SHA-1; could be OpenSSL, - * blk_SHA, Apple CommonCrypto, etc... Note that including - * SHA1_HEADER may have already defined platform_SHA_CTX for our - * own implementations like block-sha1 and ppc-sha1, so we list - * the default for OpenSSL compatible SHA-1 implementations here. - */ -#define platform_SHA_CTX SHA_CTX -#define platform_SHA1_Init SHA1_Init -#define platform_SHA1_Update SHA1_Update -#define platform_SHA1_Final SHA1_Final -#endif - -#define git_SHA_CTX platform_SHA_CTX -#define git_SHA1_Init platform_SHA1_Init -#define git_SHA1_Update platform_SHA1_Update -#define git_SHA1_Final platform_SHA1_Final - -#ifdef SHA1_MAX_BLOCK_SIZE -#include "compat/sha1-chunked.h" -#undef git_SHA1_Update -#define git_SHA1_Update git_SHA1_Update_Chunked -#endif - #include typedef struct git_zstream { z_stream z; diff --git a/hash.h b/hash.h index 7d7a864f5d..7122dea7b3 100644 --- a/hash.h +++ b/hash.h @@ -15,6 +15,31 @@ #include "block-sha1/sha1.h" #endif +#ifndef platform_SHA_CTX +/* + * platform's underlying implementation of SHA-1; could be OpenSSL, + * blk_SHA, Apple CommonCrypto, etc... Note that including + * SHA1_HEADER may have already defined platform_SHA_CTX for our + * own implementations like block-sha1 and ppc-sha1, so we list + * the default for OpenSSL compatible SHA-1 implementations here. + */ +#define platform_SHA_CTX SHA_CTX +#define platform_SHA1_Init SHA1_Init +#define platform_SHA1_Update SHA1_Update +#define platform_SHA1_Final SHA1_Final +#endif + +#define git_SHA_CTX platform_SHA_CTX +#define git_SHA1_Init platform_SHA1_Init +#define git_SHA1_Update platform_SHA1_Update +#define git_SHA1_Final platform_SHA1_Final + +#ifdef SHA1_MAX_BLOCK_SIZE +#include "compat/sha1-chunked.h" +#undef git_SHA1_Update +#define git_SHA1_Update git_SHA1_Update_Chunked +#endif + /* * Note that these constants are suitable for indexing the hash_algos array and * comparing against each other, but are otherwise arbitrary, so they should not ``` ## brian m. carlson, 2018-02-01 02:18 Subject: [PATCH v2 02/12] hash: create union for hash context allocation Message-ID: <20180201021848.533188-3-sandals@crustytoothpaste.net> URL: https://gitlist.dev/e/20180201021848.533188-3-sandals%40crustytoothpaste.net In-Reply-To: <20180201021848.533188-1-sandals@crustytoothpaste.net> ``` In various parts of our code, we want to allocate a structure representing the internal state of a hash algorithm. The original implementation of the hash algorithm abstraction assumed we would do that using heap allocations, and added a context size element to struct git_hash_algo. However, most of the existing code uses stack allocations and conversion would needlessly complicate various parts of the code. Add a union for the purpose of allocating hash contexts on the stack and a typedef for ease of use. Use this union for defining the init, update, and final functions to avoid casts. Remove the ctxsz element for struct git_hash_algo, which is no longer very useful. This does mean that stack allocations will grow slightly as additional hash functions are added, but this should not be a significant problem, since we don't allocate many hash contexts. The improved usability and benefits from avoiding dynamic allocation outweigh this small downside. Signed-off-by: brian m. carlson --- hash.h | 15 +++++++++------ sha1_file.c | 20 +++++++++----------- 2 files changed, 18 insertions(+), 17 deletions(-) diff --git a/hash.h b/hash.h index 7122dea7b3..eb30f59be3 100644 --- a/hash.h +++ b/hash.h @@ -55,9 +55,15 @@ /* Number of algorithms supported (including unknown). */ #define GIT_HASH_NALGOS (GIT_HASH_SHA1 + 1) -typedef void (*git_hash_init_fn)(void *ctx); -typedef void (*git_hash_update_fn)(void *ctx, const void *in, size_t len); -typedef void (*git_hash_final_fn)(unsigned char *hash, void *ctx); +/* A suitably aligned type for stack allocations of hash contexts. */ +union git_hash_ctx { + git_SHA_CTX sha1; +}; +typedef union git_hash_ctx git_hash_ctx; + +typedef void (*git_hash_init_fn)(git_hash_ctx *ctx); +typedef void (*git_hash_update_fn)(git_hash_ctx *ctx, const void *in, size_t len); +typedef void (*git_hash_final_fn)(unsigned char *hash, git_hash_ctx *ctx); struct git_hash_algo { /* @@ -69,9 +75,6 @@ struct git_hash_algo { /* A four-byte version identifier, used in pack indices. */ uint32_t format_id; - /* The size of a hash context (e.g. git_SHA_CTX). */ - size_t ctxsz; - /* The length of the hash in binary. */ size_t rawsz; diff --git a/sha1_file.c b/sha1_file.c index 3da70ac650..ec6ecea170 100644 --- a/sha1_file.c +++ b/sha1_file.c @@ -39,32 +39,32 @@ const struct object_id empty_blob_oid = { EMPTY_BLOB_SHA1_BIN_LITERAL }; -static void git_hash_sha1_init(void *ctx) +static void git_hash_sha1_init(git_hash_ctx *ctx) { - git_SHA1_Init((git_SHA_CTX *)ctx); + git_SHA1_Init(&ctx->sha1); } -static void git_hash_sha1_update(void *ctx, const void *data, size_t len) +static void git_hash_sha1_update(git_hash_ctx *ctx, const void *data, size_t len) { - git_SHA1_Update((git_SHA_CTX *)ctx, data, len); + git_SHA1_Update(&ctx->sha1, data, len); } -static void git_hash_sha1_final(unsigned char *hash, void *ctx) +static void git_hash_sha1_final(unsigned char *hash, git_hash_ctx *ctx) { - git_SHA1_Final(hash, (git_SHA_CTX *)ctx); + git_SHA1_Final(hash, &ctx->sha1); } -static void git_hash_unknown_init(void *ctx) +static void git_hash_unknown_init(git_hash_ctx *ctx) { die("trying to init unknown hash"); } -static void git_hash_unknown_update(void *ctx, const void *data, size_t len) +static void git_hash_unknown_update(git_hash_ctx *ctx, const void *data, size_t len) { die("trying to update unknown hash"); } -static void git_hash_unknown_final(unsigned char *hash, void *ctx) +static void git_hash_unknown_final(unsigned char *hash, git_hash_ctx *ctx) { die("trying to finalize unknown hash"); } @@ -75,7 +75,6 @@ const struct git_hash_algo hash_algos[GIT_HASH_NALGOS] = { 0x00000000, 0, 0, - 0, git_hash_unknown_init, git_hash_unknown_update, git_hash_unknown_final, @@ -86,7 +85,6 @@ const struct git_hash_algo hash_algos[GIT_HASH_NALGOS] = { "sha-1", /* "sha1", big-endian */ 0x73686131, - sizeof(git_SHA_CTX), GIT_SHA1_RAWSZ, GIT_SHA1_HEXSZ, git_hash_sha1_init, ``` ## brian m. carlson, 2018-02-01 02:18 Subject: [PATCH v2 03/12] builtin/index-pack: improve hash function abstraction Message-ID: <20180201021848.533188-4-sandals@crustytoothpaste.net> URL: https://gitlist.dev/e/20180201021848.533188-4-sandals%40crustytoothpaste.net In-Reply-To: <20180201021848.533188-1-sandals@crustytoothpaste.net> ``` Convert several uses of unsigned char [20] to struct object_id and convert various hard-coded constants and uses of SHA-1 functions to use the_hash_algo. Signed-off-by: brian m. carlson --- builtin/index-pack.c | 90 ++++++++++++++++++++++++++-------------------------- 1 file changed, 45 insertions(+), 45 deletions(-) diff --git a/builtin/index-pack.c b/builtin/index-pack.c index 4c51aec81f..40c000aca8 100644 --- a/builtin/index-pack.c +++ b/builtin/index-pack.c @@ -91,7 +91,7 @@ static unsigned int input_offset, input_len; static off_t consumed_bytes; static off_t max_input_size; static unsigned deepest_delta; -static git_SHA_CTX input_ctx; +static git_hash_ctx input_ctx; static uint32_t input_crc32; static int input_fd, output_fd; static const char *curr_pack; @@ -253,7 +253,7 @@ static void flush(void) if (input_offset) { if (output_fd >= 0) write_or_die(output_fd, input_buffer, input_offset); - git_SHA1_Update(&input_ctx, input_buffer, input_offset); + the_hash_algo->update_fn(&input_ctx, input_buffer, input_offset); memmove(input_buffer, input_buffer + input_offset, input_len); input_offset = 0; } @@ -326,7 +326,7 @@ static const char *open_pack_file(const char *pack_name) output_fd = -1; nothread_data.pack_fd = input_fd; } - git_SHA1_Init(&input_ctx); + the_hash_algo->init_fn(&input_ctx); return pack_name; } @@ -437,22 +437,22 @@ static int is_delta_type(enum object_type type) } static void *unpack_entry_data(off_t offset, unsigned long size, - enum object_type type, unsigned char *sha1) + enum object_type type, struct object_id *oid) { static char fixed_buf[8192]; int status; git_zstream stream; void *buf; - git_SHA_CTX c; + git_hash_ctx c; char hdr[32]; int hdrlen; if (!is_delta_type(type)) { hdrlen = xsnprintf(hdr, sizeof(hdr), "%s %lu", typename(type), size) + 1; - git_SHA1_Init(&c); - git_SHA1_Update(&c, hdr, hdrlen); + the_hash_algo->init_fn(&c); + the_hash_algo->update_fn(&c, hdr, hdrlen); } else - sha1 = NULL; + oid = NULL; if (type == OBJ_BLOB && size > big_file_threshold) buf = fixed_buf; else @@ -469,8 +469,8 @@ static void *unpack_entry_data(off_t offset, unsigned long size, stream.avail_in = input_len; status = git_inflate(&stream, 0); use(input_len - stream.avail_in); - if (sha1) - git_SHA1_Update(&c, last_out, stream.next_out - last_out); + if (oid) + the_hash_algo->update_fn(&c, last_out, stream.next_out - last_out); if (buf == fixed_buf) { stream.next_out = buf; stream.avail_out = sizeof(fixed_buf); @@ -479,15 +479,15 @@ static void *unpack_entry_data(off_t offset, unsigned long size, if (stream.total_out != size || status != Z_STREAM_END) bad_object(offset, _("inflate returned %d"), status); git_inflate_end(&stream); - if (sha1) - git_SHA1_Final(sha1, &c); + if (oid) + the_hash_algo->final_fn(oid->hash, &c); return buf == fixed_buf ? NULL : buf; } static void *unpack_raw_entry(struct object_entry *obj, off_t *ofs_offset, - unsigned char *ref_sha1, - unsigned char *sha1) + struct object_id *ref_oid, + struct object_id *oid) { unsigned char *p; unsigned long size, c; @@ -515,8 +515,8 @@ static void *unpack_raw_entry(struct object_entry *obj, switch (obj->type) { case OBJ_REF_DELTA: - hashcpy(ref_sha1, fill(20)); - use(20); + hashcpy(ref_oid->hash, fill(the_hash_algo->rawsz)); + use(the_hash_algo->rawsz); break; case OBJ_OFS_DELTA: p = fill(1); @@ -546,7 +546,7 @@ static void *unpack_raw_entry(struct object_entry *obj, } obj->hdr_size = consumed_bytes - obj->idx.offset; - data = unpack_entry_data(obj->idx.offset, obj->size, obj->type, sha1); + data = unpack_entry_data(obj->idx.offset, obj->size, obj->type, oid); obj->idx.crc32 = input_crc32; return data; } @@ -1119,11 +1119,11 @@ static void *threaded_second_pass(void *data) * - calculate SHA1 of all non-delta objects; * - remember base (SHA1 or offset) for all deltas. */ -static void parse_pack_objects(unsigned char *sha1) +static void parse_pack_objects(unsigned char *hash) { int i, nr_delays = 0; struct ofs_delta_entry *ofs_delta = ofs_deltas; - unsigned char ref_delta_sha1[20]; + struct object_id ref_delta_oid; struct stat st; if (verbose) @@ -1133,8 +1133,8 @@ static void parse_pack_objects(unsigned char *sha1) for (i = 0; i < nr_objects; i++) { struct object_entry *obj = &objects[i]; void *data = unpack_raw_entry(obj, &ofs_delta->offset, - ref_delta_sha1, - obj->idx.oid.hash); + &ref_delta_oid, + &obj->idx.oid); obj->real_type = obj->type; if (obj->type == OBJ_OFS_DELTA) { nr_ofs_deltas++; @@ -1142,7 +1142,7 @@ static void parse_pack_objects(unsigned char *sha1) ofs_delta++; } else if (obj->type == OBJ_REF_DELTA) { ALLOC_GROW(ref_deltas, nr_ref_deltas + 1, ref_deltas_alloc); - hashcpy(ref_deltas[nr_ref_deltas].sha1, ref_delta_sha1); + hashcpy(ref_deltas[nr_ref_deltas].sha1, ref_delta_oid.hash); ref_deltas[nr_ref_deltas].obj_no = i; nr_ref_deltas++; } else if (!data) { @@ -1160,10 +1160,10 @@ static void parse_pack_objects(unsigned char *sha1) /* Check pack integrity */ flush(); - git_SHA1_Final(sha1, &input_ctx); - if (hashcmp(fill(20), sha1)) + the_hash_algo->final_fn(hash, &input_ctx); + if (hashcmp(fill(the_hash_algo->rawsz), hash)) die(_("pack is corrupted (SHA1 mismatch)")); - use(20); + use(the_hash_algo->rawsz); /* If input_fd is a file, we should have reached its end now. */ if (fstat(input_fd, &st)) @@ -1239,21 +1239,21 @@ static void resolve_deltas(void) /* * Third pass: * - append objects to convert thin pack to full pack if required - * - write the final 20-byte SHA-1 + * - write the final pack hash */ static void fix_unresolved_deltas(struct sha1file *f); -static void conclude_pack(int fix_thin_pack, const char *curr_pack, unsigned char *pack_sha1) +static void conclude_pack(int fix_thin_pack, const char *curr_pack, unsigned char *pack_hash) { if (nr_ref_deltas + nr_ofs_deltas == nr_resolved_deltas) { stop_progress(&progress); - /* Flush remaining pack final 20-byte SHA1. */ + /* Flush remaining pack final hash. */ flush(); return; } if (fix_thin_pack) { struct sha1file *f; - unsigned char read_sha1[20], tail_sha1[20]; + unsigned char read_hash[GIT_MAX_RAWSZ], tail_hash[GIT_MAX_RAWSZ]; struct strbuf msg = STRBUF_INIT; int nr_unresolved = nr_ofs_deltas + nr_ref_deltas - nr_resolved_deltas; int nr_objects_initial = nr_objects; @@ -1270,12 +1270,12 @@ static void conclude_pack(int fix_thin_pack, const char *curr_pack, unsigned cha nr_objects - nr_objects_initial); stop_progress_msg(&progress, msg.buf); strbuf_release(&msg); - sha1close(f, tail_sha1, 0); - hashcpy(read_sha1, pack_sha1); - fixup_pack_header_footer(output_fd, pack_sha1, + sha1close(f, tail_hash, 0); + hashcpy(read_hash, pack_hash); + fixup_pack_header_footer(output_fd, pack_hash, curr_pack, nr_objects, - read_sha1, consumed_bytes-20); - if (hashcmp(read_sha1, tail_sha1) != 0) + read_hash, consumed_bytes-the_hash_algo->rawsz); + if (hashcmp(read_hash, tail_hash) != 0) die(_("Unexpected tail checksum for %s " "(disk corruption?)"), curr_pack); } @@ -1392,7 +1392,7 @@ static void fix_unresolved_deltas(struct sha1file *f) static void final(const char *final_pack_name, const char *curr_pack_name, const char *final_index_name, const char *curr_index_name, const char *keep_name, const char *keep_msg, - unsigned char *sha1) + unsigned char *hash) { const char *report = "pack"; struct strbuf pack_name = STRBUF_INIT; @@ -1413,7 +1413,7 @@ static void final(const char *final_pack_name, const char *curr_pack_name, int keep_fd, keep_msg_len = strlen(keep_msg); if (!keep_name) - keep_name = odb_pack_name(&keep_name_buf, sha1, "keep"); + keep_name = odb_pack_name(&keep_name_buf, hash, "keep"); keep_fd = odb_pack_keep(keep_name); if (keep_fd < 0) { @@ -1434,7 +1434,7 @@ static void final(const char *final_pack_name, const char *curr_pack_name, if (final_pack_name != curr_pack_name) { if (!final_pack_name) - final_pack_name = odb_pack_name(&pack_name, sha1, "pack"); + final_pack_name = odb_pack_name(&pack_name, hash, "pack"); if (finalize_object_file(curr_pack_name, final_pack_name)) die(_("cannot store pack file")); } else if (from_stdin) @@ -1442,18 +1442,18 @@ static void final(const char *final_pack_name, const char *curr_pack_name, if (final_index_name != curr_index_name) { if (!final_index_name) - final_index_name = odb_pack_name(&index_name, sha1, "idx"); + final_index_name = odb_pack_name(&index_name, hash, "idx"); if (finalize_object_file(curr_index_name, final_index_name)) die(_("cannot store index file")); } else chmod(final_index_name, 0444); if (!from_stdin) { - printf("%s\n", sha1_to_hex(sha1)); + printf("%s\n", sha1_to_hex(hash)); } else { struct strbuf buf = STRBUF_INIT; - strbuf_addf(&buf, "%s\t%s\n", report, sha1_to_hex(sha1)); + strbuf_addf(&buf, "%s\t%s\n", report, sha1_to_hex(hash)); write_or_die(1, buf.buf, buf.len); strbuf_release(&buf); @@ -1637,7 +1637,7 @@ int cmd_index_pack(int argc, const char **argv, const char *prefix) keep_name_buf = STRBUF_INIT; struct pack_idx_entry **idx_objects; struct pack_idx_option opts; - unsigned char pack_sha1[20]; + unsigned char pack_hash[GIT_MAX_RAWSZ]; unsigned foreign_nr = 1; /* zero is a "good" value, assume bad */ int report_end_of_input = 0; @@ -1768,11 +1768,11 @@ int cmd_index_pack(int argc, const char **argv, const char *prefix) if (show_stat) obj_stat = xcalloc(st_add(nr_objects, 1), sizeof(struct object_stat)); ofs_deltas = xcalloc(nr_objects, sizeof(struct ofs_delta_entry)); - parse_pack_objects(pack_sha1); + parse_pack_objects(pack_hash); if (report_end_of_input) write_in_full(2, "\0", 1); resolve_deltas(); - conclude_pack(fix_thin_pack, curr_pack, pack_sha1); + conclude_pack(fix_thin_pack, curr_pack, pack_hash); free(ofs_deltas); free(ref_deltas); if (strict) @@ -1784,14 +1784,14 @@ int cmd_index_pack(int argc, const char **argv, const char *prefix) ALLOC_ARRAY(idx_objects, nr_objects); for (i = 0; i < nr_objects; i++) idx_objects[i] = &objects[i].idx; - curr_index = write_idx_file(index_name, idx_objects, nr_objects, &opts, pack_sha1); + curr_index = write_idx_file(index_name, idx_objects, nr_objects, &opts, pack_hash); free(idx_objects); if (!verify) final(pack_name, curr_pack, index_name, curr_index, keep_name, keep_msg, - pack_sha1); + pack_hash); else close(input_fd); free(objects); ``` ## brian m. carlson, 2018-02-01 02:18 Subject: [PATCH v2 07/12] pack-check: convert various uses of SHA-1 to abstract forms Message-ID: <20180201021848.533188-8-sandals@crustytoothpaste.net> URL: https://gitlist.dev/e/20180201021848.533188-8-sandals%40crustytoothpaste.net In-Reply-To: <20180201021848.533188-1-sandals@crustytoothpaste.net> ``` Convert various explicit calls to use SHA-1 functions and constants to references to the_hash_algo. Make several strings more generic with respect to the hash algorithm used. Signed-off-by: brian m. carlson --- pack-check.c | 32 ++++++++++++++++---------------- 1 file changed, 16 insertions(+), 16 deletions(-) diff --git a/pack-check.c b/pack-check.c index 073c1fbd46..403a572567 100644 --- a/pack-check.c +++ b/pack-check.c @@ -41,7 +41,7 @@ int check_pack_crc(struct packed_git *p, struct pack_window **w_curs, } while (len); index_crc = p->index_data; - index_crc += 2 + 256 + p->num_objects * (20/4) + nr; + index_crc += 2 + 256 + p->num_objects * (the_hash_algo->rawsz/4) + nr; return data_crc != ntohl(*index_crc); } @@ -54,7 +54,7 @@ static int verify_packfile(struct packed_git *p, { off_t index_size = p->index_size; const unsigned char *index_base = p->index_data; - git_SHA_CTX ctx; + git_hash_ctx ctx; unsigned char hash[GIT_MAX_RAWSZ], *pack_sig; off_t offset = 0, pack_sig_ofs = 0; uint32_t nr_objects, i; @@ -64,24 +64,24 @@ static int verify_packfile(struct packed_git *p, if (!is_pack_valid(p)) return error("packfile %s cannot be accessed", p->pack_name); - git_SHA1_Init(&ctx); + the_hash_algo->init_fn(&ctx); do { unsigned long remaining; unsigned char *in = use_pack(p, w_curs, offset, &remaining); offset += remaining; if (!pack_sig_ofs) - pack_sig_ofs = p->pack_size - 20; + pack_sig_ofs = p->pack_size - the_hash_algo->rawsz; if (offset > pack_sig_ofs) remaining -= (unsigned int)(offset - pack_sig_ofs); - git_SHA1_Update(&ctx, in, remaining); + the_hash_algo->update_fn(&ctx, in, remaining); } while (offset < pack_sig_ofs); - git_SHA1_Final(hash, &ctx); + the_hash_algo->final_fn(hash, &ctx); pack_sig = use_pack(p, w_curs, pack_sig_ofs, NULL); if (hashcmp(hash, pack_sig)) - err = error("%s SHA1 checksum mismatch", + err = error("%s pack checksum mismatch", p->pack_name); - if (hashcmp(index_base + index_size - 40, pack_sig)) - err = error("%s SHA1 does not match its index", + if (hashcmp(index_base + index_size - the_hash_algo->hexsz, pack_sig)) + err = error("%s pack checksum does not match its index", p->pack_name); unuse_pack(w_curs); @@ -165,8 +165,8 @@ int verify_pack_index(struct packed_git *p) { off_t index_size; const unsigned char *index_base; - git_SHA_CTX ctx; - unsigned char sha1[20]; + git_hash_ctx ctx; + unsigned char hash[GIT_MAX_RAWSZ]; int err = 0; if (open_pack_index(p)) @@ -175,11 +175,11 @@ int verify_pack_index(struct packed_git *p) index_base = p->index_data; /* Verify SHA1 sum of the index file */ - git_SHA1_Init(&ctx); - git_SHA1_Update(&ctx, index_base, (unsigned int)(index_size - 20)); - git_SHA1_Final(sha1, &ctx); - if (hashcmp(sha1, index_base + index_size - 20)) - err = error("Packfile index for %s SHA1 mismatch", + the_hash_algo->init_fn(&ctx); + the_hash_algo->update_fn(&ctx, index_base, (unsigned int)(index_size - the_hash_algo->rawsz)); + the_hash_algo->final_fn(hash, &ctx); + if (hashcmp(hash, index_base + index_size - the_hash_algo->rawsz)) + err = error("Packfile index for %s hash mismatch", p->pack_name); return err; } ``` ## brian m. carlson, 2018-02-01 02:18 Subject: [PATCH v2 06/12] fast-import: switch various uses of SHA-1 to the_hash_algo Message-ID: <20180201021848.533188-7-sandals@crustytoothpaste.net> URL: https://gitlist.dev/e/20180201021848.533188-7-sandals%40crustytoothpaste.net In-Reply-To: <20180201021848.533188-1-sandals@crustytoothpaste.net> ``` Switch various uses of explicit calls to SHA-1 to use the_hash_algo. Convert various uses of 20 and the GIT_SHA1 constants as well. Signed-off-by: brian m. carlson --- fast-import.c | 42 ++++++++++++++++++++++-------------------- 1 file changed, 22 insertions(+), 20 deletions(-) diff --git a/fast-import.c b/fast-import.c index b70ac025e0..f20f30ab62 100644 --- a/fast-import.c +++ b/fast-import.c @@ -1092,15 +1092,15 @@ static int store_object( unsigned char hdr[96]; struct object_id oid; unsigned long hdrlen, deltalen; - git_SHA_CTX c; + git_hash_ctx c; git_zstream s; hdrlen = xsnprintf((char *)hdr, sizeof(hdr), "%s %lu", typename(type), (unsigned long)dat->len) + 1; - git_SHA1_Init(&c); - git_SHA1_Update(&c, hdr, hdrlen); - git_SHA1_Update(&c, dat->buf, dat->len); - git_SHA1_Final(oid.hash, &c); + the_hash_algo->init_fn(&c); + the_hash_algo->update_fn(&c, hdr, hdrlen); + the_hash_algo->update_fn(&c, dat->buf, dat->len); + the_hash_algo->final_fn(oid.hash, &c); if (oidout) oidcpy(oidout, &oid); @@ -1118,11 +1118,13 @@ static int store_object( return 1; } - if (last && last->data.buf && last->depth < max_depth && dat->len > 20) { + if (last && last->data.buf && last->depth < max_depth + && dat->len > the_hash_algo->rawsz) { + delta_count_attempts_by_type[type]++; delta = diff_delta(last->data.buf, last->data.len, dat->buf, dat->len, - &deltalen, dat->len - 20); + &deltalen, dat->len - the_hash_algo->rawsz); } else delta = NULL; @@ -1231,7 +1233,7 @@ static void stream_blob(uintmax_t len, struct object_id *oidout, uintmax_t mark) struct object_id oid; unsigned long hdrlen; off_t offset; - git_SHA_CTX c; + git_hash_ctx c; git_zstream s; struct sha1file_checkpoint checkpoint; int status = Z_OK; @@ -1246,8 +1248,8 @@ static void stream_blob(uintmax_t len, struct object_id *oidout, uintmax_t mark) hdrlen = xsnprintf((char *)out_buf, out_sz, "blob %" PRIuMAX, len) + 1; - git_SHA1_Init(&c); - git_SHA1_Update(&c, out_buf, hdrlen); + the_hash_algo->init_fn(&c); + the_hash_algo->update_fn(&c, out_buf, hdrlen); crc32_begin(pack_file); @@ -1265,7 +1267,7 @@ static void stream_blob(uintmax_t len, struct object_id *oidout, uintmax_t mark) if (!n && feof(stdin)) die("EOF in data (%" PRIuMAX " bytes remaining)", len); - git_SHA1_Update(&c, in_buf, n); + the_hash_algo->update_fn(&c, in_buf, n); s.next_in = in_buf; s.avail_in = n; len -= n; @@ -1291,7 +1293,7 @@ static void stream_blob(uintmax_t len, struct object_id *oidout, uintmax_t mark) } } git_deflate_end(&s); - git_SHA1_Final(oid.hash, &c); + the_hash_algo->final_fn(oid.hash, &c); if (oidout) oidcpy(oidout, &oid); @@ -1350,11 +1352,11 @@ static void *gfi_unpack_entry( { enum object_type type; struct packed_git *p = all_packs[oe->pack_id]; - if (p == pack_data && p->pack_size < (pack_size + 20)) { + if (p == pack_data && p->pack_size < (pack_size + the_hash_algo->rawsz)) { /* The object is stored in the packfile we are writing to * and we have modified it since the last time we scanned * back to read a previously written object. If an old - * window covered [p->pack_size, p->pack_size + 20) its + * window covered [p->pack_size, p->pack_size + rawsz) its * data is stale and is not valid. Closing all windows * and updating the packfile length ensures we can read * the newly written data. @@ -1362,13 +1364,13 @@ static void *gfi_unpack_entry( close_pack_windows(p); sha1flush(pack_file); - /* We have to offer 20 bytes additional on the end of + /* We have to offer rawsz bytes additional on the end of * the packfile as the core unpacker code assumes the * footer is present at the file end and must promise - * at least 20 bytes within any window it maps. But + * at least rawsz bytes within any window it maps. But * we don't actually create the footer here. */ - p->pack_size = pack_size + 20; + p->pack_size = pack_size + the_hash_algo->rawsz; } return unpack_entry(p, oe->idx.offset, &type, sizep); } @@ -2204,7 +2206,7 @@ static void construct_path_with_fanout(const char *hex_sha1, unsigned char fanout, char *path) { unsigned int i = 0, j = 0; - if (fanout >= 20) + if (fanout >= the_hash_algo->rawsz) die("Too large fanout (%u)", fanout); while (fanout) { path[i++] = hex_sha1[j++]; @@ -2212,8 +2214,8 @@ static void construct_path_with_fanout(const char *hex_sha1, path[i++] = '/'; fanout--; } - memcpy(path + i, hex_sha1 + j, GIT_SHA1_HEXSZ - j); - path[i + GIT_SHA1_HEXSZ - j] = '\0'; + memcpy(path + i, hex_sha1 + j, the_hash_algo->hexsz - j); + path[i + the_hash_algo->hexsz - j] = '\0'; } static uintmax_t do_change_note_fanout( ``` ## brian m. carlson, 2018-02-01 02:18 Subject: [PATCH v2 08/12] pack-write: switch various SHA-1 values to abstract forms Message-ID: <20180201021848.533188-9-sandals@crustytoothpaste.net> URL: https://gitlist.dev/e/20180201021848.533188-9-sandals%40crustytoothpaste.net In-Reply-To: <20180201021848.533188-1-sandals@crustytoothpaste.net> ``` Convert various uses of hardcoded 20- and 40-based numbers to use the_hash_algo, along with direct calls to SHA-1. Adjust the names of variables to refer to "hash" instead of "sha1". Signed-off-by: brian m. carlson --- pack-write.c | 49 +++++++++++++++++++++++++------------------------ 1 file changed, 25 insertions(+), 24 deletions(-) diff --git a/pack-write.c b/pack-write.c index fea6284192..5317678c29 100644 --- a/pack-write.c +++ b/pack-write.c @@ -122,7 +122,7 @@ const char *write_idx_file(const char *index_name, struct pack_idx_entry **objec uint32_t offset = htonl(obj->offset); sha1write(f, &offset, 4); } - sha1write(f, obj->oid.hash, 20); + sha1write(f, obj->oid.hash, the_hash_algo->rawsz); if ((opts->flags & WRITE_IDX_STRICT) && (i && !oidcmp(&list[-2]->oid, &obj->oid))) die("The same object %s appears twice in the pack", @@ -169,7 +169,7 @@ const char *write_idx_file(const char *index_name, struct pack_idx_entry **objec } } - sha1write(f, sha1, 20); + sha1write(f, sha1, the_hash_algo->rawsz); sha1close(f, NULL, ((opts->flags & WRITE_IDX_VERIFY) ? CSUM_CLOSE : CSUM_FSYNC)); return index_name; @@ -203,20 +203,20 @@ off_t write_pack_header(struct sha1file *f, uint32_t nr_entries) * interested in the resulting SHA1 of pack data above partial_pack_offset. */ void fixup_pack_header_footer(int pack_fd, - unsigned char *new_pack_sha1, + unsigned char *new_pack_hash, const char *pack_name, uint32_t object_count, - unsigned char *partial_pack_sha1, + unsigned char *partial_pack_hash, off_t partial_pack_offset) { int aligned_sz, buf_sz = 8 * 1024; - git_SHA_CTX old_sha1_ctx, new_sha1_ctx; + git_hash_ctx old_hash_ctx, new_hash_ctx; struct pack_header hdr; char *buf; ssize_t read_result; - git_SHA1_Init(&old_sha1_ctx); - git_SHA1_Init(&new_sha1_ctx); + the_hash_algo->init_fn(&old_hash_ctx); + the_hash_algo->init_fn(&new_hash_ctx); if (lseek(pack_fd, 0, SEEK_SET) != 0) die_errno("Failed seeking to start of '%s'", pack_name); @@ -228,9 +228,9 @@ void fixup_pack_header_footer(int pack_fd, pack_name); if (lseek(pack_fd, 0, SEEK_SET) != 0) die_errno("Failed seeking to start of '%s'", pack_name); - git_SHA1_Update(&old_sha1_ctx, &hdr, sizeof(hdr)); + the_hash_algo->update_fn(&old_hash_ctx, &hdr, sizeof(hdr)); hdr.hdr_entries = htonl(object_count); - git_SHA1_Update(&new_sha1_ctx, &hdr, sizeof(hdr)); + the_hash_algo->update_fn(&new_hash_ctx, &hdr, sizeof(hdr)); write_or_die(pack_fd, &hdr, sizeof(hdr)); partial_pack_offset -= sizeof(hdr); @@ -238,28 +238,28 @@ void fixup_pack_header_footer(int pack_fd, aligned_sz = buf_sz - sizeof(hdr); for (;;) { ssize_t m, n; - m = (partial_pack_sha1 && partial_pack_offset < aligned_sz) ? + m = (partial_pack_hash && partial_pack_offset < aligned_sz) ? partial_pack_offset : aligned_sz; n = xread(pack_fd, buf, m); if (!n) break; if (n < 0) die_errno("Failed to checksum '%s'", pack_name); - git_SHA1_Update(&new_sha1_ctx, buf, n); + the_hash_algo->update_fn(&new_hash_ctx, buf, n); aligned_sz -= n; if (!aligned_sz) aligned_sz = buf_sz; - if (!partial_pack_sha1) + if (!partial_pack_hash) continue; - git_SHA1_Update(&old_sha1_ctx, buf, n); + the_hash_algo->update_fn(&old_hash_ctx, buf, n); partial_pack_offset -= n; if (partial_pack_offset == 0) { - unsigned char sha1[20]; - git_SHA1_Final(sha1, &old_sha1_ctx); - if (hashcmp(sha1, partial_pack_sha1) != 0) + unsigned char hash[GIT_MAX_RAWSZ]; + the_hash_algo->final_fn(hash, &old_hash_ctx); + if (hashcmp(hash, partial_pack_hash) != 0) die("Unexpected checksum for %s " "(disk corruption?)", pack_name); /* @@ -267,23 +267,24 @@ void fixup_pack_header_footer(int pack_fd, * pack, which also means making partial_pack_offset * big enough not to matter anymore. */ - git_SHA1_Init(&old_sha1_ctx); + the_hash_algo->init_fn(&old_hash_ctx); partial_pack_offset = ~partial_pack_offset; partial_pack_offset -= MSB(partial_pack_offset, 1); } } free(buf); - if (partial_pack_sha1) - git_SHA1_Final(partial_pack_sha1, &old_sha1_ctx); - git_SHA1_Final(new_pack_sha1, &new_sha1_ctx); - write_or_die(pack_fd, new_pack_sha1, 20); + if (partial_pack_hash) + the_hash_algo->final_fn(partial_pack_hash, &old_hash_ctx); + the_hash_algo->final_fn(new_pack_hash, &new_hash_ctx); + write_or_die(pack_fd, new_pack_hash, the_hash_algo->rawsz); fsync_or_die(pack_fd, pack_name); } char *index_pack_lockfile(int ip_out) { - char packname[46]; + char packname[GIT_MAX_HEXSZ + 6]; + const int len = the_hash_algo->hexsz + 6; /* * The first thing we expect from index-pack's output @@ -292,9 +293,9 @@ char *index_pack_lockfile(int ip_out) * case, we need it to remove the corresponding .keep file * later on. If we don't get that then tough luck with it. */ - if (read_in_full(ip_out, packname, 46) == 46 && packname[45] == '\n') { + if (read_in_full(ip_out, packname, len) == len && packname[len-1] == '\n') { const char *name; - packname[45] = 0; + packname[len-1] = 0; if (skip_prefix(packname, "keep\t", &name)) return xstrfmt("%s/pack/pack-%s.keep", get_object_directory(), name); ``` ## brian m. carlson, 2018-02-01 02:18 Subject: [PATCH v2 12/12] bulk-checkin: abstract SHA-1 usage Message-ID: <20180201021848.533188-13-sandals@crustytoothpaste.net> URL: https://gitlist.dev/e/20180201021848.533188-13-sandals%40crustytoothpaste.net In-Reply-To: <20180201021848.533188-1-sandals@crustytoothpaste.net> ``` Convert uses of the direct SHA-1 functions to use the_hash_algo instead. Signed-off-by: brian m. carlson --- bulk-checkin.c | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-) diff --git a/bulk-checkin.c b/bulk-checkin.c index 5f79ed6ea3..8bcd1c8665 100644 --- a/bulk-checkin.c +++ b/bulk-checkin.c @@ -93,7 +93,7 @@ static int already_written(struct bulk_checkin_state *state, unsigned char sha1[ * with a new pack. */ static int stream_to_pack(struct bulk_checkin_state *state, - git_SHA_CTX *ctx, off_t *already_hashed_to, + git_hash_ctx *ctx, off_t *already_hashed_to, int fd, size_t size, enum object_type type, const char *path, unsigned flags) { @@ -127,7 +127,7 @@ static int stream_to_pack(struct bulk_checkin_state *state, if (rsize < hsize) hsize = rsize; if (hsize) - git_SHA1_Update(ctx, ibuf, hsize); + the_hash_algo->update_fn(ctx, ibuf, hsize); *already_hashed_to = offset; } s.next_in = ibuf; @@ -192,7 +192,7 @@ static int deflate_to_pack(struct bulk_checkin_state *state, unsigned flags) { off_t seekback, already_hashed_to; - git_SHA_CTX ctx; + git_hash_ctx ctx; unsigned char obuf[16384]; unsigned header_len; struct hashfile_checkpoint checkpoint; @@ -204,8 +204,8 @@ static int deflate_to_pack(struct bulk_checkin_state *state, header_len = xsnprintf((char *)obuf, sizeof(obuf), "%s %" PRIuMAX, typename(type), (uintmax_t)size) + 1; - git_SHA1_Init(&ctx); - git_SHA1_Update(&ctx, obuf, header_len); + the_hash_algo->init_fn(&ctx); + the_hash_algo->update_fn(&ctx, obuf, header_len); /* Note: idx is non-NULL when we are writing */ if ((flags & HASH_WRITE_OBJECT) != 0) @@ -236,7 +236,7 @@ static int deflate_to_pack(struct bulk_checkin_state *state, if (lseek(fd, seekback, SEEK_SET) == (off_t) -1) return error("cannot seek back"); } - git_SHA1_Final(result_sha1, &ctx); + the_hash_algo->final_fn(result_sha1, &ctx); if (!idx) return 0; ``` ## brian m. carlson, 2018-02-01 02:18 Subject: [PATCH v2 10/12] csum-file: rename sha1file to hashfile Message-ID: <20180201021848.533188-11-sandals@crustytoothpaste.net> URL: https://gitlist.dev/e/20180201021848.533188-11-sandals%40crustytoothpaste.net In-Reply-To: <20180201021848.533188-1-sandals@crustytoothpaste.net> ``` Rename struct sha1file to struct hashfile, along with all of its related functions. The transformation in this commit was made by global search-and-replace. --- builtin/index-pack.c | 20 +++++++++---------- builtin/pack-objects.c | 52 +++++++++++++++++++++++++------------------------- bulk-checkin.c | 16 ++++++++-------- csum-file.c | 36 +++++++++++++++++----------------- csum-file.h | 34 ++++++++++++++++----------------- fast-import.c | 28 +++++++++++++-------------- pack-bitmap-write.c | 30 ++++++++++++++--------------- pack-write.c | 32 +++++++++++++++---------------- pack.h | 4 ++-- 9 files changed, 126 insertions(+), 126 deletions(-) diff --git a/builtin/index-pack.c b/builtin/index-pack.c index 40c000aca8..0bb520c731 100644 --- a/builtin/index-pack.c +++ b/builtin/index-pack.c @@ -1241,7 +1241,7 @@ static void resolve_deltas(void) * - append objects to convert thin pack to full pack if required * - write the final pack hash */ -static void fix_unresolved_deltas(struct sha1file *f); +static void fix_unresolved_deltas(struct hashfile *f); static void conclude_pack(int fix_thin_pack, const char *curr_pack, unsigned char *pack_hash) { if (nr_ref_deltas + nr_ofs_deltas == nr_resolved_deltas) { @@ -1252,7 +1252,7 @@ static void conclude_pack(int fix_thin_pack, const char *curr_pack, unsigned cha } if (fix_thin_pack) { - struct sha1file *f; + struct hashfile *f; unsigned char read_hash[GIT_MAX_RAWSZ], tail_hash[GIT_MAX_RAWSZ]; struct strbuf msg = STRBUF_INIT; int nr_unresolved = nr_ofs_deltas + nr_ref_deltas - nr_resolved_deltas; @@ -1262,7 +1262,7 @@ static void conclude_pack(int fix_thin_pack, const char *curr_pack, unsigned cha REALLOC_ARRAY(objects, nr_objects + nr_unresolved + 1); memset(objects + nr_objects + 1, 0, nr_unresolved * sizeof(*objects)); - f = sha1fd(output_fd, curr_pack); + f = hashfd(output_fd, curr_pack); fix_unresolved_deltas(f); strbuf_addf(&msg, Q_("completed with %d local object", "completed with %d local objects", @@ -1270,7 +1270,7 @@ static void conclude_pack(int fix_thin_pack, const char *curr_pack, unsigned cha nr_objects - nr_objects_initial); stop_progress_msg(&progress, msg.buf); strbuf_release(&msg); - sha1close(f, tail_hash, 0); + hashclose(f, tail_hash, 0); hashcpy(read_hash, pack_hash); fixup_pack_header_footer(output_fd, pack_hash, curr_pack, nr_objects, @@ -1286,7 +1286,7 @@ static void conclude_pack(int fix_thin_pack, const char *curr_pack, unsigned cha nr_ofs_deltas + nr_ref_deltas - nr_resolved_deltas); } -static int write_compressed(struct sha1file *f, void *in, unsigned int size) +static int write_compressed(struct hashfile *f, void *in, unsigned int size) { git_zstream stream; int status; @@ -1300,7 +1300,7 @@ static int write_compressed(struct sha1file *f, void *in, unsigned int size) stream.next_out = outbuf; stream.avail_out = sizeof(outbuf); status = git_deflate(&stream, Z_FINISH); - sha1write(f, outbuf, sizeof(outbuf) - stream.avail_out); + hashwrite(f, outbuf, sizeof(outbuf) - stream.avail_out); } while (status == Z_OK); if (status != Z_STREAM_END) @@ -1310,7 +1310,7 @@ static int write_compressed(struct sha1file *f, void *in, unsigned int size) return size; } -static struct object_entry *append_obj_to_pack(struct sha1file *f, +static struct object_entry *append_obj_to_pack(struct hashfile *f, const unsigned char *sha1, void *buf, unsigned long size, enum object_type type) { @@ -1327,7 +1327,7 @@ static struct object_entry *append_obj_to_pack(struct sha1file *f, } header[n++] = c; crc32_begin(f); - sha1write(f, header, n); + hashwrite(f, header, n); obj[0].size = size; obj[0].hdr_size = n; obj[0].type = type; @@ -1335,7 +1335,7 @@ static struct object_entry *append_obj_to_pack(struct sha1file *f, obj[1].idx.offset = obj[0].idx.offset + n; obj[1].idx.offset += write_compressed(f, buf, size); obj[0].idx.crc32 = crc32_end(f); - sha1flush(f); + hashflush(f); hashcpy(obj->idx.oid.hash, sha1); return obj; } @@ -1347,7 +1347,7 @@ static int delta_pos_compare(const void *_a, const void *_b) return a->obj_no - b->obj_no; } -static void fix_unresolved_deltas(struct sha1file *f) +static void fix_unresolved_deltas(struct hashfile *f) { struct ref_delta_entry **sorted_by_pos; int i; diff --git a/builtin/pack-objects.c b/builtin/pack-objects.c index 6b9cfc289d..d91cbd460e 100644 --- a/builtin/pack-objects.c +++ b/builtin/pack-objects.c @@ -161,7 +161,7 @@ static unsigned long do_compress(void **pptr, unsigned long size) return stream.total_out; } -static unsigned long write_large_blob_data(struct git_istream *st, struct sha1file *f, +static unsigned long write_large_blob_data(struct git_istream *st, struct hashfile *f, const struct object_id *oid) { git_zstream stream; @@ -185,7 +185,7 @@ static unsigned long write_large_blob_data(struct git_istream *st, struct sha1fi stream.next_out = obuf; stream.avail_out = sizeof(obuf); zret = git_deflate(&stream, readlen ? 0 : Z_FINISH); - sha1write(f, obuf, stream.next_out - obuf); + hashwrite(f, obuf, stream.next_out - obuf); olen += stream.next_out - obuf; } if (stream.avail_in) @@ -230,7 +230,7 @@ static int check_pack_inflate(struct packed_git *p, stream.total_in == len) ? 0 : -1; } -static void copy_pack_data(struct sha1file *f, +static void copy_pack_data(struct hashfile *f, struct packed_git *p, struct pack_window **w_curs, off_t offset, @@ -243,14 +243,14 @@ static void copy_pack_data(struct sha1file *f, in = use_pack(p, w_curs, offset, &avail); if (avail > len) avail = (unsigned long)len; - sha1write(f, in, avail); + hashwrite(f, in, avail); offset += avail; len -= avail; } } /* Return 0 if we will bust the pack-size limit */ -static unsigned long write_no_reuse_object(struct sha1file *f, struct object_entry *entry, +static unsigned long write_no_reuse_object(struct hashfile *f, struct object_entry *entry, unsigned long limit, int usable_delta) { unsigned long size, datalen; @@ -323,8 +323,8 @@ static unsigned long write_no_reuse_object(struct sha1file *f, struct object_ent free(buf); return 0; } - sha1write(f, header, hdrlen); - sha1write(f, dheader + pos, sizeof(dheader) - pos); + hashwrite(f, header, hdrlen); + hashwrite(f, dheader + pos, sizeof(dheader) - pos); hdrlen += sizeof(dheader) - pos; } else if (type == OBJ_REF_DELTA) { /* @@ -337,8 +337,8 @@ static unsigned long write_no_reuse_object(struct sha1file *f, struct object_ent free(buf); return 0; } - sha1write(f, header, hdrlen); - sha1write(f, entry->delta->idx.oid.hash, 20); + hashwrite(f, header, hdrlen); + hashwrite(f, entry->delta->idx.oid.hash, 20); hdrlen += 20; } else { if (limit && hdrlen + datalen + 20 >= limit) { @@ -347,13 +347,13 @@ static unsigned long write_no_reuse_object(struct sha1file *f, struct object_ent free(buf); return 0; } - sha1write(f, header, hdrlen); + hashwrite(f, header, hdrlen); } if (st) { datalen = write_large_blob_data(st, f, &entry->idx.oid); close_istream(st); } else { - sha1write(f, buf, datalen); + hashwrite(f, buf, datalen); free(buf); } @@ -361,7 +361,7 @@ static unsigned long write_no_reuse_object(struct sha1file *f, struct object_ent } /* Return 0 if we will bust the pack-size limit */ -static off_t write_reuse_object(struct sha1file *f, struct object_entry *entry, +static off_t write_reuse_object(struct hashfile *f, struct object_entry *entry, unsigned long limit, int usable_delta) { struct packed_git *p = entry->in_pack; @@ -412,8 +412,8 @@ static off_t write_reuse_object(struct sha1file *f, struct object_entry *entry, unuse_pack(&w_curs); return 0; } - sha1write(f, header, hdrlen); - sha1write(f, dheader + pos, sizeof(dheader) - pos); + hashwrite(f, header, hdrlen); + hashwrite(f, dheader + pos, sizeof(dheader) - pos); hdrlen += sizeof(dheader) - pos; reused_delta++; } else if (type == OBJ_REF_DELTA) { @@ -421,8 +421,8 @@ static off_t write_reuse_object(struct sha1file *f, struct object_entry *entry, unuse_pack(&w_curs); return 0; } - sha1write(f, header, hdrlen); - sha1write(f, entry->delta->idx.oid.hash, 20); + hashwrite(f, header, hdrlen); + hashwrite(f, entry->delta->idx.oid.hash, 20); hdrlen += 20; reused_delta++; } else { @@ -430,7 +430,7 @@ static off_t write_reuse_object(struct sha1file *f, struct object_entry *entry, unuse_pack(&w_curs); return 0; } - sha1write(f, header, hdrlen); + hashwrite(f, header, hdrlen); } copy_pack_data(f, p, &w_curs, offset, datalen); unuse_pack(&w_curs); @@ -439,7 +439,7 @@ static off_t write_reuse_object(struct sha1file *f, struct object_entry *entry, } /* Return 0 if we will bust the pack-size limit */ -static off_t write_object(struct sha1file *f, +static off_t write_object(struct hashfile *f, struct object_entry *entry, off_t write_offset) { @@ -512,7 +512,7 @@ enum write_one_status { WRITE_ONE_RECURSIVE = 2 /* already scheduled to be written */ }; -static enum write_one_status write_one(struct sha1file *f, +static enum write_one_status write_one(struct hashfile *f, struct object_entry *e, off_t *offset) { @@ -731,7 +731,7 @@ static struct object_entry **compute_write_order(void) return wo; } -static off_t write_reused_pack(struct sha1file *f) +static off_t write_reused_pack(struct hashfile *f) { unsigned char buffer[8192]; off_t to_write, total; @@ -762,7 +762,7 @@ static off_t write_reused_pack(struct sha1file *f) if (read_pack > to_write) read_pack = to_write; - sha1write(f, buffer, read_pack); + hashwrite(f, buffer, read_pack); to_write -= read_pack; /* @@ -791,7 +791,7 @@ static const char no_split_warning[] = N_( static void write_pack_file(void) { uint32_t i = 0, j; - struct sha1file *f; + struct hashfile *f; off_t offset; uint32_t nr_remaining = nr_result; time_t last_mtime = 0; @@ -807,7 +807,7 @@ static void write_pack_file(void) char *pack_tmp_name = NULL; if (pack_to_stdout) - f = sha1fd_throughput(1, "", progress_state); + f = hashfd_throughput(1, "", progress_state); else f = create_tmp_packfile(&pack_tmp_name); @@ -834,11 +834,11 @@ static void write_pack_file(void) * If so, rewrite it like in fast-import */ if (pack_to_stdout) { - sha1close(f, oid.hash, CSUM_CLOSE); + hashclose(f, oid.hash, CSUM_CLOSE); } else if (nr_written == nr_remaining) { - sha1close(f, oid.hash, CSUM_FSYNC); + hashclose(f, oid.hash, CSUM_FSYNC); } else { - int fd = sha1close(f, oid.hash, 0); + int fd = hashclose(f, oid.hash, 0); fixup_pack_header_footer(fd, oid.hash, pack_tmp_name, nr_written, oid.hash, offset); close(fd); diff --git a/bulk-checkin.c b/bulk-checkin.c index 3310fd210a..5f79ed6ea3 100644 --- a/bulk-checkin.c +++ b/bulk-checkin.c @@ -12,7 +12,7 @@ static struct bulk_checkin_state { unsigned plugged:1; char *pack_tmp_name; - struct sha1file *f; + struct hashfile *f; off_t offset; struct pack_idx_option pack_idx_opts; @@ -35,9 +35,9 @@ static void finish_bulk_checkin(struct bulk_checkin_state *state) unlink(state->pack_tmp_name); goto clear_exit; } else if (state->nr_written == 1) { - sha1close(state->f, oid.hash, CSUM_FSYNC); + hashclose(state->f, oid.hash, CSUM_FSYNC); } else { - int fd = sha1close(state->f, oid.hash, 0); + int fd = hashclose(state->f, oid.hash, 0); fixup_pack_header_footer(fd, oid.hash, state->pack_tmp_name, state->nr_written, oid.hash, state->offset); @@ -149,7 +149,7 @@ static int stream_to_pack(struct bulk_checkin_state *state, return -1; } - sha1write(state->f, obuf, written); + hashwrite(state->f, obuf, written); state->offset += written; } s.next_out = obuf; @@ -195,7 +195,7 @@ static int deflate_to_pack(struct bulk_checkin_state *state, git_SHA_CTX ctx; unsigned char obuf[16384]; unsigned header_len; - struct sha1file_checkpoint checkpoint; + struct hashfile_checkpoint checkpoint; struct pack_idx_entry *idx = NULL; seekback = lseek(fd, 0, SEEK_CUR); @@ -216,7 +216,7 @@ static int deflate_to_pack(struct bulk_checkin_state *state, while (1) { prepare_to_stream(state, flags); if (idx) { - sha1file_checkpoint(state->f, &checkpoint); + hashfile_checkpoint(state->f, &checkpoint); idx->offset = state->offset; crc32_begin(state->f); } @@ -230,7 +230,7 @@ static int deflate_to_pack(struct bulk_checkin_state *state, */ if (!idx) die("BUG: should not happen"); - sha1file_truncate(state->f, &checkpoint); + hashfile_truncate(state->f, &checkpoint); state->offset = checkpoint.offset; finish_bulk_checkin(state); if (lseek(fd, seekback, SEEK_SET) == (off_t) -1) @@ -242,7 +242,7 @@ static int deflate_to_pack(struct bulk_checkin_state *state, idx->crc32 = crc32_end(state->f); if (already_written(state, result_sha1)) { - sha1file_truncate(state->f, &checkpoint); + hashfile_truncate(state->f, &checkpoint); state->offset = checkpoint.offset; free(idx); } else { diff --git a/csum-file.c b/csum-file.c index 2adae04073..e4ad6337dc 100644 --- a/csum-file.c +++ b/csum-file.c @@ -11,7 +11,7 @@ #include "progress.h" #include "csum-file.h" -static void flush(struct sha1file *f, const void *buf, unsigned int count) +static void flush(struct hashfile *f, const void *buf, unsigned int count) { if (0 <= f->check_fd && count) { unsigned char check_buffer[8192]; @@ -42,7 +42,7 @@ static void flush(struct sha1file *f, const void *buf, unsigned int count) } } -void sha1flush(struct sha1file *f) +void hashflush(struct hashfile *f) { unsigned offset = f->offset; @@ -53,11 +53,11 @@ void sha1flush(struct sha1file *f) } } -int sha1close(struct sha1file *f, unsigned char *result, unsigned int flags) +int hashclose(struct hashfile *f, unsigned char *result, unsigned int flags) { int fd; - sha1flush(f); + hashflush(f); git_SHA1_Final(f->buffer, &f->ctx); if (result) hashcpy(result, f->buffer); @@ -86,7 +86,7 @@ int sha1close(struct sha1file *f, unsigned char *result, unsigned int flags) return fd; } -void sha1write(struct sha1file *f, const void *buf, unsigned int count) +void hashwrite(struct hashfile *f, const void *buf, unsigned int count) { while (count) { unsigned offset = f->offset; @@ -118,15 +118,15 @@ void sha1write(struct sha1file *f, const void *buf, unsigned int count) } } -struct sha1file *sha1fd(int fd, const char *name) +struct hashfile *hashfd(int fd, const char *name) { - return sha1fd_throughput(fd, name, NULL); + return hashfd_throughput(fd, name, NULL); } -struct sha1file *sha1fd_check(const char *name) +struct hashfile *hashfd_check(const char *name) { int sink, check; - struct sha1file *f; + struct hashfile *f; sink = open("/dev/null", O_WRONLY); if (sink < 0) @@ -134,14 +134,14 @@ struct sha1file *sha1fd_check(const char *name) check = open(name, O_RDONLY); if (check < 0) die_errno("unable to open '%s'", name); - f = sha1fd(sink, name); + f = hashfd(sink, name); f->check_fd = check; return f; } -struct sha1file *sha1fd_throughput(int fd, const char *name, struct progress *tp) +struct hashfile *hashfd_throughput(int fd, const char *name, struct progress *tp) { - struct sha1file *f = xmalloc(sizeof(*f)); + struct hashfile *f = xmalloc(sizeof(*f)); f->fd = fd; f->check_fd = -1; f->offset = 0; @@ -153,14 +153,14 @@ struct sha1file *sha1fd_throughput(int fd, const char *name, struct progress *tp return f; } -void sha1file_checkpoint(struct sha1file *f, struct sha1file_checkpoint *checkpoint) +void hashfile_checkpoint(struct hashfile *f, struct hashfile_checkpoint *checkpoint) { - sha1flush(f); + hashflush(f); checkpoint->offset = f->total; checkpoint->ctx = f->ctx; } -int sha1file_truncate(struct sha1file *f, struct sha1file_checkpoint *checkpoint) +int hashfile_truncate(struct hashfile *f, struct hashfile_checkpoint *checkpoint) { off_t offset = checkpoint->offset; @@ -169,17 +169,17 @@ int sha1file_truncate(struct sha1file *f, struct sha1file_checkpoint *checkpoint return -1; f->total = offset; f->ctx = checkpoint->ctx; - f->offset = 0; /* sha1flush() was called in checkpoint */ + f->offset = 0; /* hashflush() was called in checkpoint */ return 0; } -void crc32_begin(struct sha1file *f) +void crc32_begin(struct hashfile *f) { f->crc32 = crc32(0, NULL, 0); f->do_crc = 1; } -uint32_t crc32_end(struct sha1file *f) +uint32_t crc32_end(struct hashfile *f) { f->do_crc = 0; return f->crc32; diff --git a/csum-file.h b/csum-file.h index 7530927d77..ceb3e5712d 100644 --- a/csum-file.h +++ b/csum-file.h @@ -4,7 +4,7 @@ struct progress; /* A SHA1-protected file */ -struct sha1file { +struct hashfile { int fd; int check_fd; unsigned int offset; @@ -18,36 +18,36 @@ struct sha1file { }; /* Checkpoint */ -struct sha1file_checkpoint { +struct hashfile_checkpoint { off_t offset; git_SHA_CTX ctx; }; -extern void sha1file_checkpoint(struct sha1file *, struct sha1file_checkpoint *); -extern int sha1file_truncate(struct sha1file *, struct sha1file_checkpoint *); +extern void hashfile_checkpoint(struct hashfile *, struct hashfile_checkpoint *); +extern int hashfile_truncate(struct hashfile *, struct hashfile_checkpoint *); -/* sha1close flags */ +/* hashclose flags */ #define CSUM_CLOSE 1 #define CSUM_FSYNC 2 -extern struct sha1file *sha1fd(int fd, const char *name); -extern struct sha1file *sha1fd_check(const char *name); -extern struct sha1file *sha1fd_throughput(int fd, const char *name, struct progress *tp); -extern int sha1close(struct sha1file *, unsigned char *, unsigned int); -extern void sha1write(struct sha1file *, const void *, unsigned int); -extern void sha1flush(struct sha1file *f); -extern void crc32_begin(struct sha1file *); -extern uint32_t crc32_end(struct sha1file *); +extern struct hashfile *hashfd(int fd, const char *name); +extern struct hashfile *hashfd_check(const char *name); +extern struct hashfile *hashfd_throughput(int fd, const char *name, struct progress *tp); +extern int hashclose(struct hashfile *, unsigned char *, unsigned int); +extern void hashwrite(struct hashfile *, const void *, unsigned int); +extern void hashflush(struct hashfile *f); +extern void crc32_begin(struct hashfile *); +extern uint32_t crc32_end(struct hashfile *); -static inline void sha1write_u8(struct sha1file *f, uint8_t data) +static inline void hashwrite_u8(struct hashfile *f, uint8_t data) { - sha1write(f, &data, sizeof(data)); + hashwrite(f, &data, sizeof(data)); } -static inline void sha1write_be32(struct sha1file *f, uint32_t data) +static inline void hashwrite_be32(struct hashfile *f, uint32_t data) { data = htonl(data); - sha1write(f, &data, sizeof(data)); + hashwrite(f, &data, sizeof(data)); } #endif diff --git a/fast-import.c b/fast-import.c index f20f30ab62..0f818cd932 100644 --- a/fast-import.c +++ b/fast-import.c @@ -316,7 +316,7 @@ static struct atom_str **atom_table; /* The .pack file being generated */ static struct pack_idx_option pack_idx_opts; static unsigned int pack_id; -static struct sha1file *pack_file; +static struct hashfile *pack_file; static struct packed_git *pack_data; static struct packed_git **all_packs; static off_t pack_size; @@ -905,12 +905,12 @@ static void start_packfile(void) p->pack_fd = pack_fd; p->do_not_close = 1; - pack_file = sha1fd(pack_fd, p->pack_name); + pack_file = hashfd(pack_fd, p->pack_name); hdr.hdr_signature = htonl(PACK_SIGNATURE); hdr.hdr_version = htonl(2); hdr.hdr_entries = 0; - sha1write(pack_file, &hdr, sizeof(hdr)); + hashwrite(pack_file, &hdr, sizeof(hdr)); pack_data = p; pack_size = sizeof(hdr); @@ -1016,7 +1016,7 @@ static void end_packfile(void) struct tag *t; close_pack_windows(pack_data); - sha1close(pack_file, cur_pack_oid.hash, 0); + hashclose(pack_file, cur_pack_oid.hash, 0); fixup_pack_header_footer(pack_data->pack_fd, pack_data->sha1, pack_data->pack_name, object_count, cur_pack_oid.hash, pack_size); @@ -1182,23 +1182,23 @@ static int store_object( hdrlen = encode_in_pack_object_header(hdr, sizeof(hdr), OBJ_OFS_DELTA, deltalen); - sha1write(pack_file, hdr, hdrlen); + hashwrite(pack_file, hdr, hdrlen); pack_size += hdrlen; hdr[pos] = ofs & 127; while (ofs >>= 7) hdr[--pos] = 128 | (--ofs & 127); - sha1write(pack_file, hdr + pos, sizeof(hdr) - pos); + hashwrite(pack_file, hdr + pos, sizeof(hdr) - pos); pack_size += sizeof(hdr) - pos; } else { e->depth = 0; hdrlen = encode_in_pack_object_header(hdr, sizeof(hdr), type, dat->len); - sha1write(pack_file, hdr, hdrlen); + hashwrite(pack_file, hdr, hdrlen); pack_size += hdrlen; } - sha1write(pack_file, out, s.total_out); + hashwrite(pack_file, out, s.total_out); pack_size += s.total_out; e->idx.crc32 = crc32_end(pack_file); @@ -1217,9 +1217,9 @@ static int store_object( return 0; } -static void truncate_pack(struct sha1file_checkpoint *checkpoint) +static void truncate_pack(struct hashfile_checkpoint *checkpoint) { - if (sha1file_truncate(pack_file, checkpoint)) + if (hashfile_truncate(pack_file, checkpoint)) die_errno("cannot truncate pack to skip duplicate"); pack_size = checkpoint->offset; } @@ -1235,7 +1235,7 @@ static void stream_blob(uintmax_t len, struct object_id *oidout, uintmax_t mark) off_t offset; git_hash_ctx c; git_zstream s; - struct sha1file_checkpoint checkpoint; + struct hashfile_checkpoint checkpoint; int status = Z_OK; /* Determine if we should auto-checkpoint. */ @@ -1243,7 +1243,7 @@ static void stream_blob(uintmax_t len, struct object_id *oidout, uintmax_t mark) || (pack_size + 60 + len) < pack_size) cycle_packfile(); - sha1file_checkpoint(pack_file, &checkpoint); + hashfile_checkpoint(pack_file, &checkpoint); offset = checkpoint.offset; hdrlen = xsnprintf((char *)out_buf, out_sz, "blob %" PRIuMAX, len) + 1; @@ -1277,7 +1277,7 @@ static void stream_blob(uintmax_t len, struct object_id *oidout, uintmax_t mark) if (!s.avail_out || status == Z_STREAM_END) { size_t n = s.next_out - out_buf; - sha1write(pack_file, out_buf, n); + hashwrite(pack_file, out_buf, n); pack_size += n; s.next_out = out_buf; s.avail_out = out_sz; @@ -1362,7 +1362,7 @@ static void *gfi_unpack_entry( * the newly written data. */ close_pack_windows(p); - sha1flush(pack_file); + hashflush(pack_file); /* We have to offer rawsz bytes additional on the end of * the packfile as the core unpacker code assumes the diff --git a/pack-bitmap-write.c b/pack-bitmap-write.c index a8df5ce2ab..e01f992884 100644 --- a/pack-bitmap-write.c +++ b/pack-bitmap-write.c @@ -440,19 +440,19 @@ void bitmap_writer_select_commits(struct commit **indexed_commits, } -static int sha1write_ewah_helper(void *f, const void *buf, size_t len) +static int hashwrite_ewah_helper(void *f, const void *buf, size_t len) { - /* sha1write will die on error */ - sha1write(f, buf, len); + /* hashwrite will die on error */ + hashwrite(f, buf, len); return len; } /** * Write the bitmap index to disk */ -static inline void dump_bitmap(struct sha1file *f, struct ewah_bitmap *bitmap) +static inline void dump_bitmap(struct hashfile *f, struct ewah_bitmap *bitmap) { - if (ewah_serialize_to(bitmap, sha1write_ewah_helper, f) < 0) + if (ewah_serialize_to(bitmap, hashwrite_ewah_helper, f) < 0) die("Failed to write bitmap index"); } @@ -462,7 +462,7 @@ static const unsigned char *sha1_access(size_t pos, void *table) return index[pos]->oid.hash; } -static void write_selected_commits_v1(struct sha1file *f, +static void write_selected_commits_v1(struct hashfile *f, struct pack_idx_entry **index, uint32_t index_nr) { @@ -477,15 +477,15 @@ static void write_selected_commits_v1(struct sha1file *f, if (commit_pos < 0) die("BUG: trying to write commit not in index"); - sha1write_be32(f, commit_pos); - sha1write_u8(f, stored->xor_offset); - sha1write_u8(f, stored->flags); + hashwrite_be32(f, commit_pos); + hashwrite_u8(f, stored->xor_offset); + hashwrite_u8(f, stored->flags); dump_bitmap(f, stored->write_as); } } -static void write_hash_cache(struct sha1file *f, +static void write_hash_cache(struct hashfile *f, struct pack_idx_entry **index, uint32_t index_nr) { @@ -494,7 +494,7 @@ static void write_hash_cache(struct sha1file *f, for (i = 0; i < index_nr; ++i) { struct object_entry *entry = (struct object_entry *)index[i]; uint32_t hash_value = htonl(entry->hash); - sha1write(f, &hash_value, sizeof(hash_value)); + hashwrite(f, &hash_value, sizeof(hash_value)); } } @@ -511,13 +511,13 @@ void bitmap_writer_finish(struct pack_idx_entry **index, static uint16_t default_version = 1; static uint16_t flags = BITMAP_OPT_FULL_DAG; struct strbuf tmp_file = STRBUF_INIT; - struct sha1file *f; + struct hashfile *f; struct bitmap_disk_header header; int fd = odb_mkstemp(&tmp_file, "pack/tmp_bitmap_XXXXXX"); - f = sha1fd(fd, tmp_file.buf); + f = hashfd(fd, tmp_file.buf); memcpy(header.magic, BITMAP_IDX_SIGNATURE, sizeof(BITMAP_IDX_SIGNATURE)); header.version = htons(default_version); @@ -525,7 +525,7 @@ void bitmap_writer_finish(struct pack_idx_entry **index, header.entry_count = htonl(writer.selected_nr); hashcpy(header.checksum, writer.pack_checksum); - sha1write(f, &header, sizeof(header)); + hashwrite(f, &header, sizeof(header)); dump_bitmap(f, writer.commits); dump_bitmap(f, writer.trees); dump_bitmap(f, writer.blobs); @@ -535,7 +535,7 @@ void bitmap_writer_finish(struct pack_idx_entry **index, if (options & BITMAP_OPT_HASH_CACHE) write_hash_cache(f, index, index_nr); - sha1close(f, NULL, CSUM_FSYNC); + hashclose(f, NULL, CSUM_FSYNC); if (adjust_shared_perm(tmp_file.buf)) die_errno("unable to make temporary bitmap file readable"); diff --git a/pack-write.c b/pack-write.c index 5317678c29..d775c7406d 100644 --- a/pack-write.c +++ b/pack-write.c @@ -46,7 +46,7 @@ const char *write_idx_file(const char *index_name, struct pack_idx_entry **objec int nr_objects, const struct pack_idx_option *opts, const unsigned char *sha1) { - struct sha1file *f; + struct hashfile *f; struct pack_idx_entry **sorted_by_sha, **list, **last; off_t last_obj_offset = 0; uint32_t array[256]; @@ -68,7 +68,7 @@ const char *write_idx_file(const char *index_name, struct pack_idx_entry **objec if (opts->flags & WRITE_IDX_VERIFY) { assert(index_name); - f = sha1fd_check(index_name); + f = hashfd_check(index_name); } else { if (!index_name) { struct strbuf tmp_file = STRBUF_INIT; @@ -80,7 +80,7 @@ const char *write_idx_file(const char *index_name, struct pack_idx_entry **objec if (fd < 0) die_errno("unable to create '%s'", index_name); } - f = sha1fd(fd, index_name); + f = hashfd(fd, index_name); } /* if last object's offset is >= 2^31 we should use index V2 */ @@ -91,7 +91,7 @@ const char *write_idx_file(const char *index_name, struct pack_idx_entry **objec struct pack_idx_header hdr; hdr.idx_signature = htonl(PACK_IDX_SIGNATURE); hdr.idx_version = htonl(index_version); - sha1write(f, &hdr, sizeof(hdr)); + hashwrite(f, &hdr, sizeof(hdr)); } /* @@ -110,7 +110,7 @@ const char *write_idx_file(const char *index_name, struct pack_idx_entry **objec array[i] = htonl(next - sorted_by_sha); list = next; } - sha1write(f, array, 256 * 4); + hashwrite(f, array, 256 * 4); /* * Write the actual SHA1 entries.. @@ -120,9 +120,9 @@ const char *write_idx_file(const char *index_name, struct pack_idx_entry **objec struct pack_idx_entry *obj = *list++; if (index_version < 2) { uint32_t offset = htonl(obj->offset); - sha1write(f, &offset, 4); + hashwrite(f, &offset, 4); } - sha1write(f, obj->oid.hash, the_hash_algo->rawsz); + hashwrite(f, obj->oid.hash, the_hash_algo->rawsz); if ((opts->flags & WRITE_IDX_STRICT) && (i && !oidcmp(&list[-2]->oid, &obj->oid))) die("The same object %s appears twice in the pack", @@ -137,7 +137,7 @@ const char *write_idx_file(const char *index_name, struct pack_idx_entry **objec for (i = 0; i < nr_objects; i++) { struct pack_idx_entry *obj = *list++; uint32_t crc32_val = htonl(obj->crc32); - sha1write(f, &crc32_val, 4); + hashwrite(f, &crc32_val, 4); } /* write the 32-bit offset table */ @@ -150,7 +150,7 @@ const char *write_idx_file(const char *index_name, struct pack_idx_entry **objec ? (0x80000000 | nr_large_offset++) : obj->offset); offset = htonl(offset); - sha1write(f, &offset, 4); + hashwrite(f, &offset, 4); } /* write the large offset table */ @@ -164,25 +164,25 @@ const char *write_idx_file(const char *index_name, struct pack_idx_entry **objec continue; split[0] = htonl(offset >> 32); split[1] = htonl(offset & 0xffffffff); - sha1write(f, split, 8); + hashwrite(f, split, 8); nr_large_offset--; } } - sha1write(f, sha1, the_hash_algo->rawsz); - sha1close(f, NULL, ((opts->flags & WRITE_IDX_VERIFY) + hashwrite(f, sha1, the_hash_algo->rawsz); + hashclose(f, NULL, ((opts->flags & WRITE_IDX_VERIFY) ? CSUM_CLOSE : CSUM_FSYNC)); return index_name; } -off_t write_pack_header(struct sha1file *f, uint32_t nr_entries) +off_t write_pack_header(struct hashfile *f, uint32_t nr_entries) { struct pack_header hdr; hdr.hdr_signature = htonl(PACK_SIGNATURE); hdr.hdr_version = htonl(PACK_VERSION); hdr.hdr_entries = htonl(nr_entries); - sha1write(f, &hdr, sizeof(hdr)); + hashwrite(f, &hdr, sizeof(hdr)); return sizeof(hdr); } @@ -333,14 +333,14 @@ int encode_in_pack_object_header(unsigned char *hdr, int hdr_len, return n; } -struct sha1file *create_tmp_packfile(char **pack_tmp_name) +struct hashfile *create_tmp_packfile(char **pack_tmp_name) { struct strbuf tmpname = STRBUF_INIT; int fd; fd = odb_mkstemp(&tmpname, "pack/tmp_pack_XXXXXX"); *pack_tmp_name = strbuf_detach(&tmpname, NULL); - return sha1fd(fd, *pack_tmp_name); + return hashfd(fd, *pack_tmp_name); } void finish_tmp_packfile(struct strbuf *name_buffer, diff --git a/pack.h b/pack.h index 8294341af1..34a9d458b4 100644 --- a/pack.h +++ b/pack.h @@ -81,7 +81,7 @@ extern const char *write_idx_file(const char *index_name, struct pack_idx_entry extern int check_pack_crc(struct packed_git *p, struct pack_window **w_curs, off_t offset, off_t len, unsigned int nr); extern int verify_pack_index(struct packed_git *); extern int verify_pack(struct packed_git *, verify_fn fn, struct progress *, uint32_t); -extern off_t write_pack_header(struct sha1file *f, uint32_t); +extern off_t write_pack_header(struct hashfile *f, uint32_t); extern void fixup_pack_header_footer(int, unsigned char *, const char *, uint32_t, unsigned char *, off_t); extern char *index_pack_lockfile(int fd); @@ -98,7 +98,7 @@ extern int encode_in_pack_object_header(unsigned char *hdr, int hdr_len, #define PH_ERROR_PROTOCOL (-3) extern int read_pack_header(int fd, struct pack_header *); -extern struct sha1file *create_tmp_packfile(char **pack_tmp_name); +extern struct hashfile *create_tmp_packfile(char **pack_tmp_name); extern void finish_tmp_packfile(struct strbuf *name_buffer, const char *pack_tmp_name, struct pack_idx_entry **written_list, uint32_t nr_written, struct pack_idx_option *pack_idx_opts, unsigned char sha1[]); #endif ``` ## brian m. carlson, 2018-02-01 02:18 Subject: [PATCH v2 11/12] csum-file: abstract uses of SHA-1 Message-ID: <20180201021848.533188-12-sandals@crustytoothpaste.net> URL: https://gitlist.dev/e/20180201021848.533188-12-sandals%40crustytoothpaste.net In-Reply-To: <20180201021848.533188-1-sandals@crustytoothpaste.net> ``` Convert several direct uses of SHA-1 to use the_hash_algo instead. Convert one use of the constant 20 as well. Signed-off-by: brian m. carlson --- csum-file.c | 10 +++++----- csum-file.h | 4 ++-- 2 files changed, 7 insertions(+), 7 deletions(-) diff --git a/csum-file.c b/csum-file.c index e4ad6337dc..5eda7fb6af 100644 --- a/csum-file.c +++ b/csum-file.c @@ -47,7 +47,7 @@ void hashflush(struct hashfile *f) unsigned offset = f->offset; if (offset) { - git_SHA1_Update(&f->ctx, f->buffer, offset); + the_hash_algo->update_fn(&f->ctx, f->buffer, offset); flush(f, f->buffer, offset); f->offset = 0; } @@ -58,12 +58,12 @@ int hashclose(struct hashfile *f, unsigned char *result, unsigned int flags) int fd; hashflush(f); - git_SHA1_Final(f->buffer, &f->ctx); + the_hash_algo->final_fn(f->buffer, &f->ctx); if (result) hashcpy(result, f->buffer); if (flags & (CSUM_CLOSE | CSUM_FSYNC)) { /* write checksum and close fd */ - flush(f, f->buffer, 20); + flush(f, f->buffer, the_hash_algo->rawsz); if (flags & CSUM_FSYNC) fsync_or_die(f->fd, f->name); if (close(f->fd)) @@ -110,7 +110,7 @@ void hashwrite(struct hashfile *f, const void *buf, unsigned int count) buf = (char *) buf + nr; left -= nr; if (!left) { - git_SHA1_Update(&f->ctx, data, offset); + the_hash_algo->update_fn(&f->ctx, data, offset); flush(f, data, offset); offset = 0; } @@ -149,7 +149,7 @@ struct hashfile *hashfd_throughput(int fd, const char *name, struct progress *tp f->tp = tp; f->name = name; f->do_crc = 0; - git_SHA1_Init(&f->ctx); + the_hash_algo->init_fn(&f->ctx); return f; } diff --git a/csum-file.h b/csum-file.h index ceb3e5712d..992e5c0141 100644 --- a/csum-file.h +++ b/csum-file.h @@ -8,7 +8,7 @@ struct hashfile { int fd; int check_fd; unsigned int offset; - git_SHA_CTX ctx; + git_hash_ctx ctx; off_t total; struct progress *tp; const char *name; @@ -20,7 +20,7 @@ struct hashfile { /* Checkpoint */ struct hashfile_checkpoint { off_t offset; - git_SHA_CTX ctx; + git_hash_ctx ctx; }; extern void hashfile_checkpoint(struct hashfile *, struct hashfile_checkpoint *); ``` ## brian m. carlson, 2018-02-01 02:18 Subject: [PATCH v2 09/12] read-cache: abstract away uses of SHA-1 Message-ID: <20180201021848.533188-10-sandals@crustytoothpaste.net> URL: https://gitlist.dev/e/20180201021848.533188-10-sandals%40crustytoothpaste.net In-Reply-To: <20180201021848.533188-1-sandals@crustytoothpaste.net> ``` Convert various uses of direct calls to SHA-1 and 20- and 40-based constants to use the_hash_algo instead. Don't yet convert the on-disk data structures, which will be handled in a future commit. Adjust some comments so as not to refer explicitly to SHA-1. Signed-off-by: brian m. carlson --- read-cache.c | 58 +++++++++++++++++++++++++++++----------------------------- 1 file changed, 29 insertions(+), 29 deletions(-) diff --git a/read-cache.c b/read-cache.c index 2eb81a66b9..15bbb9bab4 100644 --- a/read-cache.c +++ b/read-cache.c @@ -1545,8 +1545,8 @@ int verify_ce_order; static int verify_hdr(struct cache_header *hdr, unsigned long size) { - git_SHA_CTX c; - unsigned char sha1[20]; + git_hash_ctx c; + unsigned char hash[GIT_MAX_RAWSZ]; int hdr_version; if (hdr->hdr_signature != htonl(CACHE_SIGNATURE)) @@ -1558,10 +1558,10 @@ static int verify_hdr(struct cache_header *hdr, unsigned long size) if (!verify_index_checksum) return 0; - git_SHA1_Init(&c); - git_SHA1_Update(&c, hdr, size - 20); - git_SHA1_Final(sha1, &c); - if (hashcmp(sha1, (unsigned char *)hdr + size - 20)) + the_hash_algo->init_fn(&c); + the_hash_algo->update_fn(&c, hdr, size - the_hash_algo->rawsz); + the_hash_algo->final_fn(hash, &c); + if (hashcmp(hash, (unsigned char *)hdr + size - the_hash_algo->rawsz)) return error("bad index file sha1 signature"); return 0; } @@ -1791,7 +1791,7 @@ int do_read_index(struct index_state *istate, const char *path, int must_exist) die_errno("cannot stat the open index"); mmap_size = xsize_t(st.st_size); - if (mmap_size < sizeof(struct cache_header) + 20) + if (mmap_size < sizeof(struct cache_header) + the_hash_algo->rawsz) die("index file smaller than expected"); mmap = xmmap(NULL, mmap_size, PROT_READ, MAP_PRIVATE, fd, 0); @@ -1803,7 +1803,7 @@ int do_read_index(struct index_state *istate, const char *path, int must_exist) if (verify_hdr(hdr, mmap_size) < 0) goto unmap; - hashcpy(istate->sha1, (const unsigned char *)hdr + mmap_size - 20); + hashcpy(istate->sha1, (const unsigned char *)hdr + mmap_size - the_hash_algo->rawsz); istate->version = ntohl(hdr->hdr_version); istate->cache_nr = ntohl(hdr->hdr_entries); istate->cache_alloc = alloc_nr(istate->cache_nr); @@ -1831,7 +1831,7 @@ int do_read_index(struct index_state *istate, const char *path, int must_exist) istate->timestamp.sec = st.st_mtime; istate->timestamp.nsec = ST_MTIME_NSEC(st); - while (src_offset <= mmap_size - 20 - 8) { + while (src_offset <= mmap_size - the_hash_algo->rawsz - 8) { /* After an array of active_nr index entries, * there can be arbitrary number of extended * sections, each of which is prefixed with @@ -1957,11 +1957,11 @@ int unmerged_index(const struct index_state *istate) static unsigned char write_buffer[WRITE_BUFFER_SIZE]; static unsigned long write_buffer_len; -static int ce_write_flush(git_SHA_CTX *context, int fd) +static int ce_write_flush(git_hash_ctx *context, int fd) { unsigned int buffered = write_buffer_len; if (buffered) { - git_SHA1_Update(context, write_buffer, buffered); + the_hash_algo->update_fn(context, write_buffer, buffered); if (write_in_full(fd, write_buffer, buffered) < 0) return -1; write_buffer_len = 0; @@ -1969,7 +1969,7 @@ static int ce_write_flush(git_SHA_CTX *context, int fd) return 0; } -static int ce_write(git_SHA_CTX *context, int fd, void *data, unsigned int len) +static int ce_write(git_hash_ctx *context, int fd, void *data, unsigned int len) { while (len) { unsigned int buffered = write_buffer_len; @@ -1991,7 +1991,7 @@ static int ce_write(git_SHA_CTX *context, int fd, void *data, unsigned int len) return 0; } -static int write_index_ext_header(git_SHA_CTX *context, int fd, +static int write_index_ext_header(git_hash_ctx *context, int fd, unsigned int ext, unsigned int sz) { ext = htonl(ext); @@ -2000,26 +2000,26 @@ static int write_index_ext_header(git_SHA_CTX *context, int fd, (ce_write(context, fd, &sz, 4) < 0)) ? -1 : 0; } -static int ce_flush(git_SHA_CTX *context, int fd, unsigned char *sha1) +static int ce_flush(git_hash_ctx *context, int fd, unsigned char *hash) { unsigned int left = write_buffer_len; if (left) { write_buffer_len = 0; - git_SHA1_Update(context, write_buffer, left); + the_hash_algo->update_fn(context, write_buffer, left); } - /* Flush first if not enough space for SHA1 signature */ - if (left + 20 > WRITE_BUFFER_SIZE) { + /* Flush first if not enough space for hash signature */ + if (left + the_hash_algo->rawsz > WRITE_BUFFER_SIZE) { if (write_in_full(fd, write_buffer, left) < 0) return -1; left = 0; } - /* Append the SHA1 signature at the end */ - git_SHA1_Final(write_buffer + left, context); - hashcpy(sha1, write_buffer + left); - left += 20; + /* Append the hash signature at the end */ + the_hash_algo->final_fn(write_buffer + left, context); + hashcpy(hash, write_buffer + left); + left += the_hash_algo->rawsz; return (write_in_full(fd, write_buffer, left) < 0) ? -1 : 0; } @@ -2100,7 +2100,7 @@ static void copy_cache_entry_to_ondisk(struct ondisk_cache_entry *ondisk, } } -static int ce_write_entry(git_SHA_CTX *c, int fd, struct cache_entry *ce, +static int ce_write_entry(git_hash_ctx *c, int fd, struct cache_entry *ce, struct strbuf *previous_name, struct ondisk_cache_entry *ondisk) { int size; @@ -2167,7 +2167,7 @@ static int verify_index_from(const struct index_state *istate, const char *path) int fd; ssize_t n; struct stat st; - unsigned char sha1[20]; + unsigned char hash[GIT_MAX_RAWSZ]; if (!istate->initialized) return 0; @@ -2179,14 +2179,14 @@ static int verify_index_from(const struct index_state *istate, const char *path) if (fstat(fd, &st)) goto out; - if (st.st_size < sizeof(struct cache_header) + 20) + if (st.st_size < sizeof(struct cache_header) + the_hash_algo->rawsz) goto out; - n = pread_in_full(fd, sha1, 20, st.st_size - 20); - if (n != 20) + n = pread_in_full(fd, hash, the_hash_algo->rawsz, st.st_size - the_hash_algo->rawsz); + if (n != the_hash_algo->rawsz) goto out; - if (hashcmp(istate->sha1, sha1)) + if (hashcmp(istate->sha1, hash)) goto out; close(fd); @@ -2235,7 +2235,7 @@ static int do_write_index(struct index_state *istate, struct tempfile *tempfile, int strip_extensions) { int newfd = tempfile->fd; - git_SHA_CTX c; + git_hash_ctx c; struct cache_header hdr; int i, err = 0, removed, extended, hdr_version; struct cache_entry **cache = istate->cache; @@ -2273,7 +2273,7 @@ static int do_write_index(struct index_state *istate, struct tempfile *tempfile, hdr.hdr_version = htonl(hdr_version); hdr.hdr_entries = htonl(entries - removed); - git_SHA1_Init(&c); + the_hash_algo->init_fn(&c); if (ce_write(&c, newfd, &hdr, sizeof(hdr)) < 0) return -1; ``` ## brian m. carlson, 2018-02-01 02:18 Subject: [PATCH v2 04/12] builtin/unpack-objects: switch uses of SHA-1 to the_hash_algo Message-ID: <20180201021848.533188-5-sandals@crustytoothpaste.net> URL: https://gitlist.dev/e/20180201021848.533188-5-sandals%40crustytoothpaste.net In-Reply-To: <20180201021848.533188-1-sandals@crustytoothpaste.net> ``` Switch various uses of explicit calls to SHA-1 into references to the_hash_algo to better abstract away the various uses of it. Signed-off-by: brian m. carlson --- builtin/unpack-objects.c | 18 +++++++++--------- 1 file changed, 9 insertions(+), 9 deletions(-) diff --git a/builtin/unpack-objects.c b/builtin/unpack-objects.c index 62ea264c46..813ca31979 100644 --- a/builtin/unpack-objects.c +++ b/builtin/unpack-objects.c @@ -21,7 +21,7 @@ static unsigned char buffer[4096]; static unsigned int offset, len; static off_t consumed_bytes; static off_t max_input_size; -static git_SHA_CTX ctx; +static git_hash_ctx ctx; static struct fsck_options fsck_options = FSCK_OPTIONS_STRICT; /* @@ -62,7 +62,7 @@ static void *fill(int min) if (min > sizeof(buffer)) die("cannot fill %d bytes", min); if (offset) { - git_SHA1_Update(&ctx, buffer, offset); + the_hash_algo->update_fn(&ctx, buffer, offset); memmove(buffer, buffer + offset, len); offset = 0; } @@ -345,8 +345,8 @@ static void unpack_delta_entry(enum object_type type, unsigned long delta_size, struct object_id base_oid; if (type == OBJ_REF_DELTA) { - hashcpy(base_oid.hash, fill(GIT_SHA1_RAWSZ)); - use(GIT_SHA1_RAWSZ); + hashcpy(base_oid.hash, fill(the_hash_algo->rawsz)); + use(the_hash_algo->rawsz); delta_data = get_data(delta_size); if (dry_run || !delta_data) { free(delta_data); @@ -564,15 +564,15 @@ int cmd_unpack_objects(int argc, const char **argv, const char *prefix) /* We don't take any non-flag arguments now.. Maybe some day */ usage(unpack_usage); } - git_SHA1_Init(&ctx); + the_hash_algo->init_fn(&ctx); unpack_all(); - git_SHA1_Update(&ctx, buffer, offset); - git_SHA1_Final(oid.hash, &ctx); + the_hash_algo->update_fn(&ctx, buffer, offset); + the_hash_algo->final_fn(oid.hash, &ctx); if (strict) write_rest(); - if (hashcmp(fill(GIT_SHA1_RAWSZ), oid.hash)) + if (hashcmp(fill(the_hash_algo->rawsz), oid.hash)) die("final sha1 did not match"); - use(GIT_SHA1_RAWSZ); + use(the_hash_algo->rawsz); /* Write the last part of the buffer to stdout */ while (len) { ``` ## brian m. carlson, 2018-02-01 02:18 Subject: [PATCH v2 05/12] sha1_file: switch uses of SHA-1 to the_hash_algo Message-ID: <20180201021848.533188-6-sandals@crustytoothpaste.net> URL: https://gitlist.dev/e/20180201021848.533188-6-sandals%40crustytoothpaste.net In-Reply-To: <20180201021848.533188-1-sandals@crustytoothpaste.net> ``` Switch various uses of explicit calls to SHA-1 into references to the_hash_algo for better abstraction. Convert some calls to use struct object_id. Signed-off-by: brian m. carlson --- sha1_file.c | 52 ++++++++++++++++++++++++++-------------------------- 1 file changed, 26 insertions(+), 26 deletions(-) diff --git a/sha1_file.c b/sha1_file.c index ec6ecea170..ff55fb303d 100644 --- a/sha1_file.c +++ b/sha1_file.c @@ -786,16 +786,16 @@ void *xmmap(void *start, size_t length, int check_sha1_signature(const unsigned char *sha1, void *map, unsigned long size, const char *type) { - unsigned char real_sha1[20]; + struct object_id real_oid; enum object_type obj_type; struct git_istream *st; - git_SHA_CTX c; + git_hash_ctx c; char hdr[32]; int hdrlen; if (map) { - hash_sha1_file(map, size, type, real_sha1); - return hashcmp(sha1, real_sha1) ? -1 : 0; + hash_sha1_file(map, size, type, real_oid.hash); + return hashcmp(sha1, real_oid.hash) ? -1 : 0; } st = open_istream(sha1, &obj_type, &size, NULL); @@ -806,8 +806,8 @@ int check_sha1_signature(const unsigned char *sha1, void *map, hdrlen = xsnprintf(hdr, sizeof(hdr), "%s %lu", typename(obj_type), size) + 1; /* Sha1.. */ - git_SHA1_Init(&c); - git_SHA1_Update(&c, hdr, hdrlen); + the_hash_algo->init_fn(&c); + the_hash_algo->update_fn(&c, hdr, hdrlen); for (;;) { char buf[1024 * 16]; ssize_t readlen = read_istream(st, buf, sizeof(buf)); @@ -818,11 +818,11 @@ int check_sha1_signature(const unsigned char *sha1, void *map, } if (!readlen) break; - git_SHA1_Update(&c, buf, readlen); + the_hash_algo->update_fn(&c, buf, readlen); } - git_SHA1_Final(real_sha1, &c); + the_hash_algo->final_fn(real_oid.hash, &c); close_istream(st); - return hashcmp(sha1, real_sha1) ? -1 : 0; + return hashcmp(sha1, real_oid.hash) ? -1 : 0; } int git_open_cloexec(const char *name, int flags) @@ -1421,16 +1421,16 @@ static void write_sha1_file_prepare(const void *buf, unsigned long len, const char *type, unsigned char *sha1, char *hdr, int *hdrlen) { - git_SHA_CTX c; + git_hash_ctx c; /* Generate the header */ *hdrlen = xsnprintf(hdr, *hdrlen, "%s %lu", type, len)+1; /* Sha1.. */ - git_SHA1_Init(&c); - git_SHA1_Update(&c, hdr, *hdrlen); - git_SHA1_Update(&c, buf, len); - git_SHA1_Final(sha1, &c); + the_hash_algo->init_fn(&c); + the_hash_algo->update_fn(&c, hdr, *hdrlen); + the_hash_algo->update_fn(&c, buf, len); + the_hash_algo->final_fn(sha1, &c); } /* @@ -1552,8 +1552,8 @@ static int write_loose_object(const unsigned char *sha1, char *hdr, int hdrlen, int fd, ret; unsigned char compressed[4096]; git_zstream stream; - git_SHA_CTX c; - unsigned char parano_sha1[20]; + git_hash_ctx c; + struct object_id parano_oid; static struct strbuf tmp_file = STRBUF_INIT; const char *filename = sha1_file_name(sha1); @@ -1569,14 +1569,14 @@ static int write_loose_object(const unsigned char *sha1, char *hdr, int hdrlen, git_deflate_init(&stream, zlib_compression_level); stream.next_out = compressed; stream.avail_out = sizeof(compressed); - git_SHA1_Init(&c); + the_hash_algo->init_fn(&c); /* First header.. */ stream.next_in = (unsigned char *)hdr; stream.avail_in = hdrlen; while (git_deflate(&stream, 0) == Z_OK) ; /* nothing */ - git_SHA1_Update(&c, hdr, hdrlen); + the_hash_algo->update_fn(&c, hdr, hdrlen); /* Then the data itself.. */ stream.next_in = (void *)buf; @@ -1584,7 +1584,7 @@ static int write_loose_object(const unsigned char *sha1, char *hdr, int hdrlen, do { unsigned char *in0 = stream.next_in; ret = git_deflate(&stream, Z_FINISH); - git_SHA1_Update(&c, in0, stream.next_in - in0); + the_hash_algo->update_fn(&c, in0, stream.next_in - in0); if (write_buffer(fd, compressed, stream.next_out - compressed) < 0) die("unable to write sha1 file"); stream.next_out = compressed; @@ -1596,8 +1596,8 @@ static int write_loose_object(const unsigned char *sha1, char *hdr, int hdrlen, ret = git_deflate_end_gently(&stream); if (ret != Z_OK) die("deflateEnd on object %s failed (%d)", sha1_to_hex(sha1), ret); - git_SHA1_Final(parano_sha1, &c); - if (hashcmp(sha1, parano_sha1) != 0) + the_hash_algo->final_fn(parano_oid.hash, &c); + if (hashcmp(sha1, parano_oid.hash) != 0) die("confused by unstable object source data for %s", sha1_to_hex(sha1)); close_sha1_file(fd); @@ -2091,14 +2091,14 @@ static int check_stream_sha1(git_zstream *stream, const char *path, const unsigned char *expected_sha1) { - git_SHA_CTX c; + git_hash_ctx c; unsigned char real_sha1[GIT_MAX_RAWSZ]; unsigned char buf[4096]; unsigned long total_read; int status = Z_OK; - git_SHA1_Init(&c); - git_SHA1_Update(&c, hdr, stream->total_out); + the_hash_algo->init_fn(&c); + the_hash_algo->update_fn(&c, hdr, stream->total_out); /* * We already read some bytes into hdr, but the ones up to the NUL @@ -2117,7 +2117,7 @@ static int check_stream_sha1(git_zstream *stream, if (size - total_read < stream->avail_out) stream->avail_out = size - total_read; status = git_inflate(stream, Z_FINISH); - git_SHA1_Update(&c, buf, stream->next_out - buf); + the_hash_algo->update_fn(&c, buf, stream->next_out - buf); total_read += stream->next_out - buf; } git_inflate_end(stream); @@ -2132,7 +2132,7 @@ static int check_stream_sha1(git_zstream *stream, return -1; } - git_SHA1_Final(real_sha1, &c); + the_hash_algo->final_fn(real_sha1, &c); if (hashcmp(expected_sha1, real_sha1)) { error("sha1 mismatch for %s (expected %s)", path, sha1_to_hex(expected_sha1)); ``` ## Stefan Beller, 2018-02-01 22:06 Subject: Re: [PATCH v2 00/12] object_id part 11 (the_hash_algo) Message-ID: URL: https://gitlist.dev/e/CAGZ79kZ5FT7hD4q%2B%3D-AVBUzTaHUjR1sQ3ksmJGHfXzon-az2xA%40mail.gmail.com In-Reply-To: <20180201021848.533188-1-sandals@crustytoothpaste.net> ``` On Wed, Jan 31, 2018 at 6:18 PM, brian m. carlson wrote: > This series includes various changes to adopt the use of the_hash_algo > for abstracting hash algorithms away. This series looks good to me. > > Changes from v1: > * Fix comments referring to SHA-1. > * Switch hash function wrappers to take git_hash_ctx *. > * Use a const int variable for a constant value. > * Wrap an overly long line. > > brian m. carlson (12): > hash: move SHA-1 macros to hash.h > hash: create union for hash context allocation > builtin/index-pack: improve hash function abstraction > builtin/unpack-objects: switch uses of SHA-1 to the_hash_algo > sha1_file: switch uses of SHA-1 to the_hash_algo > fast-import: switch various uses of SHA-1 to the_hash_algo > pack-check: convert various uses of SHA-1 to abstract forms > pack-write: switch various SHA-1 values to abstract forms > read-cache: abstract away uses of SHA-1 > csum-file: rename sha1file to hashfile > csum-file: abstract uses of SHA-1 > bulk-checkin: abstract SHA-1 usage > > builtin/index-pack.c | 108 +++++++++++++++++++++++------------------------ > builtin/pack-objects.c | 52 +++++++++++------------ > builtin/unpack-objects.c | 18 ++++---- > bulk-checkin.c | 28 ++++++------ > cache.h | 25 ----------- > csum-file.c | 46 ++++++++++---------- > csum-file.h | 38 ++++++++--------- > fast-import.c | 70 +++++++++++++++--------------- > hash.h | 40 +++++++++++++++--- > pack-bitmap-write.c | 30 ++++++------- > pack-check.c | 32 +++++++------- > pack-write.c | 77 ++++++++++++++++----------------- > pack.h | 4 +- > read-cache.c | 58 ++++++++++++------------- > sha1_file.c | 72 +++++++++++++++---------------- > 15 files changed, 351 insertions(+), 347 deletions(-) > ``` ## Junio C Hamano, 2018-02-02 19:39 Subject: Re: [PATCH v2 01/12] hash: move SHA-1 macros to hash.h Message-ID: URL: https://gitlist.dev/e/xmqqmv0rchat.fsf%40gitster-ct.c.googlers.com In-Reply-To: <20180201021848.533188-2-sandals@crustytoothpaste.net> ``` "brian m. carlson" writes: > +#ifndef platform_SHA_CTX > +/* > + * platform's underlying implementation of SHA-1; could be OpenSSL, > + * blk_SHA, Apple CommonCrypto, etc... Note that including > + * SHA1_HEADER may have already defined platform_SHA_CTX for our > + * own implementations like block-sha1 and ppc-sha1, so we list > + * the default for OpenSSL compatible SHA-1 implementations here. > + */ This nit has been with us for a long time, but "Note that including SHA1_HEADER may have..." has been way too stale a comment for quite some time. It was made different from "including SHA1_HEADER" by f18f816c ("hash.h: move SHA-1 implementation selection into a header file", 2017-03-11). Perhaps Note that we may have already defined platform_SHA_CTX for our own implementations like block-sha1 and ppc-sha1, so we list ... or something. ``` ## Junio C Hamano, 2018-02-02 19:46 Subject: Re: [PATCH v2 00/12] object_id part 11 (the_hash_algo) Message-ID: URL: https://gitlist.dev/e/xmqqinbfcgzo.fsf%40gitster-ct.c.googlers.com In-Reply-To: <20180201021848.533188-1-sandals@crustytoothpaste.net> ``` "brian m. carlson" writes: > This series includes various changes to adopt the use of the_hash_algo > for abstracting hash algorithms away. > > Changes from v1: > * Fix comments referring to SHA-1. > * Switch hash function wrappers to take git_hash_ctx *. > * Use a const int variable for a constant value. > * Wrap an overly long line. Thanks for working on this. All changes looked sensible (even though I spotted one nit in the original, which was moved as-is, which does not count as a "change" ;-)). ``` ## brian m. carlson, 2018-02-06 00:34 Subject: Re: [PATCH v2 00/12] object_id part 11 (the_hash_algo) Message-ID: <20180206003359.GB7904@genre.crustytoothpaste.net> URL: https://gitlist.dev/e/20180206003359.GB7904%40genre.crustytoothpaste.net In-Reply-To: ``` On Fri, Feb 02, 2018 at 11:46:03AM -0800, Junio C Hamano wrote: > Thanks for working on this. All changes looked sensible (even > though I spotted one nit in the original, which was moved as-is, > which does not count as a "change" ;-)). I forgot to ask: do you want a reroll which fixes this, or just a follow-up patch on top to fix the comment? -- brian m. carlson / brian with sandals: Houston, Texas, US https://www.crustytoothpaste.net/~bmc | My opinion only OpenPGP: https://keybase.io/bk2204 ``` ## Junio C Hamano, 2018-02-06 14:52 Subject: Re: [PATCH v2 00/12] object_id part 11 (the_hash_algo) Message-ID: URL: https://gitlist.dev/e/xmqq7erq892l.fsf%40gitster-ct.c.googlers.com In-Reply-To: <20180206003359.GB7904@genre.crustytoothpaste.net> ``` "brian m. carlson" writes: > On Fri, Feb 02, 2018 at 11:46:03AM -0800, Junio C Hamano wrote: >> Thanks for working on this. All changes looked sensible (even >> though I spotted one nit in the original, which was moved as-is, >> which does not count as a "change" ;-)). > > I forgot to ask: do you want a reroll which fixes this, or just a > follow-up patch on top to fix the comment? I do not think it is in scope of the series, so a follow-up would be sufficient. Thanks. ``` ## brian m. carlson, 2018-02-08 02:48 Subject: [PATCH v2] hash: update obsolete reference to SHA1_HEADER Message-ID: <20180208024858.872624-1-sandals@crustytoothpaste.net> URL: https://gitlist.dev/e/20180208024858.872624-1-sandals%40crustytoothpaste.net In-Reply-To: ``` We moved away from SHA1_HEADER to a preprocessor if chain, but didn't update the comment discussing the platform defines. Update this comment so it reflects the current state of our codebase. Signed-off-by: brian m. carlson --- hash.h | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/hash.h b/hash.h index eb30f59be3..7c8238bc2e 100644 --- a/hash.h +++ b/hash.h @@ -18,8 +18,8 @@ #ifndef platform_SHA_CTX /* * platform's underlying implementation of SHA-1; could be OpenSSL, - * blk_SHA, Apple CommonCrypto, etc... Note that including - * SHA1_HEADER may have already defined platform_SHA_CTX for our + * blk_SHA, Apple CommonCrypto, etc... Note that the relevant + * SHA-1 header may have already defined platform_SHA_CTX for our * own implementations like block-sha1 and ppc-sha1, so we list * the default for OpenSSL compatible SHA-1 implementations here. */ ```