{"thread":{"id":"54628","subject":"[PATCH 1/3] csum-file: add hashwrite_be64()","startedAt":"2020-11-12T12:20:32Z","lastAt":"2020-11-12T14:51:17Z","messageCount":6,"participants":["René Scharfe","Derrick Stolee","Taylor Blau"],"isPatch":true,"patchVersion":1,"patchTotal":3},"messages":[{"id":"409752","messageId":"16932ced-8bcd-89bd-b927-cae1bce0365a@web.de","threadId":"54628","inReplyTo":null,"subject":"[PATCH 1/3] csum-file: add hashwrite_be64()","fromName":"René Scharfe","fromEmail":"l.s.r@web.de","sentAt":"2020-11-12T12:20:19Z","receivedAt":"2020-11-12T12:20:32Z","isPatch":true,"sender":{"key":"l.s.r@web.de","avatar":"https://avatars.githubusercontent.com/u/26122331?v=4"},"body":"Add a helper function for hashing and writing 64-bit integers in network\nbyte order. It returns the number of written bytes. This simplifies\ncallers that keep track of the file offset, even though this number is a\nconstant.\n\nSuggested-by: Derrick Stolee \nOriginal-patch-by: Taylor Blau \nSigned-off-by: René Scharfe \n---\n csum-file.h | 7 +++++++\n 1 file changed, 7 insertions(+)\n\ndiff --git a/csum-file.h b/csum-file.h\nindex f9cbd317fb..e54d53d1d0 100644\n--- a/csum-file.h\n+++ b/csum-file.h\n@@ -62,4 +62,11 @@ static inline void hashwrite_be32(struct hashfile *f, uint32_t data)\n \thashwrite(f, &data, sizeof(data));\n }\n\n+static inline size_t hashwrite_be64(struct hashfile *f, uint64_t data)\n+{\n+\tdata = htonll(data);\n+\thashwrite(f, &data, sizeof(data));\n+\treturn sizeof(data);\n+}\n+\n #endif\n--\n2.29.2\n"},{"id":"409753","messageId":"d38ac2a6-261c-4767-8691-8d97d551381c@web.de","threadId":"54628","inReplyTo":"16932ced-8bcd-89bd-b927-cae1bce0365a@web.de","subject":"[PATCH 2/3] midx: use hashwrite_be64()","fromName":"René Scharfe","fromEmail":"l.s.r@web.de","sentAt":"2020-11-12T12:22:16Z","receivedAt":"2020-11-12T12:22:25Z","isPatch":true,"sender":{"key":"l.s.r@web.de","avatar":"https://avatars.githubusercontent.com/u/26122331?v=4"},"body":"Call hashwrite_be64() to write 64-bit values instead of open-coding it\nusing hashwrite_be32() and sizeof. This shortens the code and makes its\nintent clearer.\n\nSigned-off-by: René Scharfe \n---\nAnd it saves me from headache induced by trying to remember whether \"big\nendian\" means that the most significant (\"big\") digit comes first (yes)\nor at the end, like the name suggests (no).\n\n midx.c | 7 ++-----\n 1 file changed, 2 insertions(+), 5 deletions(-)\n\ndiff --git a/midx.c b/midx.c\nindex d233b54ac7..da03c1449a 100644\n--- a/midx.c\n+++ b/midx.c\n@@ -785,9 +785,7 @@ static size_t write_midx_large_offsets(struct hashfile *f, uint32_t nr_large_off\n \t\tif (!(offset >> 31))\n \t\t\tcontinue;\n\n-\t\thashwrite_be32(f, offset >> 32);\n-\t\thashwrite_be32(f, offset & 0xffffffffUL);\n-\t\twritten += 2 * sizeof(uint32_t);\n+\t\twritten += hashwrite_be64(f, offset);\n\n \t\tnr_large_offset--;\n \t}\n@@ -975,8 +973,7 @@ static int write_midx_internal(const char *object_dir, struct multi_pack_index *\n \t\t\t chunk_offsets[i]);\n\n \t\thashwrite_be32(f, chunk_ids[i]);\n-\t\thashwrite_be32(f, chunk_offsets[i] >> 32);\n-\t\thashwrite_be32(f, chunk_offsets[i]);\n+\t\thashwrite_be64(f, chunk_offsets[i]);\n\n \t\twritten += MIDX_CHUNKLOOKUP_WIDTH;\n \t}\n--\n2.29.2\n"},{"id":"409754","messageId":"716c9fd8-273e-6bf6-ab67-1f589d2c9687@web.de","threadId":"54628","inReplyTo":"16932ced-8bcd-89bd-b927-cae1bce0365a@web.de","subject":"[PATCH 3/3] pack-write: use hashwrite_be64()","fromName":"René Scharfe","fromEmail":"l.s.r@web.de","sentAt":"2020-11-12T12:23:10Z","receivedAt":"2020-11-12T12:23:19Z","isPatch":true,"sender":{"key":"l.s.r@web.de","avatar":"https://avatars.githubusercontent.com/u/26122331?v=4"},"body":"Call hashwrite_be64() to write a 64-bit value instead of open-coding it\nusing htonl() and hashwrite(). This shortens the code, gets rid of a\nbuffer and several magic numbers, and makes the intent clearer.\n\nSigned-off-by: René Scharfe \n---\n pack-write.c | 5 +----\n 1 file changed, 1 insertion(+), 4 deletions(-)\n\ndiff --git a/pack-write.c b/pack-write.c\nindex 23e19cc1ec..3513665e1e 100644\n--- a/pack-write.c\n+++ b/pack-write.c\n@@ -151,13 +151,10 @@ const char *write_idx_file(const char *index_name, struct pack_idx_entry **objec\n \t\twhile (nr_large_offset) {\n \t\t\tstruct pack_idx_entry *obj = *list++;\n \t\t\tuint64_t offset = obj->offset;\n-\t\t\tuint32_t split[2];\n\n \t\t\tif (!need_large_offset(offset, opts))\n \t\t\t\tcontinue;\n-\t\t\tsplit[0] = htonl(offset >> 32);\n-\t\t\tsplit[1] = htonl(offset & 0xffffffff);\n-\t\t\thashwrite(f, split, 8);\n+\t\t\thashwrite_be64(f, offset);\n \t\t\tnr_large_offset--;\n \t\t}\n \t}\n--\n2.29.2\n"},{"id":"409757","messageId":"fcebe67f-f1d7-0069-01de-12844508cae1@gmail.com","threadId":"54628","inReplyTo":"16932ced-8bcd-89bd-b927-cae1bce0365a@web.de","subject":"Re: [PATCH 1/3] csum-file: add hashwrite_be64()","fromName":"Derrick Stolee","fromEmail":"stolee@gmail.com","sentAt":"2020-11-12T13:52:24Z","receivedAt":"2020-11-12T13:52:30Z","isPatch":true,"sender":{"key":"stolee@gmail.com","avatar":"https://avatars.githubusercontent.com/u/570044?v=4"},"body":"On 11/12/2020 7:20 AM, René Scharfe wrote:\n> Add a helper function for hashing and writing 64-bit integers in network\n> byte order. It returns the number of written bytes. This simplifies\n> callers that keep track of the file offset, even though this number is a\n> constant.\n> \n> Suggested-by: Derrick Stolee \n> Original-patch-by: Taylor Blau \n\nThese patches are absolutely correct, and I'm glad to see them show up\nin a very clear presentation. I had to go look to see why these were\nnot already present, with [1] being the last instance of these showing\nup on-list. They did not get into the new version after a significant\nrefactor [2].\n\nThanks,\n-Stolee\n\n[1] https://lore.kernel.org/git/20200904202226.GA21837@nand.local/\n[2] https://lore.kernel.org/git/cover.1599664389.git.me@ttaylorr.com/\n"},{"id":"409765","messageId":"X61LEUBDyhnboZAv@nand.local","threadId":"54628","inReplyTo":"fcebe67f-f1d7-0069-01de-12844508cae1@gmail.com","subject":"Re: [PATCH 1/3] csum-file: add hashwrite_be64()","fromName":"Taylor Blau","fromEmail":"me@ttaylorr.com","sentAt":"2020-11-12T14:47:45Z","receivedAt":"2020-11-12T14:47:52Z","isPatch":true,"sender":{"key":"me@ttaylorr.com","avatar":"https://avatars.githubusercontent.com/u/301000140?v=4"},"body":"On Thu, Nov 12, 2020 at 08:52:24AM -0500, Derrick Stolee wrote:\n> On 11/12/2020 7:20 AM, René Scharfe wrote:\n> > Add a helper function for hashing and writing 64-bit integers in network\n> > byte order. It returns the number of written bytes. This simplifies\n> > callers that keep track of the file offset, even though this number is a\n> > constant.\n> >\n> > Suggested-by: Derrick Stolee \n> > Original-patch-by: Taylor Blau \n>\n> These patches are absolutely correct, and I'm glad to see them show up\n> in a very clear presentation. I had to go look to see why these were\n> not already present, with [1] being the last instance of these showing\n> up on-list. They did not get into the new version after a significant\n> refactor [2].\n\nThat's right. What happened was we stopped writing the uncompressed \"has\nan unpresentable Bloom filter\" bitmap between the two versions you're\ntalking about. Since we wrote each word as a big-endian 8-byte unsigned\nvalue, introducing hashwrite_be64() was useful in the earlier version.\n\nBut in the rewritten version, there were no _new_ callers that would\nhave wanted hashwrite_be64(), so I dropped those patches since the\nseries was already large.\n\nThat all being said, these are definitely useful patches to have, so I'm\nglad to see them being dug back up. Thanks, René :-).\n\n> Thanks,\n> -Stolee\n>\n> [1] https://lore.kernel.org/git/20200904202226.GA21837@nand.local/\n> [2] https://lore.kernel.org/git/cover.1599664389.git.me@ttaylorr.com/\n\nThanks,\nTaylor\n"},{"id":"409766","messageId":"X61L4Ebd5PBpYDel@nand.local","threadId":"54628","inReplyTo":"16932ced-8bcd-89bd-b927-cae1bce0365a@web.de","subject":"Re: [PATCH 1/3] csum-file: add hashwrite_be64()","fromName":"Taylor Blau","fromEmail":"me@ttaylorr.com","sentAt":"2020-11-12T14:51:12Z","receivedAt":"2020-11-12T14:51:17Z","isPatch":true,"sender":{"key":"me@ttaylorr.com","avatar":"https://avatars.githubusercontent.com/u/301000140?v=4"},"body":"On Thu, Nov 12, 2020 at 01:20:19PM +0100, René Scharfe wrote:\n> Add a helper function for hashing and writing 64-bit integers in network\n> byte order. It returns the number of written bytes. This simplifies\n> callers that keep track of the file offset, even though this number is a\n> constant.\n>\n> Suggested-by: Derrick Stolee \n> Original-patch-by: Taylor Blau \n> Signed-off-by: René Scharfe \n> ---\n> csum-file.h | 7 +++++++\n> 1 file changed, 7 insertions(+)\n>\n> diff --git a/csum-file.h b/csum-file.h\n> index f9cbd317fb..e54d53d1d0 100644\n> --- a/csum-file.h\n> +++ b/csum-file.h\n> @@ -62,4 +62,11 @@ static inline void hashwrite_be32(struct hashfile *f, uint32_t data)\n> \thashwrite(f, &data, sizeof(data));\n> }\n>\n> +static inline size_t hashwrite_be64(struct hashfile *f, uint64_t data)\n> +{\n> +\tdata = htonll(data);\n\nGreat. This is new from my patch (which wrote the high- and low four\nbytes with two separate hashwrite_be32()'s), but I think it's a clear\nimprovement.\n\nIn addition to being more readable, we can use the bswap instruction\nonce instead of twice if it exists.\n\nPlease feel free to add my:\n\n Signed-off-by: Taylor Blau \n\nThanks,\nTaylor\n"}]}