{"thread":{"id":"52520","subject":"[PATCH 0/2] packfile: small syscall reductions","startedAt":"2019-12-26T10:42:23Z","lastAt":"2019-12-26T18:59:33Z","messageCount":6,"participants":["Eric Wong","Junio C Hamano"],"isPatch":true,"patchVersion":1,"patchTotal":2},"messages":[{"id":"388915","messageId":"20191226104220.27325-1-e@80x24.org","threadId":"52520","inReplyTo":null,"subject":"[PATCH 0/2] packfile: small syscall reductions","fromName":"Eric Wong","fromEmail":"e@80x24.org","sentAt":"2019-12-26T10:42:18Z","receivedAt":"2019-12-26T10:42:23Z","isPatch":true,"sender":{"key":"e@80x24.org","avatar":null},"body":"Being an avid user of strace, redundant syscalls clutter up the\noutput and bother me, so I removed some. Folks with CPU\nmitigations enabled might also benefit from fewer syscalls.\n\nEric Wong (2):\n packfile: remove redundant fcntl F_GETFD/F_SETFD\n packfile: replace lseek+read with pread\n\n packfile.c | 16 ++--------------\n 1 file changed, 2 insertions(+), 14 deletions(-)\n\n"},{"id":"388916","messageId":"20191226104220.27325-2-e@80x24.org","threadId":"52520","inReplyTo":"20191226104220.27325-1-e@80x24.org","subject":"[PATCH 1/2] packfile: remove redundant fcntl F_GETFD/F_SETFD","fromName":"Eric Wong","fromEmail":"e@80x24.org","sentAt":"2019-12-26T10:42:19Z","receivedAt":"2019-12-26T10:42:27Z","isPatch":true,"sender":{"key":"e@80x24.org","avatar":null},"body":"git_open sets close-on-exec since cd66ada06588f797\n(\"sha1_file: open window into packfiles with O_CLOEXEC\").\nThere's no reason to keep using fcntl to set the close-on-exec\nflag, anymore.\n\nSigned-off-by: Eric Wong \n---\n packfile.c | 11 -----------\n 1 file changed, 11 deletions(-)\n\ndiff --git a/packfile.c b/packfile.c\nindex f0dc63e92f..1821cb7a3d 100644\n--- a/packfile.c\n+++ b/packfile.c\n@@ -510,7 +510,6 @@ static int open_packed_git_1(struct packed_git *p)\n \tstruct pack_header hdr;\n \tunsigned char hash[GIT_MAX_RAWSZ];\n \tunsigned char *idx_hash;\n-\tlong fd_flag;\n \tssize_t read_result;\n \tconst unsigned hashsz = the_hash_algo->rawsz;\n \n@@ -554,16 +553,6 @@ static int open_packed_git_1(struct packed_git *p)\n \t} else if (p->pack_size != st.st_size)\n \t\treturn error(\"packfile %s size changed\", p->pack_name);\n \n-\t/* We leave these file descriptors open with sliding mmap;\n-\t * there is no point keeping them open across exec(), though.\n-\t */\n-\tfd_flag = fcntl(p->pack_fd, F_GETFD, 0);\n-\tif (fd_flag < 0)\n-\t\treturn error(\"cannot determine file descriptor flags\");\n-\tfd_flag |= FD_CLOEXEC;\n-\tif (fcntl(p->pack_fd, F_SETFD, fd_flag) == -1)\n-\t\treturn error(\"cannot set FD_CLOEXEC\");\n-\n \t/* Verify we recognize this pack file format. */\n \tread_result = read_in_full(p->pack_fd, &hdr, sizeof(hdr));\n \tif (read_result < 0)\n"},{"id":"388917","messageId":"20191226104220.27325-3-e@80x24.org","threadId":"52520","inReplyTo":"20191226104220.27325-1-e@80x24.org","subject":"[PATCH 2/2] packfile: replace lseek+read with pread","fromName":"Eric Wong","fromEmail":"e@80x24.org","sentAt":"2019-12-26T10:42:20Z","receivedAt":"2019-12-26T10:42:27Z","isPatch":true,"sender":{"key":"e@80x24.org","avatar":null},"body":"We already have pread emulation for portability, so there's\nthere's no reason to make two syscalls where one suffices.\n\nFurthermore, readers of the packfile will be using mmap\n(or pread to emulate mmap), anyways, so the file description\noffset does not matter in this case.\n\nSigned-off-by: Eric Wong \n---\n packfile.c | 5 ++---\n 1 file changed, 2 insertions(+), 3 deletions(-)\n\ndiff --git a/packfile.c b/packfile.c\nindex 1821cb7a3d..7e7c04e4d8 100644\n--- a/packfile.c\n+++ b/packfile.c\n@@ -576,9 +576,8 @@ static int open_packed_git_1(struct packed_git *p)\n \t\t\t \" while index indicates %\"PRIu32\" objects\",\n \t\t\t p->pack_name, ntohl(hdr.hdr_entries),\n \t\t\t p->num_objects);\n-\tif (lseek(p->pack_fd, p->pack_size - hashsz, SEEK_SET) == -1)\n-\t\treturn error(\"end of packfile %s is unavailable\", p->pack_name);\n-\tread_result = read_in_full(p->pack_fd, hash, hashsz);\n+\tread_result = pread_in_full(p->pack_fd, hash, hashsz,\n+\t\t\t\t\tp->pack_size - hashsz);\n \tif (read_result < 0)\n \t\treturn error_errno(\"error reading from %s\", p->pack_name);\n \tif (read_result != hashsz)\n"},{"id":"388934","messageId":"xmqqzhffkku2.fsf@gitster-ct.c.googlers.com","threadId":"52520","inReplyTo":"20191226104220.27325-3-e@80x24.org","subject":"Re: [PATCH 2/2] packfile: replace lseek+read with pread","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2019-12-26T18:23:01Z","receivedAt":"2019-12-26T18:23:06Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Eric Wong writes:\n\n> We already have pread emulation for portability, so there's\n> there's no reason to make two syscalls where one suffices.\n>\n> Furthermore, readers of the packfile will be using mmap\n> (or pread to emulate mmap), anyways, so the file description\n> offset does not matter in this case.\n\ns/description/descriptor/ probably.\n\nAfter seeking to the packfile trailer and reading the pack id hash\nusing lseek+read, this helper function does not read from the file\ndescriptor, and the sole caller of it closes the file descriptor\nimmediately after it returns, which means the read file offset after\nreading the packfile trailer does not matter.\n\nSo this conversion is correct. Thanks for a careful analysis.\n\nWill queue both patches.\n\n> Signed-off-by: Eric Wong \n> ---\n> packfile.c | 5 ++---\n> 1 file changed, 2 insertions(+), 3 deletions(-)\n>\n> diff --git a/packfile.c b/packfile.c\n> index 1821cb7a3d..7e7c04e4d8 100644\n> --- a/packfile.c\n> +++ b/packfile.c\n> @@ -576,9 +576,8 @@ static int open_packed_git_1(struct packed_git *p)\n> \t\t\t \" while index indicates %\"PRIu32\" objects\",\n> \t\t\t p->pack_name, ntohl(hdr.hdr_entries),\n> \t\t\t p->num_objects);\n> -\tif (lseek(p->pack_fd, p->pack_size - hashsz, SEEK_SET) == -1)\n> -\t\treturn error(\"end of packfile %s is unavailable\", p->pack_name);\n> -\tread_result = read_in_full(p->pack_fd, hash, hashsz);\n> +\tread_result = pread_in_full(p->pack_fd, hash, hashsz,\n> +\t\t\t\t\tp->pack_size - hashsz);\n> \tif (read_result < 0)\n> \t\treturn error_errno(\"error reading from %s\", p->pack_name);\n> \tif (read_result != hashsz)\n"},{"id":"388935","messageId":"xmqqv9q3kke4.fsf@gitster-ct.c.googlers.com","threadId":"52520","inReplyTo":"xmqqzhffkku2.fsf@gitster-ct.c.googlers.com","subject":"Re: [PATCH 2/2] packfile: replace lseek+read with pread","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2019-12-26T18:32:35Z","receivedAt":"2019-12-26T18:33:00Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Junio C Hamano writes:\n\n> Eric Wong writes:\n>\n>> We already have pread emulation for portability, so there's\n>> there's no reason to make two syscalls where one suffices.\n>>\n>> Furthermore, readers of the packfile will be using mmap\n>> (or pread to emulate mmap), anyways, so the file description\n>> offset does not matter in this case.\n>\n> s/description/descriptor/ probably.\n>\n> After seeking to the packfile trailer and reading the pack id hash\n> using lseek+read, this helper function does not read from the file\n> descriptor, and the sole caller of it closes the file descriptor\n> immediately after it returns, which means the read file offset after\n> reading the packfile trailer does not matter.\n\nOops, that was not right. When we successfully open the packfile,\nwe leave the file descriptor open, so we do need the \"we never read\nusing read(2) from the file descriptor\" guarantee for this change to\nbe correct.\n\nBut we do have the guarantee, and existing code does depend on the\nguarantee, so the patch is good.\n\nThanks.\n"},{"id":"388937","messageId":"20191226185932.GA19165@dcvr","threadId":"52520","inReplyTo":"xmqqzhffkku2.fsf@gitster-ct.c.googlers.com","subject":"Re: [PATCH 2/2] packfile: replace lseek+read with pread","fromName":"Eric Wong","fromEmail":"e@80x24.org","sentAt":"2019-12-26T18:59:32Z","receivedAt":"2019-12-26T18:59:33Z","isPatch":true,"sender":{"key":"e@80x24.org","avatar":null},"body":"Junio C Hamano wrote:\n> Eric Wong writes:\n> >\n> > Furthermore, readers of the packfile will be using mmap\n> > (or pread to emulate mmap), anyways, so the file description\n> > offset does not matter in this case.\n> \n> s/description/descriptor/ probably.\n\nNo, I meant \"description\" :) The offset is shared in case of dup{,2,3}\nsyscalls, which only creates a new descriptor, not a new description.\nBoth the Linux and POSIX lseek(2) manpages say \"description\".\n"}]}