From: George Hu <integral@archlinux.org>
Date: Sun, 15 Feb 2026 06:23:16 GMT
Subject: Re: [PATCH] copy.c: use `sendfile()` for in-kernel file copying on Linux
Message-ID: <77a887d0-51bf-4bf6-af8f-d5555dab2fe2@archlinux.org>
In-Reply-To: <bf0b3c41-9784-4494-a932-68abfa60cea6@gmail.com>

On 2/15/26 12:43 AM, Phillip Wood wrote:

> On 13/02/2026 12:46, George Hu wrote:
>> The `sendfile()` system call copies data between one file descriptor
>> and another within the kernel, which is more efficient than the
>> combination of `read()` and `write()`.
>
> Does git copy any files big enough that this makes a noticeable 
> difference?
>
>>   int copy_fd(int ifd, int ofd)
>>   {
>> +#ifdef __linux__
>
> Our normal practice when a function has platform specific 
> implementations is to host those implementations under compat/<platform>
> (see the implementations of trace2_collect_process_information() for 
> an example)
>

The Linux implementation of `trace2_collect_process_information()` 
resides in compat/linux with a stub version in compat/stub. After moving 
the Linux-specifc `copy_fd()` implementation into compat/linux, where 
should the generic implementation be placed?

>> +    struct stat ifd_st;
>> +    size_t ifd_len;
>> +    ssize_t ret = 0;
>> +
>> +    fstat(ifd, &ifd_st);
>
> What happens if fstat() fails?
>
>> +    ifd_len = ifd_st.st_size;
>> +
>> +    while (ifd_len && (ret = sendfile(ofd, ifd, NULL, ifd_len)) > 0)
>> +        ifd_len -= (size_t)ret;
>
> This does not propagate errors to the caller, if sendfile() fails the 
> function returns 0. write_in_full() handles non-blocking writes, we 
> should do the same here if we see EAGAIN. The man page lists various 
> restrictions on the file descriptors passed to sendfile() - I'm not 
> sure that they affect the uses of copy_file() in git but to be safe we 
> should fall back to the read()/write() loop if we see EINVAL.
>

According to the manual, `sendfile()` returns -1 on failure; a return 
value of 0 indicates EOF.

There are error cases besides EAGAIN and EINVAL. Maybe we should fall 
back to the read() / write() loop for errors other than EAGAIN?

Sincerely,
George

> Thanks
>
> Phillip
>
>> +#else
>>       while (1) {
>>           char buffer[8192];
>>           ssize_t len = xread(ifd, buffer, sizeof(buffer));
>> @@ -19,6 +34,8 @@ int copy_fd(int ifd, int ofd)
>>           if (write_in_full(ofd, buffer, len) < 0)
>>               return COPY_WRITE_ERROR;
>>       }
>> +#endif
>> +
>>       return 0;
>>   }
>