git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: pread() over NFS (again) [1.5.5.4]

From
Shawn O. Pearce <spearce@spearce.org>
Date
Jun 26, 2008, 22:07 UTC
Message-ID
<20080626220755.GB11793@spearce.org>
In-Reply-To
<7vod5nq2dy.fsf@gitster.siamese.dyndns.org>
Junio C Hamano <gitster@pobox.com> wrote:
Show 13 quoted lines
> "Shawn O. Pearce" <spearce@spearce.org> writes:
> 
> > We have a file position we saw once before as the start of a delta.
> > We wrote it down to disk.  We want to go back and open it up, as
> > we have the base decompressed and in memory and need to compute
> > the SHA-1 of the object that resides at this offset.
> >
> > And *wham* we get an EOF.  Where there should be data.  Where we
> > know there is data.
> 
> We have written that earlier in the same process?  Are we playing games
> with mixed mmap() and pread()?  Is fsync() or msync() or unmap/remap
> needed?
Yes, we wrote the very same file earlier in the process.
index-pack _used_ to use a mixed write/mmap game.  Today it uses
write, followed by later pread.  No mmap.  We had major performance
issues on Mac OS X with mmap, not to mention the coherency issues
that can arise from using it.  So index-pack is mmap free[*1*].
 
Show 7 quoted lines
> > Actually, I think the last time someone reported something like this
> > in Git it turned out to be an NFS kernel bug.  I didn't quote it
> > in my reply to him, but I think he did say this was a linux client,
> > linux server.
> 
> This is getting into the area Linus would immediately know the answer to,
> but he is away for this week.
Yea, I was expecting a reply from him, but that explains it.
*1* There is usage of mmap in index-pack, but only to access
    _existing_ objects and packs from the object store.
-- 
Shawn.
Previous: Junio C HamanoNext: logank@sent.com
Message 7 of 16 in “pread() over NFS (again) [1.5.5.4]”
  1. Christian HoltjeJun 26, 2008
  2. Shawn O. PearceJun 26, 2008
  3. Junio C HamanoJun 26, 2008
  4. Shawn O. PearceJun 26, 2008
  5. Christian HoltjeJun 26, 2008
  6. Junio C HamanoJun 26, 2008
  7. Shawn O. PearceJun 26, 2008
  8. logank@sent.comJun 26, 2008
  9. Junio C HamanoJun 26, 2008
  10. J. Bruce FieldsJun 27, 2008
  11. Trond MyklebustJun 27, 2008
  12. Shawn O. PearceJun 30, 2008
  13. Nicolas PitreJun 30, 2008
  14. J. Bruce FieldsJun 27, 2008
  15. Christian HoltjeJun 27, 2008
  16. Christian HoltjeJun 27, 2008

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.