git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: [PATCH 1/3] Git.pm: Add faculties to allow temp files to be cached

From
EWEric Wong <normalperson@yhbt.net>
Date
Aug 12, 2008, 03:08 UTC
Message-ID
<20080812030809.GA14051@untitled>
In-Reply-To
<1218470035-13864-2-git-send-email-marcus@griep.us>
Marcus Griep <marcus@griep.us> wrote:
Show 13 quoted lines
> This patch offers a generic interface to allow temp files to be
> cached while using an instance of the 'Git' package. If many
> temp files are created and destroyed during the execution of a
> program, this caching mechanism can help reduce the amount of
> files created and destroyed by the filesystem.
> 
> There are two methods offered for creating a new file: a no-lock and
> a acquire-lock version. The no-lock version provides no
> guarantee that a file is not in use or that the temp file may be
> stolen by a subsequent request. The acquire-lock version provides a
> weak guarantee that a temp file will not be stolen by subsequent
> requests even from a no-lock request. If a file is locked when
> another acquire request is made, a simple error is thrown.

I'm not sure if the no-lock version is worth the potential for buggy or dangerous code. I like this new idea of locking the files to prevent bugs.

Show 13 quoted lines
> +=item temp_release ( NAME [, BOOL] )
> +
> +=item temp_release ( FILEHANDLE [, BOOL] )
> +
> +Releases a lock acquired through C<temp_acquire()>. Can be called either with
> +the C<NAME> mapping used when acquiring the temp file or with the C<FILEHANDLE>
> +referencing a locked temp file.
> +
> +Warns if an attempt is made to release a file that is not locked.
> +
> +If called with C<BOOL> true, then the temp file will be truncated before being
> +released. This can help to reduce disk I/O where the system is smart enough to
> +detect the truncation while data is in the output buffers.

Always truncating on release makes the interface simpler. With locking, we can probably *only* truncate on release if you're that worried about the extra overhead :)

Show 11 quoted lines
> +=item temp_reset ( FILEHANDLE )
> +
> +Truncates and resets the position of the C<FILEHANDLE>.  Uses C<sysseek>.
> +
> +=cut
> +
> +sub temp_reset {
> +	my ($self, $temp_fd) = _maybe_self(@_);
> +
> +	truncate $temp_fd, 0
> +		or throw Error::Simple("couldn't truncate file");

I would do a regular seek() here in addition to the sysseek() below. I am not certain one of the many userspace buffering layers Perl can potentially use doesn't do anything funky with its offset accounting.

> +	sysseek $temp_fd, 0, SEEK_SET
> +		or throw Error::Simple("couldn't seek to beginning of file");

I would also put a tell() here after the sysseek and throw an error if it returns a non-zero value just in case. Yes, I'm really paranoid about this stuff and have a huge distrust of userspace I/O layers :)

-- 
Eric Wong
Previous: Eric WongNext: Marcus Griep
Message 26 of 43 in “git-svn: Make it scream by minimizing temp files”
  1. git-svn: Make it scream by minimizing temp filesMarcus Griep, Aug 8, 2008
  2. Junio C HamanoAug 8, 2008
  3. Marcus GriepAug 9, 2008
  4. Eric WongAug 9, 2008
  5. Marcus GriepAug 9, 2008
  6. Eric WongAug 10, 2008
  7. Junio C HamanoAug 10, 2008
  8. Eric WongAug 10, 2008
  9. Junio C HamanoAug 10, 2008
  10. Eric WongAug 10, 2008
  11. 0/3 git-svn and temporary file improvementsMarcus Griep, Aug 11, 2008
  12. 1/3 Git.pm: Add faculties to allow temp files to be cachedMarcus Griep, Aug 11, 2008
  13. 2/3 git-svn: Make it scream by minimizing temp filesMarcus Griep, Aug 11, 2008
  14. 3/3 git-svn: Reduce temp file usage when dealing with non-linksMarcus Griep, Aug 11, 2008
  15. Eric WongAug 12, 2008
  16. Marcus GriepAug 12, 2008
  17. 3/3 git-svn: Reduce temp file usage when dealing with non-linksMarcus Griep, Aug 12, 2008
  18. 3/3 git-svn: Reduce temp file usage when dealing with non-linksMarcus Griep, Aug 12, 2008
  19. Eric WongAug 13, 2008
  20. Marcus GriepAug 13, 2008
  21. Eric WongAug 13, 2008
  22. Eric WongAug 12, 2008
  23. Marcus GriepAug 12, 2008
  24. 2/3 git-svn: Make it incrementally faster by minimizing temp filesMarcus Griep, Aug 12, 2008
  25. Eric WongAug 13, 2008
  26. Eric WongAug 12, 2008
  27. Marcus GriepAug 12, 2008
  28. 1/3 Git.pm: Add faculties to allow temp files to be cachedMarcus Griep, Aug 12, 2008
  29. Eric WongAug 13, 2008
  30. Lea WiemannAug 13, 2008
  31. Marcus GriepAug 13, 2008
  32. Marcus GriepAug 13, 2008
  33. Junio C HamanoAug 13, 2008
  34. Lea WiemannAug 13, 2008
  35. Git.pm: require Perl 5.6.1Lea Wiemann, Aug 13, 2008
  36. Eric WongAug 14, 2008
  37. Git.pm: Make File::Spec and File::Temp requirement lazyMarcus Griep, Aug 15, 2008
  38. Bryan DonlanAug 15, 2008
  39. Marcus GriepAug 15, 2008
  40. Git.pm: Make File::Spec and File::Temp requirement lazyMarcus Griep, Aug 15, 2008
  41. Miklos VajnaAug 13, 2008
  42. Junio C HamanoAug 14, 2008
  43. Marcus GriepAug 14, 2008

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.