git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: [PATCH v2 1/1] git-p4: auto-delete named temporary file

From
Junio C Hamano <gitster@pobox.com>
Date
Aug 27, 2019, 22:31 UTC
Message-ID
<xmqq1rx62qbe.fsf@gitster-ct.c.googlers.com>
In-Reply-To
<7e59b5cec2f267820feeeeb63a20814fe67d61e3.1566876175.git.ahippo@yandex.com>
Andrey Mazo <ahippo@yandex.ru> writes:
Show 24 quoted lines
> From: "Philip.McGraw" <Philip.McGraw@bentley.com>
>
> Avoid double-open exceptions on Windows platform when
> calculating for lfs compressed size threshold
> (git-p4.largeFileCompressedThreshold) comparisons.
>
> Take new approach using the NamedTemporaryFile()
> file-like object as input to the ZipFile() which
> auto-deletes after implicit close leaving with scope.
>
> Original code had double-open exception on Windows
> platform because file still open from NamedTemporaryFile()
> using generated filename instead of object.
>
> Thanks to Andrey for patiently suggesting several
> iterations on this change for avoiding exceptions!
>
> Also print error details after resulting IOError to make
> debugging cause of exception less mysterious when it has
> nothing to do with "git version recent enough."
>
> Signed-off-by: Philip.McGraw <Philip.McGraw@bentley.com>
> Reviewed-by: Andrey Mazo <ahippo@yandex.com>
> ---
Luke, does this look good?

I know Mazo is the only other contributor who has multiple commits to git-p4.py in the past 2 years, to make Reviewed-by carry some weight ;-) but as we have so small number of people touching this script anyway, I'd rather see what the main contributor in the past 2 years thinks.

Thanks.
Show 47 quoted lines
>  git-p4.py | 13 ++++++-------
>  1 file changed, 6 insertions(+), 7 deletions(-)
>
> diff --git a/git-p4.py b/git-p4.py
> index c71a6832e2..33bdb14fd1 100755
> --- a/git-p4.py
> +++ b/git-p4.py
> @@ -1158,17 +1158,15 @@ def exceedsLargeFileThreshold(self, relPath, contents):
>          if gitConfigInt('git-p4.largeFileCompressedThreshold'):
>              contentsSize = sum(len(d) for d in contents)
>              if contentsSize <= gitConfigInt('git-p4.largeFileCompressedThreshold'):
>                  return False
>              contentTempFile = self.generateTempFile(contents)
> -            compressedContentFile = tempfile.NamedTemporaryFile(prefix='git-p4-large-file', delete=False)
> -            zf = zipfile.ZipFile(compressedContentFile.name, mode='w')
> -            zf.write(contentTempFile, compress_type=zipfile.ZIP_DEFLATED)
> -            zf.close()
> -            compressedContentsSize = zf.infolist()[0].compress_size
> +            compressedContentFile = tempfile.NamedTemporaryFile(prefix='git-p4-large-file', delete=True)
> +            with zipfile.ZipFile(compressedContentFile, mode='w') as zf:
> +                zf.write(contentTempFile, compress_type=zipfile.ZIP_DEFLATED)
> +                compressedContentsSize = zf.infolist()[0].compress_size
>              os.remove(contentTempFile)
> -            os.remove(compressedContentFile.name)
>              if compressedContentsSize > gitConfigInt('git-p4.largeFileCompressedThreshold'):
>                  return True
>          return False
>  
>      def addLargeFile(self, relPath):
> @@ -3512,12 +3510,13 @@ def importHeadRevision(self, revision):
>          details["time"] = res["time"]
>  
>          self.updateOptionDict(details)
>          try:
>              self.commit(details, self.extractFilesFromCommit(details), self.branch)
> -        except IOError:
> +        except IOError as err:
>              print("IO error with git fast-import. Is your git version recent enough?")
> +            print("IO error details: {}".format(err))
>              print(self.gitError.read())
>  
>      def openStreams(self):
>          self.importProcess = subprocess.Popen(["git", "fast-import"],
>                                                stdin=subprocess.PIPE,
>
> base-commit: 1feeaaf26bff51996f9f96c6dc41ca0f95ab5fc4
> Pull-Request: https://github.com/gitgitgadget/git/pull/303
Previous: Andrey MazoNext: Luke Diamand
Message 5 of 12 in “git-p4: auto-delete named temporary file”
  1. 0/1 git-p4: auto-delete named temporary filePhilip McGraw via GitGitGadget, Aug 1, 2019
  2. AndreyAug 2, 2019
  3. 0/1 git-p4: auto-delete named temporary filePhilip McGraw via GitGitGadget, Aug 2, 2019
  4. 1/1 git-p4: auto-delete named temporary fileAndrey Mazo, Aug 27, 2019
  5. Junio C HamanoAug 27, 2019
  6. Luke DiamandAug 28, 2019
  7. 1/1 git-p4: auto-delete named temporary fileGit Gadget, Aug 26, 2019
  8. Junio C HamanoAug 26, 2019
  9. Johannes SchindelinAug 28, 2019
  10. Junio C HamanoAug 29, 2019
  11. Johannes SchindelinAug 29, 2019
  12. Junio C HamanoOct 6, 2019

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.