git/list[1] front-page[2] threads[3] people[4] search[5] about
 

[PATCH 097/104] Fix checkout of large files to network shares on Windows XP

From
Sam Vilain <sam.vilain@catalyst.net.nz>
Date
May 26, 2010, 06:01 UTC
Message-ID
<1274853674-18521-97-git-send-email-sam.vilain@catalyst.net.nz>
In-Reply-To
<1274853674-18521-1-git-send-email-sam.vilain@catalyst.net.nz>
From: René Scharfe <rene.scharfe@lsrfire.ath.cx>

Bigger writes to network drives on Windows XP fail. Cap them at 31MB to allow them to succeed. Callers need to be prepared for write() calls that do less work than requested anyway.

On local drives, write() calls are translated to WriteFile() calls with a cap of 64KB on Windows XP and 256KB on Vista. Thus a cap of 31MB won't affect the number of WriteFile() calls which do the actual work. There's still room for some other version of Windows to use a chunk size of 1MB without increasing the number of system calls.

Signed-off-by: Rene Scharfe <rene.scharfe@lsrfire.ath.cx>
Signed-off-by: Johannes Sixt <j6t@kdbg.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
---
 compat/mingw.c       |   17 +++++++++++++++++
 compat/mingw.h       |    3 +++
 t/t5705-clone-2gb.sh |   12 +++++++++---
 3 files changed, 29 insertions(+), 3 deletions(-)
diff --git a/compat/mingw.c b/compat/mingw.c
index f90a114..9a8e336 100644
--- a/compat/mingw.c
+++ b/compat/mingw.c
@@ -140,6 +140,23 @@ int mingw_open (const char *filename, int oflags, ...)
 	return fd;
 }
 
+#undef write
+ssize_t mingw_write(int fd, const void *buf, size_t count)
+{
+	/*
+	 * While write() calls to a file on a local disk are translated
+	 * into WriteFile() calls with a maximum size of 64KB on Windows
+	 * XP and 256KB on Vista, no such cap is placed on writes to
+	 * files over the network on Windows XP.  Unfortunately, there
+	 * seems to be a limit of 32MB-28KB on X64 and 64MB-32KB on x86;
+	 * bigger writes fail on Windows XP.
+	 * So we cap to a nice 31MB here to avoid write failures over
+	 * the net without changing the number of WriteFile() calls in
+	 * the local case.
+	 */
+	return write(fd, buf, min(count, 31 * 1024 * 1024));
+}
+
 #undef fopen
 FILE *mingw_fopen (const char *filename, const char *otype)
 {
diff --git a/compat/mingw.h b/compat/mingw.h
index 7c2ab64..0e3e743 100644
--- a/compat/mingw.h
+++ b/compat/mingw.h
@@ -170,6 +170,9 @@ int link(const char *oldpath, const char *newpath);
 int mingw_open (const char *filename, int oflags, ...);
 #define open mingw_open
 
+ssize_t mingw_write(int fd, const void *buf, size_t count);
+#define write mingw_write
+
 FILE *mingw_fopen (const char *filename, const char *otype);
 #define fopen mingw_fopen
 
diff --git a/t/t5705-clone-2gb.sh b/t/t5705-clone-2gb.sh
index adfaae8..8afbdd4 100755
--- a/t/t5705-clone-2gb.sh
+++ b/t/t5705-clone-2gb.sh
@@ -12,7 +12,7 @@ test_expect_success 'setup' '
 
 	git config pack.compression 0 &&
 	git config pack.depth 0 &&
-	blobsize=$((20*1024*1024)) &&
+	blobsize=$((100*1024*1024)) &&
 	blobcount=$((2*1024*1024*1024/$blobsize+1)) &&
 	i=1 &&
 	(while test $i -le $blobcount
@@ -36,9 +36,15 @@ test_expect_success 'setup' '
 
 '
 
-test_expect_success 'clone' '
+test_expect_success 'clone - bare' '
 
-	git clone --bare --no-hardlinks . clone
+	git clone --bare --no-hardlinks . clone-bare
+
+'
+
+test_expect_success 'clone - with worktree, file:// protocol' '
+
+	git clone file://. clone-wt
 
 '
 
-- 
1.7.1.rc2.333.gb2668
Previous: Sam VilainNext: Sam Vilain
Message 15 of 23 in “tests: chmod +x t5150”
  1. 075/104 tests: chmod +x t5150Sam Vilain, May 26, 2010
  2. 076/104 t7604-merge-custom-message: shift expected output creationSam Vilain, May 26, 2010
  3. 082/104 fmt-merge-msg: add function to append shortlog onlySam Vilain, May 26, 2010
  4. 084/104 autocrlf: Make it work also for un-normalized repositoriesSam Vilain, May 26, 2010
  5. 086/104 gitweb: Use @diff_opts while using format-patchSam Vilain, May 26, 2010
  6. 087/104 hash_object: correction for zero length fileSam Vilain, May 26, 2010
  7. 088/104 for-each-ref: Field with abbreviated objectnameSam Vilain, May 26, 2010
  8. 090/104 Documentation: rebase -i ignores options passed to "git am"Sam Vilain, May 26, 2010
  9. 091/104 Documentation: fix minor inconsistencySam Vilain, May 26, 2010
  10. 092/104 Documentation/gitdiffcore: fix order in pickaxe descriptionSam Vilain, May 26, 2010
  11. 093/104 post-receive-email: document command-line modeSam Vilain, May 26, 2010
  12. 094/104 diff: fix coloring of extended diff headersSam Vilain, May 26, 2010
  13. 095/104 Fix "Out of memory? mmap failed" for files larger than 4GB on WindowsSam Vilain, May 26, 2010
  14. 096/104 start_command: close cmd->err descriptor when fork/spawn failsSam Vilain, May 26, 2010
  15. 097/104 Fix checkout of large files to network shares on Windows XPSam Vilain, May 26, 2010
  16. 098/104 mingw: use _commit to implement fsyncSam Vilain, May 26, 2010
  17. 099/104 Recent MinGW has a C99 implementation of snprintf functionsSam Vilain, May 26, 2010
  18. 100/104 Complete prototype of git_config_from_parameters()Sam Vilain, May 26, 2010
  19. 101/104 test get_git_work_tree() return value for NULLSam Vilain, May 26, 2010
  20. 102/104 t7502-commit: fix spellingSam Vilain, May 26, 2010
  21. 103/104 show-branch: use DEFAULT_ABBREV instead of 7Sam Vilain, May 26, 2010
  22. 104/104 Documentation/SubmittingPatches: clarify GMail section and SMTPSam Vilain, May 26, 2010
  23. Sverre RabbelierMay 26, 2010

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.