threads / patch / 6854

patchSupport for large files on 32bit systems.

Subject: [PATCH] Support for large files on 32bit systems.

## tl;dr

6 messages between Feb 17, 2007 and Feb 19, 2007. Diffs are folded; open one to read it.

replies: 5people: 4as markdown or json

Martin Waitz· Feb 17, 2007, 09:13 UTC · lore

Glibc uses the same size for int and off_t by default. In order to support large pack sizes (>2GB) we force Glibc to a 64bit off_t.

Signed-off-by: Martin Waitz <tali@admingilde.org>
---
 git-compat-util.h |    5 +++++
 1 files changed, 5 insertions(+), 0 deletions(-)
Show changes to git-compat-util.h +5 −0
diff --git a/git-compat-util.h b/git-compat-util.h
index bf3ceb8..38f9594 100644
--- a/git-compat-util.h
+++ b/git-compat-util.h
@@ -19,6 +19,11 @@
 #define _GNU_SOURCE 1
 #define _BSD_SOURCE 1
 
+/* support large files even on 32bit systems */
+#ifdef __GLIBC__
+#define _FILE_OFFSET_BITS=64
+#endif
+
 #include <unistd.h>
 #include <stdio.h>
 #include <sys/stat.h>

-- 
1.5.0.rc2.g08361-dirty

-- 
Martin Waitz
Martin Waitz· Feb 17, 2007, 09:39 UTC · re: Martin Waitz · lore

Re: [PATCH] Support for large files on 32bit systems.

hoi :)

arg, this patch is completely broken. I tested it by setting the macro via the command line and the = is still left in there. And __GLIBC__ is not yet defined at this point.

However, we need to set _FILE_OFFSET_BITS for glibc somehow. What is the best way to do so?

-- 
Martin Waitz
Shawn O. Pearce· Feb 17, 2007, 09:49 UTC · re: Martin Waitz · lore

Re: [PATCH] Support for large files on 32bit systems.

Martin Waitz <tali@admingilde.org> wrote:
Show 7 quoted lines
> arg, this patch is completely broken.
> I tested it by setting the macro via the command line and the
> = is still left in there.  And __GLIBC__ is not yet defined at this
> point.
> 
> However, we need to set _FILE_OFFSET_BITS for glibc somehow.
> What is the best way to do so?

I think the only way to do with this is to have the Makefile detect if -D_FILE_OFFSET_BITS=64 is required to be added to CFLAGS based on some rule (e.g. uname output?), then add that to CFLAGS when needed.

This is one area where autoconf is nice, as you can compile a thousand little C programs to see how the compiler reacts by default, and when you attempt to set this define, and then decide if you should include it or not.

It is also the part of autoconf that's not-so-nice... as it compiles a thousand little C programs just to figure out how to compile the real program. ;-)

-- 
Shawn.
Martin Waitz· Feb 17, 2007, 10:46 UTC · re: Shawn O. Pearce · lore

Re: [PATCH] Support for large files on 32bit systems.

hoi :)
On Sat, Feb 17, 2007 at 04:49:59AM -0500, Shawn O. Pearce wrote:
> I think the only way to do with this is to have the Makefile detect
> if -D_FILE_OFFSET_BITS=64 is required to be added to CFLAGS based on
> some rule (e.g. uname output?), then add that to CFLAGS when needed.
something like this?

(I'm just testing it with a large data transfer, which will take some time..., will report success later)

+++ Support for large files on 32bit systems.

Glibc uses the same size for int and off_t by default. In order to support large pack sizes (>2GB) we force Glibc to a 64bit off_t.

Signed-off-by: Martin Waitz <tali@admingilde.org>
---
 Makefile |    2 ++
 1 files changed, 2 insertions(+), 0 deletions(-)
Show changes to Makefile +2 −0
diff --git a/Makefile b/Makefile
index ebecbbd..325c19f 100644
--- a/Makefile
+++ b/Makefile
@@ -334,9 +334,11 @@ EXTLIBS = -lz
 
 ifeq ($(uname_S),Linux)
 	NO_STRLCPY = YesPlease
+	BASIC_CFLAGS = -D_FILE_OFFSET_BITS=64
 endif
 ifeq ($(uname_S),GNU/kFreeBSD)
 	NO_STRLCPY = YesPlease
+	BASIC_CFLAGS = -D_FILE_OFFSET_BITS=64
 endif
 ifeq ($(uname_S),Darwin)
 	NEEDS_SSL_WITH_CRYPTO = YesPlease

-- 
1.5.0.80.g42d14


-- 
Martin Waitz
Nicolas Pitre· Feb 17, 2007, 13:32 UTC · re: Shawn O. Pearce · lore

Re: [PATCH] Support for large files on 32bit systems.

On Sat, 17 Feb 2007, Shawn O. Pearce wrote:
Show 12 quoted lines
> Martin Waitz <tali@admingilde.org> wrote:
> > arg, this patch is completely broken.
> > I tested it by setting the macro via the command line and the
> > = is still left in there.  And __GLIBC__ is not yet defined at this
> > point.
> > 
> > However, we need to set _FILE_OFFSET_BITS for glibc somehow.
> > What is the best way to do so?
> 
> I think the only way to do with this is to have the Makefile detect
> if -D_FILE_OFFSET_BITS=64 is required to be added to CFLAGS based on
> some rule (e.g. uname output?), then add that to CFLAGS when needed.
Why not simply defining _FILE_OFFSET_BITS=64 unconditionally?

It certainly won't cause GIT to explode if compiled against something else than glibc.

Nicolas
Junio C Hamano· Feb 19, 2007, 02:44 UTC · re: Nicolas Pitre · lore

Re: [PATCH] Support for large files on 32bit systems.

Nicolas Pitre <nico@cam.org> writes:
Show 8 quoted lines
>> I think the only way to do with this is to have the Makefile detect
>> if -D_FILE_OFFSET_BITS=64 is required to be added to CFLAGS based on
>> some rule (e.g. uname output?), then add that to CFLAGS when needed.
>
> Why not simply defining _FILE_OFFSET_BITS=64 unconditionally?
>
> It certainly won't cause GIT to explode if compiled against something 
> else than glibc.
I like the simplicity of that approach.  Like this?
Show changes to git-compat-util.h +2 −0
diff --git a/git-compat-util.h b/git-compat-util.h
index 105ac28..dd5f369 100644
--- a/git-compat-util.h
+++ b/git-compat-util.h
@@ -1,6 +1,8 @@
 #ifndef GIT_COMPAT_UTIL_H
 #define GIT_COMPAT_UTIL_H
 
+#define _FILE_OFFSET_BITS 64
+
 #ifndef FLEX_ARRAY
 #if defined(__GNUC__) && (__GNUC__ < 3)
 #define FLEX_ARRAY 0

← back to recent threads