git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: [PATCH v3 2/3] Introduce GIT_MMAP_LIMIT to allow testing expected mmap size

From
Steffen Prohaska <prohaska@zib.de>
Date
Aug 22, 2014, 13:46 UTC
Message-ID
<0342479D-7C9F-42E6-9B79-745AEDE57EB5@zib.de>
In-Reply-To
<xmqq1ts9qy24.fsf@gitster.dls.corp.google.com>
On Aug 22, 2014, at 12:26 AM, Junio C Hamano <gitster@pobox.com> wrote:
Show 14 quoted lines
> Steffen Prohaska <prohaska@zib.de> writes:
> 
>> Similar to testing expectations about malloc with GIT_ALLOC_LIMIT (see
>> commit d41489), it can be useful to test expectations about mmap.
>> 
>> This introduces a new environment variable GIT_MMAP_LIMIT to limit the
>> largest allowed mmap length (in KB).  xmmap() is modified to check the
>> limit.  Together with GIT_ALLOC_LIMIT tests can now easily confirm
>> expectations about memory consumption.
>> 
>> GIT_ALLOC_LIMIT will be used in the next commit to test that data will
> 
> I smell the need for s/ALLOC/MMAP/ here, but perhaps you did mean
> ALLOC (I won't know until I check 3/3 ;-)
You are right.
Show 15 quoted lines
>> diff --git a/sha1_file.c b/sha1_file.c
>> index 00c07f2..88d64c0 100644
>> --- a/sha1_file.c
>> +++ b/sha1_file.c
>> @@ -663,10 +663,25 @@ void release_pack_memory(size_t need)
>> 		; /* nothing */
>> }
>> 
>> +static void mmap_limit_check(size_t length)
>> +{
>> +	static int limit = -1;
> 
> Perhaps you want ssize_t here?  I see mmap() as a tool to handle a
> lot more data than a single malloc() typically would ;-) so previous
> mistakes by other people would not be a good excuse.
Maybe; and ...
>> +	if (limit == -1) {
>> +		const char *env = getenv("GIT_MMAP_LIMIT");
>> +		limit = env ? atoi(env) * 1024 : 0;
... this should then be changed to atol(env), and ... 
>> +	}
>> +	if (limit && length > limit)
>> +		die("attempting to mmap %"PRIuMAX" over limit %d",
>> +		    (intmax_t)length, limit);
... here PRIuMAX and (uintmax_t); (uintmax_t) also for length.
I'll fix it and also GIT_ALLOC_LIMIT.
	Steffen
Show 13 quoted lines
>> +}
>> +
>> void *xmmap(void *start, size_t length,
>> 	int prot, int flags, int fd, off_t offset)
>> {
>> -	void *ret = mmap(start, length, prot, flags, fd, offset);
>> +	void *ret;
>> +
>> +	mmap_limit_check(length);
>> +	ret = mmap(start, length, prot, flags, fd, offset);
>> 	if (ret == MAP_FAILED) {
>> 		if (!length)
>> 			return NULL;
Previous: Junio C HamanoNext: Junio C Hamano
Message 5 of 9 in “Stream fd to clean filter, GIT_MMAP_LIMIT”
  1. 0/3 Stream fd to clean filter, GIT_MMAP_LIMITSteffen Prohaska, Aug 21, 2014
  2. 1/3 convert: Refactor would_convert_to_git() to single arg 'path'Steffen Prohaska, Aug 21, 2014
  3. 2/3 Introduce GIT_MMAP_LIMIT to allow testing expected mmap sizeSteffen Prohaska, Aug 21, 2014
  4. Junio C HamanoAug 21, 2014
  5. Steffen ProhaskaAug 22, 2014
  6. Junio C HamanoAug 22, 2014
  7. Junio C HamanoAug 22, 2014
  8. Steffen ProhaskaAug 24, 2014
  9. 3/3 convert: Stream from fd to required clean filter instead of mmapSteffen Prohaska, Aug 21, 2014

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.