git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: [PATCH v3] Prevent megablobs from gunking up git packs

From
Junio C Hamano <junkio@cox.net>
Date
May 26, 2007, 22:51 UTC
Message-ID
<7vwsyvgpvf.fsf@assigned-by-dhcp.cox.net>
In-Reply-To
<465887AB.1010001@gmail.com>
Dana How <danahow@gmail.com> writes:
Show 23 quoted lines
> diff --git a/builtin-pack-objects.c b/builtin-pack-objects.c
> index 19b0aa1..59be849 100644
> --- a/builtin-pack-objects.c
> +++ b/builtin-pack-objects.c
> ...
> @@ -371,8 +372,6 @@ static unsigned long write_object(struct sha1file *f,
>  				pack_size_limit - write_offset : 0;
>  				/* no if no delta */
>  	int usable_delta =	!entry->delta ? 0 :
> -				/* yes if unlimited packfile */
> -				!pack_size_limit ? 1 :
>  				/* no if base written to previous pack */
>  				entry->delta->offset == (off_t)-1 ? 0 :
>  				/* otherwise double-check written to this
> @@ -408,7 +407,7 @@ static unsigned long write_object(struct sha1file *f,
>  		buf = read_sha1_file(entry->sha1, &type, &size);
>  		if (!buf)
>  			die("unable to read %s", sha1_to_hex(entry->sha1));
> -		if (size != entry->size)
> +		if (size != entry->size && type == obj_type)
>  			die("object %s size inconsistency (%lu vs %lu)",
>  			    sha1_to_hex(entry->sha1), size, entry->size);
>  		if (usable_delta) {

I do not quite get how these two hunks relate to the topic of this patch. Care to enlighten?

Show 18 quoted lines
> @@ -564,6 +563,17 @@ static off_t write_one(struct sha1file *f,
>  			return 0;
>  	}
>  
> +	/* refuse to include as many megablobs as possible */
> +	if (max_blob_size && e->size >= max_blob_size) {
> +		struct stat st;
> +		/* skip if unpacked, remotely packed, or loose anywhere */
> +		if (!e->in_pack || !e->in_pack->pack_local || find_sha1_file(e->sha1, &st)) {
> +			e->offset = (off_t)-1;	/* might drop reused delta base if mbs less */
> +			written++;
> +			return offset;
> +		}
> +	}
> +
>  	e->offset = offset;
>  	size = write_object(f, e, offset);
>  	if (!size) {

I thought that you are simply ignoring the "naughty blobs"---why should it be done this late in the call sequence? I haven't followed the existing code nor your patch closely, but I wonder why the filtering is simply done inside (or by the caller of) add_object_entry(). You would need to do sha1_object_info() much earlier than the current code does, though.

Previous: Dana HowNext: Dana How
Message 2 of 6 in “Prevent megablobs from gunking up git packs”
  1. Prevent megablobs from gunking up git packsDana How, May 26, 2007
  2. Junio C HamanoMay 26, 2007
  3. Dana HowMay 26, 2007
  4. Nicolas PitreMay 27, 2007
  5. Dana HowMay 27, 2007
  6. Nicolas PitreMay 27, 2007

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.