# [PATCH] remove delta-against-self bit 6 messages from 2006-02-09 to 2006-02-10. Participants: Nicolas Pitre, Junio C Hamano. Thread: https://gitlist.dev/t/3278 ## Nicolas Pitre, 2006-02-09 22:50 Subject: [PATCH] remove delta-against-self bit Message-ID: URL: https://gitlist.dev/e/Pine.LNX.4.64.0602091736500.5397%40localhost.localdomain ``` After experimenting with code to add the ability to encode a delta against part of the deltified file, it turns out that resulting packs are _bigger_ than when this ability is not used. The raw delta output might be smaller, but it doesn't compress as well using gzip with a negative net saving on average. Said bit would in fact be more useful to allow for encoding the copying of chunks larger than 64KB providing more savings with large files. This will correspond to packs version 3. While the current code still produces packs version 2, it is made future proof so pack versions 2 and 3 are accepted. Any pack version 2 are compatible with version 3 since the redefined bit was never used before. When enough time has passed, code to use that bit to produce version 3 packs could be added. Signed-off-by: Nicolas Pitre --- diff --git a/patch-delta.c b/patch-delta.c index 98c27be..c0e1311 100644 --- a/patch-delta.c +++ b/patch-delta.c @@ -44,16 +44,15 @@ void *patch_delta(void *src_buf, unsigne cmd = *data++; if (cmd & 0x80) { unsigned long cp_off = 0, cp_size = 0; - const unsigned char *buf; if (cmd & 0x01) cp_off = *data++; if (cmd & 0x02) cp_off |= (*data++ << 8); if (cmd & 0x04) cp_off |= (*data++ << 16); if (cmd & 0x08) cp_off |= (*data++ << 24); if (cmd & 0x10) cp_size = *data++; if (cmd & 0x20) cp_size |= (*data++ << 8); + if (cmd & 0x40) cp_size |= (*data++ << 16); if (cp_size == 0) cp_size = 0x10000; - buf = (cmd & 0x40) ? dst_buf : src_buf; - memcpy(out, buf + cp_off, cp_size); + memcpy(out, src_buf + cp_off, cp_size); out += cp_size; } else { memcpy(out, data, cmd); diff --git a/pack.h b/pack.h index 657deaa..9dafa2b 100644 --- a/pack.h +++ b/pack.h @@ -21,6 +21,7 @@ enum object_type { */ #define PACK_SIGNATURE 0x5041434b /* "PACK" */ #define PACK_VERSION 2 +#define pack_version_ok(v) ((v) == htonl(2) || (v) == htonl(3)) struct pack_header { unsigned int hdr_signature; unsigned int hdr_version; diff --git a/index-pack.c b/index-pack.c index 541d7bc..babe34b 100644 --- a/index-pack.c +++ b/index-pack.c @@ -68,9 +68,9 @@ static void parse_pack_header(void) hdr = (void *)pack_base; if (hdr->hdr_signature != htonl(PACK_SIGNATURE)) die("packfile '%s' signature mismatch", pack_name); - if (hdr->hdr_version != htonl(PACK_VERSION)) - die("packfile '%s' version %d different from ours %d", - pack_name, ntohl(hdr->hdr_version), PACK_VERSION); + if (!pack_version_ok(hdr->hdr_version)) + die("packfile '%s' version %d unsupported", + pack_name, ntohl(hdr->hdr_version)); nr_objects = ntohl(hdr->hdr_entries); diff --git a/pack-check.c b/pack-check.c index 511f294..67a7ecd 100644 --- a/pack-check.c +++ b/pack-check.c @@ -16,9 +16,9 @@ static int verify_packfile(struct packed hdr = p->pack_base; if (hdr->hdr_signature != htonl(PACK_SIGNATURE)) return error("Packfile %s signature mismatch", p->pack_name); - if (hdr->hdr_version != htonl(PACK_VERSION)) - return error("Packfile version %d different from ours %d", - ntohl(hdr->hdr_version), PACK_VERSION); + if (!pack_version_ok(hdr->hdr_version)) + return error("Packfile version %d unsupported", + ntohl(hdr->hdr_version)); nr_objects = ntohl(hdr->hdr_entries); if (num_packed_objects(p) != nr_objects) return error("Packfile claims to have %d objects, " diff --git a/pack-objects.c b/pack-objects.c diff --git a/unpack-objects.c b/unpack-objects.c index 4b5b5cb..815a1b3 100644 --- a/unpack-objects.c +++ b/unpack-objects.c @@ -246,13 +246,12 @@ static void unpack_all(void) { int i; struct pack_header *hdr = fill(sizeof(struct pack_header)); - unsigned version = ntohl(hdr->hdr_version); unsigned nr_objects = ntohl(hdr->hdr_entries); if (ntohl(hdr->hdr_signature) != PACK_SIGNATURE) die("bad pack file"); - if (version != PACK_VERSION) - die("unable to handle pack file version %d", version); + if (!pack_version_ok(hdr->hdr_version)) + die("unknown pack file version %d", ntohl(hdr->hdr_version)); fprintf(stderr, "Unpacking %d objects\n", nr_objects); use(sizeof(struct pack_header)); ``` ## Junio C Hamano, 2006-02-09 23:53 Subject: Re: [PATCH] remove delta-against-self bit Message-ID: <7v7j846qco.fsf@assigned-by-dhcp.cox.net> URL: https://gitlist.dev/e/7v7j846qco.fsf%40assigned-by-dhcp.cox.net In-Reply-To: ``` Nicolas Pitre writes: > Said bit would in fact be more useful to allow for encoding the copying > of chunks larger than 64KB providing more savings with large files. > This will correspond to packs version 3. > > While the current code still produces packs version 2, it is made future > proof so pack versions 2 and 3 are accepted. Any pack version 2 are > compatible with version 3 since the redefined bit was never used before. > When enough time has passed, code to use that bit to produce version 3 > packs could be added. I agree with the general direction and this futureproofing is a good thing to have before 1.2.0, I think. The bit is however _already_ looked at by the count_delta(), to assess the extent of damage, IIRC. Should we be futureproofing that bit as well? ``` ## Junio C Hamano, 2006-02-10 01:17 Subject: [PATCH] count-delta.c: Match the delta data semantics change in version 3. Message-ID: <7vbqxg3tdd.fsf_-_@assigned-by-dhcp.cox.net> URL: https://gitlist.dev/e/7vbqxg3tdd.fsf_-_%40assigned-by-dhcp.cox.net In-Reply-To: <7v7j846qco.fsf@assigned-by-dhcp.cox.net> ``` Junio C Hamano writes: > I agree with the general direction and this futureproofing is a > good thing to have before 1.2.0, I think. > > The bit is however _already_ looked at by the count_delta(), > to assess the extent of damage, IIRC. Should we be > futureproofing that bit as well? Something like this? -- >8 -- This matches the count_delta() logic to the change previous commit introduces to patch_delta(). Signed-off-by: Junio C Hamano --- count-delta.c | 7 ++----- 1 files changed, 2 insertions(+), 5 deletions(-) 6cb2807fd27b9e970a9b012b925deab99f480d61 diff --git a/count-delta.c b/count-delta.c index 7559ff6..978a60c 100644 --- a/count-delta.c +++ b/count-delta.c @@ -50,13 +50,10 @@ int count_delta(void *delta_buf, unsigne if (cmd & 0x08) cp_off |= (*data++ << 24); if (cmd & 0x10) cp_size = *data++; if (cmd & 0x20) cp_size |= (*data++ << 8); + if (cmd & 0x40) cp_size |= (*data++ << 16); if (cp_size == 0) cp_size = 0x10000; - if (cmd & 0x40) - /* copy from dst */ - ; - else - copied_from_source += cp_size; + copied_from_source += cp_size; out += cp_size; } else { /* write literal into dst */ -- 1.1.6.gce16 ``` ## Nicolas Pitre, 2006-02-10 01:54 Subject: Re: [PATCH] count-delta.c: Match the delta data semantics change in version 3. Message-ID: URL: https://gitlist.dev/e/Pine.LNX.4.64.0602092053200.5397%40localhost.localdomain In-Reply-To: <7vbqxg3tdd.fsf_-_@assigned-by-dhcp.cox.net> ``` On Thu, 9 Feb 2006, Junio C Hamano wrote: > Junio C Hamano writes: > > > I agree with the general direction and this futureproofing is a > > good thing to have before 1.2.0, I think. > > > > The bit is however _already_ looked at by the count_delta(), > > to assess the extent of damage, IIRC. Should we be > > futureproofing that bit as well? > > Something like this? Right. Nicolas ``` ## Nicolas Pitre, 2006-02-10 15:20 Subject: Re: [PATCH] count-delta.c: Match the delta data semantics change in version 3. Message-ID: URL: https://gitlist.dev/e/Pine.LNX.4.64.0602101017420.5397%40localhost.localdomain In-Reply-To: ``` On Thu, 9 Feb 2006, Nicolas Pitre wrote: > On Thu, 9 Feb 2006, Junio C Hamano wrote: > > > Junio C Hamano writes: > > > > > The bit is however _already_ looked at by the count_delta(), > > > to assess the extent of damage, IIRC. Should we be > > > futureproofing that bit as well? > > > > Something like this? > > Right. I'm not sure to fully understand what you meant in the comment below. but if it was related to the removed code, maybe this patch would make sense: diff --git a/count-delta.c b/count-delta.c index 978a60c..058a2aa 100644 --- a/count-delta.c +++ b/count-delta.c @@ -16,11 +16,7 @@ * * Number of bytes that are _not_ copied from the source is deletion, * and number of inserted literal bytes are addition, so sum of them - * is the extent of damage. xdelta can express an edit that copies - * data inside of the destination which originally came from the - * source. We do not count that in the following routine, so we are - * undercounting the source material that remains in the final output - * that way. + * is the extent of damage. */ int count_delta(void *delta_buf, unsigned long delta_size, unsigned long *src_copied, unsigned long *literal_added) ``` ## Junio C Hamano, 2006-02-10 17:19 Subject: Re: [PATCH] count-delta.c: Match the delta data semantics change in version 3. Message-ID: <7vlkwjw2ra.fsf@assigned-by-dhcp.cox.net> URL: https://gitlist.dev/e/7vlkwjw2ra.fsf%40assigned-by-dhcp.cox.net In-Reply-To: ``` You are right about that comment. Thanks. ```