threads / rfc / 24758

RFC patch, 2 partscommit: add parse_commit_repl() to replace commits at parsing time

Subject: [RFC/PATCH 1/2] commit: add parse_commit_repl() to replace commits at parsing time

## tl;dr

8 messages between Aug 17, 2010 and Aug 20, 2010. Diffs are folded; open one to read it.

replies: 7people: 4as markdown or json

Christian Couder· Aug 17, 2010, 01:58 UTC · lore

The function parse_commit() is not safe regarding replaced commits because it uses the buffer of the replacement commit but the object part of the commit struct stay the same. Especially the sha1 is not changed so it doesn't match the content of the commit.

To fix that, this patch adds a new function that takes a "struct commit **" instead of a "struct commit *" so we can change the commit pointer that is passed to us.

Signed-off-by: Christian Couder <chriscool@tuxfamily.org>
---
 commit.c |   43 +++++++++++++++++++++++++++++++++++++++++++
 commit.h |    6 ++++++
 2 files changed, 49 insertions(+), 0 deletions(-)
Show changes to 2 files +49 −0

commit.c, commit.h

diff --git a/commit.c b/commit.c
index 652c1ba..f170179 100644
--- a/commit.c
+++ b/commit.c
@@ -316,6 +316,49 @@ int parse_commit(struct commit *item)
 	return ret;
 }
 
+int parse_commit_repl(struct commit **commit)
+{
+	enum object_type type;
+	void *buffer;
+	unsigned long size;
+	int ret;
+	const unsigned char *repl;
+	struct commit *item = *commit;
+
+	if (!item)
+		return -1;
+	if (item->object.parsed)
+		return 0;
+	buffer = read_sha1_file_repl(item->object.sha1, &type, &size, &repl);
+	if (!buffer)
+		return error("Could not read %s",
+			     sha1_to_hex(item->object.sha1));
+
+	if (item->object.sha1 != repl) {
+		struct commit *repl_item = lookup_commit(repl);
+		if (!repl_item) {
+			free(buffer);
+			return error("Bad replacement %s for commit %s",
+				     sha1_to_hex(repl),
+				     sha1_to_hex(item->object.sha1));
+		}
+		repl_item->object.flags = item->object.flags;
+		*commit = item = repl_item;
+	} else if (type != OBJ_COMMIT) {
+		free(buffer);
+		return error("Object %s not a commit",
+			     sha1_to_hex(item->object.sha1));
+	}
+
+	ret = parse_commit_buffer(item, buffer, size);
+	if (save_commit_buffer && !ret) {
+		item->buffer = buffer;
+		return 0;
+	}
+	free(buffer);
+	return ret;
+}
+
 int find_commit_subject(const char *commit_buffer, const char **subject)
 {
 	const char *eol;
diff --git a/commit.h b/commit.h
index a3618f8..d3dfebb 100644
--- a/commit.h
+++ b/commit.h
@@ -39,6 +39,12 @@ struct commit *lookup_commit_reference_gently(const unsigned char *sha1,
 
 int parse_commit_buffer(struct commit *item, void *buffer, unsigned long size);
 
+int parse_commit_repl(struct commit **item);
+
+/*
+ * parse_commit() is deprecated, because it's buggy regarding replacements.
+ * Use parse_commit_repl() instead.
+ */
 int parse_commit(struct commit *item);
 
 /* Find beginning and length of commit subject. */
-- 
1.7.2.1.351.g275bf
Junio C Hamano· Aug 17, 2010, 21:18 UTC · re: Christian Couder · lore

Re: [RFC/PATCH 1/2] commit: add parse_commit_repl() to replace commits at parsing time

Christian Couder <chriscool@tuxfamily.org> writes:
> The function parse_commit() is not safe regarding replaced commits
> because it uses the buffer of the replacement commit but the object
> part of the commit struct stay the same. Especially the sha1 is not
> changed so it doesn't match the content of the commit.
This all sounds backwards to me, if I am reading the discussion correctly.

If a replace record says commit 0123 is replaced by commit 4567 (iow, 0123 was a mistake, and pretend that its content is what is recorded in 4567), and when we are honoring the replace records (iow, we are not fsck), shouldn't read_sha1("0123") give us a piece of memory that stores what is recorded in 4567, parse_object("0123") return a struct commit whose buffer points at a block of memory that has what is recorded in 4567 _while_ its object.sha1[] say "0123"?

What problem are you trying to solve?
Nguyen Thai Ngoc Duy· Aug 18, 2010, 03:17 UTC · re: Junio C Hamano · lore

Re: [RFC/PATCH 1/2] commit: add parse_commit_repl() to replace commits at parsing time

On Wed, Aug 18, 2010 at 7:18 AM, Junio C Hamano <gitster@pobox.com> wrote:
Show 16 quoted lines
> Christian Couder <chriscool@tuxfamily.org> writes:
>
>> The function parse_commit() is not safe regarding replaced commits
>> because it uses the buffer of the replacement commit but the object
>> part of the commit struct stay the same. Especially the sha1 is not
>> changed so it doesn't match the content of the commit.
>
> This all sounds backwards to me, if I am reading the discussion correctly.
>
> If a replace record says commit 0123 is replaced by commit 4567 (iow, 0123
> was a mistake, and pretend that its content is what is recorded in 4567),
> and when we are honoring the replace records (iow, we are not fsck),
> shouldn't read_sha1("0123") give us a piece of memory that stores what is
> recorded in 4567, parse_object("0123") return a struct commit whose buffer
> points at a block of memory that has what is recorded in 4567 _while_ its
> object.sha1[] say "0123"?
1. parse_object() as it is now would return object.sha1[] = "4567".
2. lookup_commit(), then parse_commit() would return object.sha1[] = "0123".
> What problem are you trying to solve?

Inconsistency in replacing objects. I have no comments whether #1 or #2 is expected behavior. But at least it should stick to one behavior only.

-- 
Duy
Christian Couder· Aug 18, 2010, 04:07 UTC · re: Nguyen Thai Ngoc Duy · lore

Re: [RFC/PATCH 1/2] commit: add parse_commit_repl() to replace commits at parsing time

On Wednesday 18 August 2010 05:17:52 Nguyen Thai Ngoc Duy wrote:
Show 27 quoted lines
> On Wed, Aug 18, 2010 at 7:18 AM, Junio C Hamano <gitster@pobox.com> wrote:
> > Christian Couder <chriscool@tuxfamily.org> writes:
> >> The function parse_commit() is not safe regarding replaced commits
> >> because it uses the buffer of the replacement commit but the object
> >> part of the commit struct stay the same. Especially the sha1 is not
> >> changed so it doesn't match the content of the commit.
> > 
> > This all sounds backwards to me, if I am reading the discussion
> > correctly.
> > 
> > If a replace record says commit 0123 is replaced by commit 4567 (iow,
> > 0123 was a mistake, and pretend that its content is what is recorded in
> > 4567), and when we are honoring the replace records (iow, we are not
> > fsck), shouldn't read_sha1("0123") give us a piece of memory that stores
> > what is recorded in 4567, parse_object("0123") return a struct commit
> > whose buffer points at a block of memory that has what is recorded in
> > 4567 _while_ its object.sha1[] say "0 123"?
> 
> 1. parse_object() as it is now would return object.sha1[] = "4567".
> 2. lookup_commit(), then parse_commit() would return object.sha1[] =
> "0123".
> 
> > What problem are you trying to solve?
> 
> Inconsistency in replacing objects. I have no comments whether #1 or
> #2 is expected behavior. But at least it should stick to one behavior
> only.
We discussed this inconsistency in this thread:
http://thread.gmane.org/gmane.comp.version-control.git/152321/ 

So we can resolve the inconsistency with Duy's patch to make parse_object() return object.sha1[] = "0123".

It's simpler and probably safer. The downside is that the sha1 will not be consistent with the content anymore and that it will be more difficult to realize that an object has been replaced as there will be no sha1 change to be seen.

Best regards, Christian.

Jonathan Nieder· Aug 18, 2010, 04:24 UTC · re: Christian Couder · lore

Re: [RFC/PATCH 1/2] commit: add parse_commit_repl() to replace commits at parsing time

Christian Couder wrote:
> The downside is that the sha1 will not be 
> consistent with the content anymore and that it will be more difficult to 
> realize that an object has been replaced as there will be no sha1 change to be 
> seen.

Maybe in the long run it would make sense to keep a "replaced" flag and use it to mark the replace objects specially in some user-facing commands (like log --format=medium).

Nguyen Thai Ngoc Duy· Aug 18, 2010, 04:37 UTC · re: Jonathan Nieder · lore

Re: [RFC/PATCH 1/2] commit: add parse_commit_repl() to replace commits at parsing time

On Wed, Aug 18, 2010 at 2:24 PM, Jonathan Nieder <jrnieder@gmail.com> wrote:
Show 10 quoted lines
> Christian Couder wrote:
>
>> The downside is that the sha1 will not be
>> consistent with the content anymore and that it will be more difficult to
>> realize that an object has been replaced as there will be no sha1 change to be
>> seen.
>
> Maybe in the long run it would make sense to keep a "replaced" flag
> and use it to mark the replace objects specially in some user-facing
> commands (like log --format=medium).

Sounds good (if uses really need to know that). It can be used for commit grafts too.

You would need to find an available bit flag first though. Currently object.flags is used for different purposes and its bit definitions are not centralized. So it's hard to find a "good" bit that no one uses yet, to become the "replaced" bit. OK I'm getting off-topic now.

-- 
Duy
Junio C Hamano· Aug 18, 2010, 14:50 UTC · re: Christian Couder · lore

Re: [RFC/PATCH 1/2] commit: add parse_commit_repl() to replace commits at parsing time

Christian Couder <chriscool@tuxfamily.org> writes:
Show 40 quoted lines
> On Wednesday 18 August 2010 05:17:52 Nguyen Thai Ngoc Duy wrote:
>> On Wed, Aug 18, 2010 at 7:18 AM, Junio C Hamano <gitster@pobox.com> wrote:
>> > Christian Couder <chriscool@tuxfamily.org> writes:
>> >> The function parse_commit() is not safe regarding replaced commits
>> >> because it uses the buffer of the replacement commit but the object
>> >> part of the commit struct stay the same. Especially the sha1 is not
>> >> changed so it doesn't match the content of the commit.
>> > 
>> > This all sounds backwards to me, if I am reading the discussion
>> > correctly.
>> > 
>> > If a replace record says commit 0123 is replaced by commit 4567 (iow,
>> > 0123 was a mistake, and pretend that its content is what is recorded in
>> > 4567), and when we are honoring the replace records (iow, we are not
>> > fsck), shouldn't read_sha1("0123") give us a piece of memory that stores
>> > what is recorded in 4567, parse_object("0123") return a struct commit
>> > whose buffer points at a block of memory that has what is recorded in
>> > 4567 _while_ its object.sha1[] say "0 123"?
>> 
>> 1. parse_object() as it is now would return object.sha1[] = "4567".
>> 2. lookup_commit(), then parse_commit() would return object.sha1[] =
>> "0123".
>> 
>> > What problem are you trying to solve?
>> 
>> Inconsistency in replacing objects. I have no comments whether #1 or
>> #2 is expected behavior. But at least it should stick to one behavior
>> only.
>
> We discussed this inconsistency in this thread:
>
> http://thread.gmane.org/gmane.comp.version-control.git/152321/ 
>
> So we can resolve the inconsistency with Duy's patch to make parse_object() 
> return object.sha1[] = "0123".
>
> It's simpler and probably safer. The downside is that the sha1 will not be 
> consistent with the content anymore and that it will be more difficult to 
> realize that an object has been replaced as there will be no sha1 change to be 
> seen.
I do not see it as a downside at all.

If the user wants to take replaced objects, they should be shown just like an ordinary objects at the machinery level.

Of course, the user is free to add comments on the commit log to note the fact that a new commit is replacing some other commit and for what purpose. Also if somebody really wants to, cat-file piped to hash-object can be used to see the difference.

Christian Couder· Aug 20, 2010, 04:04 UTC · re: Junio C Hamano · lore

Re: [RFC/PATCH 1/2] commit: add parse_commit_repl() to replace commits at parsing time

On Wednesday 18 August 2010 16:50:24 Junio C Hamano wrote:
> Christian Couder <chriscool@tuxfamily.org> writes:
> > On Wednesday 18 August 2010 05:17:52 Nguyen Thai Ngoc Duy wrote:
> >> On Wed, Aug 18, 2010 at 7:18 AM, Junio C Hamano <gitster@pobox.com> 
wrote:
Show 48 quoted lines
> >> > Christian Couder <chriscool@tuxfamily.org> writes:
> >> >> The function parse_commit() is not safe regarding replaced commits
> >> >> because it uses the buffer of the replacement commit but the object
> >> >> part of the commit struct stay the same. Especially the sha1 is not
> >> >> changed so it doesn't match the content of the commit.
> >> > 
> >> > This all sounds backwards to me, if I am reading the discussion
> >> > correctly.
> >> > 
> >> > If a replace record says commit 0123 is replaced by commit 4567 (iow,
> >> > 0123 was a mistake, and pretend that its content is what is recorded
> >> > in 4567), and when we are honoring the replace records (iow, we are
> >> > not fsck), shouldn't read_sha1("0123") give us a piece of memory that
> >> > stores what is recorded in 4567, parse_object("0123") return a struct
> >> > commit whose buffer points at a block of memory that has what is
> >> > recorded in 4567 _while_ its object.sha1[] say "0 123"?
> >> 
> >> 1. parse_object() as it is now would return object.sha1[] = "4567".
> >> 2. lookup_commit(), then parse_commit() would return object.sha1[] =
> >> "0123".
> >> 
> >> > What problem are you trying to solve?
> >> 
> >> Inconsistency in replacing objects. I have no comments whether #1 or
> >> #2 is expected behavior. But at least it should stick to one behavior
> >> only.
> > 
> > We discussed this inconsistency in this thread:
> > 
> > http://thread.gmane.org/gmane.comp.version-control.git/152321/
> > 
> > So we can resolve the inconsistency with Duy's patch to make
> > parse_object() return object.sha1[] = "0123".
> > 
> > It's simpler and probably safer. The downside is that the sha1 will not
> > be consistent with the content anymore and that it will be more
> > difficult to realize that an object has been replaced as there will be
> > no sha1 change to be seen.
> 
> I do not see it as a downside at all.
> 
> If the user wants to take replaced objects, they should be shown just like
> an ordinary objects at the machinery level.
> 
> Of course, the user is free to add comments on the commit log to note the
> fact that a new commit is replacing some other commit and for what
> purpose.  Also if somebody really wants to, cat-file piped to hash-object
> can be used to see the difference.
Ok so please apply Duy's patch perhaps with an improved commit message.

Thanks, Christian.

← back to recent threads