{"thread":{"id":"50069","subject":"[PATCH 0/1] commit-graph: writing missing parents is a BUG","startedAt":"2018-12-19T20:14:09Z","lastAt":"2019-01-02T23:02:04Z","messageCount":3,"participants":["Derrick Stolee via GitGitGadget","Junio C Hamano"],"isPatch":true,"patchVersion":1,"patchTotal":1},"messages":[{"id":"365594","messageId":"pull.102.git.gitgitgadget@gmail.com","threadId":"50069","inReplyTo":null,"subject":"[PATCH 0/1] commit-graph: writing missing parents is a BUG","fromName":"Derrick Stolee via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2018-12-19T20:14:05Z","receivedAt":"2018-12-19T20:14:09Z","isPatch":true,"sender":{"key":"stolee@gmail.com","avatar":"https://avatars.githubusercontent.com/u/570044?v=4"},"body":"A user complained that they had the following message in a git command:\n\nfatal: invalid parent position 2147483647\n\nIn hex, this value is 0x7fffffff, corresponding to the GRAPH_MISSING_PARENT\nconstant. This constant was intended as a way to have the commit-graph store\ncommits with parents that are not also in the commit-graph. During\ndevelopment, however, we chose to require the commit-graph to be closed\nunder reachability. Thus, this value should never be written, and we don't\nfall back to parsing usual commits when we see the constant.\n\nThis actually happened, and the root cause is unknown. This either means the\nreachable closure logic is broken somewhere, or something caused the binary\nsearch to find the parent in our list of commits. This second problem is\nmore likely, as we have seen RAM issues cause corrupted memory before. I'm\nstill investigating the root cause, but for now we can hit a BUG() statement\ninstead of writing bad data.\n\nThanks, -Stolee\n\nDerrick Stolee (1):\n commit-graph: writing missing parents is a BUG\n\n commit-graph.c | 17 +++++++++++------\n 1 file changed, 11 insertions(+), 6 deletions(-)\n\n\nbase-commit: 0d0ac3826a3bbb9247e39e12623bbcfdd722f24c\nPublished-As: https://github.com/gitgitgadget/git/releases/tags/pr-102%2Fderrickstolee%2Fcommit-graph-bug-v1\nFetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-102/derrickstolee/commit-graph-bug-v1\nPull-Request: https://github.com/gitgitgadget/git/pull/102\n-- \ngitgitgadget\n"},{"id":"365595","messageId":"907a24d2c45acec17607971b577782798c4adcbc.1545250444.git.gitgitgadget@gmail.com","threadId":"50069","inReplyTo":"pull.102.git.gitgitgadget@gmail.com","subject":"[PATCH 1/1] commit-graph: writing missing parents is a BUG","fromName":"Derrick Stolee via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2018-12-19T20:14:07Z","receivedAt":"2018-12-19T20:14:13Z","isPatch":true,"sender":{"key":"stolee@gmail.com","avatar":"https://avatars.githubusercontent.com/u/570044?v=4"},"body":"From: Derrick Stolee \n\nWhen writing a commit-graph, we write GRAPH_MISSING_PARENT if the\nparent's object id does not appear in the list of commits to be\nwritten into the commit-graph. This was done as the initial design\nallowed commits to have missing parents, but the final version\nrequires the commit-graph to be closed under reachability. Thus,\nthis GRAPH_MISSING_PARENT value should never be written.\n\nHowever, there are reasons why it could be written! These range\nfrom a bug in the reachable-closure code to a memory error causing\nthe binary search into the list of object ids to fail. In either\ncase, we should fail fast and avoid writing the commit-graph file\nwith bad data.\n\nRemove the GRAPH_MISSING_PARENT constant in favor of the constant\nGRAPH_EDGE_LAST_MASK, which has the same value.\n\nSigned-off-by: Derrick Stolee \n---\n commit-graph.c | 17 +++++++++++------\n 1 file changed, 11 insertions(+), 6 deletions(-)\n\ndiff --git a/commit-graph.c b/commit-graph.c\nindex 40c855f185..c14ada6918 100644\n--- a/commit-graph.c\n+++ b/commit-graph.c\n@@ -34,7 +34,6 @@\n #define GRAPH_OID_LEN GRAPH_OID_LEN_SHA1\n \n #define GRAPH_OCTOPUS_EDGES_NEEDED 0x80000000\n-#define GRAPH_PARENT_MISSING 0x7fffffff\n #define GRAPH_EDGE_LAST_MASK 0x7fffffff\n #define GRAPH_PARENT_NONE 0x70000000\n \n@@ -496,7 +495,9 @@ static void write_graph_chunk_data(struct hashfile *f, int hash_len,\n \t\t\t\t\t commit_to_sha1);\n \n \t\t\tif (edge_value < 0)\n-\t\t\t\tedge_value = GRAPH_PARENT_MISSING;\n+\t\t\t\tBUG(\"missing parent %s for commit %s\",\n+\t\t\t\t oid_to_hex(&parent->item->object.oid),\n+\t\t\t\t oid_to_hex(&(*list)->object.oid));\n \t\t}\n \n \t\thashwrite_be32(f, edge_value);\n@@ -514,7 +515,9 @@ static void write_graph_chunk_data(struct hashfile *f, int hash_len,\n \t\t\t\t\t nr_commits,\n \t\t\t\t\t commit_to_sha1);\n \t\t\tif (edge_value < 0)\n-\t\t\t\tedge_value = GRAPH_PARENT_MISSING;\n+\t\t\t\tBUG(\"missing parent %s for commit %s\",\n+\t\t\t\t oid_to_hex(&parent->item->object.oid),\n+\t\t\t\t oid_to_hex(&(*list)->object.oid));\n \t\t}\n \n \t\thashwrite_be32(f, edge_value);\n@@ -567,7 +570,9 @@ static void write_graph_chunk_large_edges(struct hashfile *f,\n \t\t\t\t\t\t commit_to_sha1);\n \n \t\t\tif (edge_value < 0)\n-\t\t\t\tedge_value = GRAPH_PARENT_MISSING;\n+\t\t\t\tBUG(\"missing parent %s for commit %s\",\n+\t\t\t\t oid_to_hex(&parent->item->object.oid),\n+\t\t\t\t oid_to_hex(&(*list)->object.oid));\n \t\t\telse if (!parent->next)\n \t\t\t\tedge_value |= GRAPH_LAST_EDGE;\n \n@@ -864,7 +869,7 @@ void write_commit_graph(const char *obj_dir,\n \t\t\tcount_distinct++;\n \t}\n \n-\tif (count_distinct >= GRAPH_PARENT_MISSING)\n+\tif (count_distinct >= GRAPH_EDGE_LAST_MASK)\n \t\tdie(_(\"the commit graph format cannot write %d commits\"), count_distinct);\n \n \tcommits.nr = 0;\n@@ -891,7 +896,7 @@ void write_commit_graph(const char *obj_dir,\n \t}\n \tnum_chunks = num_extra_edges ? 4 : 3;\n \n-\tif (commits.nr >= GRAPH_PARENT_MISSING)\n+\tif (commits.nr >= GRAPH_EDGE_LAST_MASK)\n \t\tdie(_(\"too many commits to write graph\"));\n \n \tcompute_generation_numbers(&commits, report_progress);\n-- \ngitgitgadget\n"},{"id":"366057","messageId":"xmqqwonmiryw.fsf@gitster-ct.c.googlers.com","threadId":"50069","inReplyTo":"907a24d2c45acec17607971b577782798c4adcbc.1545250444.git.gitgitgadget@gmail.com","subject":"Re: [PATCH 1/1] commit-graph: writing missing parents is a BUG","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2019-01-02T23:01:59Z","receivedAt":"2019-01-02T23:02:04Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"\"Derrick Stolee via GitGitGadget\" writes:\n\n> From: Derrick Stolee \n>\n> When writing a commit-graph, we write GRAPH_MISSING_PARENT if the\n> parent's object id does not appear in the list of commits to be\n> written into the commit-graph. This was done as the initial design\n> allowed commits to have missing parents, but the final version\n> requires the commit-graph to be closed under reachability. Thus,\n> this GRAPH_MISSING_PARENT value should never be written.\n>\n> However, there are reasons why it could be written! These range\n> from a bug in the reachable-closure code to a memory error causing\n> the binary search into the list of object ids to fail. In either\n> case, we should fail fast and avoid writing the commit-graph file\n> with bad data.\n>\n> Remove the GRAPH_MISSING_PARENT constant in favor of the constant\n> GRAPH_EDGE_LAST_MASK, which has the same value.\n>\n> Signed-off-by: Derrick Stolee \n> ---\n\nThanks, will queue.\n"}]}