{"thread":{"id":"370","subject":"[0/5] Updates for library, fsck-cache","startedAt":"2005-04-28T05:00:24Z","lastAt":"2005-04-28T06:01:56Z","messageCount":10,"participants":["Daniel Barkalow","Linus Torvalds"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"1993","messageId":"Pine.LNX.4.21.0504280055180.30848-100000@iabervon.org","threadId":"370","inReplyTo":null,"subject":"[0/5] Updates for library, fsck-cache","fromName":"Daniel Barkalow","fromEmail":"barkalow@iabervon.org","sentAt":"2005-04-28T05:00:24Z","receivedAt":"2005-04-28T05:00:24Z","isPatch":false,"sender":{"key":"barkalow@iabervon.org","avatar":"https://avatars.githubusercontent.com/u/55364219?v=4"},"body":"This patch series updates the parser library for some new things, adds a\nfunction to parse an object of a type that's understood by the library but\nnot known in advance by the program, updates fsck-cache to use it, and\nadds support for fsck-cache getting the root set out of .git/refs/.\n\n 1: Consider a blob to be parsed\n 2: Add tag header/parser to library\n 3: Add function to parse an object of unspecified type.\n 4: Rework fsck-cache to use parse_object().\n 5: Make fsck-cache read heads out of .git/refs/*/*\n\n\t-Daniel\n*This .sig left intentionally blank*\n\n"},{"id":"1994","messageId":"Pine.LNX.4.21.0504280100360.30848-100000@iabervon.org","threadId":"370","inReplyTo":"Pine.LNX.4.21.0504280055180.30848-100000@iabervon.org","subject":"[1/5] Consider a blob to be parsed","fromName":"Daniel Barkalow","fromEmail":"barkalow@iabervon.org","sentAt":"2005-04-28T05:04:57Z","receivedAt":"2005-04-28T05:04:57Z","isPatch":false,"sender":{"key":"barkalow@iabervon.org","avatar":"https://avatars.githubusercontent.com/u/55364219?v=4"},"body":"We don't parse blobs at all, so any that we've got are as parsed as\nthey're going to get. Don't make fsck-cache mark them.\n\nSigned-Off-By: Daniel Barkalow <barkalow@iabervon.org>\nIndex: blob.c\n===================================================================\n--- 89fdfd09b281fdf5071bc13a30ef683bd6851b61/blob.c  (mode:100644 sha1:3d99b93f020d84c5410c2b1056f2d7446b647d1e)\n+++ 40a5a5657d67569bdc0e37b8ea2b76ec42824604/blob.c  (mode:100644 sha1:2fad8a5eb4c553190736870519f65c265dfb8526)\n@@ -12,6 +12,7 @@\n \t\tmemset(ret, 0, sizeof(struct blob));\n \t\tcreated_object(sha1, &ret->object);\n \t\tret->object.type = blob_type;\n+\t\tret->object.parsed = 1;\n \t\treturn ret;\n \t}\n \tif (obj->parsed && obj->type != blob_type) {\n\n"},{"id":"1995","messageId":"Pine.LNX.4.21.0504280105010.30848-100000@iabervon.org","threadId":"370","inReplyTo":"Pine.LNX.4.21.0504280055180.30848-100000@iabervon.org","subject":"[2/5] Add tag header/parser to library","fromName":"Daniel Barkalow","fromEmail":"barkalow@iabervon.org","sentAt":"2005-04-28T05:06:50Z","receivedAt":"2005-04-28T05:06:50Z","isPatch":false,"sender":{"key":"barkalow@iabervon.org","avatar":"https://avatars.githubusercontent.com/u/55364219?v=4"},"body":"This adds preliminary support for tags in the library. It doesn't even\nstore the signature, however, let alone provide any way of checking it.\n\nSigned-Off-By: Daniel Barkalow <barkalow@iabervon.org>\nIndex: Makefile\n===================================================================\n--- 40a5a5657d67569bdc0e37b8ea2b76ec42824604/Makefile  (mode:100644 sha1:2d2913b6b98ac836b43755b1304d2a838dad87dd)\n+++ e08498de8d0216a24378c454597ecb38f65c4ee1/Makefile  (mode:100644 sha1:b533a2834da564f2fc8d00680aeac59c9d197bd8)\n@@ -25,7 +25,7 @@\n install: $(PROG) $(SCRIPTS)\n \tinstall $(PROG) $(SCRIPTS) $(HOME)/bin/\n \n-LIB_OBJS=read-cache.o sha1_file.o usage.o object.o commit.o tree.o blob.o\n+LIB_OBJS=read-cache.o sha1_file.o usage.o object.o commit.o tree.o blob.o tag.o\n LIB_FILE=libgit.a\n LIB_H=cache.h object.h\n \nIndex: tag.c\n===================================================================\n--- /dev/null  (tree:40a5a5657d67569bdc0e37b8ea2b76ec42824604)\n+++ e08498de8d0216a24378c454597ecb38f65c4ee1/tag.c  (mode:100644 sha1:f079d83a9de5c3d6d98b2a29339ec1ef3a25f333)\n@@ -0,0 +1,73 @@\n+#include \"tag.h\"\n+#include \"cache.h\"\n+\n+const char *tag_type = \"tag\";\n+\n+struct tag *lookup_tag(unsigned char *sha1)\n+{\n+        struct object *obj = lookup_object(sha1);\n+        if (!obj) {\n+                struct tag *ret = xmalloc(sizeof(struct tag));\n+                memset(ret, 0, sizeof(struct tag));\n+                created_object(sha1, &ret->object);\n+                ret->object.type = tag_type;\n+                return ret;\n+        }\n+        if (obj->type != tag_type) {\n+                error(\"Object %s is a %s, not a tree\", \n+                      sha1_to_hex(sha1), obj->type);\n+                return NULL;\n+        }\n+        return (struct tag *) obj;\n+}\n+\n+int parse_tag(struct tag *item)\n+{\n+        char type[20];\n+        void *data, *bufptr;\n+        unsigned long size;\n+        if (item->object.parsed)\n+                return 0;\n+        item->object.parsed = 1;\n+        data = bufptr = read_sha1_file(item->object.sha1, type, &size);\n+        if (!data)\n+                return error(\"Could not read %s\",\n+                             sha1_to_hex(item->object.sha1));\n+        if (strcmp(type, tag_type))\n+                return error(\"Object %s not a tag\",\n+                             sha1_to_hex(item->object.sha1));\n+\n+\tint typelen, taglen;\n+\tunsigned char object[20];\n+\tconst char *type_line, *tag_line, *sig_line;\n+\n+\tif (size < 64)\n+\t\treturn -1;\n+\tif (memcmp(\"object \", data, 7) || get_sha1_hex(data + 7, object))\n+\t\treturn -1;\n+\n+\titem->tagged = parse_object(object);\n+\n+\ttype_line = data + 48;\n+\tif (memcmp(\"\\ntype \", type_line-1, 6))\n+\t\treturn -1;\n+\n+\ttag_line = strchr(type_line, '\\n');\n+\tif (!tag_line || memcmp(\"tag \", ++tag_line, 4))\n+\t\treturn -1;\n+\n+\tsig_line = strchr(tag_line, '\\n');\n+\tif (!sig_line)\n+\t\treturn -1;\n+\tsig_line++;\n+\n+\ttypelen = tag_line - type_line - strlen(\"type \\n\");\n+\tif (typelen >= 20)\n+\t\treturn -1;\n+\ttaglen = sig_line - tag_line - strlen(\"tag \\n\");\n+\titem->tag = xmalloc(taglen + 1);\n+\tmemcpy(item->tag, tag_line + 4, taglen);\n+\titem->tag[taglen] = '\\0';\n+\n+\treturn 0;\n+}\nIndex: tag.h\n===================================================================\n--- /dev/null  (tree:40a5a5657d67569bdc0e37b8ea2b76ec42824604)\n+++ e08498de8d0216a24378c454597ecb38f65c4ee1/tag.h  (mode:100644 sha1:7ae7864d8a8477bfd22d4e29d78119d345f2bbb9)\n@@ -0,0 +1,15 @@\n+#ifndef TAG_H\n+#define TAG_H\n+\n+#include \"object.h\"\n+\n+extern const char *tag_type;\n+\n+struct tag {\n+\tstruct object object;\n+\tstruct object *tagged;\n+\tchar *tag;\n+\tchar *signature; /* not actually implemented */\n+};\n+\n+#endif /* TAG_H */\n\n"},{"id":"1996","messageId":"Pine.LNX.4.21.0504280106540.30848-100000@iabervon.org","threadId":"370","inReplyTo":"Pine.LNX.4.21.0504280055180.30848-100000@iabervon.org","subject":"[3/5] Add function to parse an object of unspecified type","fromName":"Daniel Barkalow","fromEmail":"barkalow@iabervon.org","sentAt":"2005-04-28T05:11:44Z","receivedAt":"2005-04-28T05:11:44Z","isPatch":false,"sender":{"key":"barkalow@iabervon.org","avatar":"https://avatars.githubusercontent.com/u/55364219?v=4"},"body":"This adds a function that parses an object from the database when we have\nto look up its actual type. It also checks the hash of the file, due to\nits heritage as part of fsck-cache.\n\nSigned-Off-By: Daniel Barkalow <barkalow@iabervon.org>\nIndex: object.c\n===================================================================\n--- e08498de8d0216a24378c454597ecb38f65c4ee1/object.c  (mode:100644 sha1:91bbc6e5e2eadfb0a66b14d992eac260d07267f8)\n+++ dd06c679f8acb720976073de90a9eeabbc11e951/object.c  (mode:100644 sha1:787cda1f5667cdd4e92b7c18eb115cf125f6df9c)\n@@ -1,5 +1,9 @@\n #include \"object.h\"\n+#include \"blob.h\"\n+#include \"tree.h\"\n+#include \"commit.h\"\n #include \"cache.h\"\n+#include \"tag.h\"\n #include <stdlib.h>\n #include <string.h>\n \n@@ -94,3 +98,39 @@\n \t\tp = p->next;\n \t}\n }\n+\n+struct object *parse_object(unsigned char *sha1)\n+{\n+\tunsigned long mapsize;\n+\tvoid *map = map_sha1_file(sha1, &mapsize);\n+\tif (map) {\n+\t\tchar type[100];\n+\t\tunsigned long size;\n+\t\tvoid *buffer = unpack_sha1_file(map, mapsize, type, &size);\n+\t\tif (!buffer)\n+\t\t\treturn NULL;\n+\t\tif (check_sha1_signature(sha1, buffer, size, type) < 0)\n+\t\t\tprintf(\"sha1 mismatch %s\\n\", sha1_to_hex(sha1));\n+\t\tmunmap(map, mapsize);\n+\t\tif (!strcmp(type, \"blob\")) {\n+\t\t\tstruct blob *ret = lookup_blob(sha1);\n+\t\t\t//parse_blob(ret);\n+\t\t\treturn &ret->object;\n+\t\t} else if (!strcmp(type, \"tree\")) {\n+\t\t\tstruct tree *ret = lookup_tree(sha1);\n+\t\t\tparse_tree(ret);\n+\t\t\treturn &ret->object;\n+\t\t} else if (!strcmp(type, \"commit\")) {\n+\t\t\tstruct commit *ret = lookup_commit(sha1);\n+\t\t\tparse_commit(ret);\n+\t\t\treturn &ret->object;\n+\t\t} else if (!strcmp(type, \"tag\")) {\n+\t\t\tstruct tag *ret = lookup_tag(sha1);\n+\t\t\tparse_tag(ret);\n+\t\t\treturn &ret->object;\n+\t\t} else {\n+\t\t\treturn NULL;\n+\t\t}\n+\t}\n+\treturn NULL;\n+}\nIndex: object.h\n===================================================================\n--- e08498de8d0216a24378c454597ecb38f65c4ee1/object.h  (mode:100644 sha1:bc607fd55f6ce4e56ce87766369b5d4d55ec79af)\n+++ dd06c679f8acb720976073de90a9eeabbc11e951/object.h  (mode:100644 sha1:d53a35a4d7321b5ec970103208ac576f9f722dff)\n@@ -22,6 +22,9 @@\n \n void created_object(unsigned char *sha1, struct object *obj);\n \n+/** Returns the object, having parsed it to find out what it is. **/\n+struct object *parse_object(unsigned char *sha1);\n+\n void add_ref(struct object *refer, struct object *target);\n \n void mark_reachable(struct object *obj, unsigned int mask);\n\n"},{"id":"1997","messageId":"Pine.LNX.4.21.0504280111490.30848-100000@iabervon.org","threadId":"370","inReplyTo":"Pine.LNX.4.21.0504280055180.30848-100000@iabervon.org","subject":"[4/5] Rework fsck-cache to use parse_object()","fromName":"Daniel Barkalow","fromEmail":"barkalow@iabervon.org","sentAt":"2005-04-28T05:13:46Z","receivedAt":"2005-04-28T05:13:46Z","isPatch":false,"sender":{"key":"barkalow@iabervon.org","avatar":"https://avatars.githubusercontent.com/u/55364219?v=4"},"body":"With support for parse_object() and tags in the core, fsck_cache can just\ncall them, and can be simplified a bit.\n\nSigned-Off-By: Daniel Barkalow <barkalow@iabervon.org>\nIndex: fsck-cache.c\n===================================================================\n--- dd06c679f8acb720976073de90a9eeabbc11e951/fsck-cache.c  (mode:100644 sha1:f9b1431dd8f4f3b426a7e410de952277aaa11401)\n+++ ca8a271be1370ab0795a869c15114e566bdd15d8/fsck-cache.c  (mode:100644 sha1:280a104050b665515418c00c33af8e6b0b0e2101)\n@@ -6,6 +6,7 @@\n #include \"commit.h\"\n #include \"tree.h\"\n #include \"blob.h\"\n+#include \"tag.h\"\n \n #define REACHABLE 0x0001\n \n@@ -21,6 +22,7 @@\n \t/* Look up all the requirements, warn about missing objects.. */\n \tfor (i = 0; i < nr_objs; i++) {\n \t\tstruct object *obj = objs[i];\n+\t\tstruct object_list *refs;\n \n \t\tif (show_unreachable && !(obj->flags & REACHABLE)) {\n \t\t\tprintf(\"unreachable %s %s\\n\", obj->type, sha1_to_hex(obj->sha1));\n@@ -35,101 +37,49 @@\n \t\t\tprintf(\"dangling %s %s\\n\", obj->type, \n \t\t\t       sha1_to_hex(obj->sha1));\n \t\t}\n+\t\tfor (refs = obj->refs; refs; refs = refs->next) {\n+\t\t\tif (!refs->item->parsed) {\n+\t\t\t\tprintf(\"broken link from %s\\n\",\n+\t\t\t\t       sha1_to_hex(obj->sha1));\n+\t\t\t\tprintf(\"              to %s\\n\",\n+\t\t\t\t       sha1_to_hex(refs->item->sha1));\n+\t\t\t}\n+\t\t}\n \t}\n }\n \n-static int fsck_tree(unsigned char *sha1, void *data, unsigned long size)\n+static int fsck_tree(struct tree *item)\n {\n-\tstruct tree *item = lookup_tree(sha1);\n-\tif (parse_tree(item))\n-\t\treturn -1;\n \tif (item->has_full_path) {\n \t\tfprintf(stderr, \"warning: fsck-cache: tree %s \"\n-\t\t\t\"has full pathnames in it\\n\", sha1_to_hex(sha1));\n+\t\t\t\"has full pathnames in it\\n\", \n+\t\t\tsha1_to_hex(item->object.sha1));\n \t}\n \treturn 0;\n }\n \n-static int fsck_commit(unsigned char *sha1, void *data, unsigned long size)\n+static int fsck_commit(struct commit *commit)\n {\n-\tstruct commit *commit = lookup_commit(sha1);\n-\tif (parse_commit(commit))\n-\t\treturn -1;\n \tif (!commit->tree)\n \t\treturn -1;\n \tif (!commit->parents && show_root)\n-\t\tprintf(\"root %s\\n\", sha1_to_hex(sha1));\n+\t\tprintf(\"root %s\\n\", sha1_to_hex(commit->object.sha1));\n \tif (!commit->date)\n-\t\tprintf(\"bad commit date in %s\\n\", sha1_to_hex(sha1));\n-\treturn 0;\n-}\n-\n-static int fsck_blob(unsigned char *sha1, void *data, unsigned long size)\n-{\n-\tstruct blob *blob = lookup_blob(sha1);\n-\tblob->object.parsed = 1;\n+\t\tprintf(\"bad commit date in %s\\n\", \n+\t\t       sha1_to_hex(commit->object.sha1));\n \treturn 0;\n }\n \n-static int fsck_tag(unsigned char *sha1, void *data, unsigned long size)\n+static int fsck_tag(struct tag *tag)\n {\n-\tint typelen, taglen;\n-\tunsigned char object[20];\n-\tchar object_hex[60];\n-\tconst char *type_line, *tag_line, *sig_line;\n-\n-\tif (size < 64)\n-\t\treturn -1;\n-\tif (memcmp(\"object \", data, 7) || get_sha1_hex(data + 7, object))\n-\t\treturn -1;\n-\n-\ttype_line = data + 48;\n-\tif (memcmp(\"\\ntype \", type_line-1, 6))\n-\t\treturn -1;\n-\n-\ttag_line = strchr(type_line, '\\n');\n-\tif (!tag_line || memcmp(\"tag \", ++tag_line, 4))\n-\t\treturn -1;\n-\n-\tsig_line = strchr(tag_line, '\\n');\n-\tif (!sig_line)\n-\t\treturn -1;\n-\tsig_line++;\n-\n-\ttypelen = tag_line - type_line - strlen(\"type \\n\");\n-\tif (typelen >= 20)\n-\t\treturn -1;\n-\ttaglen = sig_line - tag_line - strlen(\"tag \\n\");\n-\n \tif (!show_tags)\n \t\treturn 0;\n \n-\tstrcpy(object_hex, sha1_to_hex(object));\n-\tprintf(\"tagged %.*s %s (%.*s) in %s\\n\",\n-\t\ttypelen, type_line + 5,\n-\t\tobject_hex,\n-\t\ttaglen, tag_line + 4,\n-\t\tsha1_to_hex(sha1));\n-\treturn 0;\n-}\n-\n-static int fsck_entry(unsigned char *sha1, char *tag, void *data, \n-\t\t      unsigned long size)\n-{\n-\tif (!strcmp(tag, \"blob\")) {\n-\t\tif (fsck_blob(sha1, data, size) < 0)\n-\t\t\treturn -1;\n-\t} else if (!strcmp(tag, \"tree\")) {\n-\t\tif (fsck_tree(sha1, data, size) < 0)\n-\t\t\treturn -1;\n-\t} else if (!strcmp(tag, \"commit\")) {\n-\t\tif (fsck_commit(sha1, data, size) < 0)\n-\t\t\treturn -1;\n-\t} else if (!strcmp(tag, \"tag\")) {\n-\t\tif (fsck_tag(sha1, data, size) < 0)\n-\t\t\treturn -1;\n-\t} else\n-\t\treturn -1;\n+\tprintf(\"tagged %s %s\",\n+\t       tag->tagged->type,\n+\t       sha1_to_hex(tag->tagged->sha1));\n+\tprintf(\" (%s) in %s\\n\",\n+\t       tag->tag, sha1_to_hex(tag->object.sha1));\n \treturn 0;\n }\n \n@@ -137,20 +87,17 @@\n {\n \tunsigned char sha1[20];\n \tif (!get_sha1_hex(hex, sha1)) {\n-\t\tunsigned long mapsize;\n-\t\tvoid *map = map_sha1_file(sha1, &mapsize);\n-\t\tif (map) {\n-\t\t\tchar type[100];\n-\t\t\tunsigned long size;\n-\t\t\tvoid *buffer = unpack_sha1_file(map, mapsize, type, &size);\n-\t\t\tif (!buffer)\n-\t\t\t\treturn -1;\n-\t\t\tif (check_sha1_signature(sha1, buffer, size, type) < 0)\n-\t\t\t\tprintf(\"sha1 mismatch %s\\n\", sha1_to_hex(sha1));\n-\t\t\tmunmap(map, mapsize);\n-\t\t\tif (!fsck_entry(sha1, type, buffer, size))\n-\t\t\t\treturn 0;\n-\t\t}\n+\t\tstruct object *obj = parse_object(sha1);\n+\t\tif (!obj)\n+\t\t\treturn -1;\n+\t\tif (obj->type == blob_type)\n+\t\t\treturn 0;\n+\t\tif (obj->type == tree_type)\n+\t\t\treturn fsck_tree((struct tree *) obj);\n+\t\tif (obj->type == commit_type)\n+\t\t\treturn fsck_commit((struct commit *) obj);\n+\t\tif (obj->type == tag_type)\n+\t\t\treturn fsck_tag((struct tag *) obj);\n \t}\n \treturn -1;\n }\n\n"},{"id":"1999","messageId":"Pine.LNX.4.21.0504280113560.30848-100000@iabervon.org","threadId":"370","inReplyTo":"Pine.LNX.4.21.0504280055180.30848-100000@iabervon.org","subject":"[5/5] Make fsck-cache read heads out of .git/refs/*/*","fromName":"Daniel Barkalow","fromEmail":"barkalow@iabervon.org","sentAt":"2005-04-28T05:17:24Z","receivedAt":"2005-04-28T05:17:24Z","isPatch":false,"sender":{"key":"barkalow@iabervon.org","avatar":"https://avatars.githubusercontent.com/u/55364219?v=4"},"body":"This makes fsck-cache expect the root set of the reachability graph (i.e.,\nyour heads and tags) to be .git/refs/*/*, which contain hex versions of\nhashs of roots (like .get/HEAD has always been).\n\nSigned-Off-By: Daniel Barkalow <barkalow@iabervon.org>\nIndex: fsck-cache.c\n===================================================================\n--- ca8a271be1370ab0795a869c15114e566bdd15d8/fsck-cache.c  (mode:100644 sha1:280a104050b665515418c00c33af8e6b0b0e2101)\n+++ c53109e78c0b0d8925a4198fdec8295620f8f349/fsck-cache.c  (mode:100644 sha1:c4ff5bf3d71c35236e769d30035b00acf08d452b)\n@@ -135,6 +135,67 @@\n \treturn 0;\n }\n \n+int read_refs_dirs(char *path)\n+{\n+\tDIR *dir = opendir(path);\n+\tchar filename[NAME_MAX+1];\n+\tstruct dirent *de;\n+\tint total = 0;\n+\tchar *base;\n+\n+\tif (!dir) {\n+\t\terror(\"missing refs directory '%s'\", path);\n+\t\treturn 0;\n+\t}\n+\tstrcpy(filename, path);\n+\tbase = filename + strlen(path);\n+\t*(base++) = '/';\n+\twhile ((de = readdir(dir)) != NULL) {\n+\t\tDIR *subdir;\n+\t\tchar *subbase;\n+\t\tif (de->d_name[0] == '.')\n+\t\t\tcontinue;\n+\t\tstrcpy(base, de->d_name);\n+\t\tsubbase = base + strlen(de->d_name);\n+\t\tsubdir = opendir(filename);\n+\t\tif (!subdir) {\n+\t\t\tfprintf(stderr, \"can't open subdir %s\\n\", filename);\n+\t\t\tcontinue;\n+\t\t}\n+\t\t*(subbase++) = '/';\n+\t\twhile ((de = readdir(subdir)) != NULL) {\n+\t\t\tchar hex[41];\n+\t\t\tchar sha1[20];\n+\t\t\tint fd;\n+\t\t\tstruct object *obj;\n+\t\t\tif (de->d_name[0] == '.')\n+\t\t\t\tcontinue;\n+\t\t\tstrcpy(subbase, de->d_name);\n+\t\t\tfd = open(filename, O_RDONLY);\n+\t\t\tif (fd < 0) {\n+\t\t\t\tfprintf(stderr, \"Couldn't open %s\\n\", filename);\n+\t\t\t\tcontinue;\n+\t\t\t}\n+\t\t\tif ((read(fd, hex, 41) < 41) ||\n+\t\t\t    (hex[40] != '\\n') ||\n+\t\t\t    get_sha1_hex(hex, sha1)) {\n+\t\t\t\tfprintf(stderr, \"Couldn't read a hash from %s\\n\",\n+\t\t\t\t\tfilename);\n+\t\t\t\tcontinue;\n+\t\t\t}\n+\t\t\tobj = parse_object(sha1);\n+\t\t\tobj->used = 1;\n+\t\t\tmark_reachable(obj, REACHABLE);\n+\t\t\t\n+\t\t\ttotal++;\n+\t\t}\n+\t\tclosedir(subdir);\n+\t}\n+\tclosedir(dir);\n+\n+\treturn total;\n+}\n+\n int main(int argc, char **argv)\n {\n \tint i, heads;\n@@ -183,6 +244,8 @@\n \t\terror(\"expected sha1, got %s\", arg);\n \t}\n \n+\theads += read_refs_dirs(\".git/refs\");\n+\n \tif (!heads) {\n \t\tif (show_unreachable) {\n \t\t\tfprintf(stderr, \"unable to do reachability without a head\\n\");\n\n"},{"id":"2000","messageId":"Pine.LNX.4.58.0504272217240.18901@ppc970.osdl.org","threadId":"370","inReplyTo":"Pine.LNX.4.21.0504280100360.30848-100000@iabervon.org","subject":"Re: [1/5] Consider a blob to be parsed","fromName":"Linus Torvalds","fromEmail":"torvalds@osdl.org","sentAt":"2005-04-28T05:28:05Z","receivedAt":"2005-04-28T05:28:05Z","isPatch":false,"sender":{"key":"torvalds@linux-foundation.org","avatar":"https://avatars.githubusercontent.com/u/1024025?v=4"},"body":"\n\nOn Thu, 28 Apr 2005, Daniel Barkalow wrote:\n>\n> We don't parse blobs at all, so any that we've got are as parsed as\n> they're going to get. Don't make fsck-cache mark them.\n\nNO NO NO!\n\nThis is WRONG, dammit. I fixed it once, you are re-introducing the same\nbug.\n\nDaniel, the problem is that you parse them only when you SEE them, and \nthat is totally different from having seen a REFERENCE to them. One says \n\"I've seen this object\", the other says \"I _want_ to see this object\". \nThey are two totally different things.\n\nYou now mark all \"blob\" objects parsed regardless of whether you have \nactually seen the blob or not. Ie you mark a blob parsed just from having \nseen a _reference_ to it, and fsck can never know whether it actually \nreally saw the object or not.\n\nThis is the commit that already fixed this bug once, and that you are now \nre-introducing:\n\n\tcommit 4728b861ace127dc39c648f3bea64c3b86bbabc5\n\ttree 242227fc3c3a74d070ed36496e790335dd00c44a\n\tparent da6abf5d9c342a74dffbcc2015b9c27d7819a900\n\tauthor Linus Torvalds <torvalds@ppc970.osdl.org> Sun, 24 Apr 2005 14:10:55 -0700\n\tcommitter Linus Torvalds <torvalds@ppc970.osdl.org> Sun, 24 Apr 2005 14:10:55 -0700\n\t\n\t    fsck-cache: notice missing \"blob\" objects.\n\t    \n\t    We should _not_ mark a blob object \"parsed\" just because we\n\t    looked it up: it gets marked that way only once we've actually\n\t    seen it. Otherwise we can never notice a missing blob.\n\nplease think about it.\n\nTry to make some test-cases for fsck. They are quite easy to make: copy a \ngood directory, and \n - remove one commit (in the middle)\n - remove at least on tree\n - remove at least one blob\n - corrupt a file obviously (make it not uncompress ok)\n - corrupt a file in a subtle way (make it uncompress ok and have the\n   right signature, but be the wrong type - for example, make a \"commit\" \n   object that points to a \"tree\" object that actually is a \"blob\")\n\nAnd you'll see how this \"consider a blob parsed\" totally destroys fsck's \nability to notice that the blob doesn't even _exist_ any more (case 3 \nabove).\n\n\"parsing\" and \"looking up\" are two totally independent operations. They\nare independent for commits and trees, and they are independent for blobs.  \n\nTo mark a blob parsed, you _need_ to have actually looked it up and\nverified that it exists and that the object header is valid (and if you're \nfsck, that the sha1 matches). You MUST NOT do it in \"lookup_blob()\".\n\n\t\tLinus\n"},{"id":"2004","messageId":"Pine.LNX.4.21.0504280131400.30848-100000@iabervon.org","threadId":"370","inReplyTo":"Pine.LNX.4.58.0504272217240.18901@ppc970.osdl.org","subject":"Re: [1/5] Consider a blob to be parsed","fromName":"Daniel Barkalow","fromEmail":"barkalow@iabervon.org","sentAt":"2005-04-28T05:49:07Z","receivedAt":"2005-04-28T05:49:07Z","isPatch":false,"sender":{"key":"barkalow@iabervon.org","avatar":"https://avatars.githubusercontent.com/u/55364219?v=4"},"body":"On Wed, 27 Apr 2005, Linus Torvalds wrote:\n\n> \n> \n> On Thu, 28 Apr 2005, Daniel Barkalow wrote:\n> >\n> > We don't parse blobs at all, so any that we've got are as parsed as\n> > they're going to get. Don't make fsck-cache mark them.\n> \n> NO NO NO!\n> \n> This is WRONG, dammit. I fixed it once, you are re-introducing the same\n> bug.\n\n\n> Daniel, the problem is that you parse them only when you SEE them, and \n> that is totally different from having seen a REFERENCE to them. One says \n> \"I've seen this object\", the other says \"I _want_ to see this object\". \n> They are two totally different things.\n\nGood point; the right fix is actually to have a parse_blob that checks to\nmake sure it's there, and not have a special case for the simple case.\n\nReplacement [1/5] to follow...\n\n\t-Daniel\n*This .sig left intentionally blank*\n\n"},{"id":"2005","messageId":"Pine.LNX.4.21.0504280155320.30848-100000@iabervon.org","threadId":"370","inReplyTo":"Pine.LNX.4.21.0504280131400.30848-100000@iabervon.org","subject":"[1'/5] Mark blobs as parsed when they're actually parsed","fromName":"Daniel Barkalow","fromEmail":"barkalow@iabervon.org","sentAt":"2005-04-28T05:58:08Z","receivedAt":"2005-04-28T05:58:08Z","isPatch":false,"sender":{"key":"barkalow@iabervon.org","avatar":"https://avatars.githubusercontent.com/u/55364219?v=4"},"body":"This eliminates the special case for blobs versus other types of\nobjects. Now the scheme is entirely regular and I won't introduce stupid\nbugs. (And fsck-cache doesn't have to do the do-nothing parse)\n\nSigned-Off-By: Daniel Barkalow <barkalow@iabervon.org>\nIndex: blob.c\n===================================================================\n--- 89fdfd09b281fdf5071bc13a30ef683bd6851b61/blob.c  (mode:100644 sha1:3d99b93f020d84c5410c2b1056f2d7446b647d1e)\n+++ bcb43fecd82fa6f851e266b7e3873973068d51f3/blob.c  (mode:100644 sha1:d4af4a309433744d2fe819886d66741ab016f62b)\n@@ -14,10 +14,28 @@\n \t\tret->object.type = blob_type;\n \t\treturn ret;\n \t}\n-\tif (obj->parsed && obj->type != blob_type) {\n+\tif (obj->type != blob_type) {\n \t\terror(\"Object %s is a %s, not a blob\", \n \t\t      sha1_to_hex(sha1), obj->type);\n \t\treturn NULL;\n \t}\n \treturn (struct blob *) obj;\n }\n+\n+int parse_blob(struct blob *item)\n+{\n+        char type[20];\n+        void *buffer;\n+        unsigned long size;\n+        if (item->object.parsed)\n+                return 0;\n+        item->object.parsed = 1;\n+        buffer = read_sha1_file(item->object.sha1, type, &size);\n+        if (!buffer)\n+                return error(\"Could not read %s\",\n+                             sha1_to_hex(item->object.sha1));\n+        if (strcmp(type, blob_type))\n+                return error(\"Object %s not a blob\",\n+                             sha1_to_hex(item->object.sha1));\n+\treturn 0;\n+}\nIndex: blob.h\n===================================================================\n--- 89fdfd09b281fdf5071bc13a30ef683bd6851b61/blob.h  (mode:100644 sha1:5cbf6d65ee88e1e0c0f1153af4aa7f80c3c48c16)\n+++ bcb43fecd82fa6f851e266b7e3873973068d51f3/blob.h  (mode:100644 sha1:4afad0f067e1979d7ccc4778e254cce6962ad136)\n@@ -11,4 +11,6 @@\n \n struct blob *lookup_blob(unsigned char *sha1);\n \n+int parse_blob(struct blob *item);\n+\n #endif /* BLOB_H */\n\n"},{"id":"2006","messageId":"Pine.LNX.4.21.0504280158230.30848-100000@iabervon.org","threadId":"370","inReplyTo":"Pine.LNX.4.21.0504280106540.30848-100000@iabervon.org","subject":"[3'/5] Add function to parse an object of unspecified type (take 2)","fromName":"Daniel Barkalow","fromEmail":"barkalow@iabervon.org","sentAt":"2005-04-28T06:01:56Z","receivedAt":"2005-04-28T06:01:56Z","isPatch":false,"sender":{"key":"barkalow@iabervon.org","avatar":"https://avatars.githubusercontent.com/u/55364219?v=4"},"body":"This adds a function that parses an object from the database when we have\nto look up its actual type. It also checks the hash of the file, due to\nits heritage as part of fsck-cache.\n\nSigned-Off-By: Daniel Barkalow <barkalow@iabervon.org>\nIndex: object.c\n===================================================================\n--- bcb43fecd82fa6f851e266b7e3873973068d51f3/object.c  (mode:100644 sha1:91bbc6e5e2eadfb0a66b14d992eac260d07267f8)\n+++ 52505514b78c9bd77fcd701663c1967919d1cec3/object.c  (mode:100644 sha1:ca4af8fa2dc0672b92310a3ebdd4d14bf070dd69)\n@@ -1,5 +1,9 @@\n #include \"object.h\"\n+#include \"blob.h\"\n+#include \"tree.h\"\n+#include \"commit.h\"\n #include \"cache.h\"\n+#include \"tag.h\"\n #include <stdlib.h>\n #include <string.h>\n \n@@ -94,3 +98,39 @@\n \t\tp = p->next;\n \t}\n }\n+\n+struct object *parse_object(unsigned char *sha1)\n+{\n+\tunsigned long mapsize;\n+\tvoid *map = map_sha1_file(sha1, &mapsize);\n+\tif (map) {\n+\t\tchar type[100];\n+\t\tunsigned long size;\n+\t\tvoid *buffer = unpack_sha1_file(map, mapsize, type, &size);\n+\t\tif (!buffer)\n+\t\t\treturn NULL;\n+\t\tif (check_sha1_signature(sha1, buffer, size, type) < 0)\n+\t\t\tprintf(\"sha1 mismatch %s\\n\", sha1_to_hex(sha1));\n+\t\tmunmap(map, mapsize);\n+\t\tif (!strcmp(type, \"blob\")) {\n+\t\t\tstruct blob *ret = lookup_blob(sha1);\n+\t\t\tparse_blob(ret);\n+\t\t\treturn &ret->object;\n+\t\t} else if (!strcmp(type, \"tree\")) {\n+\t\t\tstruct tree *ret = lookup_tree(sha1);\n+\t\t\tparse_tree(ret);\n+\t\t\treturn &ret->object;\n+\t\t} else if (!strcmp(type, \"commit\")) {\n+\t\t\tstruct commit *ret = lookup_commit(sha1);\n+\t\t\tparse_commit(ret);\n+\t\t\treturn &ret->object;\n+\t\t} else if (!strcmp(type, \"tag\")) {\n+\t\t\tstruct tag *ret = lookup_tag(sha1);\n+\t\t\tparse_tag(ret);\n+\t\t\treturn &ret->object;\n+\t\t} else {\n+\t\t\treturn NULL;\n+\t\t}\n+\t}\n+\treturn NULL;\n+}\nIndex: object.h\n===================================================================\n--- bcb43fecd82fa6f851e266b7e3873973068d51f3/object.h  (mode:100644 sha1:bc607fd55f6ce4e56ce87766369b5d4d55ec79af)\n+++ 52505514b78c9bd77fcd701663c1967919d1cec3/object.h  (mode:100644 sha1:d53a35a4d7321b5ec970103208ac576f9f722dff)\n@@ -22,6 +22,9 @@\n \n void created_object(unsigned char *sha1, struct object *obj);\n \n+/** Returns the object, having parsed it to find out what it is. **/\n+struct object *parse_object(unsigned char *sha1);\n+\n void add_ref(struct object *refer, struct object *target);\n \n void mark_reachable(struct object *obj, unsigned int mask);\n\n"}]}