{"thread":{"id":"4769","subject":"Experimental \"git prune\"","startedAt":"2006-07-04T22:41:58Z","lastAt":"2006-07-05T16:24:45Z","messageCount":4,"participants":["Linus Torvalds","Junio C Hamano","Johannes Schindelin"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"23252","messageId":"Pine.LNX.4.64.0607041539550.12404@g5.osdl.org","threadId":"4769","inReplyTo":null,"subject":"Experimental \"git prune\"","fromName":"Linus Torvalds","fromEmail":"torvalds@osdl.org","sentAt":"2006-07-04T22:41:58Z","receivedAt":"2006-07-04T22:41:58Z","isPatch":false,"sender":{"key":"torvalds@linux-foundation.org","avatar":"https://avatars.githubusercontent.com/u/1024025?v=4"},"body":"\nThis is an example of how \"git prune\" could be speeded up a lot.\n\nThis needs the builtin magic to actually enable it, and the \"printf\" needs \nto be made a real unlink, but as far as I can tell it actually works.\n\nComments?\n\n\t\tLinus\n\n----\n/*\n * builtin-prune.c\n */\n#include \"cache.h\"\n#include \"refs.h\"\n#include \"tag.h\"\n#include \"commit.h\"\n#include \"tree.h\"\n#include \"blob.h\"\n#include \"tree-walk.h\"\n#include \"diff.h\"\n#include \"revision.h\"\n#include \"builtin.h\"\n#include \"cache-tree.h\"\n\nstatic struct rev_info revs;\n\nstatic int prune_object(char *path, const char *filename, const unsigned char *sha1)\n{\n\tprintf(\"%s/%s should be pruned\\n\", path, filename);\n\treturn 0;\n}\n\nstatic int prune_dir(int i, char *path)\n{\n\tDIR *dir = opendir(path);\n\tstruct dirent *de;\n\n\tif (!dir)\n\t\treturn 0;\n\n\twhile ((de = readdir(dir)) != NULL) {\n\t\tchar name[100];\n\t\tunsigned char sha1[20];\n\t\tint len = strlen(de->d_name);\n\n\t\tswitch (len) {\n\t\tcase 2:\n\t\t\tif (de->d_name[1] != '.')\n\t\t\t\tbreak;\n\t\tcase 1:\n\t\t\tif (de->d_name[0] != '.')\n\t\t\t\tbreak;\n\t\t\tcontinue;\n\t\tcase 38:\n\t\t\tsprintf(name, \"%02x\", i);\n\t\t\tmemcpy(name+2, de->d_name, len+1);\n\t\t\tif (get_sha1_hex(name, sha1) < 0)\n\t\t\t\tbreak;\n\n\t\t\t/*\n\t\t\t * Do we know about this object?\n\t\t\t * It must have been reachable\n\t\t\t */\n\t\t\tif (lookup_object(sha1))\n\t\t\t\tcontinue;\n\n\t\t\tprune_object(path, de->d_name, sha1);\n\t\t\tcontinue;\n\t\t}\n\t\tfprintf(stderr, \"bad sha1 file: %s/%s\\n\", path, de->d_name);\n\t}\n\tclosedir(dir);\n\treturn 0;\n}\n\nstatic void prune_object_dir(const char *path)\n{\n\tint i;\n\tfor (i = 0; i < 256; i++) {\n\t\tstatic char dir[4096];\n\t\tsprintf(dir, \"%s/%02x\", path, i);\n\t\tprune_dir(i, dir);\n\t}\n}\n\nstatic void process_blob(struct blob *blob,\n\t\t\t struct object_array *p,\n\t\t\t struct name_path *path,\n\t\t\t const char *name)\n{\n\tstruct object *obj = &blob->object;\n\n\tif (obj->flags & SEEN)\n\t\treturn;\n\tobj->flags |= SEEN;\n\t/* Nothing to do, really .. The blob lookup was the important part */\n}\n\nstatic void process_tree(struct tree *tree,\n\t\t\t struct object_array *p,\n\t\t\t struct name_path *path,\n\t\t\t const char *name)\n{\n\tstruct object *obj = &tree->object;\n\tstruct tree_desc desc;\n\tstruct name_entry entry;\n\tstruct name_path me;\n\n\tif (obj->flags & SEEN)\n\t\treturn;\n\tobj->flags |= SEEN;\n\tif (parse_tree(tree) < 0)\n\t\tdie(\"bad tree object %s\", sha1_to_hex(obj->sha1));\n\tname = strdup(name);\n\tadd_object(obj, p, path, name);\n\tme.up = path;\n\tme.elem = name;\n\tme.elem_len = strlen(name);\n\n\tdesc.buf = tree->buffer;\n\tdesc.size = tree->size;\n\n\twhile (tree_entry(&desc, &entry)) {\n\t\tif (S_ISDIR(entry.mode))\n\t\t\tprocess_tree(lookup_tree(entry.sha1), p, &me, entry.path);\n\t\telse\n\t\t\tprocess_blob(lookup_blob(entry.sha1), p, &me, entry.path);\n\t}\n\tfree(tree->buffer);\n\ttree->buffer = NULL;\n}\n\nstatic void process_tag(struct tag *tag, struct object_array *p, const char *name)\n{\n\tstruct object *obj = &tag->object;\n\tstruct name_path me;\n\n\tif (obj->flags & SEEN)\n\t\treturn;\n\tobj->flags |= SEEN;\n\n\tme.up = NULL;\n\tme.elem = \"tag:/\";\n\tme.elem_len = 5;\n\n\tif (parse_tag(tag) < 0)\n\t\tdie(\"bad tag object %s\", sha1_to_hex(obj->sha1));\n\tadd_object(tag->tagged, p, NULL, name);\n}\n\nstatic void walk_commit_list(struct rev_info *revs)\n{\n\tint i;\n\tstruct commit *commit;\n\tstruct object_array objects = { 0, 0, NULL };\n\n\t/* Walk all commits, process their trees */\n\twhile ((commit = get_revision(revs)) != NULL)\n\t\tprocess_tree(commit->tree, &objects, NULL, \"\");\n\n\t/* Then walk all the pending objects, recursively processing them too */\n\tfor (i = 0; i < revs->pending.nr; i++) {\n\t\tstruct object_array_entry *pending = revs->pending.objects + i;\n\t\tstruct object *obj = pending->item;\n\t\tconst char *name = pending->name;\n\t\tif (obj->type == TYPE_TAG) {\n\t\t\tprocess_tag((struct tag *) obj, &objects, name);\n\t\t\tcontinue;\n\t\t}\n\t\tif (obj->type == TYPE_TREE) {\n\t\t\tprocess_tree((struct tree *)obj, &objects, NULL, name);\n\t\t\tcontinue;\n\t\t}\n\t\tif (obj->type == TYPE_BLOB) {\n\t\t\tprocess_blob((struct blob *)obj, &objects, NULL, name);\n\t\t\tcontinue;\n\t\t}\n\t\tdie(\"unknown pending object %s (%s)\", sha1_to_hex(obj->sha1), name);\n\t}\n}\n\nstatic int add_one_ref(const char *path, const unsigned char *sha1)\n{\n\tstruct object *object = parse_object(sha1);\n\tif (!object)\n\t\tdie(\"bad object ref: %s:%s\", path, sha1_to_hex(sha1));\n\tadd_pending_object(&revs, object, \"\");\n\treturn 0;        \n}\n\nstatic void add_one_tree(const unsigned char *sha1)\n{\n\tstruct tree *tree = lookup_tree(sha1);\n\tadd_pending_object(&revs, &tree->object, \"\");\n}\n\nstatic void add_cache_tree(struct cache_tree *it)\n{\n\tint i;\n\n\tif (it->entry_count >= 0)\n\t\tadd_one_tree(it->sha1);\n\tfor (i = 0; i < it->subtree_nr; i++)\n\t\tadd_cache_tree(it->down[i]->cache_tree);\n}\n\nstatic void add_cache_refs(void)\n{\n\tint i;\n\n \tread_cache();\n\tfor (i = 0; i < active_nr; i++) {\n\t\tlookup_blob(active_cache[i]->sha1);\n\t\t/*\n\t\t * We could add the blobs to the pending list, but quite\n\t\t * frankly, we don't care. Once we've looked them up, and\n\t\t * added them as objects, we've really done everything \n\t\t * there is to do for a blob\n\t\t */\n\t}\n\tif (active_cache_tree)\n\t\tadd_cache_tree(active_cache_tree);\n}\n\nint cmd_prune(int argc, const char **argv, char **envp)\n{\n\t/*\n\t * Set up revision parsing, and mark us as being interested\n\t * in all object types, not just commits.\n\t */\n\tinit_revisions(&revs);\n\trevs.tag_objects = 1;\n\trevs.blob_objects = 1;\n\trevs.tree_objects = 1;\n\n\t/* Add all external refs */\n\tfor_each_ref(add_one_ref);\n\n\t/* Add all refs from the index file */\n\tadd_cache_refs();\n\n\t/*\n\t * Set up the revision walk - this will move all commits\n\t * from the pending list to the commit walking list.\n\t */\n\tprepare_revision_walk(&revs);\n\n\twalk_commit_list(&revs);\n\n\tprune_object_dir(get_object_directory());\n\n\treturn 0;\n}\n"},{"id":"23255","messageId":"7vac7pf09e.fsf@assigned-by-dhcp.cox.net","threadId":"4769","inReplyTo":"Pine.LNX.4.64.0607041539550.12404@g5.osdl.org","subject":"Re: Experimental \"git prune\"","fromName":"Junio C Hamano","fromEmail":"junkio@cox.net","sentAt":"2006-07-04T22:55:41Z","receivedAt":"2006-07-04T22:55:41Z","isPatch":false,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Linus Torvalds <torvalds@osdl.org> writes:\n\n> This is an example of how \"git prune\" could be speeded up a lot.\n\nAh, you are avoiding the verification cost of fsck-objects.  Neat.\n"},{"id":"23270","messageId":"Pine.LNX.4.63.0607050956240.29667@wbgn013.biozentrum.uni-wuerzburg.de","threadId":"4769","inReplyTo":"Pine.LNX.4.64.0607041539550.12404@g5.osdl.org","subject":"Re: Experimental \"git prune\"","fromName":"Johannes Schindelin","fromEmail":"johannes.schindelin@gmx.de","sentAt":"2006-07-05T07:57:57Z","receivedAt":"2006-07-05T07:57:57Z","isPatch":false,"sender":{"key":"johannes.schindelin@gmx.de","avatar":"https://avatars.githubusercontent.com/u/127790?v=4"},"body":"Hi,\n\nOn Tue, 4 Jul 2006, Linus Torvalds wrote:\n\n> static void process_tag(struct tag *tag, struct object_array *p, const char *name)\n> {\n> \tstruct object *obj = &tag->object;\n> \tstruct name_path me;\n\nAFAICS this variable is set, but not used:\n\n> \n> \tif (obj->flags & SEEN)\n> \t\treturn;\n> \tobj->flags |= SEEN;\n> \n> \tme.up = NULL;\n> \tme.elem = \"tag:/\";\n> \tme.elem_len = 5;\n> \n> \tif (parse_tag(tag) < 0)\n> \t\tdie(\"bad tag object %s\", sha1_to_hex(obj->sha1));\n> \tadd_object(tag->tagged, p, NULL, name);\n> }\n\nAs for the rest, I like it.\n\nCiao,\nDscho\n"},{"id":"23281","messageId":"Pine.LNX.4.64.0607050923360.12404@g5.osdl.org","threadId":"4769","inReplyTo":"Pine.LNX.4.63.0607050956240.29667@wbgn013.biozentrum.uni-wuerzburg.de","subject":"Re: Experimental \"git prune\"","fromName":"Linus Torvalds","fromEmail":"torvalds@osdl.org","sentAt":"2006-07-05T16:24:45Z","receivedAt":"2006-07-05T16:24:45Z","isPatch":false,"sender":{"key":"torvalds@linux-foundation.org","avatar":"https://avatars.githubusercontent.com/u/1024025?v=4"},"body":"\n\nOn Wed, 5 Jul 2006, Johannes Schindelin wrote:\n\n> Hi,\n> \n> On Tue, 4 Jul 2006, Linus Torvalds wrote:\n> \n> > static void process_tag(struct tag *tag, struct object_array *p, const char *name)\n> > {\n> > \tstruct object *obj = &tag->object;\n> > \tstruct name_path me;\n> \n> AFAICS this variable is set, but not used:\n> \n> > \n> > \tif (obj->flags & SEEN)\n> > \t\treturn;\n> > \tobj->flags |= SEEN;\n> > \n> > \tme.up = NULL;\n> > \tme.elem = \"tag:/\";\n> > \tme.elem_len = 5;\n> > \n> > \tif (parse_tag(tag) < 0)\n> > \t\tdie(\"bad tag object %s\", sha1_to_hex(obj->sha1));\n> > \tadd_object(tag->tagged, p, NULL, name);\n\nYeah, that NULL should be \"&me\".\n\nThat said, we don't actually use the object name logic at all, so we could \ndrop it entirely. Having it _might_ allow some of the error messages to be \nbetter, who knows..\n\n\t\t\tLinus\n"}]}