{"thread":{"id":"8928","subject":"[PATCH 2/4] export add_ref_decoration","startedAt":"2007-07-08T16:23:23Z","lastAt":"2007-07-09T14:59:48Z","messageCount":25,"participants":["skimo@liacs.nl","Johannes Schindelin","Sven Verdoolaege","Steven Grimm","Johannes Sixt","Jeff King"],"isPatch":true,"patchVersion":1,"patchTotal":4},"messages":[{"id":"46763","messageId":"11839118073186-git-send-email-skimo@liacs.nl","threadId":"8928","inReplyTo":null,"subject":"[PATCH 0/4] Add git-rewrite-commits","fromName":"","fromEmail":"skimo@liacs.nl","sentAt":"2007-07-08T16:23:23Z","receivedAt":"2007-07-08T16:23:23Z","isPatch":true,"sender":{"key":"skimo@kotnet.org","avatar":null},"body":"From: Sven Verdoolaege \n\nThis patch series adds git-rewrite-commits\nand depends (for the tests in the fourth part)\non my earlier patch to allow negative matches\n(or an extended patch by Johannes).\n\n[PATCH 1/4] export get_short_sha1\n[PATCH 2/4] export add_ref_decoration\n[PATCH 3/4] revision: mark commits that didn't match a pattern for later use\n[PATCH 4/4] Add git-rewrite-commits\n\nThe first two should be fairly uncontroversial.\nThe third may be considered a waste of a precious bit.\nIf so, any suggestions for other ways of passing on this information\nare welcomed.\nThe fourth contains the actual git-rewrite-commits builtin.\nMy main motivation was that cg-admin-rewritehist doesn't\nchange the SHA1's in commit message and I don't like shell\nprogramming.\n\nskimo\n"},{"id":"46766","messageId":"11839118081240-git-send-email-skimo@liacs.nl","threadId":"8928","inReplyTo":"11839118073186-git-send-email-skimo@liacs.nl","subject":"[PATCH 1/4] export get_short_sha1","fromName":"","fromEmail":"skimo@liacs.nl","sentAt":"2007-07-08T16:23:24Z","receivedAt":"2007-07-08T16:23:24Z","isPatch":true,"sender":{"key":"skimo@kotnet.org","avatar":null},"body":"From: Sven Verdoolaege \n\nSometimes it is useful to check whether a given string\nis exactly a short SHA1.\n\nSigned-off-by: Sven Verdoolaege \n---\n cache.h | 1 +\n sha1_name.c | 3 +--\n 2 files changed, 2 insertions(+), 2 deletions(-)\n\ndiff --git a/cache.h b/cache.h\nindex e64071e..8fda8ee 100644\n--- a/cache.h\n+++ b/cache.h\n@@ -391,6 +391,7 @@ static inline unsigned int hexval(unsigned char c)\n \n extern int get_sha1(const char *str, unsigned char *sha1);\n extern int get_sha1_with_mode(const char *str, unsigned char *sha1, unsigned *mode);\n+extern int get_short_sha1(const char *name, int len, unsigned char *sha1, int quietly);\n extern int get_sha1_hex(const char *hex, unsigned char *sha1);\n extern char *sha1_to_hex(const unsigned char *sha1);\t/* static buffer result! */\n extern int read_ref(const char *filename, unsigned char *sha1);\ndiff --git a/sha1_name.c b/sha1_name.c\nindex 858f08c..0bed79d 100644\n--- a/sha1_name.c\n+++ b/sha1_name.c\n@@ -154,8 +154,7 @@ static int find_unique_short_object(int len, char *canonical,\n \treturn 0;\n }\n \n-static int get_short_sha1(const char *name, int len, unsigned char *sha1,\n-\t\t\t int quietly)\n+int get_short_sha1(const char *name, int len, unsigned char *sha1, int quietly)\n {\n \tint i, status;\n \tchar canonical[40];\n-- \n1.5.3.rc0.68.geec71-dirty\n"},{"id":"46762","messageId":"11839118083737-git-send-email-skimo@liacs.nl","threadId":"8928","inReplyTo":"11839118073186-git-send-email-skimo@liacs.nl","subject":"[PATCH 2/4] export add_ref_decoration","fromName":"","fromEmail":"skimo@liacs.nl","sentAt":"2007-07-08T16:23:25Z","receivedAt":"2007-07-08T16:23:25Z","isPatch":true,"sender":{"key":"skimo@kotnet.org","avatar":null},"body":"From: Sven Verdoolaege \n\nadd_ref_decoration is also useful outside of git-log.\nSince the name_decoration declaration appears in commit.h,\nthe function is moved to commit.c.\n\nSigned-off-by: Sven Verdoolaege \n---\n builtin-log.c | 25 -------------------------\n commit.c | 27 +++++++++++++++++++++++++++\n commit.h | 3 +++\n log-tree.c | 2 --\n 4 files changed, 30 insertions(+), 27 deletions(-)\n\ndiff --git a/builtin-log.c b/builtin-log.c\nindex 13bae31..c14eea5 100644\n--- a/builtin-log.c\n+++ b/builtin-log.c\n@@ -21,31 +21,6 @@ static const char *fmt_patch_subject_prefix = \"PATCH\";\n /* this is in builtin-diff.c */\n void add_head(struct rev_info *revs);\n \n-static void add_name_decoration(const char *prefix, const char *name, struct object *obj)\n-{\n-\tint plen = strlen(prefix);\n-\tint nlen = strlen(name);\n-\tstruct name_decoration *res = xmalloc(sizeof(struct name_decoration) + plen + nlen);\n-\tmemcpy(res->name, prefix, plen);\n-\tmemcpy(res->name + plen, name, nlen + 1);\n-\tres->next = add_decoration(&name_decoration, obj, res);\n-}\n-\n-static int add_ref_decoration(const char *refname, const unsigned char *sha1, int flags, void *cb_data)\n-{\n-\tstruct object *obj = parse_object(sha1);\n-\tif (!obj)\n-\t\treturn 0;\n-\tadd_name_decoration(\"\", refname, obj);\n-\twhile (obj->type == OBJ_TAG) {\n-\t\tobj = ((struct tag *)obj)->tagged;\n-\t\tif (!obj)\n-\t\t\tbreak;\n-\t\tadd_name_decoration(\"tag: \", refname, obj);\n-\t}\n-\treturn 0;\n-}\n-\n static void cmd_log_init(int argc, const char **argv, const char *prefix,\n \t\t struct rev_info *rev)\n {\ndiff --git a/commit.c b/commit.c\nindex 03436b1..24d7dd4 100644\n--- a/commit.c\n+++ b/commit.c\n@@ -1553,3 +1553,30 @@ int in_merge_bases(struct commit *commit, struct commit **reference, int num)\n \tfree_commit_list(bases);\n \treturn ret;\n }\n+\n+struct decoration name_decoration = { \"object names\" };\n+\n+static void add_name_decoration(const char *prefix, const char *name, struct object *obj)\n+{\n+\tint plen = strlen(prefix);\n+\tint nlen = strlen(name);\n+\tstruct name_decoration *res = xmalloc(sizeof(struct name_decoration) + plen + nlen);\n+\tmemcpy(res->name, prefix, plen);\n+\tmemcpy(res->name + plen, name, nlen + 1);\n+\tres->next = add_decoration(&name_decoration, obj, res);\n+}\n+\n+int add_ref_decoration(const char *refname, const unsigned char *sha1, int flags, void *cb_data)\n+{\n+\tstruct object *obj = parse_object(sha1);\n+\tif (!obj)\n+\t\treturn 0;\n+\tadd_name_decoration(\"\", refname, obj);\n+\twhile (obj->type == OBJ_TAG) {\n+\t\tobj = ((struct tag *)obj)->tagged;\n+\t\tif (!obj)\n+\t\t\tbreak;\n+\t\tadd_name_decoration(\"tag: \", refname, obj);\n+\t}\n+\treturn 0;\n+}\ndiff --git a/commit.h b/commit.h\nindex 467872e..bf23535 100644\n--- a/commit.h\n+++ b/commit.h\n@@ -122,4 +122,7 @@ extern struct commit_list *get_shallow_commits(struct object_array *heads,\n \t\tint depth, int shallow_flag, int not_shallow_flag);\n \n int in_merge_bases(struct commit *, struct commit **, int);\n+\n+extern int add_ref_decoration(const char *refname, const unsigned char *sha1, int flags, void *cb_data);\n+\n #endif /* COMMIT_H */\ndiff --git a/log-tree.c b/log-tree.c\nindex 8624d5a..b69f029 100644\n--- a/log-tree.c\n+++ b/log-tree.c\n@@ -4,8 +4,6 @@\n #include \"log-tree.h\"\n #include \"reflog-walk.h\"\n \n-struct decoration name_decoration = { \"object names\" };\n-\n static void show_parents(struct commit *commit, int abbrev)\n {\n \tstruct commit_list *p;\n-- \n1.5.3.rc0.68.geec71-dirty\n"},{"id":"46765","messageId":"1183911808110-git-send-email-skimo@liacs.nl","threadId":"8928","inReplyTo":"11839118073186-git-send-email-skimo@liacs.nl","subject":"[PATCH 3/4] revision: mark commits that didn't match a pattern for later use","fromName":"","fromEmail":"skimo@liacs.nl","sentAt":"2007-07-08T16:23:26Z","receivedAt":"2007-07-08T16:23:26Z","isPatch":true,"sender":{"key":"skimo@kotnet.org","avatar":null},"body":"From: Sven Verdoolaege \n\nSigned-off-by: Sven Verdoolaege \n---\n revision.c | 4 +++-\n revision.h | 1 +\n 2 files changed, 4 insertions(+), 1 deletions(-)\n\ndiff --git a/revision.c b/revision.c\nindex 0035d40..8d3bed2 100644\n--- a/revision.c\n+++ b/revision.c\n@@ -1421,8 +1421,10 @@ static struct commit *get_revision_1(struct rev_info *revs)\n \t\tif (revs->no_merges &&\n \t\t commit->parents && commit->parents->next)\n \t\t\tcontinue;\n-\t\tif (!commit_match(commit, revs))\n+\t\tif (!commit_match(commit, revs)) {\n+\t\t\tcommit->object.flags |= PRUNED;\n \t\t\tcontinue;\n+\t\t}\n \t\tif (revs->prune_fn && revs->dense) {\n \t\t\t/* Commit without changes? */\n \t\t\tif (!(commit->object.flags & TREECHANGE)) {\ndiff --git a/revision.h b/revision.h\nindex 9728d4c..d3609ab 100644\n--- a/revision.h\n+++ b/revision.h\n@@ -10,6 +10,7 @@\n #define CHILD_SHOWN\t(1u<<6)\n #define ADDED\t\t(1u<<7)\t/* Parents already parsed and added? */\n #define SYMMETRIC_LEFT\t(1u<<8)\n+#define PRUNED\t\t(1u<<9)\n \n struct rev_info;\n struct log_info;\n-- \n1.5.3.rc0.68.geec71-dirty\n"},{"id":"46764","messageId":"1183911808787-git-send-email-skimo@liacs.nl","threadId":"8928","inReplyTo":"11839118073186-git-send-email-skimo@liacs.nl","subject":"[PATCH 4/4] Add git-rewrite-commits","fromName":"","fromEmail":"skimo@liacs.nl","sentAt":"2007-07-08T16:23:27Z","receivedAt":"2007-07-08T16:23:27Z","isPatch":true,"sender":{"key":"skimo@kotnet.org","avatar":null},"body":"From: Sven Verdoolaege \n\nThis builtin is similar to git-filter-branch (and cg-admin-rewritehist).\nThe main difference is that git-rewrite-commits will automatically\nrewrite any SHA1 in a commit message to the rewritten SHA1 as\nwell as any reference (in .git/refs/) that points to a rewritten commit.\n\nIt's also a lot faster than git-filter-branch if no external command\nis called. For example, running either to eliminating a commit specified\nas a graft results in the following timings, both being performed\non a freshly cloned copy of a small repo:\n\n\tbash-3.00$ time git-filter-branch test\n\tRewrite 274fe3dfb8e8c7d0a6ce05138bdb650de7b459ea (425/425)\n\tRewritten history saved to the test branch\n\n\treal 0m30.845s\n\tuser 0m13.400s\n\tsys 0m19.640s\n\n\tbash-3.00$ time git-rewrite-commits\n\n\treal 0m0.223s\n\tuser 0m0.080s\n\tsys 0m0.140s\n\nThe command line is more reminiscent of git-log.\nFor example you can say\n\n\tgit-rewrite-commits --all\n\nto incorporate grafts in all branches, or\n\n\tgit rewrite-commits --author='!Darl McBribe' --all\n\nto remove all commits by Darl McBribe.\n\nSigned-off-by: Sven Verdoolaege \n---\n .gitignore | 1 +\n Documentation/cmd-list.perl | 1 +\n Documentation/git-rewrite-commits.txt | 147 +++++++++++\n Makefile | 1 +\n builtin-rewrite-commits.c | 451 +++++++++++++++++++++++++++++++++\n builtin.h | 1 +\n git.c | 1 +\n t/t7005-rewrite-commits.sh | 74 ++++++\n 8 files changed, 677 insertions(+), 0 deletions(-)\n create mode 100644 Documentation/git-rewrite-commits.txt\n create mode 100644 builtin-rewrite-commits.c\n create mode 100755 t/t7005-rewrite-commits.sh\n\ndiff --git a/.gitignore b/.gitignore\nindex 20ee642..bcd95a9 100644\n--- a/.gitignore\n+++ b/.gitignore\n@@ -109,6 +109,7 @@ git-reset\n git-rev-list\n git-rev-parse\n git-revert\n+git-rewrite-commits\n git-rm\n git-runstatus\n git-send-email\ndiff --git a/Documentation/cmd-list.perl b/Documentation/cmd-list.perl\nindex 2143995..63911b7 100755\n--- a/Documentation/cmd-list.perl\n+++ b/Documentation/cmd-list.perl\n@@ -166,6 +166,7 @@ git-reset mainporcelain\n git-revert mainporcelain\n git-rev-list plumbinginterrogators\n git-rev-parse ancillaryinterrogators\n+git-rewrite-commits mainporcelain\n git-rm mainporcelain\n git-runstatus ancillaryinterrogators\n git-send-email foreignscminterface\ndiff --git a/Documentation/git-rewrite-commits.txt b/Documentation/git-rewrite-commits.txt\nnew file mode 100644\nindex 0000000..d82b777\n--- /dev/null\n+++ b/Documentation/git-rewrite-commits.txt\n@@ -0,0 +1,147 @@\n+git-rewrite-commits(1)\n+======================\n+\n+NAME\n+----\n+git-rewrite-commits - Rewrite commits\n+\n+SYNOPSIS\n+--------\n+'git-rewrite-commits' [--index-map ] [--commit-map ]\n+\t[...]\n+\n+DESCRIPTION\n+-----------\n+Lets you rewrite the commits selected by the gitlink:git-rev-list[1]\n+options, optionally applying custom maps on each of them.\n+These maps either modify the tree associated to a commit\n+(through manipulations of the index) or the (raw) commit itself.\n+Any commit within the specified range that is filtered out\n+through a pattern is removed from its children (assuming they\n+are in range) and replaced by the closest ancestors that\n+are either not being rewritten or not filtered out.\n+\n+Any branch pointing to any of the rewritten commits is replaced\n+by a pointer to the new commit. The original pointers are saved\n+in the refs/rewritten hierarchy. Any SHA1 in the commit message\n+of any of the rewritten commits that points to (another) rewritten\n+commit is replaced by the SHA1 of the corresponding new commit.\n+\n+Besides the actions specified by the maps, the new commits\n+will also reflect any grafts that may apply to any of the selected commits.\n+\n+*WARNING*! The rewritten history will have different object names for all\n+the objects and will not converge with the original branch. You will not\n+be able to easily push and distribute the rewritten branch on top of the\n+original branch. Please do not use this command if you do not know the\n+full implications, and avoid using it anyway, if a simple single commit\n+would suffice to fix your problem.\n+\n+Maps\n+~~~~\n+\n+The maps are applied in the order as listed below. The \n+argument is run as \"sh -c '\".\n+\n+\n+OPTIONS\n+-------\n+\n+--index-map ::\n+\tThis map should only modify the index specified by 'GIT_INDEX_FILE'.\n+\tPrior to running the map, this temporary index is populated\n+\twith the tree of the commit that is being rewritten.\n+\tThis tree will be replaced according to the new state of this\n+\ttemporary index after the map finishes.\n+\n+--commit-map ::\n+\tThis map receives the (raw) commit on stdin and should produce\n+\tthe (possibly) modified commit on stdout. In the input commit,\n+\tthe tree has already been modified by the index map (if any)\n+\tand has all references to older commits (including the parents)\n+\tchanged to the possibly rewritten commits.\n+\n+::\n+\tSelects the commits to be rewritten, defaulting to the history\n+\tthat lead to HEAD. If commits are filtered using a (negative)\n+\tpattern then all the commits filtered out will be removed\n+\tfrom the history of the selected commits.\n+\n+\n+Examples\n+--------\n+\n+Suppose you want to remove a file (containing confidential information\n+or copyright violation) from all commits:\n+\n+--------------------------------------------------------------------\n+git rewrite-commits --index-map 'git update-index --remove filename'\n+--------------------------------------------------------------------\n+\n+Now, you will get the rewritten history saved in your current branch\n+(the old branch is saved in refs/rewritten).\n+\n+To set a commit \"$graft-id\" (which typically is at the tip of another\n+history) to be the parent of the current initial commit \"$commit-id\", in\n+order to paste the other history behind the current history:\n+\n+-----------------------------------------------\n+echo \"$commit-id $graft-id\" >> .git/info/grafts\n+git rewrite-commits\n+-----------------------------------------------\n+\n+To remove commits authored by \"Darl McBribe\" from the history:\n+\n+--------------------------------------------\n+git rewrite-commits --author='!Darl McBribe'\n+--------------------------------------------\n+\n+Note that the changes introduced by the commits, and not reverted by\n+subsequent commits, will still be in the rewritten branch. If you want\n+to throw out _changes_ together with the commits, you should use the\n+interactive mode of gitlink:git-rebase[1].\n+\n+Consider this history:\n+\n+------------------\n+ D--E--F--G--H\n+ / /\n+A--B-----C\n+------------------\n+\n+To rewrite only commits D,E,F,G,H, but leave A, B and C alone, use:\n+\n+--------------------------------\n+git rewrite-commits ... C..H\n+--------------------------------\n+\n+To rewrite commits E,F,G,H, use one of these:\n+\n+----------------------------------------\n+git rewrite-commits ... C..H --not D\n+git rewrite-commits ... D..H --not C\n+----------------------------------------\n+\n+To move the whole tree into a subdirectory, or remove it from there:\n+\n+---------------------------------------------------------------\n+git rewrite-commits --index-map \\\n+\t'git ls-files -s | sed \"s-\\t-&newsubdir/-\" |\n+\t\tGIT_INDEX_FILE=$GIT_INDEX_FILE.new \\\n+\t\t\tgit update-index --index-info &&\n+\t mv $GIT_INDEX_FILE.new $GIT_INDEX_FILE'\n+---------------------------------------------------------------\n+\n+\n+Author\n+------\n+Written by Sven Verdoolaege.\n+Inspired by cg-admin-rewritehist by Petr \"Pasky\" Baudis .\n+\n+Documentation\n+--------------\n+Documentation by Petr Baudis, Sven Verdoolaege and the git list.\n+\n+GIT\n+---\n+Part of the gitlink:git[7] suite\ndiff --git a/Makefile b/Makefile\nindex 5b30e5c..f84bcd7 100644\n--- a/Makefile\n+++ b/Makefile\n@@ -370,6 +370,7 @@ BUILTIN_OBJS = \\\n \tbuiltin-rev-list.o \\\n \tbuiltin-rev-parse.o \\\n \tbuiltin-revert.o \\\n+\tbuiltin-rewrite-commits.o \\\n \tbuiltin-rm.o \\\n \tbuiltin-runstatus.o \\\n \tbuiltin-shortlog.o \\\ndiff --git a/builtin-rewrite-commits.c b/builtin-rewrite-commits.c\nnew file mode 100644\nindex 0000000..eebf816\n--- /dev/null\n+++ b/builtin-rewrite-commits.c\n@@ -0,0 +1,451 @@\n+#include \"cache.h\"\n+#include \"refs.h\"\n+#include \"builtin.h\"\n+#include \"commit.h\"\n+#include \"diff.h\"\n+#include \"revision.h\"\n+#include \"list-objects.h\"\n+#include \"run-command.h\"\n+#include \"cache-tree.h\"\n+#include \"grep.h\"\n+\n+struct decoration rewrite_decoration = { \"rewritten as\" };\n+static const char *rewritten_prefix = \"rewritten\";\n+static const char *commit_map;\n+static const char *index_map;\n+\n+struct rewrite_decoration {\n+\tstruct rewrite_decoration *next;\n+\tunsigned char sha1[20];\n+};\n+\n+static void add_rewrite_decoration(struct object *obj, unsigned char *sha1)\n+{\n+\tstruct rewrite_decoration *deco = xmalloc(sizeof(struct rewrite_decoration));\n+\thashcpy(deco->sha1, sha1);\n+\tdeco->next = add_decoration(&rewrite_decoration, obj, deco);\n+}\n+\n+static int get_rewritten_sha1(unsigned char *sha1)\n+{\n+\tstruct rewrite_decoration *deco;\n+\tstruct object *obj = lookup_object(sha1);\n+\n+\tif (!obj)\n+\t\treturn 1;\n+\n+\tdeco = lookup_decoration(&rewrite_decoration, obj);\n+\tif (!deco)\n+\t\treturn 1;\n+\n+\thashcpy(sha1, deco->sha1);\n+\treturn 0;\n+}\n+\n+static char *add_parents(char *dest, struct commit_list *parents)\n+{\n+\tunsigned char sha1[20];\n+\tstruct commit_list *list;\n+\n+\tfor (list = parents; list; list = list->next) {\n+\t\thashcpy(sha1, list->item->object.sha1);\n+\t\tget_rewritten_sha1(sha1);\n+\t\tmemcpy(dest, \"parent \", 7);\n+\t\tdest += 7;\n+\t\tmemcpy(dest, sha1_to_hex(sha1), 40);\n+\t\tdest += 40;\n+\t\t*dest++ = '\\n';\n+\t}\n+\treturn dest;\n+}\n+\n+static int get_one_line(char *buf, unsigned long len)\n+{\n+\tchar *end = memchr(buf, '\\n', len);\n+\tif (end)\n+\t\treturn end - buf + 1;\n+\treturn len;\n+}\n+\n+static char *map_index(char *orig_hex)\n+{\n+\tint argc;\n+\tconst char *argv[10];\n+\tstatic char index_env[16+PATH_MAX];\n+\tchar *tmp_index = git_path(\"rewrite_index\");\n+\tconst char *env[] = { index_env, NULL };\n+\tchar *hex;\n+\n+\tmemcpy(index_env, \"GIT_INDEX_FILE=\", 15);\n+\tstrcpy(index_env+15, tmp_index);\n+\ttmp_index = index_env+15;\n+\n+\t/* First write out tree to temporary index */\n+\targc = 0;\n+\targv[argc++] = \"read-tree\";\n+\targv[argc++] = orig_hex;\n+\targv[argc] = NULL;\n+\tif (run_command_v_opt_cd_env(argv, RUN_GIT_CMD, NULL, env))\n+\t\tdie(\"Cannot write index '%s' for map\", tmp_index);\n+\n+\t/* Then map the index */\n+\targc = 0;\n+\targv[argc++] = \"sh\";\n+\targv[argc++] = \"-c\";\n+\targv[argc++] = index_map;\n+\targv[argc] = NULL;\n+\tif (run_command_v_opt_cd_env(argv, 0, NULL, env))\n+\t\tdie(\"Index map '%s' failed\", index_map);\n+\n+\t/* Finally read it back in */\n+\tif (read_cache_from(tmp_index) < 0)\n+\t\tdie(\"Error reading index '%s'\", tmp_index);\n+\tactive_cache_tree = cache_tree();\n+\tif (cache_tree_update(active_cache_tree, active_cache, active_nr,\n+\t\t\t 0, 0) < 0)\n+\t\tdie(\"Error building trees\");\n+\thex = sha1_to_hex(active_cache_tree->sha1);\n+\tdiscard_cache();\n+\n+\tunlink(tmp_index);\n+\n+\treturn hex;\n+}\n+\n+static char *rewrite_header(char *dest, unsigned long *len_p, char **buf_p,\n+\t\t\t struct commit_list *parents)\n+{\n+\tint linelen;\n+\tdo {\n+\t\tchar *line = *buf_p;\n+\t\tlinelen = get_one_line(*buf_p, *len_p);\n+\n+\t\tif (!linelen)\n+\t\t\treturn dest;\n+\t\t*buf_p += linelen;\n+\t\t*len_p -= linelen;\n+\n+\t\tif (index_map && !memcmp(line, \"tree \", 5)) {\n+\t\t\tif (linelen != 46)\n+\t\t\t\tdie(\"bad tree line in commit\");\n+\t\t\tmemcpy(dest, \"tree \", 5);\n+\t\t\tline[45] = '\\0';\n+\t\t\tmemcpy(dest+5, map_index(line+5), 40);\n+\t\t\tline[45] = '\\n';\n+\t\t\tdest[45] = '\\n';\n+\t\t\tdest += 46;\n+\t\t\tcontinue;\n+\t\t}\n+\n+\t\t/* drop old parents */\n+\t\tif (!memcmp(line, \"parent \", 7)) {\n+\t\t\tif (linelen != 48)\n+\t\t\t\tdie(\"bad parent line in commit\");\n+\t\t\tcontinue;\n+\t\t}\n+\n+\t\t/* insert new parents before author */\n+\t\tif (!memcmp(line, \"author \", 7))\n+\t\t\tdest = add_parents(dest, parents);\n+\n+\t\tmemcpy(dest, line, linelen);\n+\t\tdest += linelen;\n+\t} while (linelen > 1);\n+\treturn dest;\n+}\n+\n+static size_t memspn(const char *s, const char *accept, size_t n)\n+{\n+\tsize_t offset;\n+\n+\tfor (offset = 0; offset < n; ++offset)\n+\t\tif (!strchr(accept, s[offset]))\n+\t\t\tbreak;\n+\treturn offset;\n+}\n+\n+static size_t memcspn(const char *s, const char *accept, size_t n)\n+{\n+\tsize_t offset;\n+\n+\tfor (offset = 0; offset < n; ++offset)\n+\t\tif (strchr(accept, s[offset]))\n+\t\t\tbreak;\n+\treturn offset;\n+}\n+\n+/* Replace any (short) sha1 of a rewritten commit by the new (short) sha1 */\n+static char *rewrite_body(char *dest, unsigned long len, char *buf)\n+{\n+\tstatic const char hex[] = \"0123456789abcdef\";\n+\tunsigned char sha1[20];\n+\n+\twhile (len) {\n+\t\tsize_t ll = memcspn(buf, hex, len);\n+\t\tmemcpy(dest, buf, ll);\n+\t\tdest += ll;\n+\t\tbuf += ll;\n+\t\tlen -= ll;\n+\n+\t\tll = memspn(buf, hex, len);\n+\t\tif (ll >= 8 && ll <= 40 &&\n+\t\t !get_short_sha1(buf, ll, sha1, 1) &&\n+\t\t !get_rewritten_sha1(sha1))\n+\t\t\tmemcpy(dest, sha1_to_hex(sha1), ll);\n+\t\telse\n+\t\t\tmemcpy(dest, buf, ll);\n+\t\tdest += ll;\n+\t\tbuf += ll;\n+\t\tlen -= ll;\n+\t}\n+\treturn dest;\n+}\n+\n+static void write_ref_sha1_or_die(const char *ref, const unsigned char *old_sha1,\n+\t\t\t\t const unsigned char *new_sha1,\n+\t\t\t\t const char *logmsg, int flags)\n+{\n+\tstruct ref_lock *lock;\n+\n+\tlock = lock_any_ref_for_update(ref, old_sha1, flags);\n+\tif (!lock)\n+\t\tdie(\"%s: cannot lock the ref\", ref);\n+\tif (write_ref_sha1(lock, new_sha1, logmsg) < 0)\n+\t\tdie(\"%s: cannot update the ref\", ref);\n+}\n+\n+static int is_ref_to_be_rewritten(const char *ref)\n+{\n+\tunsigned char sha1[20];\n+\tint flag;\n+\n+\tif (prefixcmp(ref, \"refs/\"))\n+\t\treturn 0;\n+\tif (!prefixcmp(ref, \"refs/remotes/\"))\n+\t\treturn 0;\n+\n+\tresolve_ref(ref, sha1, 0, &flag);\n+\tif (flag & REF_ISSYMREF)\n+\t\treturn 0;\n+\n+\treturn 1;\n+}\n+\n+static void rewrite_sha1(struct object *obj, unsigned char *new_sha1)\n+{\n+\tstruct name_decoration *deco;\n+\tint prefix_len;\n+\n+\tif (!hashcmp(obj->sha1, new_sha1))\n+\t\treturn;\n+\n+\tadd_rewrite_decoration(obj, new_sha1);\n+\n+\tprefix_len = strlen(rewritten_prefix);\n+\tdeco = lookup_decoration(&name_decoration, obj);\n+\tfor (; deco; deco = deco->next) {\n+\t\tchar buffer[256];\n+\t\tchar *p;\n+\t\tint len = strlen(deco->name);\n+\n+\t\tif (!is_ref_to_be_rewritten(deco->name))\n+\t\t\tcontinue;\n+\t\tif (len + prefix_len + 1 + 1 > sizeof(buffer))\n+\t\t\tdie(\"rewrite ref of '%s' too long\", deco->name);\n+\t\tp = buffer;\n+\t\tmemcpy(p, \"refs/\", 5);\n+\t\tp += 5;\n+\t\tmemcpy(p, rewritten_prefix, prefix_len);\n+\t\tp += prefix_len;\n+\t\tmemcpy(p, deco->name+4, len-4 + 1);\n+\n+\t\twrite_ref_sha1_or_die(buffer, NULL, obj->sha1,\n+\t\t\t\t\t\"copied during rewrite\", 0);\n+\n+\t\tif (is_null_sha1(new_sha1))\n+\t\t\tdelete_ref(deco->name, obj->sha1);\n+\t\telse\n+\t\t\twrite_ref_sha1_or_die(deco->name, obj->sha1, new_sha1,\n+\t\t\t\t\t\t\"rewritten\", 0);\n+\t}\n+}\n+\n+static void map_and_write_commit(char *commit, size_t len, unsigned char *sha1)\n+{\n+\tchar commit_path[PATH_MAX];\n+\tstruct child_process cmd;\n+\tint argc;\n+\tconst char *argv[10];\n+\tint fd;\n+\n+\tfd = git_mkstemp(commit_path, sizeof(commit_path), \".commit_XXXXXX\");;\n+\twrite_or_die(fd, commit, len);\n+\n+\targc = 0;\n+\targv[argc++] = \"sh\";\n+\targv[argc++] = \"-c\";\n+\targv[argc++] = commit_map;\n+\targv[argc] = NULL;\n+\tmemset(&cmd, 0, sizeof(cmd));\n+\tcmd.in = open(commit_path, O_RDONLY);\n+\tif (cmd.in < 0)\n+\t\tdie(\"Unable to read commit from file '%s'\", commit_path);\n+\tunlink(commit_path);\n+\tcmd.out = -1;\n+\tcmd.argv = argv;\n+\tif (start_command(&cmd))\n+\t\tdie(\"Commit map '%s' failed to start\", commit_map);\n+\tif (index_pipe(sha1, cmd.out, commit_type, 1))\n+\t\tdie(\"Unable to write new commit\");\n+\tif (finish_command(&cmd))\n+\t\tdie(\"Commit map '%s' failed\", commit_map);\n+\tclose(cmd.in);\n+}\n+\n+/*\n+ * Replace any parent that has been removed by its parents\n+ * and return the number of new parents.\n+ * We directly modify the parent list, so any libification\n+ * should probably adapt this function.\n+ */\n+static int rewrite_parents(struct commit *commit)\n+{\n+\tint n;\n+\tstruct commit_list *list, *parents, **prev;\n+\tunsigned char sha1[20];\n+\n+\tfor (n = 0, prev = &commit->parents; *prev; ++n) {\n+\t\tlist = *prev;\n+\n+\t\tif (!(list->item->object.flags & PRUNED)) {\n+\t\t\tprev = &list->next;\n+\t\t\tcontinue;\n+\t\t}\n+\n+\t\thashcpy(sha1, list->item->object.sha1);\n+\t\tget_rewritten_sha1(sha1);\n+\t\tif (!is_null_sha1(sha1)) {\n+\t\t\thashclr(sha1);\n+\t\t\trewrite_sha1(&list->item->object, sha1);\n+\t\t}\n+\n+\t\tparents = list->item->parents;\n+\t\tif (!parents) {\n+\t\t\t*prev = list->next;\n+\t\t\tfree(list);\n+\t\t\t--n;\n+\t\t\tcontinue;\n+\t\t}\n+\t\tlist->item = parents->item;\n+\t\tprev = &list->next;\n+\t\tlist = list->next;\n+\t\twhile ((parents = parents->next)) {\n+\t\t\tcommit_list_insert(parents->item, prev);\n+\t\t\tprev = &(*prev)->next;\n+\t\t\t++n;\n+\t\t}\n+\t\t*prev = list;\n+\t}\n+\n+\treturn n;\n+}\n+\n+static void rewrite_commit(struct commit *commit)\n+{\n+\tchar *buf, *p;\n+\tint n;\n+\tchar *orig_buf = commit->buffer;\n+\tunsigned long orig_len = strlen(orig_buf);\n+\tunsigned char sha1[20];\n+\n+\tn = rewrite_parents(commit);\n+\n+\t/* Make enough remove for n (possibly extra) parents */\n+\tp = buf = xmalloc(orig_len + n*48);\n+\tp = rewrite_header(p, &orig_len, &orig_buf, commit->parents);\n+\tp = rewrite_body(p, orig_len, orig_buf);\n+\tif (!commit_map) {\n+\t\tif (write_sha1_file(buf, p-buf, commit_type, sha1))\n+\t\t\tdie(\"Unable to write new commit\");\n+\t} else\n+\t\tmap_and_write_commit(buf, p-buf, sha1);\n+\tfree(buf);\n+\n+\trewrite_sha1(&commit->object, sha1);\n+}\n+\n+static void move_head_forward(char *ref, unsigned char *old_sha1, int detached)\n+{\n+\tunsigned char new_sha1[20];\n+\tint rewritten;\n+\n+\thashcpy(new_sha1, old_sha1);\n+\trewritten = !get_rewritten_sha1(new_sha1);\n+\tif (rewritten && !is_bare_repository()) {\n+\t\tint argc;\n+\t\tconst char *argv[10];\n+\n+\t\targc = 0;\n+\t\targv[argc++] = \"read-tree\";\n+\t\targv[argc++] = \"-m\";\n+\t\targv[argc++] = \"-u\";\n+\t\targv[argc++] = sha1_to_hex(old_sha1);\n+\t\targv[argc++] = sha1_to_hex(new_sha1);\n+\t\targv[argc] = NULL;\n+\t\tif (run_command_v_opt(argv, RUN_GIT_CMD))\n+\t\t\tdie(\"Cannot move HEAD forward\");\n+\t}\n+\tif (!detached)\n+\t\tcreate_symref(\"HEAD\", ref, \"reattached after rewrite\");\n+\telse if (rewritten)\n+\t\twrite_ref_sha1_or_die(\"HEAD\", NULL, new_sha1,\n+\t\t\t\t \"rewritten\", REF_NODEREF);\n+}\n+\n+int cmd_rewrite_commits(int argc, const char **argv, const char *prefix)\n+{\n+\tstruct rev_info rev;\n+\tstruct commit *commit;\n+\tunsigned char HEAD_sha1[20];\n+\tchar *HEAD_ref;\n+\tint flag;\n+\tint i, j;\n+\n+\tinit_revisions(&rev, prefix);\n+\n+\tfor (i = 1, j = 1; i < argc; ++i) {\n+\t\tif (!strcmp(argv[i], \"--index-map\")) {\n+\t\t\tif (++i == argc)\n+\t\t\t\tdie(\"Argument required for --index-map\");\n+\t\t\tindex_map = argv[i];\n+\t\t} else if (!strcmp(argv[i], \"--commit-map\")) {\n+\t\t\tif (++i == argc)\n+\t\t\t\tdie(\"Argument required for --commit-map\");\n+\t\t\tcommit_map = argv[i];\n+\t\t} else\n+\t\t\targv[j++] = argv[i];\n+\t}\n+\targc = j;\n+\targc = setup_revisions(argc, argv, &rev, \"HEAD\");\n+\trev.ignore_merges = 0;\n+\trev.topo_order = 1;\n+\trev.reverse = 1;\n+\n+\tfor_each_ref(add_ref_decoration, NULL);\n+\n+\tHEAD_ref = xstrdup(resolve_ref(\"HEAD\", HEAD_sha1, 1, &flag));\n+\tif (flag & REF_ISSYMREF) {\n+\t\t/* Detach HEAD at its current position */\n+\t\twrite_ref_sha1_or_die(\"HEAD\", NULL, HEAD_sha1,\n+\t\t\t\t\t\"detached for rewrite\", REF_NODEREF);\n+\t}\n+\n+\tprepare_revision_walk(&rev);\n+\twhile ((commit = get_revision(&rev)) != NULL) {\n+\t\trewrite_commit(commit);\n+\t}\n+\n+\tmove_head_forward(HEAD_ref, HEAD_sha1, !(flag & REF_ISSYMREF));\n+\n+\treturn 0;\n+}\ndiff --git a/builtin.h b/builtin.h\nindex 661a92f..e01d337 100644\n--- a/builtin.h\n+++ b/builtin.h\n@@ -63,6 +63,7 @@ extern int cmd_rerere(int argc, const char **argv, const char *prefix);\n extern int cmd_rev_list(int argc, const char **argv, const char *prefix);\n extern int cmd_rev_parse(int argc, const char **argv, const char *prefix);\n extern int cmd_revert(int argc, const char **argv, const char *prefix);\n+extern int cmd_rewrite_commits(int argc, const char **argv, const char *prefix);\n extern int cmd_rm(int argc, const char **argv, const char *prefix);\n extern int cmd_runstatus(int argc, const char **argv, const char *prefix);\n extern int cmd_shortlog(int argc, const char **argv, const char *prefix);\ndiff --git a/git.c b/git.c\nindex a647f9c..ce02b0b 100644\n--- a/git.c\n+++ b/git.c\n@@ -356,6 +356,7 @@ static void handle_internal_command(int argc, const char **argv)\n \t\t{ \"rev-list\", cmd_rev_list, RUN_SETUP },\n \t\t{ \"rev-parse\", cmd_rev_parse, RUN_SETUP },\n \t\t{ \"revert\", cmd_revert, RUN_SETUP | NEED_WORK_TREE },\n+\t\t{ \"rewrite-commits\", cmd_rewrite_commits, RUN_SETUP },\n \t\t{ \"rm\", cmd_rm, RUN_SETUP | NEED_WORK_TREE },\n \t\t{ \"runstatus\", cmd_runstatus, RUN_SETUP | NEED_WORK_TREE },\n \t\t{ \"shortlog\", cmd_shortlog, RUN_SETUP | USE_PAGER },\ndiff --git a/t/t7005-rewrite-commits.sh b/t/t7005-rewrite-commits.sh\nnew file mode 100755\nindex 0000000..93f438f\n--- /dev/null\n+++ b/t/t7005-rewrite-commits.sh\n@@ -0,0 +1,74 @@\n+#!/bin/sh\n+\n+test_description='git-rewrite-commits'\n+. ./test-lib.sh\n+\n+make_commit () {\n+\tlower=$(echo $1 | tr A-Z a-z)\n+\techo $lower > $lower\n+\tgit add $lower\n+\ttest_tick\n+\tgit commit -m $1\n+\tgit tag $1\n+}\n+\n+test_expect_success 'setup' '\n+\tmake_commit A\n+\tmake_commit B\n+\tgit checkout -b branch B\n+\tmake_commit D\n+\tmake_commit E\n+\tgit checkout master\n+\tmake_commit C\n+\tgit checkout branch\n+\tgit merge C\n+\tgit tag F\n+\tmake_commit G\n+\tmake_commit H\n+'\n+\n+orig_H=$(git rev-parse H)\n+test_expect_success 'rewrite identically' '\n+\tgit-rewrite-commits\n+'\n+\n+test_expect_success 'result is really identical' '\n+\ttest $orig_H = $(git rev-parse H)\n+'\n+\n+# for lack of 'git-mv --cached d doh'\n+test_expect_success 'rewrite, renaming a specific file' '\n+\tgit-rewrite-commits --index-map \\\n+\t\t\"git-ls-files -s | sed \\\"s-\\\\td\\\\\\$-\\\\tdoh-\\\" |\n+\t GIT_INDEX_FILE=\\$GIT_INDEX_FILE.new \\\n+\t\t git-update-index --index-info &&\n+\t\t mv \\$GIT_INDEX_FILE.new \\$GIT_INDEX_FILE\"\n+'\n+\n+test_expect_success 'test that the file was renamed' '\n+\ttest d = $(git-show H:doh) &&\n+\t! git-show H:d --\n+'\n+\n+orig_H=$(git rev-parse H)\n+test_expect_success 'use index-map to move into a subdirectory' '\n+\tgit-rewrite-commits --index-map \\\n+\t\t \"git ls-files -s | sed \\\"s-\\\\t-&newsubdir/-\\\" |\n+\t GIT_INDEX_FILE=\\$GIT_INDEX_FILE.new \\\n+\t\t\tgit update-index --index-info &&\n+\t\t mv \\$GIT_INDEX_FILE.new \\$GIT_INDEX_FILE\" &&\n+\ttest -z \"$(git diff $orig_H H:newsubdir)\"'\n+\n+test_expect_success 'remove merge commit' '\n+\tgit-rewrite-commits --grep=\"!Merge\" &&\n+\ttest 2 = `git-log ^G^@ G --pretty=format:%P | wc -w`'\n+\n+test_expect_success 'remove first commit' '\n+\tgit-rewrite-commits --grep=\"!A\"'\n+\n+test_expect_success 'stops when index map fails' '\n+\t! git-rewrite-commits --index-map false &&\n+\tgit-checkout branch\n+'\n+\n+test_done\n-- \n1.5.3.rc0.68.geec71-dirty\n"},{"id":"46767","messageId":"Pine.LNX.4.64.0707081729040.4248@racer.site","threadId":"8928","inReplyTo":"1183911808787-git-send-email-skimo@liacs.nl","subject":"Re: [PATCH 4/4] Add git-rewrite-commits","fromName":"Johannes Schindelin","fromEmail":"johannes.schindelin@gmx.de","sentAt":"2007-07-08T16:37:22Z","receivedAt":"2007-07-08T16:37:22Z","isPatch":true,"sender":{"key":"johannes.schindelin@gmx.de","avatar":"https://avatars.githubusercontent.com/u/127790?v=4"},"body":"Hi,\n\nOn Sun, 8 Jul 2007, skimo@liacs.nl wrote:\n\n> \tbash-3.00$ time git-filter-branch test\n> \tRewrite 274fe3dfb8e8c7d0a6ce05138bdb650de7b459ea (425/425)\n> \tRewritten history saved to the test branch\n> \n> \treal 0m30.845s\n> \tuser 0m13.400s\n> \tsys 0m19.640s\n> \n> \tbash-3.00$ time git-rewrite-commits\n> \n> \treal 0m0.223s\n> \tuser 0m0.080s\n> \tsys 0m0.140s\n\nThat is to be expected. After all, the first is a script. However, I \nreally have ask: how often per hour do you want to run that program?\n\n> The command line is more reminiscent of git-log.\n> For example you can say\n> \n> \tgit-rewrite-commits --all\n> \n> to incorporate grafts in all branches, or\n> \n> \tgit rewrite-commits --author='!Darl McBribe' --all\n> \n> to remove all commits by Darl McBribe.\n\nI am really unhappy that so much is talked about filtering out commits. \nThat is amost certainly not what you want in most cases. In particular, I \nsuspect that most users would expect the _changes_ filtered out by such a \ncommand, which is just not true.\n\nFurther, I do not see much value in making this operation faster. It is \nmeant to be a one-time operation, for example when you open-source a \nproduct and have to cull a lot of history that you must not show for legal \nreasons. It is a one-shot operation. And having a script which works \nreliably is more valuable than having two programs, each with its own set \nof bugs.\n\nSo there are two things I see here that filter-branch cannot do yet. The \nfirst is to rewrite _all_ branches, which should be easy to do, it only \nhas to be done.\n\nThe second is to rewrite the commit messages so that the hashes are \nmapped, too. But that should be relatively easy, too: you can provide a \nmessage filter, and you can use the provided \"map\" function. If this \nseems to be what many people need, you can write a simple function and put \nit into filter-branch for common use.\n\nBTW I am not at all opposed to changing the name from filter-branch to \nsomething that fits better.\n\nCiao,\nDscho\n"},{"id":"46768","messageId":"20070708173027.GK1528MdfPADPa@greensroom.kotnet.org","threadId":"8928","inReplyTo":"Pine.LNX.4.64.0707081729040.4248@racer.site","subject":"Re: [PATCH 4/4] Add git-rewrite-commits","fromName":"Sven Verdoolaege","fromEmail":"skimo@kotnet.org","sentAt":"2007-07-08T17:30:27Z","receivedAt":"2007-07-08T17:30:27Z","isPatch":true,"sender":{"key":"skimo@kotnet.org","avatar":null},"body":"On Sun, Jul 08, 2007 at 05:37:22PM +0100, Johannes Schindelin wrote:\n> That is to be expected. After all, the first is a script. However, I \n> really have ask: how often per hour do you want to run that program?\n\nI have a project that needs some cleaning-up and I'd like to do\nit incrementally. I think I'll have to run it about a dozen times.\n\n> I am really unhappy that so much is talked about filtering out commits. \n> That is amost certainly not what you want in most cases. In particular, I \n> suspect that most users would expect the _changes_ filtered out by such a \n> command, which is just not true.\n\nI don't care about that either. I'm just mentioning it because\nit's mentioned in the git-filter-branch documentation (which you\nadded).\n\n> The second is to rewrite the commit messages so that the hashes are \n> mapped, too. But that should be relatively easy, too: you can provide a \n> message filter, and you can use the provided \"map\" function. If this \n> seems to be what many people need, you can write a simple function and put \n> it into filter-branch for common use.\n\nIt's not going to be me (as I sais, I don't like shell programming).\n\nskimo\n"},{"id":"46770","messageId":"46912726.5080807@midwinter.com","threadId":"8928","inReplyTo":"Pine.LNX.4.64.0707081729040.4248@racer.site","subject":"Re: [PATCH 4/4] Add git-rewrite-commits","fromName":"Steven Grimm","fromEmail":"koreth@midwinter.com","sentAt":"2007-07-08T18:04:22Z","receivedAt":"2007-07-08T18:04:22Z","isPatch":true,"sender":{"key":"koreth@midwinter.com","avatar":"https://gravatar.com/avatar/71b4d2e8b62f168bdc9e9205341159e3567003b4f9e2127c617c5fa0a1f5bad2?d=mp&s=160"},"body":"Johannes Schindelin wrote:\n> I am really unhappy that so much is talked about filtering out commits. \n> That is amost certainly not what you want in most cases. In particular, I \n> suspect that most users would expect the _changes_ filtered out by such a \n> command, which is just not true.\n> \n\nI agree that unless it's named and documented very carefully, users \nmight expect this to tweak history such that the commits in question \nnever happened (unlike revert, which of course adds a new commit and \nleaves the old ones alone.) The documentation for this command could \nstand to be more explicit about that.\n\n> Further, I do not see much value in making this operation faster. It is \n> meant to be a one-time operation, for example when you open-source a \n> product and have to cull a lot of history that you must not show for legal \n> reasons. It is a one-shot operation.\n> \n\nYour recent changes to git-rebase (which, BTW, are great) include a \nfeature that's very similar to this: the \"squash these commits together \nin my history\" feature. That'd be my use case for this, when I want to \npublish my changes to other developers who don't care about all my \nintermediate checkpoints of work in progress, and when the commits I'm \nremoving haven't been published anywhere else yet.\n\nWith this command, I could do something like:\n\ngit rewrite-commits --grep=\"!@@@checkpoint\"\ngit push\n\nand it would strip out all my intermediate checkpoint commits (assuming \nI've marked them as such in my commit comments, which I always do) \nbefore pushing to my project's shared repo. Right now that's a much more \ncumbersome, and very manual, operation. Even with the new git-rebase \nchanges, I still have to pick out those commits by hand, and it assumes \nthat I otherwise want to do a rebase in the first place.\n\n> So there are two things I see here that filter-branch cannot do yet. The \n> first is to rewrite _all_ branches, which should be easy to do, it only \n> has to be done.\n> \n\nI wonder if it makes sense to go that direction, though, or to make this \ncommand do the things that filter-branch can do, for the simple reason \nthat filter-branch is a shell script and this is already a nice \nnon-shell-dependent C program. Obviously you end up in the same place \neither way eventually once filter-branch percolates to the top of the \n\"port these scripts to C\" list, but it seems odd to me to port features \nfrom a C program back to a shell script only to have to convert the \nshell script to C later on.\n\nIronically, this app doesn't really speed up the one thing I find too \nslow in filter-branch: the \"remove a file from the tree in all \nrevisions\" case. To do that you still have to launch a filter app for \nevery commit, which is especially bad when the file in question only \nappears in a few revisions deep in the history of a repo.\n\nThis command points us in the direction of a \"remove/rename this file in \nhistory\" feature that doesn't require forking tens of thousands of child \nprocesses on a repo with lots of history. For that alone I think it's \nworthwhile, even though it's not there yet; that will never happen with \na shell script. And yeah, that's not a frequent operation, but it's sure \nnice when even the infrequent operations are lightning fast.\n\n-Steve\n"},{"id":"46771","messageId":"20070708181557.GL1528MdfPADPa@greensroom.kotnet.org","threadId":"8928","inReplyTo":"46912726.5080807@midwinter.com","subject":"Re: [PATCH 4/4] Add git-rewrite-commits","fromName":"Sven Verdoolaege","fromEmail":"skimo@kotnet.org","sentAt":"2007-07-08T18:15:57Z","receivedAt":"2007-07-08T18:15:57Z","isPatch":true,"sender":{"key":"skimo@kotnet.org","avatar":null},"body":"On Sun, Jul 08, 2007 at 11:04:22AM -0700, Steven Grimm wrote:\n> This command points us in the direction of a \"remove/rename this file in \n> history\" feature that doesn't require forking tens of thousands of child \n> processes on a repo with lots of history. For that alone I think it's \n> worthwhile, even though it's not there yet; that will never happen with \n> a shell script. And yeah, that's not a frequent operation, but it's sure \n> nice when even the infrequent operations are lightning fast.\n\nI couldn't come up with a clean interface to specify removes/renames\non the command line (other than copying the external command idea from\ncg-admin-rewritehist), but this is one of the operations I have to\nperform on the project I'm cleaning up, so if you have any suggestions,\nI'd be interested to hear them.\n\nskimo\n"},{"id":"46773","messageId":"Pine.LNX.4.64.0707081911160.4248@racer.site","threadId":"8928","inReplyTo":"20070708173027.GK1528MdfPADPa@greensroom.kotnet.org","subject":"Re: [PATCH 4/4] Add git-rewrite-commits","fromName":"Johannes Schindelin","fromEmail":"johannes.schindelin@gmx.de","sentAt":"2007-07-08T18:17:16Z","receivedAt":"2007-07-08T18:17:16Z","isPatch":true,"sender":{"key":"johannes.schindelin@gmx.de","avatar":"https://avatars.githubusercontent.com/u/127790?v=4"},"body":"Hi,\n\nOn Sun, 8 Jul 2007, Sven Verdoolaege wrote:\n\n> On Sun, Jul 08, 2007 at 05:37:22PM +0100, Johannes Schindelin wrote:\n> > That is to be expected. After all, the first is a script. However, I \n> > really have ask: how often per hour do you want to run that program?\n> \n> I have a project that needs some cleaning-up and I'd like to do it \n> incrementally. I think I'll have to run it about a dozen times.\n\nA dozen times seems not that bad, especially since you can run both \nprograms on a limited set of commits. So the cost is not that big.\n\n> > I am really unhappy that so much is talked about filtering out \n> > commits. That is amost certainly not what you want in most cases. \n> > In particular, I suspect that most users would expect the _changes_ \n> > filtered out by such a command, which is just not true.\n> \n> I don't care about that either. I'm just mentioning it because it's \n> mentioned in the git-filter-branch documentation (which you added).\n\nWhich I copied. And this is not the first, let alone the only example in \nfilter-branch's documentation.\n\n> > The second is to rewrite the commit messages so that the hashes are \n> > mapped, too. But that should be relatively easy, too: you can provide \n> > a message filter, and you can use the provided \"map\" function. If \n> > this seems to be what many people need, you can write a simple \n> > function and put it into filter-branch for common use.\n> \n> It's not going to be me (as I sais, I don't like shell programming).\n\nYes, you made that clear.\n\nHowever, this leaves things only in half-finished states.\n\n- \"git filter-branch\" did not learn the useful features that you seem to \n need, and\n\n- your builtin is at most a start of a builtin replacement for \n filter-branch, which changes the semantics, to be sure.\n\nI have no doubts that it will stay that way for a while, since this \nbuiltin seems to be good enough for what you want it to do.\n\nCiao,\nDscho\n"},{"id":"46779","messageId":"Pine.LNX.4.64.0707081920410.4248@racer.site","threadId":"8928","inReplyTo":"46912726.5080807@midwinter.com","subject":"Re: [PATCH 4/4] Add git-rewrite-commits","fromName":"Johannes Schindelin","fromEmail":"johannes.schindelin@gmx.de","sentAt":"2007-07-08T18:41:53Z","receivedAt":"2007-07-08T18:41:53Z","isPatch":true,"sender":{"key":"johannes.schindelin@gmx.de","avatar":"https://avatars.githubusercontent.com/u/127790?v=4"},"body":"Hi,\n\nOn Sun, 8 Jul 2007, Steven Grimm wrote:\n\n> Johannes Schindelin wrote:\n>\n> > I am really unhappy that so much is talked about filtering out \n> > commits. That is amost certainly not what you want in most cases. In \n> > particular, I suspect that most users would expect the _changes_ \n> > filtered out by such a command, which is just not true.\n> > \n> \n> I agree that unless it's named and documented very carefully, users \n> might expect this to tweak history such that the commits in question \n> never happened (unlike revert, which of course adds a new commit and \n> leaves the old ones alone.) The documentation for this command could \n> stand to be more explicit about that.\n\nIn the documentation of filter-branch, you have at least a couple of more \nuseful examples, and the misunderstanding is cleared up. At least a \nlittle.\n\n> > Further, I do not see much value in making this operation faster. It \n> > is meant to be a one-time operation, for example when you open-source \n> > a product and have to cull a lot of history that you must not show for \n> > legal reasons. It is a one-shot operation.\n> \n> Your recent changes to git-rebase (which, BTW, are great) include a \n> feature that's very similar to this: the \"squash these commits together \n> in my history\" feature. That'd be my use case for this, when I want to \n> publish my changes to other developers who don't care about all my \n> intermediate checkpoints of work in progress, and when the commits I'm \n> removing haven't been published anywhere else yet.\n> \n> With this command, I could do something like:\n> \n> git rewrite-commits --grep=\"!@@@checkpoint\"\n> git push\n\ngit reset --head HEAD~ &&\ngit rev-list --pretty=oneline ..HEAD@{1} | while read sha1 message\ndo\n\tgit cherry-pick -n $sha1 || exit\n\tcase \"$message\" in\n\t*@@@checkpoint*) ;;\n\t*) git commit -C $sha1;;\n\tesac\ndone\n\nYes, it is longer than the command line you gave. But it does not need \nany fancy new stuff, so it works even on 1.4.4.4. And you can make that a \nscript, or if you really need to, an alias. Since you want to avoid the \ninteractivity, rebase -i is kind of the wrong place to look.\n\n> and it would strip out all my intermediate checkpoint commits (assuming \n> I've marked them as such in my commit comments, which I always do) \n> before pushing to my project's shared repo. Right now that's a much more \n> cumbersome, and very manual, operation. Even with the new git-rebase \n> changes, I still have to pick out those commits by hand, and it assumes \n> that I otherwise want to do a rebase in the first place.\n\nIf you _have_ to use some fancy stuff, you can fake an editor by \nresetting VISUAL, like illustrated in t3404.\n\n> > So there are two things I see here that filter-branch cannot do yet. \n> > The first is to rewrite _all_ branches, which should be easy to do, it \n> > only has to be done.\n> \n> I wonder if it makes sense to go that direction, though, or to make this \n> command do the things that filter-branch can do, for the simple reason \n> that filter-branch is a shell script and this is already a nice \n> non-shell-dependent C program.\n\nExactly it is a C program. And if you are fleshing out issues, it is \nalways easier to work on a shell script. That is why git-remote is still \na perl script; the semantics are not yet flashed out, and it is orders of \nmagnitudes slower to change them in C. And to test them in C.\n\nBesides, it is way easier to fsck up some obscure detail in C, since you \nhave low-level access to everything. And those bugs are harder to spot. \nIn shell, you have a limited interface, but for prototyping, that is \nactually an advantage.\n\n> Obviously you end up in the same place either way eventually once \n> filter-branch percolates to the top of the \"port these scripts to C\" \n> list, but it seems odd to me to port features from a C program back to a \n> shell script only to have to convert the shell script to C later on.\n\nHey, if skimo would be willing to work until it is a complete replacement \nfor filter-branch, and more, I will stop working on filter-branch at once. \nSomehow I did not get this impression, however.\n\n> Ironically, this app doesn't really speed up the one thing I find too \n> slow in filter-branch: the \"remove a file from the tree in all \n> revisions\" case. To do that you still have to launch a filter app for \n> every commit, which is especially bad when the file in question only \n> appears in a few revisions deep in the history of a repo.\n\nSemantics. That is the stage we are at. Trying to find the best \nsemantics.\n\nFiltering out single files might well suffice for you. However, I think \nthat this is only a very special corner case of something much bigger and \nmore versatile. ATM I am quite certain that the eval'ed filters are the \nway to go.\n\nSince even in your corner case, you usually do it _once_.\n\n> This command points us in the direction of a \"remove/rename this file in \n> history\" feature that doesn't require forking tens of thousands of child \n> processes on a repo with lots of history. For that alone I think it's \n> worthwhile, even though it's not there yet; that will never happen with \n> a shell script. And yeah, that's not a frequent operation, but it's sure \n> nice when even the infrequent operations are lightning fast.\n\nIt is sure nice when the frequent operations get lightning fast _first_.\n\nCiao,\nDscho\n"},{"id":"46780","messageId":"20070708191104.GM1528MdfPADPa@greensroom.kotnet.org","threadId":"8928","inReplyTo":"Pine.LNX.4.64.0707081911160.4248@racer.site","subject":"Re: [PATCH 4/4] Add git-rewrite-commits","fromName":"Sven Verdoolaege","fromEmail":"skimo@kotnet.org","sentAt":"2007-07-08T19:11:04Z","receivedAt":"2007-07-08T19:11:04Z","isPatch":true,"sender":{"key":"skimo@kotnet.org","avatar":null},"body":"On Sun, Jul 08, 2007 at 07:17:16PM +0100, Johannes Schindelin wrote:\n> On Sun, 8 Jul 2007, Sven Verdoolaege wrote:\n> > On Sun, Jul 08, 2007 at 05:37:22PM +0100, Johannes Schindelin wrote:\n> > > I am really unhappy that so much is talked about filtering out \n> > > commits. That is amost certainly not what you want in most cases. \n> > > In particular, I suspect that most users would expect the _changes_ \n> > > filtered out by such a command, which is just not true.\n> > \n> > I don't care about that either. I'm just mentioning it because it's \n> > mentioned in the git-filter-branch documentation (which you added).\n> \n> Which I copied. And this is not the first, let alone the only example in \n> filter-branch's documentation.\n\nAll I'm saying is that you shouldn't blame me for doing something\nyou have done yourself.\nAnd if you're not blaming me, but just making a general comment,\nthen all I can say is that I agree with your comment.\n\n> However, this leaves things only in half-finished states.\n> \n> - \"git filter-branch\" did not learn the useful features that you seem to \n> need, and\n> \n> - your builtin is at most a start of a builtin replacement for \n> filter-branch, which changes the semantics, to be sure.\n> \n> I have no doubts that it will stay that way for a while, since this \n> builtin seems to be good enough for what you want it to do.\n\nIf people find rewrite-commits useful, but think that something\nis missing, then I'd be willing to look into that.\nI'm personally not likely to work on fiter-branch, but maybe\nsomeone else, possibly inspired by rewrite-commits, will.\nBut it is true that rewrite-commits does everything I want now.\n\nskimo\n"},{"id":"46791","messageId":"20070708211034.GO1528MdfPADPa@greensroom.kotnet.org","threadId":"8928","inReplyTo":"Pine.LNX.4.64.0707081920410.4248@racer.site","subject":"Re: [PATCH 4/4] Add git-rewrite-commits","fromName":"Sven Verdoolaege","fromEmail":"skimo@kotnet.org","sentAt":"2007-07-08T21:10:34Z","receivedAt":"2007-07-08T21:10:34Z","isPatch":true,"sender":{"key":"skimo@kotnet.org","avatar":null},"body":"On Sun, Jul 08, 2007 at 07:41:53PM +0100, Johannes Schindelin wrote:\n> Hey, if skimo would be willing to work until it is a complete replacement \n> for filter-branch, and more, I will stop working on filter-branch at once. \n\nAs I said, I'm willing to put some more time into this.\nHowever, I didn't want to wait until I had all the bells and whistles\nbefore sending something out. If people don't like the direction\nI'm going, then there is no point continuing and then I'll just\nuse it for myself.\n\nI guess the major thing that is missing is --subdirectory-filter.\nAnything else?\n\nskimo\n"},{"id":"46807","messageId":"Pine.LNX.4.64.0707090011070.4248@racer.site","threadId":"8928","inReplyTo":"1183911808787-git-send-email-skimo@liacs.nl","subject":"Re: [PATCH 4/4] Add git-rewrite-commits","fromName":"Johannes Schindelin","fromEmail":"johannes.schindelin@gmx.de","sentAt":"2007-07-08T23:56:04Z","receivedAt":"2007-07-08T23:56:04Z","isPatch":true,"sender":{"key":"johannes.schindelin@gmx.de","avatar":"https://avatars.githubusercontent.com/u/127790?v=4"},"body":"Hi,\n\nI am way too tired to comment in detail, but here are some preliminary \nfindings:\n\n- your command line interface is very nice and elegant.\n\n- you use a lot of what was in cg-admin-rewritehist (including \n adjustments I made in the documentation for filter-branch), but you also \n make it more confusing for people used to that tool:\n\n\t- instead of leaving the original branches as they are, you \n\t overwrite them. That's okay. But then you put the originals into \n\t refs/rewritten. Without cg-admin-rewritehist using that name \n\t for the _result_, you could explain your way out of confusion. \n\t As it is, you cannot.\n\n\t- in spite of doing the same as cg-admin-rewritehist with filters, \n\t you call them maps. But they are no maps. They are manipulators, \n\t you can call them mutators or filters, too. Given what people \n\t know of cg-admin-rewritehist, you really should keep the name \n\t \"filter\".\n\n\t- the name \"map\" itself is used in cg-admin-rewritehist, to map \n\t commit names from old to new. By using that name differently, \n\t again you contribute to confusion, for no good reason.\n\n- neat idea to abuse the decorator for rewriting purposes.\n\n- get_one_line() is a misnomer. It wants to be named get_linelen().\n\n- instead of spawning read-tree, you could use unpack_trees() to boost \n performance even more. But I guess it is probably left for later, to \n make it easier to review the patch.\n\n- your memspn() and memcspn() functions are very inefficient. Better walk \n the memory, introduce a GIT_HEXCHAR into ctype.c, ishexchar() into \n git-compat-util.h, and use that.\n\n- The example you give with \"git update-index --remove\" can fail, right? \n Tell the user about that.\n\n- The commit filter again deviates from the usage in cg-admin-rewritehist. \n I can see that you wanted to make it more versatile. However, it makes \n the tool potentially also a bit more cumbersome to use. Besides, you use \n a temporary file where there is no need to.\n\n- \"map\" is missing. This is a function that you can use in all filters in \n cg-admin-rewritehist, except (unfortunately) in the commit filter (for \n technical reasons). Alas, these technical reasons also prevent such a \n function to be usable by any of the filters (\"maps\", as you call them).\n\n- the more fundamental problem with the missing \"map\", I do not see a \n reasonable way to get the same functionality from any of the code \n snippets passed to rewrite-commits. Indeed, even the workaround of \n cg-admin-rewritehist, to read $TEMP/map/$sha1, does not work, since you \n are keeping it all in memory. On IRC, gitster suggested to use a \n bidirectional pipe (such as stdin/stdout) to get at the new commit \n names, but because of buffering, I guess this is no joy.\n\nAs commented on IRC, the env/tree/parent/msg filters of \ncg-admin-rewritehist could be all emulated by commit filters. However, \nthat would be really inconvenient. So at a later stage, these would have \nto be integrated into rewrite-commits (even if it would be possible to \ndrive rewrite-commits by a shell porcelain, but I guess you are opposed to \nthat idea, since you want to do everything else in C, too).\n\nHowever, the biggest and very real problem is that your filters do not \nhave a \"map\" function to get the rewritten sha1 for a given sha1. That is \nwhat makes the filters so versatile, though, since you can skip revisions \nby much more complex rules than just greps on the commit message or \nheader.\n\nBut hey, maybe it _is_ time to rethink the whole filter business, and \nintroduce some kind of regular expression based action language. Something \nlike\n\n\tgit rewrite-commits -e '/^author Darl McBribe/skip-commit' \\\n\t\t-e 'substitute/^author Joahnnes/author Johannes/header' \\\n\t\t-e 'substitute/poreclain/porcelain/body' \\\n\t\t-e 'rewrite-commit-names'\n\nHmm?\n\nCiao,\nDscho\n"},{"id":"46845","messageId":"4691F96E.D869DF97@eudaptics.com","threadId":"8928","inReplyTo":"20070708211034.GO1528MdfPADPa@greensroom.kotnet.org","subject":"Re: [PATCH 4/4] Add git-rewrite-commits","fromName":"Johannes Sixt","fromEmail":"j.sixt@eudaptics.com","sentAt":"2007-07-09T09:01:34Z","receivedAt":"2007-07-09T09:01:34Z","isPatch":true,"sender":{"key":"j6t@kdbg.org","avatar":"https://avatars.githubusercontent.com/u/14810926?v=4"},"body":"Sven Verdoolaege wrote:\n> I guess the major thing that is missing is --subdirectory-filter.\n> Anything else?\n\nYes, how about this:\n\n $ git rewrite-commits --index-map '\n testresult=$($HOME/bin/expensive-test);\n [ $testresult = t ] && $HOME/bin/tweak-index ' \\\n --commit-map '\n [ $testresult = t ] && $HOME/bin/tweak-commit '\n\n:-P\n\n-- Hannes\n"},{"id":"46847","messageId":"20070709094703.GP1528MdfPADPa@greensroom.kotnet.org","threadId":"8928","inReplyTo":"Pine.LNX.4.64.0707090011070.4248@racer.site","subject":"Re: [PATCH 4/4] Add git-rewrite-commits","fromName":"Sven Verdoolaege","fromEmail":"skimo@kotnet.org","sentAt":"2007-07-09T09:47:03Z","receivedAt":"2007-07-09T09:47:03Z","isPatch":true,"sender":{"key":"skimo@kotnet.org","avatar":null},"body":"Thanks for the review.\n\nOn Mon, Jul 09, 2007 at 12:56:04AM +0100, Johannes Schindelin wrote:\n> - you use a lot of what was in cg-admin-rewritehist (including \n> adjustments I made in the documentation for filter-branch), but you also \n> make it more confusing for people used to that tool:\n> \n> \t- instead of leaving the original branches as they are, you \n> \t overwrite them. That's okay. But then you put the originals into \n> \t refs/rewritten. Without cg-admin-rewritehist using that name \n> \t for the _result_, you could explain your way out of confusion. \n> \t As it is, you cannot.\n\nI thought you had to specify the name of the new branch on the command\nline. Anyway, I don't really care about the name of this hierarchy.\nI just picked a name that is somewhat related to \"rewrite-commits\".\nSuggestions are welcome. I could also just not create them.\nThe old values are also available in reflog.\n\n> \t- in spite of doing the same as cg-admin-rewritehist with filters, \n> \t you call them maps. But they are no maps. They are manipulators, \n> \t you can call them mutators or filters, too. Given what people \n> \t know of cg-admin-rewritehist, you really should keep the name \n> \t \"filter\".\n\nNonesense. They are not filters\n(http://en.wikipedia.org/wiki/Filter_%28higher-order_function%29).\nThey are maps (http://en.wikipedia.org/wiki/Map_%28higher-order_function%29).\n(In cg-admin-rewritehist some of them are (partial) filters,\nbut the ones I have are not filters).\nI could extend the commit-map to remove the commit it the output is\nempty, but it'd still be closer to a map than to a filter.\n(You can map a commit to nothing, but a filter can't alter\nthe elements of a list, it only determines which elements are kept.)\n\n> \t- the name \"map\" itself is used in cg-admin-rewritehist, to map \n> \t commit names from old to new. By using that name differently, \n> \t again you contribute to confusion, for no good reason.\n\nThere is a good reason to call what I call maps maps. They _are_ maps.\nStill, I'm open for suggestions.\n\n> - get_one_line() is a misnomer. It wants to be named get_linelen().\n\nHmmm... I guess you missed Linus' mistake when he introduced it\nin commit.c (e3bc7a3bc7b77f44d686003f5a9346a135529f73).\nDo you want me to rename that one as well?\n\n> - instead of spawning read-tree, you could use unpack_trees() to boost \n> performance even more. But I guess it is probably left for later, to \n> make it easier to review the patch.\n\nYeah, it looked a bit tricky for an initial implementation, especially\nwhere I move the HEAD forward.\n\n> - The example you give with \"git update-index --remove\" can fail, right? \n\nYes. Spectacularly, even.\n\n> - The commit filter again deviates from the usage in cg-admin-rewritehist. \n> I can see that you wanted to make it more versatile. However, it makes \n> the tool potentially also a bit more cumbersome to use. Besides, you use \n> a temporary file where there is no need to.\n\nAre you saying I should use two pipes?\nWhat if the commit message is larger than the pipe buffer?\n\n> - the more fundamental problem with the missing \"map\", I do not see a \n> reasonable way to get the same functionality from any of the code \n> snippets passed to rewrite-commits. Indeed, even the workaround of \n> cg-admin-rewritehist, to read $TEMP/map/$sha1, does not work, since you \n> are keeping it all in memory. On IRC, gitster suggested to use a \n> bidirectional pipe (such as stdin/stdout) to get at the new commit \n> names, but because of buffering, I guess this is no joy.\n\nI could add an option to write $TEMP/map/$sha1, but it's not clear\nto me when such a map would be useful. Please enlighten me.\n\n> As commented on IRC, the env/tree/parent/msg filters of \n> cg-admin-rewritehist could be all emulated by commit filters. However, \n> that would be really inconvenient. So at a later stage, these would have \n> to be integrated into rewrite-commits (even if it would be possible to \n> drive rewrite-commits by a shell porcelain, but I guess you are opposed to \n> that idea, since you want to do everything else in C, too).\n\nI'm not opposed to running a few commands and connecting stuff in shell.\n(See git-submodule add, although I admit that I would have preferred to\ndo all of it in C.)\n\n> However, the biggest and very real problem is that your filters do not \n> have a \"map\" function to get the rewritten sha1 for a given sha1. That is \n> what makes the filters so versatile, though, since you can skip revisions \n> by much more complex rules than just greps on the commit message or \n> header.\n\nI thought your were opposed to the idea of skipping commits, since\nyou still carry along the changes in those commits.\nDo you have a use case?\n\n> But hey, maybe it _is_ time to rethink the whole filter business, and \n> introduce some kind of regular expression based action language. Something \n> like\n> \n> \tgit rewrite-commits -e '/^author Darl McBribe/skip-commit' \\\n\nWhat's wrong with --author='!Darl McBribe' ?\n\n> \t\t-e 'substitute/^author Joahnnes/author Johannes/header' \\\n> \t\t-e 'substitute/poreclain/porcelain/body' \\\n> \t\t-e 'rewrite-commit-names'\n\nHmmm... some of these would basically need a builtin sed.\nI was thinking about adding --remove and --rename, though.\n\nskimo\n"},{"id":"46848","messageId":"20070709094833.GQ1528MdfPADPa@greensroom.kotnet.org","threadId":"8928","inReplyTo":"4691F96E.D869DF97@eudaptics.com","subject":"Re: [PATCH 4/4] Add git-rewrite-commits","fromName":"Sven Verdoolaege","fromEmail":"skimo@kotnet.org","sentAt":"2007-07-09T09:48:33Z","receivedAt":"2007-07-09T09:48:33Z","isPatch":true,"sender":{"key":"skimo@kotnet.org","avatar":null},"body":"On Mon, Jul 09, 2007 at 11:01:34AM +0200, Johannes Sixt wrote:\n> Sven Verdoolaege wrote:\n> > I guess the major thing that is missing is --subdirectory-filter.\n> > Anything else?\n> \n> Yes, how about this:\n> \n> $ git rewrite-commits --index-map '\n> testresult=$($HOME/bin/expensive-test);\n> [ $testresult = t ] && $HOME/bin/tweak-index ' \\\n> --commit-map '\n> [ $testresult = t ] && $HOME/bin/tweak-commit '\n> \n> :-P\n\nYou can write the result of expensive-test to a temporary file.\n\nskimo\n"},{"id":"46857","messageId":"Pine.LNX.4.64.0707091256430.5546@racer.site","threadId":"8928","inReplyTo":"4691F96E.D869DF97@eudaptics.com","subject":"Re: [PATCH 4/4] Add git-rewrite-commits","fromName":"Johannes Schindelin","fromEmail":"johannes.schindelin@gmx.de","sentAt":"2007-07-09T11:57:38Z","receivedAt":"2007-07-09T11:57:38Z","isPatch":true,"sender":{"key":"johannes.schindelin@gmx.de","avatar":"https://avatars.githubusercontent.com/u/127790?v=4"},"body":"Hi,\n\nOn Mon, 9 Jul 2007, Johannes Sixt wrote:\n\n> Sven Verdoolaege wrote:\n> > I guess the major thing that is missing is --subdirectory-filter.\n> > Anything else?\n> \n> Yes, how about this:\n> \n> $ git rewrite-commits --index-map '\n> testresult=$($HOME/bin/expensive-test);\n> [ $testresult = t ] && $HOME/bin/tweak-index ' \\\n> --commit-map '\n> [ $testresult = t ] && $HOME/bin/tweak-commit '\n\nAs skimo almost hinted: this will not work. Once the index \"map\" exits, \ntestresult is forgotten.\n\nCiao,\nDscho\n"},{"id":"46858","messageId":"20070709123626.GC23494@sigill.intra.peff.net","threadId":"8928","inReplyTo":"Pine.LNX.4.64.0707090011070.4248@racer.site","subject":"Re: [PATCH 4/4] Add git-rewrite-commits","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2007-07-09T12:36:27Z","receivedAt":"2007-07-09T12:36:27Z","isPatch":true,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Mon, Jul 09, 2007 at 12:56:04AM +0100, Johannes Schindelin wrote:\n\n> But hey, maybe it _is_ time to rethink the whole filter business, and \n> introduce some kind of regular expression based action language. Something \n> like\n> \n> \tgit rewrite-commits -e '/^author Darl McBribe/skip-commit' \\\n> \t\t-e 'substitute/^author Joahnnes/author Johannes/header' \\\n> \t\t-e 'substitute/poreclain/porcelain/body' \\\n> \t\t-e 'rewrite-commit-names'\n\nThis is starting to look an awful lot like sed. Which is good, but I\nwonder if we can get sed to do the heavy lifting. I have had success\nwith similar systems by writing the data structure out into a canonical\nformat, editing it as text, and then \"applying\" the result.\n\nSomething like:\n git rewrite-generate oldbranch |\n sed 's/^author Darl McBribe/skip-commit/'\n sed 's/^author Joahnnes/author Johannes/' |\n git-rewrite-commit-names |\n git rewrite-apply newbranch\n\nwhere git-rewrite-generate would generate something like git-log output,\nand git-rewrite-apply would, given a log-ish input, write a new history\nbranch. A nice advantage is that it makes things like this very natural:\n git rewrite-generate oldbranch >history\n vi history\n git rewrite-apply newbranch On Mon, Jul 09, 2007 at 12:56:04AM +0100, Johannes Schindelin wrote:\n>\n> > - you use a lot of what was in cg-admin-rewritehist (including \n> > adjustments I made in the documentation for filter-branch), but you\n> > also make it more confusing for people used to that tool:\n> > \n> > \t- instead of leaving the original branches as they are, you \n> > \t overwrite them. That's okay. But then you put the originals into \n> > \t refs/rewritten. Without cg-admin-rewritehist using that name \n> > \t for the _result_, you could explain your way out of confusion. \n> > \t As it is, you cannot.\n> \n> I thought you had to specify the name of the new branch on the command\n> line. Anyway, I don't really care about the name of this hierarchy.\n> I just picked a name that is somewhat related to \"rewrite-commits\".\n> Suggestions are welcome.\n\nHow much more explicit should I formulate my suggestion?\n\n> I could also just not create them. The old values are also available in \n> reflog.\n\nThere are two types of convenience. One for the programmer, and one for \nthe user. As a user I would definitely like to see which branches were \nactually changed. So even if you _have_ to keep your current behaviour, \nnamely that the results are stored under the original names, you should \nstore the original refs in something like refs/original (overridable by a \ncommand line option), but _only if_ they actually changed.\n\n> > \t- in spite of doing the same as cg-admin-rewritehist with filters, \n> > \t you call them maps. But they are no maps. They are manipulators, \n> > \t you can call them mutators or filters, too. Given what people \n> > \t know of cg-admin-rewritehist, you really should keep the name \n> > \t \"filter\".\n> \n> Nonesense.\n\nQuite a strong word. Let's see if you can defend it:\n\n> They are not filters \n> (http://en.wikipedia.org/wiki/Filter_%28higher-order_function%29). They \n> are maps (http://en.wikipedia.org/wiki/Map_%28higher-order_function%29).\n\nAha. Functional programming. Since when do we do functional programming \nhere? C++ is much closer to what we do. So let's see:\n\nhttp://en.wikipedia.org/wiki/Map_%28C%2B%2B_container%29\n\nDoes not apply at all here. But does something else apply? Ah, there is \nsomething about mathematics. The basis of programming. Maybe it is \nhelpful:\n\nhttp://en.wikipedia.org/wiki/Map_%28mathematics%29\n\nAh, no! It only says that it is a function. But there is a mention about \npartial maps! And you said something about that:\n\n> (In cg-admin-rewritehist some of them are (partial) filters,\n> but the ones I have are not filters).\n\nAlas, Wikipedia says about a partial function that it is a function which \nmaps from one space into another space (not necessarily different spaces, \nthough), but does not necessarily have a result for every input.\n\nBut in that sense, everything is a map. Even rewrite-commits. And \ncommits. (Maybe you should rename that to map-maps?)\n\nSo maybe more luck with \"filter\"? Since we still do not write in Haskell \nor something similar obscure, let's see if Wikipedia has something which \nis more likely applying to us. Yes. Software. That surely \nmust apply:\n\nhttp://en.wikipedia.org/wiki/Filter_%28software%29\n\nand even more to the point: UNIX filters:\n\nhttp://en.wikipedia.org/wiki/Filter_%28Unix%29\n\nYou will see that both definitions apply _perfectly_ to what you do here. \nYou pipe the commit into the \"map\" and write it out as a new commit! \nThe _perfect_ example for a UNIX filter.\n\n> I could extend the commit-map to remove the commit it the output is \n> empty, but it'd still be closer to a map than to a filter. (You can map \n> a commit to nothing, but a filter can't alter the elements of a list, it \n> only determines which elements are kept.)\n\nThis is not what the commit-filter of cg-admin-rewritehist does. If at \nall, your commit-map is a better example for a filter than rewritehist's \ncommit-filter is.\n\nSo, you cannot defend that bold statement.\n\nEnough ranted about an unmerited rant, let's get back to the interesting \nstuff:\n\nAll you demonstrated here is that the way \"commit-map\"s work right now is \nnot sufficient to make more complicated rewrite rules. You cannot split \none commit in two, and you cannot skip it.\n\n> > \t- the name \"map\" itself is used in cg-admin-rewritehist, to map \n> > \t commit names from old to new. By using that name differently, \n> > \t again you contribute to confusion, for no good reason.\n> \n> There is a good reason to call what I call maps maps. They _are_ maps.\n> Still, I'm open for suggestions.\n\nAre you? Well, my suggestion still stands... I mean, the suggestion I \nalready gave in the post you quoted. You know, to keep the name as it was \nin the program you based your work on? Okay, I'll be explicit for once: \nFilter.\n\n> > - get_one_line() is a misnomer. It wants to be named get_linelen().\n> \n> Hmmm... I guess you missed Linus' mistake when he introduced it\n> in commit.c (e3bc7a3bc7b77f44d686003f5a9346a135529f73).\n\nYes. But finger pointing will not help you here. I reviewed _your_ code.\n\n> Do you want me to rename that one as well?\n\nBe my guest. But let's do that independently, will ya?\n\n> > - instead of spawning read-tree, you could use unpack_trees() to boost\n> > performance even more. But I guess it is probably left for later, to \n> > make it easier to review the patch.\n> \n> Yeah, it looked a bit tricky for an initial implementation, especially\n> where I move the HEAD forward.\n\nLater.\n\n> > - The example you give with \"git update-index --remove\" can fail, right? \n> \n> Yes. Spectacularly, even.\n\nDoes that mean you acknowledge that the man page should tell about this? \nAnd probably also reveal the \"|| :\" remedy?\n\n> > - The commit filter again deviates from the usage in cg-admin-rewritehist. \n> > I can see that you wanted to make it more versatile. However, it makes \n> > the tool potentially also a bit more cumbersome to use. Besides, you use \n> > a temporary file where there is no need to.\n> \n> Are you saying I should use two pipes?\n\nUmm. Why not?\n\n> What if the commit message is larger than the pipe buffer?\n\nYou start_command(). Then you write() until it is all written, or the \npipe is broken. Then you get the output via index_pipe(). Which is a \nsingle sha1. I do not understand your question.\n\n> > - the more fundamental problem with the missing \"map\", I do not see a \n> > reasonable way to get the same functionality from any of the code \n> > snippets passed to rewrite-commits. Indeed, even the workaround of \n> > cg-admin-rewritehist, to read $TEMP/map/$sha1, does not work, since you \n> > are keeping it all in memory. On IRC, gitster suggested to use a \n> > bidirectional pipe (such as stdin/stdout) to get at the new commit \n> > names, but because of buffering, I guess this is no joy.\n> \n> I could add an option to write $TEMP/map/$sha1, but it's not clear\n> to me when such a map would be useful. Please enlighten me.\n\nTo be flexible, you have to have a way to ask \"what was this commit \nrewritten to?\" from within a filter.\n\nExample: maybe you expect the rewritten commit to have a tree identical to \ncommits in a certain branch, which was _also_ possibly rewritten. Maybe \nyou want to mark these pairs, for example by a tag. Then you need to get \nthe sha1 of the rewritten commits.\n\n> > As commented on IRC, the env/tree/parent/msg filters of \n> > cg-admin-rewritehist could be all emulated by commit filters. However, \n> > that would be really inconvenient. So at a later stage, these would \n> > have to be integrated into rewrite-commits (even if it would be \n> > possible to drive rewrite-commits by a shell porcelain, but I guess \n> > you are opposed to that idea, since you want to do everything else in \n> > C, too).\n> \n> I'm not opposed to running a few commands and connecting stuff in shell. \n> (See git-submodule add, although I admit that I would have preferred to \n> do all of it in C.)\n\nYes, but your original version was very intrusive and hard to review. In \ncontrast, the simple shell script we have now is hackable by many, and \neasily reviewed. Indeed, it takes but a couple of minutes to verify that \nit does what it should do and has no side effects.\n\nBut I agree, it is easy enough to run the commands from rewrite-commits.c. \nThe harder part is to provide an infrastructure which makes it useful.\n\nTo enhance on the above example: you're rewriting the commit messages so \nthat commit names are rewritten to match the rewritten commits. That is \npossible by a message filter in cg-admin-rewritehist.\n\nBut now somebody comes along, and says \"I have a history I need to \nrewrite. All bug fixes. The commit names were all abbreviated in the \ncommit messages, but they always had 'commit ' in front of them. I want \nto rewrite them, too.\"\n\nNo chance without a \"map\" function. Except to bug you to change the C \ncode for this very special need.\n\nBy contrast, filter-branch does not need to be changed for that at all.\n\n> > However, the biggest and very real problem is that your filters do not \n> > have a \"map\" function to get the rewritten sha1 for a given sha1. That \n> > is what makes the filters so versatile, though, since you can skip \n> > revisions by much more complex rules than just greps on the commit \n> > message or header.\n> \n> I thought your were opposed to the idea of skipping commits, since\n> you still carry along the changes in those commits.\n\nAre you trying to misunderstand me?\n\nYou can tell the user how to skip single commits. But you have to warn \nthem that they might be wanting something different, because the word \n\"commit\" not only implies a \"revision\", which you would skip in this \ncontext, but also a \"commit diff\", which you would _not_.\n\nThe \"much more complex\" rules clearly are for other use cases, which \nbrings me to your next question:\n\n> Do you have a use case?\n\n_You_ already mentioned it. Subdirectory filters.\n\nAnd if you think a little further, you can easily see that for this to \nbecome useful, you _have_ to have a \"map\" function, too. Because you are \nlikely wanting to use rewrite-commits, or filter-branch, to transform big \nrepositories into super/subprojects.\n\n> > But hey, maybe it _is_ time to rethink the whole filter business, and \n> > introduce some kind of regular expression based action language. \n> > Something like\n> > \n> > \tgit rewrite-commits -e '/^author Darl McBribe/skip-commit' \\\n> \n> What's wrong with --author='!Darl McBribe' ?\n\nIt is a very special use case. Not always will you be able to get all the \ninformation from the commit object you need for conditional operations. \nMy example only showed that you can do the same with that syntax. But \nimagine what you could do if we just added a small syntactical sugar:\n\n\t-e '?has-path:README?substitute/v2/v3/'\n\n> > \t\t-e 'substitute/^author Joahnnes/author Johannes/header' \\\n> > \t\t-e 'substitute/poreclain/porcelain/body' \\\n> > \t\t-e 'rewrite-commit-names'\n> \n> Hmmm... some of these would basically need a builtin sed. I was thinking \n> about adding --remove and --rename, though.\n\nIMHO you will not get happy by introducing ever more, but still restricted \nfunctionality.\n\nAt the moment, cg-admin-rewritehist can do much more than what \nrewrite-commits can do, because the eval'ed filters give you a much \ngreater freedom in what you can do (since \"map\" is available, and you can \neven modify the commit name mapping by the filters on the go: they can \nedit .git-rewrite/!), and how you can do it (no need to work around \ncertain limitations by using temporary files).\n\nA lot of the speed of rewrite-commits stems from the fact that it is \nlimited in that way.\n\nDon't get me wrong. I _want_ rewrite-commits to replace filter-branch. \nBut there are fundamental problems with filter-branch, and there are \nfundamental problems with having the same functionality from within C. \nAnd it appears to me that you do not even try to address the latter \nfundamental problems.\n\nCiao,\nDscho\n"},{"id":"46863","messageId":"Pine.LNX.4.64.0707091409480.5546@racer.site","threadId":"8928","inReplyTo":"20070709123626.GC23494@sigill.intra.peff.net","subject":"Re: [PATCH 4/4] Add git-rewrite-commits","fromName":"Johannes Schindelin","fromEmail":"johannes.schindelin@gmx.de","sentAt":"2007-07-09T13:16:19Z","receivedAt":"2007-07-09T13:16:19Z","isPatch":true,"sender":{"key":"johannes.schindelin@gmx.de","avatar":"https://avatars.githubusercontent.com/u/127790?v=4"},"body":"Hi,\n\nOn Mon, 9 Jul 2007, Jeff King wrote:\n\n> On Mon, Jul 09, 2007 at 12:56:04AM +0100, Johannes Schindelin wrote:\n> \n> > But hey, maybe it _is_ time to rethink the whole filter business, and \n> > introduce some kind of regular expression based action language. Something \n> > like\n> > \n> > \tgit rewrite-commits -e '/^author Darl McBribe/skip-commit' \\\n> > \t\t-e 'substitute/^author Joahnnes/author Johannes/header' \\\n> > \t\t-e 'substitute/poreclain/porcelain/body' \\\n> > \t\t-e 'rewrite-commit-names'\n> \n> This is starting to look an awful lot like sed.\n\nYep. Fully intended.\n\n> Which is good, but I wonder if we can get sed to do the heavy lifting. I \n> have had success with similar systems by writing the data structure out \n> into a canonical format, editing it as text, and then \"applying\" the \n> result.\n> \n> Something like:\n> git rewrite-generate oldbranch |\n> sed 's/^author Darl McBribe/skip-commit/'\n> sed 's/^author Joahnnes/author Johannes/' |\n> git-rewrite-commit-names |\n> git rewrite-apply newbranch\n> \n> where git-rewrite-generate would generate something like git-log output,\n> and git-rewrite-apply would, given a log-ish input, write a new history\n> branch. A nice advantage is that it makes things like this very natural:\n> git rewrite-generate oldbranch >history\n> vi history\n> git rewrite-apply newbranch \n> which allows interactive editing.\n\nGranted. It is a really nice idea, but again you lack the map function.\n\n> Of course, this is:\n> - possibly inefficient, since rewrite-apply doesn't know what you\n> changed and what you didn't change; it would have to recalculate\n> a lot of sha1 hashes.\n> - doesn't really deal with actual tree rewriting, unless there is some\n> canonical text format for that, and then we are talking about making\n> things _really_ inefficient\n> \n> Hmm. Which makes me think that maybe 'git-format-patch' is really\n> git-rewrite-generate, and 'git-am' is really git-rewrite-apply (but with\n> some extensions to preserve committer info).\n\nOf course, you miss out the merge commits.\n\n> So maybe a bad idea, but I thought I would throw it out there.\n\nNot so bad. What you describe as possibly inefficient has been \nimplemented very, very efficiently already: git-fast-import!\n\nSo your idea brings me to another idea: Why not write git-fast-export? \nActually, let's not all it that, since repo.or.cz has that name already \n(curiously enough, it is used for scripts exporting from _other_ SCMs, \nfeeding to git-fast-import...), but git-fast-dump.\n\nThe output should be _exactly_ as expected by git-fast-import, so that \n\"git fast-dump | git fast-import\" would be a nop, data-wise.\n\nOf course, there is a more fundamental problem with that approach: how to \nact on the commit message, conditional on the commit header? I know, with \nperl it would be really easy. But then you have to write a complete perl \nscript, and the whole purpose of this \nadmin-rewritehist/filter-branch/rewrite-commits frackass is that it should \nbe _easy_ to use.\n\nCiao,\nDscho\n"},{"id":"46865","messageId":"20070709134918.GT1528MdfPADPa@greensroom.kotnet.org","threadId":"8928","inReplyTo":"Pine.LNX.4.64.0707091257470.5546@racer.site","subject":"Re: [PATCH 4/4] Add git-rewrite-commits","fromName":"Sven Verdoolaege","fromEmail":"skimo@kotnet.org","sentAt":"2007-07-09T13:49:18Z","receivedAt":"2007-07-09T13:49:18Z","isPatch":true,"sender":{"key":"skimo@kotnet.org","avatar":null},"body":"On Mon, Jul 09, 2007 at 01:57:10PM +0100, Johannes Schindelin wrote:\n> On Mon, 9 Jul 2007, Sven Verdoolaege wrote:\n> > On Mon, Jul 09, 2007 at 12:56:04AM +0100, Johannes Schindelin wrote:\n> > I thought you had to specify the name of the new branch on the command\n> > line. Anyway, I don't really care about the name of this hierarchy.\n> > I just picked a name that is somewhat related to \"rewrite-commits\".\n> > Suggestions are welcome.\n> \n> How much more explicit should I formulate my suggestion?\n[..]\n> actually changed. So even if you _have_ to keep your current behaviour, \n> namely that the results are stored under the original names, you should \n> store the original refs in something like refs/original (overridable by a \n\nI suppose I can live with \"original\".\n\n> command line option), but _only if_ they actually changed.\n\nThat's what I do right now.\n\n> You will see that both definitions apply _perfectly_ to what you do here. \n> You pipe the commit into the \"map\" and write it out as a new commit! \n> The _perfect_ example for a UNIX filter.\n\nOK. Makes sense.\nI've always found that naming confusing, but it has a long tradition.\nSorry for the strong word.\n\n> > > - The example you give with \"git update-index --remove\" can fail, right? \n> > \n> > Yes. Spectacularly, even.\n> \n> Does that mean you acknowledge that the man page should tell about this? \n> And probably also reveal the \"|| :\" remedy?\n\nYes. (Am I on trial, here?)\n\n> > > - The commit filter again deviates from the usage in cg-admin-rewritehist. \n> > > I can see that you wanted to make it more versatile. However, it makes \n> > > the tool potentially also a bit more cumbersome to use. Besides, you use \n> > > a temporary file where there is no need to.\n> > \n> > Are you saying I should use two pipes?\n> \n> Umm. Why not?\n> \n> > What if the commit message is larger than the pipe buffer?\n> \n> You start_command(). Then you write() until it is all written, or the \n> pipe is broken. Then you get the output via index_pipe(). Which is a \n> single sha1. I do not understand your question.\n\nAh, but that is not how my commit \"filter\" works right now.\nIt should produce the whole commit on stdout (as mentioned in the documentation).\nSo the default filter is \"cat\" (and not \"git-hash-object -t commit --stdin\",\nas you seem to assume).\n\nHow about I change that to a filter that accepts a single SHA1 and\nproduces zero or more SHA1's as output?\nA filter for the current rewrite-commits would then be replaced\nby \"xargs | git-cat-file commit | original-filter | git-hash-object -t commit --stdin\"\n\n> To enhance on the above example: you're rewriting the commit messages so \n> that commit names are rewritten to match the rewritten commits. That is \n> possible by a message filter in cg-admin-rewritehist.\n> \n> But now somebody comes along, and says \"I have a history I need to \n> rewrite. All bug fixes. The commit names were all abbreviated in the \n> commit messages, but they always had 'commit ' in front of them. I want \n> to rewrite them, too.\"\n\nThe current git-rewrite-commits will rewrite all SHA1's it can find,\nirrespective of any 'commit ' that may precede it.\nI guess I'm trying to misunderstand you again.\n\n> Are you trying to misunderstand me?\n\nNo.\n\n> > > But hey, maybe it _is_ time to rethink the whole filter business, and \n> > > introduce some kind of regular expression based action language. \n> > > Something like\n> > > \n> > > \tgit rewrite-commits -e '/^author Darl McBribe/skip-commit' \\\n> > \n> > What's wrong with --author='!Darl McBribe' ?\n> \n> It is a very special use case. Not always will you be able to get all the \n> information from the commit object you need for conditional operations. \n> My example only showed that you can do the same with that syntax. But \n> imagine what you could do if we just added a small syntactical sugar:\n> \n> \t-e '?has-path:README?substitute/v2/v3/'\n\nSo you want to introduce a whole language?\nIsn't that a bit over-engineering?\n\nskimo\n"},{"id":"46867","messageId":"Pine.LNX.4.64.0707091450440.5546@racer.site","threadId":"8928","inReplyTo":"20070709134918.GT1528MdfPADPa@greensroom.kotnet.org","subject":"Re: [PATCH 4/4] Add git-rewrite-commits","fromName":"Johannes Schindelin","fromEmail":"johannes.schindelin@gmx.de","sentAt":"2007-07-09T14:11:45Z","receivedAt":"2007-07-09T14:11:45Z","isPatch":true,"sender":{"key":"johannes.schindelin@gmx.de","avatar":"https://avatars.githubusercontent.com/u/127790?v=4"},"body":"Hi,\n\nOn Mon, 9 Jul 2007, Sven Verdoolaege wrote:\n\n> On Mon, Jul 09, 2007 at 01:57:10PM +0100, Johannes Schindelin wrote:\n>\n> > On Mon, 9 Jul 2007, Sven Verdoolaege wrote:\n> >\n> > > What if the commit message is larger than the pipe buffer?\n> > \n> > You start_command(). Then you write() until it is all written, or the \n> > pipe is broken. Then you get the output via index_pipe(). Which is a \n> > single sha1. I do not understand your question.\n> \n> Ah, but that is not how my commit \"filter\" works right now. It should \n> produce the whole commit on stdout (as mentioned in the documentation). \n> So the default filter is \"cat\" (and not \"git-hash-object -t commit \n> --stdin\", as you seem to assume).\n\nSorry. Somehow I misread that index_pipe() for a read_pipe().\n\n> How about I change that to a filter that accepts a single SHA1 and\n> produces zero or more SHA1's as output?\n> A filter for the current rewrite-commits would then be replaced\n> by \"xargs | git-cat-file commit | original-filter | git-hash-object -t commit --stdin\"\n\nYou still have not addressed the fundamental problem! If the commit \nfilter takes the things as a whole commit buffer or as a single sha1, or \nif it writes one, two or -1 commits is not really a fundamental problem.\n\nThe real problem is that this filter can only act on _one_ commit. Yes, \nit wants to change only one commit at a time, but it might need \ninformation from _other_ commits, too!\n\n> > To enhance on the above example: you're rewriting the commit messages \n> > so that commit names are rewritten to match the rewritten commits. \n> > That is possible by a message filter in cg-admin-rewritehist.\n> > \n> > But now somebody comes along, and says \"I have a history I need to \n> > rewrite. All bug fixes. The commit names were all abbreviated in the \n> > commit messages, but they always had 'commit ' in front of them. I \n> > want to rewrite them, too.\"\n> \n> The current git-rewrite-commits will rewrite all SHA1's it can find,\n> irrespective of any 'commit ' that may precede it.\n\nEven abbreviated ones?\n\n> > > What's wrong with --author='!Darl McBribe' ?\n> > \n> > It is a very special use case. Not always will you be able to get all \n> > the information from the commit object you need for conditional \n> > operations. My example only showed that you can do the same with that \n> > syntax. But imagine what you could do if we just added a small \n> > syntactical sugar:\n> > \n> > \t-e '?has-path:README?substitute/v2/v3/'\n> \n> So you want to introduce a whole language?\n> Isn't that a bit over-engineering?\n\nWe already have a tool which is powerful enough to do that. Yes, it is a \nlittle complicated to operate, and yes, it is slower than your version. \nBut darn it, it _does_ more than your version.\n\nIt is a pity that you did not address the fundamental problem, so I have \nto spend time (that I should really spend differently) thinking about it.\n\nAlas, I think I have a solution. You need a flag:\n\n\t--write-sha1-mappings=\n\nYes, it makes your code slower again, but only if you need those mappings.\n\nYes, it is way less convenient than the \"map\" function, but then you could \nautomatically write a script into that directory, providing the \nconvenience functions, and exporting the path so that you can say\n\n\t--commit-filter '. \"$G\"; ... map \"$sha1\"'\n\nand also add convenience functions \"save \" and \"restore \n\" so that finally a bit of convenience is restored to the filter \nwriters.\n\nBut those are no longer fundamental problems. I'd be glad if you could \nput these suggestions to use in rewrite-commits.\n\nWhile at it, you should also change the semantics for the commit filter, \nprobably even less so than you suggested: since you rewrite the parents, \nand the tree, you should continue to pipe this information into the commit \nfilter. That would be another place for a convenience function \"commit\", \nwhich does the same as \"git hash-object -w --stdin\".\n\nOkay now. To be precise, here is my wish list:\n\n* rename the darned things to \"filter\" again.\n\n* --write-sha1-mappings= (or --write-commit-mappings), possibly \n defaulting to .git/mappings/. Be careful not to overwrite an existing \n such directory.\n\n* change the semantics of the commit filter: the output is a list \n (possibly empty) of replacement sha1's for this commit.\n\n* if any filters are called, provide a script with convenience functions, \n and an environment variable pointing to it. These functions should \n include:\n\n\t* map\n\t* commit\n\t* save\n\t* restore\n\nCiao,\nDscho\n"},{"id":"46869","messageId":"20070709144248.GW1528MdfPADPa@greensroom.kotnet.org","threadId":"8928","inReplyTo":"Pine.LNX.4.64.0707091450440.5546@racer.site","subject":"Re: [PATCH 4/4] Add git-rewrite-commits","fromName":"Sven Verdoolaege","fromEmail":"skimo@kotnet.org","sentAt":"2007-07-09T14:42:48Z","receivedAt":"2007-07-09T14:42:48Z","isPatch":true,"sender":{"key":"skimo@kotnet.org","avatar":null},"body":"On Mon, Jul 09, 2007 at 03:11:45PM +0100, Johannes Schindelin wrote:\n> On Mon, 9 Jul 2007, Sven Verdoolaege wrote:\n> > The current git-rewrite-commits will rewrite all SHA1's it can find,\n> > irrespective of any 'commit ' that may precede it.\n> \n> Even abbreviated ones?\n\nYes. I'll add that to the documentation.\n\n> * rename the darned things to \"filter\" again.\n> \n> * --write-sha1-mappings= (or --write-commit-mappings), possibly \n> defaulting to .git/mappings/. Be careful not to overwrite an existing \n> such directory.\n> \n> * change the semantics of the commit filter: the output is a list \n> (possibly empty) of replacement sha1's for this commit.\n> \n> * if any filters are called, provide a script with convenience functions, \n> and an environment variable pointing to it. These functions should \n> include:\n> \n> \t* map\n> \t* commit\n> \t* save\n> \t* restore\n\nHmm... you're tricking me into write shell code.\n\nskimo\n"},{"id":"46870","messageId":"Pine.LNX.4.64.0707091558090.5546@racer.site","threadId":"8928","inReplyTo":"20070709144248.GW1528MdfPADPa@greensroom.kotnet.org","subject":"Re: [PATCH 4/4] Add git-rewrite-commits","fromName":"Johannes Schindelin","fromEmail":"johannes.schindelin@gmx.de","sentAt":"2007-07-09T14:59:48Z","receivedAt":"2007-07-09T14:59:48Z","isPatch":true,"sender":{"key":"johannes.schindelin@gmx.de","avatar":"https://avatars.githubusercontent.com/u/127790?v=4"},"body":"Hi,\n\nOn Mon, 9 Jul 2007, Sven Verdoolaege wrote:\n\n> On Mon, Jul 09, 2007 at 03:11:45PM +0100, Johannes Schindelin wrote:\n> > On Mon, 9 Jul 2007, Sven Verdoolaege wrote:\n> > > The current git-rewrite-commits will rewrite all SHA1's it can find,\n> > > irrespective of any 'commit ' that may precede it.\n> > \n> > Even abbreviated ones?\n> \n> Yes. I'll add that to the documentation.\n\nThat is definitely something you want to control. I have seen commit \nmessages referencing certain hexadecimal numbers, and they were definitely \nno commit names. The shorter they are, the more likely they are to be \nrewritten by your magic.\n\n> > * rename the darned things to \"filter\" again.\n> > \n> > * --write-sha1-mappings= (or --write-commit-mappings), possibly \n> > defaulting to .git/mappings/. Be careful not to overwrite an existing \n> > such directory.\n> > \n> > * change the semantics of the commit filter: the output is a list \n> > (possibly empty) of replacement sha1's for this commit.\n> > \n> > * if any filters are called, provide a script with convenience functions, \n> > and an environment variable pointing to it. These functions should \n> > include:\n> > \n> > \t* map\n> > \t* commit\n> > \t* save\n> > \t* restore\n> \n> Hmm... you're tricking me into write shell code.\n\nAh, oh well. If you do the rest, I'll do the shell code.\n\nCiao,\nDscho\n"}]}