{"thread":{"id":"34587","subject":"Rewriting git-repack.sh in C","startedAt":"2013-08-02T13:48:59Z","lastAt":"2013-08-08T06:40:51Z","messageCount":17,"participants":["Stefan Beller","Duy Nguyen","Fredrik Gustafsson","Matthieu Moy","Junio C Hamano","Antoine Pelisse"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"224449","messageId":"51FBB8CB.8020600@googlemail.com","threadId":"34587","inReplyTo":null,"subject":"Rewriting git-repack.sh in C","fromName":"Stefan Beller","fromEmail":"stefanbeller@googlemail.com","sentAt":"2013-08-02T13:48:59Z","receivedAt":"2013-08-02T13:48:59Z","isPatch":false,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"Hello,\n\nI'd like to rewrite the repack shell script in C.\nSo I tried the naive approach reading the man page and \nthe script itself and write C program by matching each block/line \nof the script with a function in C\n\nNow I stumble upon other git commands (git pack-objects).\nWhat's the best way to approach such a plumbing command?\n\nI don't think just calling cmd_pack_objects(argc, **argv) would \nbe the right thing to do, as we're not using all the command \nline parameters, so some of the logic in cmd_pack_object could \nbe skipped.\nAnother approach would be to use some of the functions as used \nby cmd_pack_objects, but these mostly reside in builtin/pack_objects.c\nThey'd need to be moved up to pack.h/pack.c.\n\nSo my question is, how you'd generally approach rewriting a \nshell script in C.\n\nStefan\n\n\n\n\n\n"},{"id":"224450","messageId":"CACsJy8CaTA2vT0CxOAm0FacCWjNDJjZhg6mwSyspTChia-5ppQ@mail.gmail.com","threadId":"34587","inReplyTo":"51FBB8CB.8020600@googlemail.com","subject":"Re: Rewriting git-repack.sh in C","fromName":"Duy Nguyen","fromEmail":"pclouds@gmail.com","sentAt":"2013-08-02T14:10:59Z","receivedAt":"2013-08-02T14:10:59Z","isPatch":false,"sender":{"key":"pclouds@gmail.com","avatar":"https://avatars.githubusercontent.com/u/720?v=4"},"body":"On Fri, Aug 2, 2013 at 8:48 PM, Stefan Beller\n<stefanbeller@googlemail.com> wrote:\n> Hello,\n>\n> I'd like to rewrite the repack shell script in C.\n> So I tried the naive approach reading the man page and\n> the script itself and write C program by matching each block/line\n> of the script with a function in C\n>\n> Now I stumble upon other git commands (git pack-objects).\n> What's the best way to approach such a plumbing command?\n>\n> I don't think just calling cmd_pack_objects(argc, **argv) would\n> be the right thing to do, as we're not using all the command\n> line parameters, so some of the logic in cmd_pack_object could\n> be skipped.\n> Another approach would be to use some of the functions as used\n> by cmd_pack_objects, but these mostly reside in builtin/pack_objects.c\n> They'd need to be moved up to pack.h/pack.c.\n>\n> So my question is, how you'd generally approach rewriting a\n> shell script in C.\n\nStart a new process via start_command/run_command interface. It's\nsafer to retain the process boundary at this stage. You can try to\nintegrate further later.\n-- \nDuy\n"},{"id":"224456","messageId":"20130802163659.GA28693@lanh","threadId":"34587","inReplyTo":"CACsJy8CaTA2vT0CxOAm0FacCWjNDJjZhg6mwSyspTChia-5ppQ@mail.gmail.com","subject":"Re: Rewriting git-repack.sh in C","fromName":"Duy Nguyen","fromEmail":"pclouds@gmail.com","sentAt":"2013-08-02T16:36:59Z","receivedAt":"2013-08-02T16:36:59Z","isPatch":false,"sender":{"key":"pclouds@gmail.com","avatar":"https://avatars.githubusercontent.com/u/720?v=4"},"body":"On Fri, Aug 02, 2013 at 09:10:59PM +0700, Duy Nguyen wrote:\n> On Fri, Aug 2, 2013 at 8:48 PM, Stefan Beller\n> <stefanbeller@googlemail.com> wrote:\n> > Hello,\n> >\n> > I'd like to rewrite the repack shell script in C.\n> > So I tried the naive approach reading the man page and\n> > the script itself and write C program by matching each block/line\n> > of the script with a function in C\n> >\n> > ...\n> >\n> > So my question is, how you'd generally approach rewriting a\n> > shell script in C.\n> \n> Start a new process via start_command/run_command interface. It's\n> safer to retain the process boundary at this stage. You can try to\n> integrate further later.\n\nI was in the middle of something and somehow read this as \"rewriting\ngit-rebase.sh\" :-X\n\nFor git-repack, because it ends with a single pack-objects call, we\nmight not need a new process after all (very much like tail call\noptimization). This is what I got for some time, but still not polish\nit for submission (and it may be a bit broken after the last\nrebase). Maybe you can work off this, or from scratch if you think\nit's too messy. It basically teaches pack-objects extra features that\nrepack needs, then make repack a wrapper of pack-objects.\n\n-- 8< --\ncommit 25569a3958c3272b3eb5fa50dea680948f7a2768 (build-in-repack)\nAuthor: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>\nDate:   Wed Nov 9 19:21:39 2011 +0700\n\n    Build in git-repack\n    \n    pack-objects learns a few more options to take over what's been done\n    by git-repack.sh. cmd_repack() becomes a wrapper around\n    cmd_pack_objects().\n\ndiff --git a/Makefile b/Makefile\nindex 0f931a2..b4010a6 100644\n--- a/Makefile\n+++ b/Makefile\n@@ -460,7 +460,6 @@ SCRIPT_SH += git-mergetool.sh\n SCRIPT_SH += git-pull.sh\n SCRIPT_SH += git-quiltimport.sh\n SCRIPT_SH += git-rebase.sh\n-SCRIPT_SH += git-repack.sh\n SCRIPT_SH += git-request-pull.sh\n SCRIPT_SH += git-stash.sh\n SCRIPT_SH += git-submodule.sh\n@@ -584,6 +583,7 @@ BUILT_INS += git-init$X\n BUILT_INS += git-merge-subtree$X\n BUILT_INS += git-peek-remote$X\n BUILT_INS += git-repo-config$X\n+BUILT_INS += git-repack$X\n BUILT_INS += git-show$X\n BUILT_INS += git-stage$X\n BUILT_INS += git-status$X\ndiff --git a/builtin.h b/builtin.h\nindex 64bab6b..feb958f 100644\n--- a/builtin.h\n+++ b/builtin.h\n@@ -117,6 +117,7 @@ extern int cmd_reflog(int argc, const char **argv, const char *prefix);\n extern int cmd_remote(int argc, const char **argv, const char *prefix);\n extern int cmd_remote_ext(int argc, const char **argv, const char *prefix);\n extern int cmd_remote_fd(int argc, const char **argv, const char *prefix);\n+extern int cmd_repack(int argc, const char **argv, const char *prefix);\n extern int cmd_repo_config(int argc, const char **argv, const char *prefix);\n extern int cmd_rerere(int argc, const char **argv, const char *prefix);\n extern int cmd_reset(int argc, const char **argv, const char *prefix);\ndiff --git a/builtin/pack-objects.c b/builtin/pack-objects.c\nindex f069462..1742ea1 100644\n--- a/builtin/pack-objects.c\n+++ b/builtin/pack-objects.c\n@@ -18,10 +18,17 @@\n #include \"refs.h\"\n #include \"streaming.h\"\n #include \"thread-utils.h\"\n+#include \"sigchain.h\"\n \n static const char *pack_usage[] = {\n \tN_(\"git pack-objects --stdout [options...] [< ref-list | < object-list]\"),\n \tN_(\"git pack-objects [options...] base-name [< ref-list | < object-list]\"),\n+\tN_(\"git pack-objects --repack [options...]\"),\n+\tNULL\n+};\n+\n+static char const * const repack_usage[] = {\n+\tN_(\"git repack [options]\"),\n \tNULL\n };\n \n@@ -103,6 +110,15 @@ static struct object_entry *locate_object_entry(const unsigned char *sha1);\n static uint32_t written, written_delta;\n static uint32_t reused, reused_delta;\n \n+#define REPACK_IN_PROGRESS\t\t(1 << 0)\n+#define REPACK_UPDATE_INFO\t\t(1 << 1)\n+#define REPACK_ALL_INTO_ONE\t\t(1 << 2)\n+#define REPACK_REMOVE_REDUNDANT\t\t(1 << 3)\n+\n+static int repack_flags, nr_written_packs;\n+static int repack_usedeltabaseoffset;\n+static struct string_list written_packs;\n+static struct string_list backup_files;\n \n static void *get_delta(struct object_entry *entry)\n {\n@@ -792,9 +808,19 @@ static void write_pack_file(void)\n \t\t\tsnprintf(tmpname, sizeof(tmpname), \"%s-\", base_name);\n \t\t\tfinish_tmp_packfile(tmpname, pack_tmp_name,\n \t\t\t\t\t    written_list, nr_written,\n-\t\t\t\t\t    &pack_idx_opts, sha1);\n+\t\t\t\t\t    &pack_idx_opts, sha1,\n+\t\t\t\t\t    repack_flags & REPACK_IN_PROGRESS ?\n+\t\t\t\t\t    &backup_files : NULL);\n \t\t\tfree(pack_tmp_name);\n-\t\t\tputs(sha1_to_hex(sha1));\n+\t\t\tif (repack_flags & REPACK_IN_PROGRESS) {\n+\t\t\t\tint len = strlen(tmpname);\n+\t\t\t\tchar *s = xmalloc(len + 2);\n+\t\t\t\tmemcpy(s, tmpname, len - 4);\n+\t\t\t\tmemcpy(s + len - 4, \".pack\", 6);\n+\t\t\t\tstring_list_append(&written_packs, s);\n+\t\t\t\tnr_written_packs++;\n+\t\t\t} else\n+\t\t\t\tputs(sha1_to_hex(sha1));\n \t\t}\n \n \t\t/* mark written objects as written to previous pack */\n@@ -2359,7 +2385,8 @@ static void get_object_list(int ac, const char **av)\n \tsave_commit_buffer = 0;\n \tsetup_revisions(ac, av, &revs, NULL);\n \n-\twhile (fgets(line, sizeof(line), stdin) != NULL) {\n+\twhile (!(repack_flags & REPACK_IN_PROGRESS) &&\n+\t       fgets(line, sizeof(line), stdin) != NULL) {\n \t\tint len = strlen(line);\n \t\tif (len && line[len - 1] == '\\n')\n \t\t\tline[--len] = 0;\n@@ -2387,6 +2414,30 @@ static void get_object_list(int ac, const char **av)\n \t\tloosen_unused_packed_objects(&revs);\n }\n \n+static void rollback_repack(void)\n+{\n+\tstruct strbuf dst = STRBUF_INIT;\n+\tint i, ret;\n+\tfor (i = 0; i < backup_files.nr; i++) {\n+\t\tconst char *src = backup_files.items[i].string;\n+\t\tstrbuf_addstr(&dst, src);\n+\t\tstrbuf_setlen(&dst, dst.len - 4); /* remove .old */\n+\t\tret = rename(src, dst.buf);\n+\t\tif (ret)\n+\t\t\twarning(\"failed to restore %s: %s\", src, strerror(errno));\n+\t\tstrbuf_setlen(&dst, 0);\n+\t}\n+\tstrbuf_release(&dst);\n+\tstring_list_clear(&backup_files, 0);\n+}\n+\n+static void rollback_repack_on_signal(int signo)\n+{\n+\trollback_repack();\n+\tsigchain_pop(signo);\n+\traise(signo);\n+}\n+\n static int option_parse_index_version(const struct option *opt,\n \t\t\t\t      const char *arg, int unset)\n {\n@@ -2436,11 +2487,12 @@ static int option_parse_ulong(const struct option *opt,\n \n int cmd_pack_objects(int argc, const char **argv, const char *prefix)\n {\n+\tstruct strbuf repack_base_name = STRBUF_INIT;\n \tint use_internal_rev_list = 0;\n \tint thin = 0;\n \tint all_progress_implied = 0;\n \tconst char *rp_av[6];\n-\tint rp_ac = 0;\n+\tint i, rp_ac = 0;\n \tint rev_list_unpacked = 0, rev_list_all = 0, rev_list_reflog = 0;\n \tstruct option pack_objects_options[] = {\n \t\tOPT_SET_INT('q', \"quiet\", &progress,\n@@ -2505,6 +2557,16 @@ int cmd_pack_objects(int argc, const char **argv, const char *prefix)\n \t\t\t    N_(\"pack compression level\")),\n \t\tOPT_SET_INT(0, \"keep-true-parents\", &grafts_replace_parents,\n \t\t\t    N_(\"do not hide commits by grafts\"), 0),\n+\n+\t\tOPT_BIT(0, \"repack\", &repack_flags,\n+\t\t\tN_(\"repack mode\"), REPACK_IN_PROGRESS),\n+\t\tOPT_BIT(0, \"repack-all\", &repack_flags,\n+\t\t\tN_(\"repack everything into one pack\"), REPACK_ALL_INTO_ONE),\n+\t\tOPT_BIT(0, \"remove-redundant\", &repack_flags,\n+\t\t\tN_(\"remove redundant objects after repack\"), REPACK_REMOVE_REDUNDANT),\n+\t\tOPT_BIT(0, \"update-info\", &repack_flags,\n+\t\t\tN_(\"run git-update-server-info after repack\"), REPACK_UPDATE_INFO),\n+\n \t\tOPT_END(),\n \t};\n \n@@ -2556,6 +2618,36 @@ int cmd_pack_objects(int argc, const char **argv, const char *prefix)\n \tif (delta_search_threads != 1)\n \t\twarning(\"no threads support, ignoring --threads\");\n #endif\n+\tif ((repack_flags & REPACK_IN_PROGRESS) == 0 &&\n+\t    (repack_flags & ~REPACK_IN_PROGRESS))\n+\t\tdie(\"--repack must be given for any repack related options\");\n+\tif (repack_flags & REPACK_IN_PROGRESS) {\n+\t\tif (pack_to_stdout)\n+\t\t\tdie(\"--stdout cannot be used with --repack\");\n+\t\tif (argc)\n+\t\t\tdie(\"base name cannot be used with --repack\");\n+\n+\t\trp_av[rp_ac++] = \"--all\";\n+\t\trp_av[rp_ac++] = \"--reflog\";\n+\t\tuse_internal_rev_list = 1;\n+\n+\t\tgrafts_replace_parents = 0; /* --keep-true-parents */\n+\t\tignore_packed_keep = 1;\t    /* --honor-pack-keep */\n+\t\tnon_empty = 1;\t\t    /* --non-empty */\n+\n+\t\tif (!(repack_flags & REPACK_ALL_INTO_ONE)) {\n+\t\t\tincremental = 1; /* --incremental */\n+\t\t\trp_av[rp_ac++] = \"--unpacked\";\n+\t\t}\n+\n+\t\tstrbuf_addf(&repack_base_name,\n+\t\t\t    \"%s/pack/pack\", get_object_directory());\n+\t\tbase_name = repack_base_name.buf;\n+\n+\t\tsigchain_push_common(rollback_repack_on_signal);\n+\t\tatexit(rollback_repack);\n+\t}\n+\n \tif (!pack_to_stdout && !pack_size_limit)\n \t\tpack_size_limit = pack_size_limit_cfg;\n \tif (pack_to_stdout && pack_size_limit)\n@@ -2598,5 +2690,184 @@ int cmd_pack_objects(int argc, const char **argv, const char *prefix)\n \t\tfprintf(stderr, \"Total %\"PRIu32\" (delta %\"PRIu32\"),\"\n \t\t\t\" reused %\"PRIu32\" (delta %\"PRIu32\")\\n\",\n \t\t\twritten, written_delta, reused, reused_delta);\n+\n+\tif (!(repack_flags & REPACK_IN_PROGRESS))\n+\t\treturn 0;\n+\n+\tif (!nr_written_packs) {\n+\t\tprintf(_(\"Nothing new to pack.\\n\"));\n+\t\treturn 0;\n+\t}\n+\n+\t/* At this point all new packs should be in place. We can\n+\t   safely remove old ones */\n+\tfor (i = 0; i < backup_files.nr; i++) {\n+\t\tconst char *s = backup_files.items[i].string;\n+\t\tint ret = unlink(s);\n+\t\tif (ret)\n+\t\t\twarning(\"failed to remove %s: %s\", s, strerror(errno));\n+\t}\n+\tstring_list_clear(&backup_files, 0);\n+\n+\tif (repack_flags & REPACK_REMOVE_REDUNDANT) {\n+\t\tstruct packed_git *p;\n+\t\tstruct string_list to_be_removed = STRING_LIST_INIT_DUP;\n+\n+\t\t/* free_pack_by_name() may have freed a few packs in\n+\t\t   write_pack_file() */\n+\t\treprepare_packed_git();\n+\t\tfor (p = packed_git; p; p = p->next) {\n+\t\t\tif (!p->pack_local || p->pack_keep)\n+\t\t\t\tcontinue;\n+\n+\t\t\tfor (i = 0; i < written_packs.nr; i++) {\n+\t\t\t\tchar *s = written_packs.items[i].string;\n+\t\t\t\tif (!strcmp(s, p->pack_name))\n+\t\t\t\t\tbreak;\n+\t\t\t}\n+\t\t\tif (i < written_packs.nr)\n+\t\t\t\tcontinue;\n+\n+\t\t\tstring_list_append(&to_be_removed, p->pack_name);\n+\t\t}\n+\t\twritten_packs.strdup_strings = 1;\n+\t\tstring_list_clear(&written_packs, 0);\n+\n+\t\tfor (i = 0; i < to_be_removed.nr; i++) {\n+\t\t\tchar *path = to_be_removed.items[i].string;\n+\n+\t\t\t/* Windows limitation on unlink().\n+\t\t\t   See c74faea19e39ca933492f697596310397175c329 */\n+\t\t\tfree_pack_by_name(path);\n+\n+\t\t\tif (unlink(path))\n+\t\t\t\twarning(\"failed to remove %s: %s\", path, strerror(errno));\n+\t\t\tstrcpy(path + strlen(path)-5, \".idx\");\n+\t\t\tif (unlink(path))\n+\t\t\t\twarning(\"failed to remove %s: %s\", path, strerror(errno));\n+\t\t}\n+\t\tstring_list_clear(&to_be_removed, 0);\n+\n+\t\treprepare_packed_git();\n+\t\tprune_packed_objects(progress ? PRUNE_PACKED_VERBOSE : 0);\n+\t}\n+\n+\tif (repack_flags & REPACK_UPDATE_INFO)\n+\t\tupdate_server_info(0);\n+\n \treturn 0;\n }\n+\n+static int repack_config(const char *k, const char *v, void *cb)\n+{\n+\tif (!strcasecmp(k, \"repack.usedeltabaseoffset\")) {\n+\t\trepack_usedeltabaseoffset = git_config_bool(k, v);\n+\t\treturn 0;\n+\t}\n+\treturn git_default_config(k, v, cb);\n+}\n+\n+int cmd_repack(int argc, const char **argv, const char *prefix)\n+{\n+\tint all_in_one = 0;\n+\tint all_in_one_and_unreachable = 0;\n+\tint unpack_unreachable = 0;\n+\tint remove_redundant = 0;\n+\tint no_reuse_delta = 0;\n+\tint no_reuse_object = 0;\n+\tint no_update = 0;\n+\tint quiet = 0;\n+\tint local = 0;\n+\n+\tstruct option opts[] = {\n+\t\tOPT_BOOL('a', NULL, &all_in_one,\n+\t\t\t \"pack everything in a single pack\"),\n+\t\tOPT_BOOL('A', NULL, &all_in_one_and_unreachable,\n+\t\t\t \"same as -a, and turn unreachable objects loose\"),\n+\t\tOPT_BOOL('d', NULL, &remove_redundant,\n+\t\t\t \"remove redundant packs, and run git-prune-packed\"),\n+\t\tOPT_BOOL('f', NULL, &no_reuse_delta,\n+\t\t\t \"pass --no-reuse-delta to git-pack-objects\"),\n+\t\tOPT_BOOL('F', NULL, &no_reuse_object,\n+\t\t\t \"pass --no-reuse-object to git-pack-objects\"),\n+\t\tOPT_BOOL('n', NULL, &no_update,\n+\t\t\t \"do not run git-update-server-info\"),\n+\t\tOPT_BOOL('q', NULL, &quiet, \"be quiet\"),\n+\t\tOPT_BOOL('l', NULL, &local,\n+\t\t\t \"pass --local to git-pack-objects\"),\n+\t\t{ OPTION_ARGUMENT, 0, \"window\", NULL, \"n\",\n+\t\t \"size of the window used for delta compression\", 0 },\n+\t\t{ OPTION_ARGUMENT, 0, \"window-memory\", NULL, \"n\",\n+\t\t  \"same as the above, but limit memory size instead of entries count\", 0 },\n+\t\t{ OPTION_ARGUMENT, 0, \"depth\", NULL, \"n\",\n+\t\t  \"limits the maximum delta depth\", 0 },\n+\t\t{ OPTION_ARGUMENT, 0, \"max-pack-size\", NULL, \"n\",\n+\t\t  \"maximum size of each packfile\", 0},\n+\t\tOPT_END(),\n+\t};\n+\n+\tconst char *av[] = { \"pack-objects\", \"--repack\",\n+\t\t\t     NULL, /* --[no-]update-info */\n+\t\t\t     NULL, /* --delta-base-offset */\n+\t\t\t     NULL, /* --repack-all */\n+\t\t\t     NULL, /* --remove-redundant */\n+\t\t\t     NULL, /* --no-reuse-delta */\n+\t\t\t     NULL, /* --no-reuse-object */\n+\t\t\t     NULL, /* --local */\n+\t\t\t     NULL, /* -q */\n+\t\t\t     NULL, /* --unpack-unreachable */\n+\t\t\t     NULL, /* --window */\n+\t\t\t     NULL, /* --window-memory */\n+\t\t\t     NULL, /* --depth */\n+\t\t\t     NULL, /* --max-pack-size */\n+\t\t\t     NULL\n+\t};\n+\tint ac = 2;\n+\n+\tgit_config(repack_config, NULL);\n+\n+\targc = parse_options(argc, argv, prefix, opts, repack_usage, 0);\n+\n+\tif (no_update)\n+\t\tav[ac++] = \"--no-update-info\";\n+\telse\n+\t\tav[ac++] = \"--update-info\";\n+\tif (repack_usedeltabaseoffset)\n+\t\tav[ac++] = \"--delta-base-offset\";\n+\tif (all_in_one_and_unreachable) {\n+\t\tav[ac++] = \"--repack-all\";\n+\t\tall_in_one = 1;\n+\t\tunpack_unreachable = 1;\n+\t}\n+\tif (all_in_one)\n+\t\tav[ac++] = \"--repack-all\";\n+\tif (remove_redundant)\n+\t\tav[ac++] = \"--remove-redundant\";\n+\tif (no_reuse_delta)\n+\t\tav[ac++] = \"--no-reuse-delta\";\n+\tif (no_reuse_object)\n+\t\tav[ac++] = \"--no-reuse-object\";\n+\tif (local)\n+\t\tav[ac++] = \"--local\";\n+\tif (quiet)\n+\t\tav[ac++] = \"-q\";\n+\tif ((ac + argc) * sizeof(*av) > sizeof(av))\n+\t\tdie(\"Too many options\");\n+\tmemcpy(av + ac, argv, argc * sizeof(*argv));\n+\tac += argc;\n+\n+\tif (all_in_one && remove_redundant) {\n+\t\tstruct packed_git *p;\n+\n+\t\tprepare_packed_git();\n+\t\tfor (p = packed_git; p; p = p->next) {\n+\t\t\tif (!p->pack_keep &&\n+\t\t\t    unpack_unreachable && remove_redundant) {\n+\t\t\t\tav[ac++] = \"--unpack-unreachable\";\n+\t\t\t\tbreak;\n+\t\t\t}\n+\t\t}\n+\t}\n+\n+\treturn cmd_pack_objects(ac, av, prefix);\n+}\ndiff --git a/bulk-checkin.c b/bulk-checkin.c\nindex 6b0b6d4..3ca3c55 100644\n--- a/bulk-checkin.c\n+++ b/bulk-checkin.c\n@@ -46,7 +46,7 @@ static void finish_bulk_checkin(struct bulk_checkin_state *state)\n \tsprintf(packname, \"%s/pack/pack-\", get_object_directory());\n \tfinish_tmp_packfile(packname, state->pack_tmp_name,\n \t\t\t    state->written, state->nr_written,\n-\t\t\t    &state->pack_idx_opts, sha1);\n+\t\t\t    &state->pack_idx_opts, sha1, NULL);\n \tfor (i = 0; i < state->nr_written; i++)\n \t\tfree(state->written[i]);\n \ndiff --git a/git-repack.sh b/contrib/examples/git-repack.sh\nsimilarity index 100%\nrename from git-repack.sh\nrename to contrib/examples/git-repack.sh\ndiff --git a/git.c b/git.c\nindex 1ada169..db4e4b3 100644\n--- a/git.c\n+++ b/git.c\n@@ -389,6 +389,7 @@ static void handle_internal_command(int argc, const char **argv)\n \t\t{ \"remote-ext\", cmd_remote_ext },\n \t\t{ \"remote-fd\", cmd_remote_fd },\n \t\t{ \"replace\", cmd_replace, RUN_SETUP },\n+\t\t{ \"repack\", cmd_repack, RUN_SETUP },\n \t\t{ \"repo-config\", cmd_repo_config, RUN_SETUP_GENTLY },\n \t\t{ \"rerere\", cmd_rerere, RUN_SETUP },\n \t\t{ \"reset\", cmd_reset, RUN_SETUP },\ndiff --git a/pack-write.c b/pack-write.c\nindex ca9e63b..e6aa7e3 100644\n--- a/pack-write.c\n+++ b/pack-write.c\n@@ -1,6 +1,7 @@\n #include \"cache.h\"\n #include \"pack.h\"\n #include \"csum-file.h\"\n+#include \"string-list.h\"\n \n void reset_pack_idx_option(struct pack_idx_option *opts)\n {\n@@ -348,10 +349,12 @@ void finish_tmp_packfile(char *name_buffer,\n \t\t\t struct pack_idx_entry **written_list,\n \t\t\t uint32_t nr_written,\n \t\t\t struct pack_idx_option *pack_idx_opts,\n-\t\t\t unsigned char sha1[])\n+\t\t\t unsigned char sha1[],\n+\t\t\t struct string_list *backup_files)\n {\n \tconst char *idx_tmp_name;\n \tchar *end_of_name_prefix = strrchr(name_buffer, 0);\n+\tstruct stat st;\n \n \tif (adjust_shared_perm(pack_tmp_name))\n \t\tdie_errno(\"unable to make temporary pack file readable\");\n@@ -368,6 +371,14 @@ void finish_tmp_packfile(char *name_buffer,\n \t\tdie_errno(\"unable to rename temporary pack file\");\n \n \tsprintf(end_of_name_prefix, \"%s.idx\", sha1_to_hex(sha1));\n+\tif (backup_files && !stat(name_buffer, &st)) {\n+\t\tstruct strbuf old = STRBUF_INIT;\n+\t\tstrbuf_addf(&old, \"%s.old\", name_buffer);\n+\t\tif (rename(name_buffer, old.buf))\n+\t\t\tdie_errno(\"unable to rename pack %s\", name_buffer);\n+\t\tstring_list_append(backup_files, strbuf_detach(&old, NULL));\n+\t}\n+\tbackup_file(name_buffer);\n \tif (rename(idx_tmp_name, name_buffer))\n \t\tdie_errno(\"unable to rename temporary index file\");\n \ndiff --git a/pack.h b/pack.h\nindex aa6ee7d..d3f92ad 100644\n--- a/pack.h\n+++ b/pack.h\n@@ -90,7 +90,9 @@ extern int encode_in_pack_object_header(enum object_type, uintmax_t, unsigned ch\n #define PH_ERROR_PROTOCOL\t(-3)\n extern int read_pack_header(int fd, struct pack_header *);\n \n+struct string_list;\n+\n extern struct sha1file *create_tmp_packfile(char **pack_tmp_name);\n-extern void finish_tmp_packfile(char *name_buffer, const char *pack_tmp_name, struct pack_idx_entry **written_list, uint32_t nr_written, struct pack_idx_option *pack_idx_opts, unsigned char sha1[]);\n+extern void finish_tmp_packfile(char *name_buffer, const char *pack_tmp_name, struct pack_idx_entry **written_list, uint32_t nr_written, struct pack_idx_option *pack_idx_opts, unsigned char sha1[], struct string_list *backup_files);\n \n #endif\n-- 8< --\n"},{"id":"224489","messageId":"20130803063325.GJ19369@paksenarrion.iveqy.com","threadId":"34587","inReplyTo":"CACsJy8CaTA2vT0CxOAm0FacCWjNDJjZhg6mwSyspTChia-5ppQ@mail.gmail.com","subject":"Re: Rewriting git-repack.sh in C","fromName":"Fredrik Gustafsson","fromEmail":"iveqy@iveqy.com","sentAt":"2013-08-03T06:33:25Z","receivedAt":"2013-08-03T06:33:25Z","isPatch":false,"sender":{"key":"iveqy@iveqy.com","avatar":"https://avatars.githubusercontent.com/u/761743?v=4"},"body":"On Fri, Aug 02, 2013 at 09:10:59PM +0700, Duy Nguyen wrote:\n> > So my question is, how you'd generally approach rewriting a\n> > shell script in C.\n> \n> Start a new process via start_command/run_command interface. It's\n> safer to retain the process boundary at this stage. You can try to\n> integrate further later.\n\nIs it really the right approach to just replace sh with C? IMHO forking\nand wait for the result should not be done if it can be avoided. It just\nadds overhead.\n\nOf course you can argue that just replace sh with C is a good first step\ntowards to actually do the command in \"full C\". Altough I'm afraid that\nthat will get such low priority that it won't be done.\n\n-- \nMed vänliga hälsningar\nFredrik Gustafsson\n\ntel: 0733-608274\ne-post: iveqy@iveqy.com\n"},{"id":"224501","messageId":"CACsJy8A-xo97MC-vBmqCrvvSDMWKd_MxnRrTBSeSqcQ1HifzJA@mail.gmail.com","threadId":"34587","inReplyTo":"20130803063325.GJ19369@paksenarrion.iveqy.com","subject":"Re: Rewriting git-repack.sh in C","fromName":"Duy Nguyen","fromEmail":"pclouds@gmail.com","sentAt":"2013-08-03T10:03:52Z","receivedAt":"2013-08-03T10:03:52Z","isPatch":false,"sender":{"key":"pclouds@gmail.com","avatar":"https://avatars.githubusercontent.com/u/720?v=4"},"body":"On Sat, Aug 3, 2013 at 1:33 PM, Fredrik Gustafsson <iveqy@iveqy.com> wrote:\n> On Fri, Aug 02, 2013 at 09:10:59PM +0700, Duy Nguyen wrote:\n>> > So my question is, how you'd generally approach rewriting a\n>> > shell script in C.\n>>\n>> Start a new process via start_command/run_command interface. It's\n>> safer to retain the process boundary at this stage. You can try to\n>> integrate further later.\n>\n> Is it really the right approach to just replace sh with C? IMHO forking\n> and wait for the result should not be done if it can be avoided. It just\n> adds overhead.\n\nAgreed. As I said in another post, I misread this as rewriting\ngit-rebase.sh, which is a lot more complicated.\n\n> Of course you can argue that just replace sh with C is a good first step\n> towards to actually do the command in \"full C\". Altough I'm afraid that\n> that will get such low priority that it won't be done.\n\nOne of the reasons I started porting git-repack.sh since 2011 and\nnever finished it.\n-- \nDuy\n"},{"id":"224568","messageId":"vpqy58g2zpo.fsf@anie.imag.fr","threadId":"34587","inReplyTo":"51FBB8CB.8020600@googlemail.com","subject":"Re: Rewriting git-repack.sh in C","fromName":"Matthieu Moy","fromEmail":"matthieu.moy@grenoble-inp.fr","sentAt":"2013-08-05T10:34:27Z","receivedAt":"2013-08-05T10:34:27Z","isPatch":false,"sender":{"key":"matthieu.moy@grenoble-inp.fr","avatar":"https://gravatar.com/avatar/72c8a2705971a25dfaff23cece15130d405685845d911aedd5667ace277f3fc5?d=mp&s=160"},"body":"Stefan Beller <stefanbeller@googlemail.com> writes:\n\n> Hello,\n>\n> I'd like to rewrite the repack shell script in C.\n> So I tried the naive approach reading the man page and \n> the script itself and write C program by matching each block/line \n> of the script with a function in C\n\nYou should add one step here: check that tests are good, and possibly\nadd some to improve coverage. It's easier to add tests when you already\nhave a reference implementation.\n\n-- \nMatthieu Moy\nhttp://www-verimag.imag.fr/~moy/\n"},{"id":"224748","messageId":"1375884049-14961-1-git-send-email-stefanbeller@googlemail.com","threadId":"34587","inReplyTo":"CACsJy8A-xo97MC-vBmqCrvvSDMWKd_MxnRrTBSeSqcQ1HifzJA@mail.gmail.com","subject":"[PATCH 0/4] Re: Rewriting git-repack.sh in C","fromName":"Stefan Beller","fromEmail":"stefanbeller@googlemail.com","sentAt":"2013-08-07T14:00:45Z","receivedAt":"2013-08-07T14:00:45Z","isPatch":true,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"Hi Duy,\n\nI applied your patch on the current master and added 3 patches, \nso git compiles and the testsuite works without additional breakages.\n\nThe functionality is obviously not yet completed as the backup_file\nfunction is still empty. What do you intend that function will do?\n\nStefan\n\nNguyễn Thái Ngọc Duy (1):\n  Build in git-repack\n\nStefan Beller (3):\n  backup_file dummy function\n  pack-objects: do not print usage when repacking\n  repack: add unpack-unreachable\n\n Makefile                       |   2 +-\n builtin.h                      |   1 +\n builtin/pack-objects.c         | 284 ++++++++++++++++++++++++++++++++++++++++-\n bulk-checkin.c                 |   2 +-\n contrib/examples/git-repack.sh | 194 ++++++++++++++++++++++++++++\n git-repack.sh                  | 194 ----------------------------\n git.c                          |   1 +\n pack-write.c                   |  18 ++-\n pack.h                         |   4 +-\n 9 files changed, 497 insertions(+), 203 deletions(-)\n create mode 100755 contrib/examples/git-repack.sh\n delete mode 100755 git-repack.sh\n\n-- \n1.8.4.rc1.25.g2793d0a\n"},{"id":"224749","messageId":"1375884049-14961-2-git-send-email-stefanbeller@googlemail.com","threadId":"34587","inReplyTo":"1375884049-14961-1-git-send-email-stefanbeller@googlemail.com","subject":"[PATCH 1/4] Build in git-repack","fromName":"Stefan Beller","fromEmail":"stefanbeller@googlemail.com","sentAt":"2013-08-07T14:00:46Z","receivedAt":"2013-08-07T14:00:46Z","isPatch":true,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"From: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>\n\npack-objects learns a few more options to take over what's been done\nby git-repack.sh. cmd_repack() becomes a wrapper around\ncmd_pack_objects().\n---\n Makefile                       |   2 +-\n builtin.h                      |   1 +\n builtin/pack-objects.c         | 279 ++++++++++++++++++++++++++++++++++++++++-\n bulk-checkin.c                 |   2 +-\n contrib/examples/git-repack.sh | 194 ++++++++++++++++++++++++++++\n git-repack.sh                  | 194 ----------------------------\n git.c                          |   1 +\n pack-write.c                   |  13 +-\n pack.h                         |   4 +-\n 9 files changed, 488 insertions(+), 202 deletions(-)\n create mode 100755 contrib/examples/git-repack.sh\n delete mode 100755 git-repack.sh\n\ndiff --git a/Makefile b/Makefile\nindex 3588ca1..8bd122b 100644\n--- a/Makefile\n+++ b/Makefile\n@@ -464,7 +464,6 @@ SCRIPT_SH += git-pull.sh\n SCRIPT_SH += git-quiltimport.sh\n SCRIPT_SH += git-rebase.sh\n SCRIPT_SH += git-remote-testgit.sh\n-SCRIPT_SH += git-repack.sh\n SCRIPT_SH += git-request-pull.sh\n SCRIPT_SH += git-stash.sh\n SCRIPT_SH += git-submodule.sh\n@@ -596,6 +595,7 @@ BUILT_INS += git-get-tar-commit-id$X\n BUILT_INS += git-init$X\n BUILT_INS += git-merge-subtree$X\n BUILT_INS += git-peek-remote$X\n+BUILT_INS += git-repack$X\n BUILT_INS += git-repo-config$X\n BUILT_INS += git-show$X\n BUILT_INS += git-stage$X\ndiff --git a/builtin.h b/builtin.h\nindex 8afa2de..b56cf07 100644\n--- a/builtin.h\n+++ b/builtin.h\n@@ -102,6 +102,7 @@ extern int cmd_reflog(int argc, const char **argv, const char *prefix);\n extern int cmd_remote(int argc, const char **argv, const char *prefix);\n extern int cmd_remote_ext(int argc, const char **argv, const char *prefix);\n extern int cmd_remote_fd(int argc, const char **argv, const char *prefix);\n+extern int cmd_repack(int argc, const char **argv, const char *prefix);\n extern int cmd_repo_config(int argc, const char **argv, const char *prefix);\n extern int cmd_rerere(int argc, const char **argv, const char *prefix);\n extern int cmd_reset(int argc, const char **argv, const char *prefix);\ndiff --git a/builtin/pack-objects.c b/builtin/pack-objects.c\nindex f069462..1742ea1 100644\n--- a/builtin/pack-objects.c\n+++ b/builtin/pack-objects.c\n@@ -18,10 +18,17 @@\n #include \"refs.h\"\n #include \"streaming.h\"\n #include \"thread-utils.h\"\n+#include \"sigchain.h\"\n \n static const char *pack_usage[] = {\n \tN_(\"git pack-objects --stdout [options...] [< ref-list | < object-list]\"),\n \tN_(\"git pack-objects [options...] base-name [< ref-list | < object-list]\"),\n+\tN_(\"git pack-objects --repack [options...]\"),\n+\tNULL\n+};\n+\n+static char const * const repack_usage[] = {\n+\tN_(\"git repack [options]\"),\n \tNULL\n };\n \n@@ -103,6 +110,15 @@ static struct object_entry *locate_object_entry(const unsigned char *sha1);\n static uint32_t written, written_delta;\n static uint32_t reused, reused_delta;\n \n+#define REPACK_IN_PROGRESS\t\t(1 << 0)\n+#define REPACK_UPDATE_INFO\t\t(1 << 1)\n+#define REPACK_ALL_INTO_ONE\t\t(1 << 2)\n+#define REPACK_REMOVE_REDUNDANT\t\t(1 << 3)\n+\n+static int repack_flags, nr_written_packs;\n+static int repack_usedeltabaseoffset;\n+static struct string_list written_packs;\n+static struct string_list backup_files;\n \n static void *get_delta(struct object_entry *entry)\n {\n@@ -792,9 +808,19 @@ static void write_pack_file(void)\n \t\t\tsnprintf(tmpname, sizeof(tmpname), \"%s-\", base_name);\n \t\t\tfinish_tmp_packfile(tmpname, pack_tmp_name,\n \t\t\t\t\t    written_list, nr_written,\n-\t\t\t\t\t    &pack_idx_opts, sha1);\n+\t\t\t\t\t    &pack_idx_opts, sha1,\n+\t\t\t\t\t    repack_flags & REPACK_IN_PROGRESS ?\n+\t\t\t\t\t    &backup_files : NULL);\n \t\t\tfree(pack_tmp_name);\n-\t\t\tputs(sha1_to_hex(sha1));\n+\t\t\tif (repack_flags & REPACK_IN_PROGRESS) {\n+\t\t\t\tint len = strlen(tmpname);\n+\t\t\t\tchar *s = xmalloc(len + 2);\n+\t\t\t\tmemcpy(s, tmpname, len - 4);\n+\t\t\t\tmemcpy(s + len - 4, \".pack\", 6);\n+\t\t\t\tstring_list_append(&written_packs, s);\n+\t\t\t\tnr_written_packs++;\n+\t\t\t} else\n+\t\t\t\tputs(sha1_to_hex(sha1));\n \t\t}\n \n \t\t/* mark written objects as written to previous pack */\n@@ -2359,7 +2385,8 @@ static void get_object_list(int ac, const char **av)\n \tsave_commit_buffer = 0;\n \tsetup_revisions(ac, av, &revs, NULL);\n \n-\twhile (fgets(line, sizeof(line), stdin) != NULL) {\n+\twhile (!(repack_flags & REPACK_IN_PROGRESS) &&\n+\t       fgets(line, sizeof(line), stdin) != NULL) {\n \t\tint len = strlen(line);\n \t\tif (len && line[len - 1] == '\\n')\n \t\t\tline[--len] = 0;\n@@ -2387,6 +2414,30 @@ static void get_object_list(int ac, const char **av)\n \t\tloosen_unused_packed_objects(&revs);\n }\n \n+static void rollback_repack(void)\n+{\n+\tstruct strbuf dst = STRBUF_INIT;\n+\tint i, ret;\n+\tfor (i = 0; i < backup_files.nr; i++) {\n+\t\tconst char *src = backup_files.items[i].string;\n+\t\tstrbuf_addstr(&dst, src);\n+\t\tstrbuf_setlen(&dst, dst.len - 4); /* remove .old */\n+\t\tret = rename(src, dst.buf);\n+\t\tif (ret)\n+\t\t\twarning(\"failed to restore %s: %s\", src, strerror(errno));\n+\t\tstrbuf_setlen(&dst, 0);\n+\t}\n+\tstrbuf_release(&dst);\n+\tstring_list_clear(&backup_files, 0);\n+}\n+\n+static void rollback_repack_on_signal(int signo)\n+{\n+\trollback_repack();\n+\tsigchain_pop(signo);\n+\traise(signo);\n+}\n+\n static int option_parse_index_version(const struct option *opt,\n \t\t\t\t      const char *arg, int unset)\n {\n@@ -2436,11 +2487,12 @@ static int option_parse_ulong(const struct option *opt,\n \n int cmd_pack_objects(int argc, const char **argv, const char *prefix)\n {\n+\tstruct strbuf repack_base_name = STRBUF_INIT;\n \tint use_internal_rev_list = 0;\n \tint thin = 0;\n \tint all_progress_implied = 0;\n \tconst char *rp_av[6];\n-\tint rp_ac = 0;\n+\tint i, rp_ac = 0;\n \tint rev_list_unpacked = 0, rev_list_all = 0, rev_list_reflog = 0;\n \tstruct option pack_objects_options[] = {\n \t\tOPT_SET_INT('q', \"quiet\", &progress,\n@@ -2505,6 +2557,16 @@ int cmd_pack_objects(int argc, const char **argv, const char *prefix)\n \t\t\t    N_(\"pack compression level\")),\n \t\tOPT_SET_INT(0, \"keep-true-parents\", &grafts_replace_parents,\n \t\t\t    N_(\"do not hide commits by grafts\"), 0),\n+\n+\t\tOPT_BIT(0, \"repack\", &repack_flags,\n+\t\t\tN_(\"repack mode\"), REPACK_IN_PROGRESS),\n+\t\tOPT_BIT(0, \"repack-all\", &repack_flags,\n+\t\t\tN_(\"repack everything into one pack\"), REPACK_ALL_INTO_ONE),\n+\t\tOPT_BIT(0, \"remove-redundant\", &repack_flags,\n+\t\t\tN_(\"remove redundant objects after repack\"), REPACK_REMOVE_REDUNDANT),\n+\t\tOPT_BIT(0, \"update-info\", &repack_flags,\n+\t\t\tN_(\"run git-update-server-info after repack\"), REPACK_UPDATE_INFO),\n+\n \t\tOPT_END(),\n \t};\n \n@@ -2556,6 +2618,36 @@ int cmd_pack_objects(int argc, const char **argv, const char *prefix)\n \tif (delta_search_threads != 1)\n \t\twarning(\"no threads support, ignoring --threads\");\n #endif\n+\tif ((repack_flags & REPACK_IN_PROGRESS) == 0 &&\n+\t    (repack_flags & ~REPACK_IN_PROGRESS))\n+\t\tdie(\"--repack must be given for any repack related options\");\n+\tif (repack_flags & REPACK_IN_PROGRESS) {\n+\t\tif (pack_to_stdout)\n+\t\t\tdie(\"--stdout cannot be used with --repack\");\n+\t\tif (argc)\n+\t\t\tdie(\"base name cannot be used with --repack\");\n+\n+\t\trp_av[rp_ac++] = \"--all\";\n+\t\trp_av[rp_ac++] = \"--reflog\";\n+\t\tuse_internal_rev_list = 1;\n+\n+\t\tgrafts_replace_parents = 0; /* --keep-true-parents */\n+\t\tignore_packed_keep = 1;\t    /* --honor-pack-keep */\n+\t\tnon_empty = 1;\t\t    /* --non-empty */\n+\n+\t\tif (!(repack_flags & REPACK_ALL_INTO_ONE)) {\n+\t\t\tincremental = 1; /* --incremental */\n+\t\t\trp_av[rp_ac++] = \"--unpacked\";\n+\t\t}\n+\n+\t\tstrbuf_addf(&repack_base_name,\n+\t\t\t    \"%s/pack/pack\", get_object_directory());\n+\t\tbase_name = repack_base_name.buf;\n+\n+\t\tsigchain_push_common(rollback_repack_on_signal);\n+\t\tatexit(rollback_repack);\n+\t}\n+\n \tif (!pack_to_stdout && !pack_size_limit)\n \t\tpack_size_limit = pack_size_limit_cfg;\n \tif (pack_to_stdout && pack_size_limit)\n@@ -2598,5 +2690,184 @@ int cmd_pack_objects(int argc, const char **argv, const char *prefix)\n \t\tfprintf(stderr, \"Total %\"PRIu32\" (delta %\"PRIu32\"),\"\n \t\t\t\" reused %\"PRIu32\" (delta %\"PRIu32\")\\n\",\n \t\t\twritten, written_delta, reused, reused_delta);\n+\n+\tif (!(repack_flags & REPACK_IN_PROGRESS))\n+\t\treturn 0;\n+\n+\tif (!nr_written_packs) {\n+\t\tprintf(_(\"Nothing new to pack.\\n\"));\n+\t\treturn 0;\n+\t}\n+\n+\t/* At this point all new packs should be in place. We can\n+\t   safely remove old ones */\n+\tfor (i = 0; i < backup_files.nr; i++) {\n+\t\tconst char *s = backup_files.items[i].string;\n+\t\tint ret = unlink(s);\n+\t\tif (ret)\n+\t\t\twarning(\"failed to remove %s: %s\", s, strerror(errno));\n+\t}\n+\tstring_list_clear(&backup_files, 0);\n+\n+\tif (repack_flags & REPACK_REMOVE_REDUNDANT) {\n+\t\tstruct packed_git *p;\n+\t\tstruct string_list to_be_removed = STRING_LIST_INIT_DUP;\n+\n+\t\t/* free_pack_by_name() may have freed a few packs in\n+\t\t   write_pack_file() */\n+\t\treprepare_packed_git();\n+\t\tfor (p = packed_git; p; p = p->next) {\n+\t\t\tif (!p->pack_local || p->pack_keep)\n+\t\t\t\tcontinue;\n+\n+\t\t\tfor (i = 0; i < written_packs.nr; i++) {\n+\t\t\t\tchar *s = written_packs.items[i].string;\n+\t\t\t\tif (!strcmp(s, p->pack_name))\n+\t\t\t\t\tbreak;\n+\t\t\t}\n+\t\t\tif (i < written_packs.nr)\n+\t\t\t\tcontinue;\n+\n+\t\t\tstring_list_append(&to_be_removed, p->pack_name);\n+\t\t}\n+\t\twritten_packs.strdup_strings = 1;\n+\t\tstring_list_clear(&written_packs, 0);\n+\n+\t\tfor (i = 0; i < to_be_removed.nr; i++) {\n+\t\t\tchar *path = to_be_removed.items[i].string;\n+\n+\t\t\t/* Windows limitation on unlink().\n+\t\t\t   See c74faea19e39ca933492f697596310397175c329 */\n+\t\t\tfree_pack_by_name(path);\n+\n+\t\t\tif (unlink(path))\n+\t\t\t\twarning(\"failed to remove %s: %s\", path, strerror(errno));\n+\t\t\tstrcpy(path + strlen(path)-5, \".idx\");\n+\t\t\tif (unlink(path))\n+\t\t\t\twarning(\"failed to remove %s: %s\", path, strerror(errno));\n+\t\t}\n+\t\tstring_list_clear(&to_be_removed, 0);\n+\n+\t\treprepare_packed_git();\n+\t\tprune_packed_objects(progress ? PRUNE_PACKED_VERBOSE : 0);\n+\t}\n+\n+\tif (repack_flags & REPACK_UPDATE_INFO)\n+\t\tupdate_server_info(0);\n+\n \treturn 0;\n }\n+\n+static int repack_config(const char *k, const char *v, void *cb)\n+{\n+\tif (!strcasecmp(k, \"repack.usedeltabaseoffset\")) {\n+\t\trepack_usedeltabaseoffset = git_config_bool(k, v);\n+\t\treturn 0;\n+\t}\n+\treturn git_default_config(k, v, cb);\n+}\n+\n+int cmd_repack(int argc, const char **argv, const char *prefix)\n+{\n+\tint all_in_one = 0;\n+\tint all_in_one_and_unreachable = 0;\n+\tint unpack_unreachable = 0;\n+\tint remove_redundant = 0;\n+\tint no_reuse_delta = 0;\n+\tint no_reuse_object = 0;\n+\tint no_update = 0;\n+\tint quiet = 0;\n+\tint local = 0;\n+\n+\tstruct option opts[] = {\n+\t\tOPT_BOOL('a', NULL, &all_in_one,\n+\t\t\t \"pack everything in a single pack\"),\n+\t\tOPT_BOOL('A', NULL, &all_in_one_and_unreachable,\n+\t\t\t \"same as -a, and turn unreachable objects loose\"),\n+\t\tOPT_BOOL('d', NULL, &remove_redundant,\n+\t\t\t \"remove redundant packs, and run git-prune-packed\"),\n+\t\tOPT_BOOL('f', NULL, &no_reuse_delta,\n+\t\t\t \"pass --no-reuse-delta to git-pack-objects\"),\n+\t\tOPT_BOOL('F', NULL, &no_reuse_object,\n+\t\t\t \"pass --no-reuse-object to git-pack-objects\"),\n+\t\tOPT_BOOL('n', NULL, &no_update,\n+\t\t\t \"do not run git-update-server-info\"),\n+\t\tOPT_BOOL('q', NULL, &quiet, \"be quiet\"),\n+\t\tOPT_BOOL('l', NULL, &local,\n+\t\t\t \"pass --local to git-pack-objects\"),\n+\t\t{ OPTION_ARGUMENT, 0, \"window\", NULL, \"n\",\n+\t\t \"size of the window used for delta compression\", 0 },\n+\t\t{ OPTION_ARGUMENT, 0, \"window-memory\", NULL, \"n\",\n+\t\t  \"same as the above, but limit memory size instead of entries count\", 0 },\n+\t\t{ OPTION_ARGUMENT, 0, \"depth\", NULL, \"n\",\n+\t\t  \"limits the maximum delta depth\", 0 },\n+\t\t{ OPTION_ARGUMENT, 0, \"max-pack-size\", NULL, \"n\",\n+\t\t  \"maximum size of each packfile\", 0},\n+\t\tOPT_END(),\n+\t};\n+\n+\tconst char *av[] = { \"pack-objects\", \"--repack\",\n+\t\t\t     NULL, /* --[no-]update-info */\n+\t\t\t     NULL, /* --delta-base-offset */\n+\t\t\t     NULL, /* --repack-all */\n+\t\t\t     NULL, /* --remove-redundant */\n+\t\t\t     NULL, /* --no-reuse-delta */\n+\t\t\t     NULL, /* --no-reuse-object */\n+\t\t\t     NULL, /* --local */\n+\t\t\t     NULL, /* -q */\n+\t\t\t     NULL, /* --unpack-unreachable */\n+\t\t\t     NULL, /* --window */\n+\t\t\t     NULL, /* --window-memory */\n+\t\t\t     NULL, /* --depth */\n+\t\t\t     NULL, /* --max-pack-size */\n+\t\t\t     NULL\n+\t};\n+\tint ac = 2;\n+\n+\tgit_config(repack_config, NULL);\n+\n+\targc = parse_options(argc, argv, prefix, opts, repack_usage, 0);\n+\n+\tif (no_update)\n+\t\tav[ac++] = \"--no-update-info\";\n+\telse\n+\t\tav[ac++] = \"--update-info\";\n+\tif (repack_usedeltabaseoffset)\n+\t\tav[ac++] = \"--delta-base-offset\";\n+\tif (all_in_one_and_unreachable) {\n+\t\tav[ac++] = \"--repack-all\";\n+\t\tall_in_one = 1;\n+\t\tunpack_unreachable = 1;\n+\t}\n+\tif (all_in_one)\n+\t\tav[ac++] = \"--repack-all\";\n+\tif (remove_redundant)\n+\t\tav[ac++] = \"--remove-redundant\";\n+\tif (no_reuse_delta)\n+\t\tav[ac++] = \"--no-reuse-delta\";\n+\tif (no_reuse_object)\n+\t\tav[ac++] = \"--no-reuse-object\";\n+\tif (local)\n+\t\tav[ac++] = \"--local\";\n+\tif (quiet)\n+\t\tav[ac++] = \"-q\";\n+\tif ((ac + argc) * sizeof(*av) > sizeof(av))\n+\t\tdie(\"Too many options\");\n+\tmemcpy(av + ac, argv, argc * sizeof(*argv));\n+\tac += argc;\n+\n+\tif (all_in_one && remove_redundant) {\n+\t\tstruct packed_git *p;\n+\n+\t\tprepare_packed_git();\n+\t\tfor (p = packed_git; p; p = p->next) {\n+\t\t\tif (!p->pack_keep &&\n+\t\t\t    unpack_unreachable && remove_redundant) {\n+\t\t\t\tav[ac++] = \"--unpack-unreachable\";\n+\t\t\t\tbreak;\n+\t\t\t}\n+\t\t}\n+\t}\n+\n+\treturn cmd_pack_objects(ac, av, prefix);\n+}\ndiff --git a/bulk-checkin.c b/bulk-checkin.c\nindex 6b0b6d4..3ca3c55 100644\n--- a/bulk-checkin.c\n+++ b/bulk-checkin.c\n@@ -46,7 +46,7 @@ static void finish_bulk_checkin(struct bulk_checkin_state *state)\n \tsprintf(packname, \"%s/pack/pack-\", get_object_directory());\n \tfinish_tmp_packfile(packname, state->pack_tmp_name,\n \t\t\t    state->written, state->nr_written,\n-\t\t\t    &state->pack_idx_opts, sha1);\n+\t\t\t    &state->pack_idx_opts, sha1, NULL);\n \tfor (i = 0; i < state->nr_written; i++)\n \t\tfree(state->written[i]);\n \ndiff --git a/contrib/examples/git-repack.sh b/contrib/examples/git-repack.sh\nnew file mode 100755\nindex 0000000..7579331\n--- /dev/null\n+++ b/contrib/examples/git-repack.sh\n@@ -0,0 +1,194 @@\n+#!/bin/sh\n+#\n+# Copyright (c) 2005 Linus Torvalds\n+#\n+\n+OPTIONS_KEEPDASHDASH=\n+OPTIONS_SPEC=\"\\\n+git repack [options]\n+--\n+a               pack everything in a single pack\n+A               same as -a, and turn unreachable objects loose\n+d               remove redundant packs, and run git-prune-packed\n+f               pass --no-reuse-delta to git-pack-objects\n+F               pass --no-reuse-object to git-pack-objects\n+n               do not run git-update-server-info\n+q,quiet         be quiet\n+l               pass --local to git-pack-objects\n+unpack-unreachable=  with -A, do not loosen objects older than this\n+ Packing constraints\n+window=         size of the window used for delta compression\n+window-memory=  same as the above, but limit memory size instead of entries count\n+depth=          limits the maximum delta depth\n+max-pack-size=  maximum size of each packfile\n+\"\n+SUBDIRECTORY_OK='Yes'\n+. git-sh-setup\n+\n+no_update_info= all_into_one= remove_redundant= unpack_unreachable=\n+local= no_reuse= extra=\n+while test $# != 0\n+do\n+\tcase \"$1\" in\n+\t-n)\tno_update_info=t ;;\n+\t-a)\tall_into_one=t ;;\n+\t-A)\tall_into_one=t\n+\t\tunpack_unreachable=--unpack-unreachable ;;\n+\t--unpack-unreachable)\n+\t\tunpack_unreachable=\"--unpack-unreachable=$2\"; shift ;;\n+\t-d)\tremove_redundant=t ;;\n+\t-q)\tGIT_QUIET=t ;;\n+\t-f)\tno_reuse=--no-reuse-delta ;;\n+\t-F)\tno_reuse=--no-reuse-object ;;\n+\t-l)\tlocal=--local ;;\n+\t--max-pack-size|--window|--window-memory|--depth)\n+\t\textra=\"$extra $1=$2\"; shift ;;\n+\t--) shift; break;;\n+\t*)\tusage ;;\n+\tesac\n+\tshift\n+done\n+\n+case \"`git config --bool repack.usedeltabaseoffset || echo true`\" in\n+true)\n+\textra=\"$extra --delta-base-offset\" ;;\n+esac\n+\n+PACKDIR=\"$GIT_OBJECT_DIRECTORY/pack\"\n+PACKTMP=\"$PACKDIR/.tmp-$$-pack\"\n+rm -f \"$PACKTMP\"-*\n+trap 'rm -f \"$PACKTMP\"-*' 0 1 2 3 15\n+\n+# There will be more repacking strategies to come...\n+case \",$all_into_one,\" in\n+,,)\n+\targs='--unpacked --incremental'\n+\t;;\n+,t,)\n+\targs= existing=\n+\tif [ -d \"$PACKDIR\" ]; then\n+\t\tfor e in `cd \"$PACKDIR\" && find . -type f -name '*.pack' \\\n+\t\t\t| sed -e 's/^\\.\\///' -e 's/\\.pack$//'`\n+\t\tdo\n+\t\t\tif [ -e \"$PACKDIR/$e.keep\" ]; then\n+\t\t\t\t: keep\n+\t\t\telse\n+\t\t\t\texisting=\"$existing $e\"\n+\t\t\tfi\n+\t\tdone\n+\t\tif test -n \"$existing\" -a -n \"$unpack_unreachable\" -a \\\n+\t\t\t-n \"$remove_redundant\"\n+\t\tthen\n+\t\t\t# This may have arbitrary user arguments, so we\n+\t\t\t# have to protect it against whitespace splitting\n+\t\t\t# when it gets run as \"pack-objects $args\" later.\n+\t\t\t# Fortunately, we know it's an approxidate, so we\n+\t\t\t# can just use dots instead.\n+\t\t\targs=\"$args $(echo \"$unpack_unreachable\" | tr ' ' .)\"\n+\t\tfi\n+\tfi\n+\t;;\n+esac\n+\n+mkdir -p \"$PACKDIR\" || exit\n+\n+args=\"$args $local ${GIT_QUIET:+-q} $no_reuse$extra\"\n+names=$(git pack-objects --keep-true-parents --honor-pack-keep --non-empty --all --reflog $args </dev/null \"$PACKTMP\") ||\n+\texit 1\n+if [ -z \"$names\" ]; then\n+\tsay Nothing new to pack.\n+fi\n+\n+# Ok we have prepared all new packfiles.\n+\n+# First see if there are packs of the same name and if so\n+# if we can move them out of the way (this can happen if we\n+# repacked immediately after packing fully.\n+rollback=\n+failed=\n+for name in $names\n+do\n+\tfor sfx in pack idx\n+\tdo\n+\t\tfile=pack-$name.$sfx\n+\t\ttest -f \"$PACKDIR/$file\" || continue\n+\t\trm -f \"$PACKDIR/old-$file\" &&\n+\t\tmv \"$PACKDIR/$file\" \"$PACKDIR/old-$file\" || {\n+\t\t\tfailed=t\n+\t\t\tbreak\n+\t\t}\n+\t\trollback=\"$rollback $file\"\n+\tdone\n+\ttest -z \"$failed\" || break\n+done\n+\n+# If renaming failed for any of them, roll the ones we have\n+# already renamed back to their original names.\n+if test -n \"$failed\"\n+then\n+\trollback_failure=\n+\tfor file in $rollback\n+\tdo\n+\t\tmv \"$PACKDIR/old-$file\" \"$PACKDIR/$file\" ||\n+\t\trollback_failure=\"$rollback_failure $file\"\n+\tdone\n+\tif test -n \"$rollback_failure\"\n+\tthen\n+\t\techo >&2 \"WARNING: Some packs in use have been renamed by\"\n+\t\techo >&2 \"WARNING: prefixing old- to their name, in order to\"\n+\t\techo >&2 \"WARNING: replace them with the new version of the\"\n+\t\techo >&2 \"WARNING: file.  But the operation failed, and\"\n+\t\techo >&2 \"WARNING: attempt to rename them back to their\"\n+\t\techo >&2 \"WARNING: original names also failed.\"\n+\t\techo >&2 \"WARNING: Please rename them in $PACKDIR manually:\"\n+\t\tfor file in $rollback_failure\n+\t\tdo\n+\t\t\techo >&2 \"WARNING:   old-$file -> $file\"\n+\t\tdone\n+\tfi\n+\texit 1\n+fi\n+\n+# Now the ones with the same name are out of the way...\n+fullbases=\n+for name in $names\n+do\n+\tfullbases=\"$fullbases pack-$name\"\n+\tchmod a-w \"$PACKTMP-$name.pack\"\n+\tchmod a-w \"$PACKTMP-$name.idx\"\n+\tmv -f \"$PACKTMP-$name.pack\" \"$PACKDIR/pack-$name.pack\" &&\n+\tmv -f \"$PACKTMP-$name.idx\"  \"$PACKDIR/pack-$name.idx\" ||\n+\texit\n+done\n+\n+# Remove the \"old-\" files\n+for name in $names\n+do\n+\trm -f \"$PACKDIR/old-pack-$name.idx\"\n+\trm -f \"$PACKDIR/old-pack-$name.pack\"\n+done\n+\n+# End of pack replacement.\n+\n+if test \"$remove_redundant\" = t\n+then\n+\t# We know $existing are all redundant.\n+\tif [ -n \"$existing\" ]\n+\tthen\n+\t\t( cd \"$PACKDIR\" &&\n+\t\t  for e in $existing\n+\t\t  do\n+\t\t\tcase \" $fullbases \" in\n+\t\t\t*\" $e \"*) ;;\n+\t\t\t*)\trm -f \"$e.pack\" \"$e.idx\" \"$e.keep\" ;;\n+\t\t\tesac\n+\t\t  done\n+\t\t)\n+\tfi\n+\tgit prune-packed ${GIT_QUIET:+-q}\n+fi\n+\n+case \"$no_update_info\" in\n+t) : ;;\n+*) git update-server-info ;;\n+esac\ndiff --git a/git-repack.sh b/git-repack.sh\ndeleted file mode 100755\nindex 7579331..0000000\n--- a/git-repack.sh\n+++ /dev/null\n@@ -1,194 +0,0 @@\n-#!/bin/sh\n-#\n-# Copyright (c) 2005 Linus Torvalds\n-#\n-\n-OPTIONS_KEEPDASHDASH=\n-OPTIONS_SPEC=\"\\\n-git repack [options]\n---\n-a               pack everything in a single pack\n-A               same as -a, and turn unreachable objects loose\n-d               remove redundant packs, and run git-prune-packed\n-f               pass --no-reuse-delta to git-pack-objects\n-F               pass --no-reuse-object to git-pack-objects\n-n               do not run git-update-server-info\n-q,quiet         be quiet\n-l               pass --local to git-pack-objects\n-unpack-unreachable=  with -A, do not loosen objects older than this\n- Packing constraints\n-window=         size of the window used for delta compression\n-window-memory=  same as the above, but limit memory size instead of entries count\n-depth=          limits the maximum delta depth\n-max-pack-size=  maximum size of each packfile\n-\"\n-SUBDIRECTORY_OK='Yes'\n-. git-sh-setup\n-\n-no_update_info= all_into_one= remove_redundant= unpack_unreachable=\n-local= no_reuse= extra=\n-while test $# != 0\n-do\n-\tcase \"$1\" in\n-\t-n)\tno_update_info=t ;;\n-\t-a)\tall_into_one=t ;;\n-\t-A)\tall_into_one=t\n-\t\tunpack_unreachable=--unpack-unreachable ;;\n-\t--unpack-unreachable)\n-\t\tunpack_unreachable=\"--unpack-unreachable=$2\"; shift ;;\n-\t-d)\tremove_redundant=t ;;\n-\t-q)\tGIT_QUIET=t ;;\n-\t-f)\tno_reuse=--no-reuse-delta ;;\n-\t-F)\tno_reuse=--no-reuse-object ;;\n-\t-l)\tlocal=--local ;;\n-\t--max-pack-size|--window|--window-memory|--depth)\n-\t\textra=\"$extra $1=$2\"; shift ;;\n-\t--) shift; break;;\n-\t*)\tusage ;;\n-\tesac\n-\tshift\n-done\n-\n-case \"`git config --bool repack.usedeltabaseoffset || echo true`\" in\n-true)\n-\textra=\"$extra --delta-base-offset\" ;;\n-esac\n-\n-PACKDIR=\"$GIT_OBJECT_DIRECTORY/pack\"\n-PACKTMP=\"$PACKDIR/.tmp-$$-pack\"\n-rm -f \"$PACKTMP\"-*\n-trap 'rm -f \"$PACKTMP\"-*' 0 1 2 3 15\n-\n-# There will be more repacking strategies to come...\n-case \",$all_into_one,\" in\n-,,)\n-\targs='--unpacked --incremental'\n-\t;;\n-,t,)\n-\targs= existing=\n-\tif [ -d \"$PACKDIR\" ]; then\n-\t\tfor e in `cd \"$PACKDIR\" && find . -type f -name '*.pack' \\\n-\t\t\t| sed -e 's/^\\.\\///' -e 's/\\.pack$//'`\n-\t\tdo\n-\t\t\tif [ -e \"$PACKDIR/$e.keep\" ]; then\n-\t\t\t\t: keep\n-\t\t\telse\n-\t\t\t\texisting=\"$existing $e\"\n-\t\t\tfi\n-\t\tdone\n-\t\tif test -n \"$existing\" -a -n \"$unpack_unreachable\" -a \\\n-\t\t\t-n \"$remove_redundant\"\n-\t\tthen\n-\t\t\t# This may have arbitrary user arguments, so we\n-\t\t\t# have to protect it against whitespace splitting\n-\t\t\t# when it gets run as \"pack-objects $args\" later.\n-\t\t\t# Fortunately, we know it's an approxidate, so we\n-\t\t\t# can just use dots instead.\n-\t\t\targs=\"$args $(echo \"$unpack_unreachable\" | tr ' ' .)\"\n-\t\tfi\n-\tfi\n-\t;;\n-esac\n-\n-mkdir -p \"$PACKDIR\" || exit\n-\n-args=\"$args $local ${GIT_QUIET:+-q} $no_reuse$extra\"\n-names=$(git pack-objects --keep-true-parents --honor-pack-keep --non-empty --all --reflog $args </dev/null \"$PACKTMP\") ||\n-\texit 1\n-if [ -z \"$names\" ]; then\n-\tsay Nothing new to pack.\n-fi\n-\n-# Ok we have prepared all new packfiles.\n-\n-# First see if there are packs of the same name and if so\n-# if we can move them out of the way (this can happen if we\n-# repacked immediately after packing fully.\n-rollback=\n-failed=\n-for name in $names\n-do\n-\tfor sfx in pack idx\n-\tdo\n-\t\tfile=pack-$name.$sfx\n-\t\ttest -f \"$PACKDIR/$file\" || continue\n-\t\trm -f \"$PACKDIR/old-$file\" &&\n-\t\tmv \"$PACKDIR/$file\" \"$PACKDIR/old-$file\" || {\n-\t\t\tfailed=t\n-\t\t\tbreak\n-\t\t}\n-\t\trollback=\"$rollback $file\"\n-\tdone\n-\ttest -z \"$failed\" || break\n-done\n-\n-# If renaming failed for any of them, roll the ones we have\n-# already renamed back to their original names.\n-if test -n \"$failed\"\n-then\n-\trollback_failure=\n-\tfor file in $rollback\n-\tdo\n-\t\tmv \"$PACKDIR/old-$file\" \"$PACKDIR/$file\" ||\n-\t\trollback_failure=\"$rollback_failure $file\"\n-\tdone\n-\tif test -n \"$rollback_failure\"\n-\tthen\n-\t\techo >&2 \"WARNING: Some packs in use have been renamed by\"\n-\t\techo >&2 \"WARNING: prefixing old- to their name, in order to\"\n-\t\techo >&2 \"WARNING: replace them with the new version of the\"\n-\t\techo >&2 \"WARNING: file.  But the operation failed, and\"\n-\t\techo >&2 \"WARNING: attempt to rename them back to their\"\n-\t\techo >&2 \"WARNING: original names also failed.\"\n-\t\techo >&2 \"WARNING: Please rename them in $PACKDIR manually:\"\n-\t\tfor file in $rollback_failure\n-\t\tdo\n-\t\t\techo >&2 \"WARNING:   old-$file -> $file\"\n-\t\tdone\n-\tfi\n-\texit 1\n-fi\n-\n-# Now the ones with the same name are out of the way...\n-fullbases=\n-for name in $names\n-do\n-\tfullbases=\"$fullbases pack-$name\"\n-\tchmod a-w \"$PACKTMP-$name.pack\"\n-\tchmod a-w \"$PACKTMP-$name.idx\"\n-\tmv -f \"$PACKTMP-$name.pack\" \"$PACKDIR/pack-$name.pack\" &&\n-\tmv -f \"$PACKTMP-$name.idx\"  \"$PACKDIR/pack-$name.idx\" ||\n-\texit\n-done\n-\n-# Remove the \"old-\" files\n-for name in $names\n-do\n-\trm -f \"$PACKDIR/old-pack-$name.idx\"\n-\trm -f \"$PACKDIR/old-pack-$name.pack\"\n-done\n-\n-# End of pack replacement.\n-\n-if test \"$remove_redundant\" = t\n-then\n-\t# We know $existing are all redundant.\n-\tif [ -n \"$existing\" ]\n-\tthen\n-\t\t( cd \"$PACKDIR\" &&\n-\t\t  for e in $existing\n-\t\t  do\n-\t\t\tcase \" $fullbases \" in\n-\t\t\t*\" $e \"*) ;;\n-\t\t\t*)\trm -f \"$e.pack\" \"$e.idx\" \"$e.keep\" ;;\n-\t\t\tesac\n-\t\t  done\n-\t\t)\n-\tfi\n-\tgit prune-packed ${GIT_QUIET:+-q}\n-fi\n-\n-case \"$no_update_info\" in\n-t) : ;;\n-*) git update-server-info ;;\n-esac\ndiff --git a/git.c b/git.c\nindex 2025f77..72d1dc9 100644\n--- a/git.c\n+++ b/git.c\n@@ -397,6 +397,7 @@ static void handle_internal_command(int argc, const char **argv)\n \t\t{ \"remote-ext\", cmd_remote_ext },\n \t\t{ \"remote-fd\", cmd_remote_fd },\n \t\t{ \"replace\", cmd_replace, RUN_SETUP },\n+\t\t{ \"repack\", cmd_repack, RUN_SETUP },\n \t\t{ \"repo-config\", cmd_repo_config, RUN_SETUP_GENTLY },\n \t\t{ \"rerere\", cmd_rerere, RUN_SETUP },\n \t\t{ \"reset\", cmd_reset, RUN_SETUP },\ndiff --git a/pack-write.c b/pack-write.c\nindex ca9e63b..e6aa7e3 100644\n--- a/pack-write.c\n+++ b/pack-write.c\n@@ -1,6 +1,7 @@\n #include \"cache.h\"\n #include \"pack.h\"\n #include \"csum-file.h\"\n+#include \"string-list.h\"\n \n void reset_pack_idx_option(struct pack_idx_option *opts)\n {\n@@ -348,10 +349,12 @@ void finish_tmp_packfile(char *name_buffer,\n \t\t\t struct pack_idx_entry **written_list,\n \t\t\t uint32_t nr_written,\n \t\t\t struct pack_idx_option *pack_idx_opts,\n-\t\t\t unsigned char sha1[])\n+\t\t\t unsigned char sha1[],\n+\t\t\t struct string_list *backup_files)\n {\n \tconst char *idx_tmp_name;\n \tchar *end_of_name_prefix = strrchr(name_buffer, 0);\n+\tstruct stat st;\n \n \tif (adjust_shared_perm(pack_tmp_name))\n \t\tdie_errno(\"unable to make temporary pack file readable\");\n@@ -368,6 +371,14 @@ void finish_tmp_packfile(char *name_buffer,\n \t\tdie_errno(\"unable to rename temporary pack file\");\n \n \tsprintf(end_of_name_prefix, \"%s.idx\", sha1_to_hex(sha1));\n+\tif (backup_files && !stat(name_buffer, &st)) {\n+\t\tstruct strbuf old = STRBUF_INIT;\n+\t\tstrbuf_addf(&old, \"%s.old\", name_buffer);\n+\t\tif (rename(name_buffer, old.buf))\n+\t\t\tdie_errno(\"unable to rename pack %s\", name_buffer);\n+\t\tstring_list_append(backup_files, strbuf_detach(&old, NULL));\n+\t}\n+\tbackup_file(name_buffer);\n \tif (rename(idx_tmp_name, name_buffer))\n \t\tdie_errno(\"unable to rename temporary index file\");\n \ndiff --git a/pack.h b/pack.h\nindex aa6ee7d..d3f92ad 100644\n--- a/pack.h\n+++ b/pack.h\n@@ -90,7 +90,9 @@ extern int encode_in_pack_object_header(enum object_type, uintmax_t, unsigned ch\n #define PH_ERROR_PROTOCOL\t(-3)\n extern int read_pack_header(int fd, struct pack_header *);\n \n+struct string_list;\n+\n extern struct sha1file *create_tmp_packfile(char **pack_tmp_name);\n-extern void finish_tmp_packfile(char *name_buffer, const char *pack_tmp_name, struct pack_idx_entry **written_list, uint32_t nr_written, struct pack_idx_option *pack_idx_opts, unsigned char sha1[]);\n+extern void finish_tmp_packfile(char *name_buffer, const char *pack_tmp_name, struct pack_idx_entry **written_list, uint32_t nr_written, struct pack_idx_option *pack_idx_opts, unsigned char sha1[], struct string_list *backup_files);\n \n #endif\n-- \n1.8.4.rc1.25.g2793d0a\n"},{"id":"224750","messageId":"1375884049-14961-3-git-send-email-stefanbeller@googlemail.com","threadId":"34587","inReplyTo":"1375884049-14961-1-git-send-email-stefanbeller@googlemail.com","subject":"[PATCH 2/4] backup_file dummy function","fromName":"Stefan Beller","fromEmail":"stefanbeller@googlemail.com","sentAt":"2013-08-07T14:00:47Z","receivedAt":"2013-08-07T14:00:47Z","isPatch":true,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"---\n pack-write.c | 5 +++++\n 1 file changed, 5 insertions(+)\n\ndiff --git a/pack-write.c b/pack-write.c\nindex e6aa7e3..b728ea2 100644\n--- a/pack-write.c\n+++ b/pack-write.c\n@@ -344,6 +344,11 @@ struct sha1file *create_tmp_packfile(char **pack_tmp_name)\n \treturn sha1fd(fd, *pack_tmp_name);\n }\n \n+void backup_file(char *filename)\n+{\n+\n+}\n+\n void finish_tmp_packfile(char *name_buffer,\n \t\t\t const char *pack_tmp_name,\n \t\t\t struct pack_idx_entry **written_list,\n-- \n1.8.4.rc1.25.g2793d0a\n"},{"id":"224751","messageId":"1375884049-14961-4-git-send-email-stefanbeller@googlemail.com","threadId":"34587","inReplyTo":"1375884049-14961-1-git-send-email-stefanbeller@googlemail.com","subject":"[PATCH 3/4] pack-objects: do not print usage when repacking","fromName":"Stefan Beller","fromEmail":"stefanbeller@googlemail.com","sentAt":"2013-08-07T14:00:48Z","receivedAt":"2013-08-07T14:00:48Z","isPatch":true,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"---\n builtin/pack-objects.c | 2 +-\n 1 file changed, 1 insertion(+), 1 deletion(-)\n\ndiff --git a/builtin/pack-objects.c b/builtin/pack-objects.c\nindex 1742ea1..0bd8f3b 100644\n--- a/builtin/pack-objects.c\n+++ b/builtin/pack-objects.c\n@@ -2585,7 +2585,7 @@ int cmd_pack_objects(int argc, const char **argv, const char *prefix)\n \t\tbase_name = argv[0];\n \t\targc--;\n \t}\n-\tif (pack_to_stdout != !base_name || argc)\n+\tif (!(repack_flags & REPACK_IN_PROGRESS) && (pack_to_stdout != !base_name || argc))\n \t\tusage_with_options(pack_usage, pack_objects_options);\n \n \trp_av[rp_ac++] = \"pack-objects\";\n-- \n1.8.4.rc1.25.g2793d0a\n"},{"id":"224752","messageId":"1375884049-14961-5-git-send-email-stefanbeller@googlemail.com","threadId":"34587","inReplyTo":"1375884049-14961-1-git-send-email-stefanbeller@googlemail.com","subject":"[PATCH 4/4] repack: add unpack-unreachable","fromName":"Stefan Beller","fromEmail":"stefanbeller@googlemail.com","sentAt":"2013-08-07T14:00:49Z","receivedAt":"2013-08-07T14:00:49Z","isPatch":true,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"---\n builtin/pack-objects.c | 3 +++\n 1 file changed, 3 insertions(+)\n\ndiff --git a/builtin/pack-objects.c b/builtin/pack-objects.c\nindex 0bd8f3b..0fe01fc 100644\n--- a/builtin/pack-objects.c\n+++ b/builtin/pack-objects.c\n@@ -2795,6 +2795,9 @@ int cmd_repack(int argc, const char **argv, const char *prefix)\n \t\tOPT_BOOL('q', NULL, &quiet, \"be quiet\"),\n \t\tOPT_BOOL('l', NULL, &local,\n \t\t\t \"pass --local to git-pack-objects\"),\n+\t\t{ OPTION_CALLBACK, 0, \"unpack-unreachable\", NULL, N_(\"time\"),\n+\t\t  N_(\"unpack unreachable objects newer than <time>\"),\n+\t\t  PARSE_OPT_OPTARG, option_parse_unpack_unreachable },\n \t\t{ OPTION_ARGUMENT, 0, \"window\", NULL, \"n\",\n \t\t \"size of the window used for delta compression\", 0 },\n \t\t{ OPTION_ARGUMENT, 0, \"window-memory\", NULL, \"n\",\n-- \n1.8.4.rc1.25.g2793d0a\n"},{"id":"224753","messageId":"vpqzjsth8wt.fsf@anie.imag.fr","threadId":"34587","inReplyTo":"1375884049-14961-2-git-send-email-stefanbeller@googlemail.com","subject":"Re: [PATCH 1/4] Build in git-repack","fromName":"Matthieu Moy","fromEmail":"matthieu.moy@grenoble-inp.fr","sentAt":"2013-08-07T14:28:50Z","receivedAt":"2013-08-07T14:28:50Z","isPatch":true,"sender":{"key":"matthieu.moy@grenoble-inp.fr","avatar":"https://gravatar.com/avatar/72c8a2705971a25dfaff23cece15130d405685845d911aedd5667ace277f3fc5?d=mp&s=160"},"body":"[ It's cool you're working on this, I'd really like a git-repack in C.\n  That would fix this\n  http://thread.gmane.org/gmane.comp.version-control.git/226458 ]\n\nStefan Beller <stefanbeller@googlemail.com> writes:\n\n> From: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>\n>\n> pack-objects learns a few more options to take over what's been done\n> by git-repack.sh. cmd_repack() becomes a wrapper around\n> cmd_pack_objects().\n\nI think the patch would read easier if these were split into two\npatches: one doing the real stuff in pack-objects, and then getting rid\nof git-repack.sh to replace it with a trivial built-in.\n\nActually, I'm wondering why pack-objects requires so much changes.\ngit-repack.sh was already a relatively small wrapper around\npack-objects, and did not need the new options you add, so why are they\nneeded? In particular adding the new --update-info option that just does\n\n> +\tif (repack_flags & REPACK_UPDATE_INFO)\n> +\t\tupdate_server_info(0);\n\nseems overkill to me: why don't you just let cmd_repack call\nupdate_server_info(0)?\n\n-- \nMatthieu Moy\nhttp://www-verimag.imag.fr/~moy/\n"},{"id":"224756","messageId":"7v38qlec2w.fsf@alter.siamese.dyndns.org","threadId":"34587","inReplyTo":"vpqzjsth8wt.fsf@anie.imag.fr","subject":"Re: [PATCH 1/4] Build in git-repack","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2013-08-07T15:48:39Z","receivedAt":"2013-08-07T15:48:39Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Matthieu Moy <Matthieu.Moy@grenoble-inp.fr> writes:\n\n> [ It's cool you're working on this, I'd really like a git-repack in C.\n>   That would fix this\n>   http://thread.gmane.org/gmane.comp.version-control.git/226458 ]\n>\n> Stefan Beller <stefanbeller@googlemail.com> writes:\n>\n>> From: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>\n>>\n>> pack-objects learns a few more options to take over what's been done\n>> by git-repack.sh. cmd_repack() becomes a wrapper around\n>> cmd_pack_objects().\n>\n> I think the patch would read easier if these were split into two\n> patches: one doing the real stuff in pack-objects, and then getting rid\n> of git-repack.sh to replace it with a trivial built-in.\n>\n> Actually, I'm wondering why pack-objects requires so much changes.\n> git-repack.sh was already a relatively small wrapper around\n> pack-objects, and did not need the new options you add, so why are they\n> needed? In particular adding the new --update-info option that just does\n>\n>> +\tif (repack_flags & REPACK_UPDATE_INFO)\n>> +\t\tupdate_server_info(0);\n>\n> seems overkill to me: why don't you just let cmd_repack call\n> update_server_info(0)?\n\nMy feeling exactly.  I would rather see a patch that does not touch\npack-objects at all, and use run_command() interface to spawn it.\nOnce we do have to pack, the necessary processing cycle will dwarf\nthe fork/exec latency anyway, no?\n"},{"id":"224761","messageId":"520279BE.90704@googlemail.com","threadId":"34587","inReplyTo":"7v38qlec2w.fsf@alter.siamese.dyndns.org","subject":"Re: [PATCH 1/4] Build in git-repack","fromName":"Stefan Beller","fromEmail":"stefanbeller@googlemail.com","sentAt":"2013-08-07T16:45:50Z","receivedAt":"2013-08-07T16:45:50Z","isPatch":true,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"On 08/07/2013 05:48 PM, Junio C Hamano wrote:\n> Matthieu Moy <Matthieu.Moy@grenoble-inp.fr> writes:\n>>\n>> seems overkill to me: why don't you just let cmd_repack call\n>> update_server_info(0)?\n> \n> My feeling exactly.  I would rather see a patch that does not touch\n> pack-objects at all, and use run_command() interface to spawn it.\n> Once we do have to pack, the necessary processing cycle will dwarf\n> the fork/exec latency anyway, no?\n> \n\nThanks for clarification. That was my initial idea as well, \nto not touch the pack-objects logic. \n\nHowever Duy send that patch (basically as is, \nI just made it apply again), \nand I guessed that I'd get to results\nfaster with an already existing approach.\n\nI will reconsider and try to remove the additional logic from \npack-objects again (so it will not get touched) and move it to the\nrepack itself. It is just a way to understand, 'what needs to be done'.\n\nStefan\n\n"},{"id":"224783","messageId":"CACsJy8D3FL0rG9YHuSOtobNffiqffU6J7_gfc85CiGuwUdRNnQ@mail.gmail.com","threadId":"34587","inReplyTo":"7v38qlec2w.fsf@alter.siamese.dyndns.org","subject":"Re: [PATCH 1/4] Build in git-repack","fromName":"Duy Nguyen","fromEmail":"pclouds@gmail.com","sentAt":"2013-08-08T02:44:50Z","receivedAt":"2013-08-08T02:44:50Z","isPatch":true,"sender":{"key":"pclouds@gmail.com","avatar":"https://avatars.githubusercontent.com/u/720?v=4"},"body":"On Wed, Aug 7, 2013 at 10:48 PM, Junio C Hamano <gitster@pobox.com> wrote:\n> Matthieu Moy <Matthieu.Moy@grenoble-inp.fr> writes:\n>\n>> [ It's cool you're working on this, I'd really like a git-repack in C.\n>>   That would fix this\n>>   http://thread.gmane.org/gmane.comp.version-control.git/226458 ]\n>>\n>> Stefan Beller <stefanbeller@googlemail.com> writes:\n>>\n>>> From: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>\n>>>\n>>> pack-objects learns a few more options to take over what's been done\n>>> by git-repack.sh. cmd_repack() becomes a wrapper around\n>>> cmd_pack_objects().\n>>\n>> I think the patch would read easier if these were split into two\n>> patches: one doing the real stuff in pack-objects, and then getting rid\n>> of git-repack.sh to replace it with a trivial built-in.\n>>\n>> Actually, I'm wondering why pack-objects requires so much changes.\n>> git-repack.sh was already a relatively small wrapper around\n>> pack-objects, and did not need the new options you add, so why are they\n>> needed? In particular adding the new --update-info option that just does\n>>\n>>> +    if (repack_flags & REPACK_UPDATE_INFO)\n>>> +            update_server_info(0);\n>>\n>> seems overkill to me: why don't you just let cmd_repack call\n>> update_server_info(0)?\n>\n> My feeling exactly.  I would rather see a patch that does not touch\n> pack-objects at all, and use run_command() interface to spawn it.\n> Once we do have to pack, the necessary processing cycle will dwarf\n> the fork/exec latency anyway, no?\n\nI'm not opposed to run_command(). I think the reason I wanted to move\nrepack functionality to pack-objects is to avoid reading sha-1 from\npack-objects and reconstruct the paths again from the sha-1. But for\nsimplicity, perhaps we should not touch pack-objects at all. Then we\ncould have builtin/repack.c instead of stuffing cmd_repack in\nbuiltin/pack-objects.c\n\n@Stefan, if you want to push this work, feel free to take it as _your_\npatch, rewrite as will. You don't need to retain my name.\n-- \nDuy\n"},{"id":"224784","messageId":"CACsJy8BfUJL6C2tignfgP5iQq2Egz03ZL7vDvVBJpaFWtD3XNQ@mail.gmail.com","threadId":"34587","inReplyTo":"1375884049-14961-3-git-send-email-stefanbeller@googlemail.com","subject":"Re: [PATCH 2/4] backup_file dummy function","fromName":"Duy Nguyen","fromEmail":"pclouds@gmail.com","sentAt":"2013-08-08T02:45:46Z","receivedAt":"2013-08-08T02:45:46Z","isPatch":true,"sender":{"key":"pclouds@gmail.com","avatar":"https://avatars.githubusercontent.com/u/720?v=4"},"body":"On Wed, Aug 7, 2013 at 9:00 PM, Stefan Beller\n<stefanbeller@googlemail.com> wrote:\n> ---\n>  pack-write.c | 5 +++++\n>  1 file changed, 5 insertions(+)\n>\n> diff --git a/pack-write.c b/pack-write.c\n> index e6aa7e3..b728ea2 100644\n> --- a/pack-write.c\n> +++ b/pack-write.c\n> @@ -344,6 +344,11 @@ struct sha1file *create_tmp_packfile(char **pack_tmp_name)\n>         return sha1fd(fd, *pack_tmp_name);\n>  }\n>\n> +void backup_file(char *filename)\n> +{\n> +\n> +}\n> +\n\nI think the function looks like this (before it got lost after rebase)\n\n+static void backup_file(const char *path)\n+{\n+       struct stat st;\n+       if (repack_flags & REPACK_IN_PROGRESS &&\n+           !stat(path, &st)) {\n+               struct strbuf old = STRBUF_INIT;\n+               strbuf_addf(&old, \"%s.old\", path);\n+               if (rename(path, old.buf))\n+                       die_errno(\"unable to rename pack %s\", path);\n+               string_list_append(&backup_files, strbuf_detach(&old, NULL));\n+       }\n+}\n+\n-- \nDuy\n"},{"id":"224790","messageId":"CALWbr2yQBK9D3HJ2k9xPryKL=TFh_u7V2K5qOZJQw=J=qYXs+g@mail.gmail.com","threadId":"34587","inReplyTo":"1375884049-14961-4-git-send-email-stefanbeller@googlemail.com","subject":"Re: [PATCH 3/4] pack-objects: do not print usage when repacking","fromName":"Antoine Pelisse","fromEmail":"apelisse@gmail.com","sentAt":"2013-08-08T06:40:51Z","receivedAt":"2013-08-08T06:40:51Z","isPatch":true,"sender":{"key":"apelisse@gmail.com","avatar":"https://avatars.githubusercontent.com/u/1929644?v=4"},"body":"On Wed, Aug 7, 2013 at 4:00 PM, Stefan Beller\n<stefanbeller@googlemail.com> wrote:\n> ---\n>  builtin/pack-objects.c | 2 +-\n>  1 file changed, 1 insertion(+), 1 deletion(-)\n>\n> diff --git a/builtin/pack-objects.c b/builtin/pack-objects.c\n> index 1742ea1..0bd8f3b 100644\n> --- a/builtin/pack-objects.c\n> +++ b/builtin/pack-objects.c\n> @@ -2585,7 +2585,7 @@ int cmd_pack_objects(int argc, const char **argv, const char *prefix)\n>                 base_name = argv[0];\n>                 argc--;\n>         }\n> -       if (pack_to_stdout != !base_name || argc)\n> +       if (!(repack_flags & REPACK_IN_PROGRESS) && (pack_to_stdout != !base_name || argc))\n>                 usage_with_options(pack_usage, pack_objects_options);\n>\n>         rp_av[rp_ac++] = \"pack-objects\";\n\nHello Stefan,\nI'm not sure I understand that and why it's needed, would you mind\nexplain it to me ? (and/or maybe add it to the commit message)\n\nThanks,\n"}]}