{"thread":{"id":"49985","subject":"[RFC PATCH 00/24] Add backup log","startedAt":"2018-12-09T10:45:05Z","lastAt":"2018-12-09T10:45:38Z","messageCount":25,"participants":["Nguyễn Thái Ngọc Duy"],"isPatch":true,"patchVersion":1,"patchTotal":24},"messages":[{"id":"364843","messageId":"20181209104419.12639-1-pclouds@gmail.com","threadId":"49985","inReplyTo":null,"subject":"[RFC PATCH 00/24] Add backup log","fromName":"Nguyễn Thái Ngọc Duy","fromEmail":"pclouds@gmail.com","sentAt":"2018-12-09T10:43:55Z","receivedAt":"2018-12-09T10:45:05Z","isPatch":true,"sender":{"key":"pclouds@gmail.com","avatar":"https://avatars.githubusercontent.com/u/720?v=4"},"body":"\"Backup log\" is similar to reflog. But instead of keeping track of ref\nchanges, it keeps track of file content changes. These could be from\nthe index (e.g. \"git add\" replacing something in the index), worktree\n(\"git reset --hard\" deleting everything) or in gitdir (\"git config\n--edit\", or deleted reflog).\n\nBackup log, when enabled, keeps the backup versions so you can undo if\nneeded. Head for 01/24 to have a better picture what it does, when\nbackups are made... This series adds a new plumbing command 'git\nbackup-log' to manage these backups.\n\nA couple things left to do:\n\n- high level UI design, including maybe extended SHA-1 syntax\n- whether \"git checkout <paths>\" should keep backups. I think doing it\n  unconditionally may be too much, but maybe keep backups of files\n  with \"precious\" attribute on\n- a UI to edit $GIT_DIR/info/excludes and gitattributes so we can make\n  backups of them\n- whether we should keep command causing the changes in the backup log\n  (e.g. this change is made by git-add, that one git-rebase...).\n  Reflog has this. I did not add it because it complicates the parsing\n  a bit and not sure if it's worth it.\n\nNguyễn Thái Ngọc Duy (24):\n  doc: introduce new \"backup log\" concept\n  backup-log: add \"update\" subcommand\n  read-cache.c: new flag for add_index_entry() to write to backup log\n  add: support backup log\n  update-index: support backup log with --keep-backup\n  commit: support backup log\n  apply: support backup log with --keep-backup\n  add--interactive: support backup log\n  backup-log.c: add API for walking backup log\n  backup-log: add cat command\n  backup-log: add diff command\n  backup-log: add log command\n  backup-log: add prune command\n  gc: prune backup logs\n  backup-log: keep all blob references around\n  sha1-file.c: let index_path() accept NULL istate\n  config --edit: support backup log\n  refs: keep backup of deleted reflog\n  unpack-trees.c: keep backup of ignored files being overwritten\n  reset --hard: keep backup of overwritten files\n  checkout -f: keep backup of overwritten files\n  am: keep backup of overwritten files on --skip or --abort\n  rebase: keep backup of overwritten files on --skip or --abort\n  FIXME\n\n .gitignore                         |   1 +\n Documentation/config/core.txt      |   5 +\n Documentation/git-apply.txt        |   3 +\n Documentation/git-backup-log.txt   | 109 ++++++++\n Documentation/git-update-index.txt |   3 +\n Makefile                           |   2 +\n apply.c                            |  38 ++-\n apply.h                            |   1 +\n backup-log.c                       | 388 +++++++++++++++++++++++++++++\n backup-log.h                       |  38 +++\n builtin.h                          |   1 +\n builtin/add.c                      |   5 +\n builtin/am.c                       |   3 +\n builtin/backup-log.c               | 371 +++++++++++++++++++++++++++\n builtin/checkout.c                 |   4 +\n builtin/commit.c                   |  16 +-\n builtin/config.c                   |  27 +-\n builtin/gc.c                       |   3 +\n builtin/pack-objects.c             |   9 +-\n builtin/rebase.c                   |   6 +-\n builtin/repack.c                   |   1 +\n builtin/reset.c                    |   2 +\n builtin/update-index.c             |   7 +\n cache.h                            |   2 +\n command-list.txt                   |   1 +\n git-add--interactive.perl          |  14 +-\n git.c                              |   1 +\n merge-recursive.c                  |   2 +-\n merge.c                            |   2 +\n parse-options.c                    |   2 +-\n reachable.c                        |   3 +\n read-cache.c                       |  49 +++-\n refs/files-backend.c               |  32 +++\n revision.c                         |   3 +\n sha1-file.c                        |   8 +-\n t/t2080-backup-log.sh              | 228 +++++++++++++++++\n unpack-trees.c                     | 143 +++++++++--\n unpack-trees.h                     |   6 +-\n 38 files changed, 1488 insertions(+), 51 deletions(-)\n create mode 100644 Documentation/git-backup-log.txt\n create mode 100644 backup-log.c\n create mode 100644 backup-log.h\n create mode 100644 builtin/backup-log.c\n create mode 100755 t/t2080-backup-log.sh\n\n-- \n2.20.0.rc2.486.g9832c05c3d\n\n"},{"id":"364844","messageId":"20181209104419.12639-2-pclouds@gmail.com","threadId":"49985","inReplyTo":"20181209104419.12639-1-pclouds@gmail.com","subject":"[PATCH 01/24] doc: introduce new \"backup log\" concept","fromName":"Nguyễn Thái Ngọc Duy","fromEmail":"pclouds@gmail.com","sentAt":"2018-12-09T10:43:56Z","receivedAt":"2018-12-09T10:45:06Z","isPatch":true,"sender":{"key":"pclouds@gmail.com","avatar":"https://avatars.githubusercontent.com/u/720?v=4"},"body":"A reflog file records changes of a certain ref over time. A backup log\nfile does a similar job, but it records changes of some files over time.\nThis is the main idea of it.\n\nThis is added so that we can support undoing certain changes. For\nexample, if you have carefully prepared your index with \"git add -p\"\nthen accidentally do \"git commit -a\", your well crafted index is lost\nwith no easy way to recover it. We could go the other way and make\n\"git commit -a\" complain loudly, but that has other bad side effects,\nthe big one is we may complain too loudly at the wrong time and too\noften. This \"do the right things (most of the time) but allow the\nuser to undo when we get it wrong\" approach seems more inline with how\ngit handles other things.\n\nThe current plan is to have three backup log files:\n\n- $GIT_DIR/index.bkl contains \"interesting\" changes made in the index.\n\n- $GIT_DIR/worktree.bkl contains \"interesting\" changes made in\n  worktree.\n\n- $GIT_DIR/common/gitdir.bkl contains changes made in other files\n  inside $GIT_DIR such as config, those in info/ directory, or reflog\n  file deletion (aka the mystical reflog graveyard)\n\nAll these only record \"interesting\" changes which will be defined\nlater. But a few examples of them are: \"git add -p\" is interesting,\nbut \"git reset HEAD\" is not. Similarly changes made in $GIT_DIR/config\nare usually interesting.\n\nThis patch does none of that! It adds a new man page for a new\nplumbing command \"git backup-log\" which does show what this\nfunctionality looks like.\n\nSigned-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>\n---\n Documentation/config/core.txt    |   5 ++\n Documentation/git-backup-log.txt | 107 +++++++++++++++++++++++++++++++\n 2 files changed, 112 insertions(+)\n create mode 100644 Documentation/git-backup-log.txt\n\ndiff --git a/Documentation/config/core.txt b/Documentation/config/core.txt\nindex d0e6635fe0..63b78cc048 100644\n--- a/Documentation/config/core.txt\n+++ b/Documentation/config/core.txt\n@@ -316,6 +316,11 @@ This value is true by default in a repository that has\n a working directory associated with it, and false by\n default in a bare repository.\n \n+core.backupLog::\n+\tIf true, many commands will keep backup content in object database\n+\tbefore they modify some file. See linkgit:git-backup-log[1] for more\n+\tinformation.\n+\n core.repositoryFormatVersion::\n \tInternal variable identifying the repository format and layout\n \tversion.\ndiff --git a/Documentation/git-backup-log.txt b/Documentation/git-backup-log.txt\nnew file mode 100644\nindex 0000000000..98998156c1\n--- /dev/null\n+++ b/Documentation/git-backup-log.txt\n@@ -0,0 +1,107 @@\n+git-backup-log(1)\n+=================\n+\n+NAME\n+----\n+git-backup-log - Manage backup log files\n+\n+\n+SYNOPSIS\n+--------\n+[verse]\n+'git backup-log' [--id=<id> | --path=<path>] log [<rev-options>] [[--] <path>...]\n+'git backup-log' [--id=<id> | --path=<path>] cat [--before] [--hash] <change-id> <path>\n+'git backup-log' [--id=<id> | --path=<path>] diff [<diff-options>] <change-id>\n+'git backup-log' [--id=<id> | --path=<path>] prune [--expire=<time>]\n+'git backup-log' [--id=<id> | --path=<path>] update <path> <old-hash> <new-hash>\n+\n+DESCRIPTION\n+-----------\n+Backup log records changes of certain files in the object database so\n+that if some file is overwritten by accident, you could still get the\n+original content back.\n+\n+Backup log is enabled by setting core.backupLog to true and the\n+following commands will save backups:\n+\n+- linkgit:git-add[1] keeps all index changes. File removal is not\n+  recorded.\n+- linkgit:git-commit[1] keep all index changes in `-a` or partial\n+  commit mode.\n+- linkgit:git-apply[1] and linkgit:git-update-index[1] will keep\n+  changes if `--keep-backup` is specified\n+- Changes of `$GIT_DIR/config` made by `git config --edit` are kept.\n+- Deleted reflogs are kept. References from this deleted reflog will\n+  not be kept at the next garbage collection though. This is mostly\n+  meant to immediately undo an accidental branch deletion.\n+- linkgit:git-checkout[1] when switching branches and\n+  linkgit:git-merge[1] will make a backup before overwriting\n+  ignored files.\n+- linkgit:git-checkout[1] with `--force`, linkgit:git-reset[1] with\n+  `--hard` or linkgit:git-am[1] and linkgit:git-rebase[1] with\n+  `--skip` or `--abort` will make a backup before overwriting non\n+  up-to-date files.\n+\n+Backups are split in three groups, changes related in the index, in\n+working directory or in $GIT_DIR. These can be selected with `--id`\n+parameter as `index`, `worktree` and `gitdir` respectively.\n+Alternatively file path of these are `$GIT_DIR/index.bkl`,\n+`$GIT_DIR/worktree.bkl` and `$GIT_DIR/common/gitdir.bkl` which could\n+be specified with `--path`\n+\n+This command is split into subcommands:\n+\n+update::\n+\tAdd a new change associated with `<path>` from `<old-hash>` to\n+\t`<new-hash>` to the selected backup log file. `<path>` must be\n+\tnormalized.\n+\n+log::\n+\tView the selected backup log (optionally filtered by pathspec).\n+\tBy default, the diff of the change is shown.\n+\n+cat::\n+\tOutput the file content or their object name of a specific\n+\tchange.\n+\n+diff::\n+\tOutput the diff of a specific change.\n+\n+prune::\n+\tPrune the backup log, delete updates older than a specific\n+\ttime or invalid entries. The actual backup content is still in\n+\tthe object database and may need to be pruned separatedly.\n+\n+OPTIONS\n+-------\n+\n+--id=<id>::\n+\tThe name of the the backup log file. Supported names are\n+\t`index`, `worktree` and `gitdir`.\n+\n+--path=<path>::\n+\tThe path of a backup log file.\n+\n+--before::\n+\tShow the version before the change instead. This is most\n+\tuseful when the change is a file deletion.\n+\n+--hash::\n+\tShow the object name instead of content.\n+\n+--expire=<time>::\n+\tThe cutoff time for pruning backup log. The default is three\n+\tmonths ago.\n+\n+<path>::\n+\tThe path of the file where the change is made.\n+\n+<old-hash>::\n+\tThe blob hash of the content before the change.\n+\n+<old-hash>::\n+\tThe blob hash of the content after the change.\n+\n+GIT\n+---\n+Part of the linkgit:git[1] suite\n-- \n2.20.0.rc2.486.g9832c05c3d\n\n"},{"id":"364845","messageId":"20181209104419.12639-3-pclouds@gmail.com","threadId":"49985","inReplyTo":"20181209104419.12639-1-pclouds@gmail.com","subject":"[PATCH 02/24] backup-log: add \"update\" subcommand","fromName":"Nguyễn Thái Ngọc Duy","fromEmail":"pclouds@gmail.com","sentAt":"2018-12-09T10:43:57Z","receivedAt":"2018-12-09T10:45:08Z","isPatch":true,"sender":{"key":"pclouds@gmail.com","avatar":"https://avatars.githubusercontent.com/u/720?v=4"},"body":"This defines backup log file format and adds basic support for writing\nnew entries to backup log files. The format is the same as reflog\nexcept that \"message\" field becomes \"path\".\n\nSimilar to reflog, updating is done by appending to the end of the file\ninstead of creating a branch new file and do an atomic rename. If the\nbackup log file gets large, regenerating whole file could take longer,\nand we should keep backup log overhead to minimum since it will be\ncalled by a bunch of commands later.\n\nSigned-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>\n---\n .gitignore            |  1 +\n Makefile              |  2 ++\n backup-log.c          | 46 ++++++++++++++++++++++++\n backup-log.h          | 13 +++++++\n builtin.h             |  1 +\n builtin/backup-log.c  | 81 +++++++++++++++++++++++++++++++++++++++++++\n command-list.txt      |  1 +\n git.c                 |  1 +\n parse-options.c       |  2 +-\n t/t2080-backup-log.sh | 20 +++++++++++\n 10 files changed, 167 insertions(+), 1 deletion(-)\n create mode 100644 backup-log.c\n create mode 100644 backup-log.h\n create mode 100644 builtin/backup-log.c\n create mode 100755 t/t2080-backup-log.sh\n\ndiff --git a/.gitignore b/.gitignore\nindex 0d77ea5894..9c0427c709 100644\n--- a/.gitignore\n+++ b/.gitignore\n@@ -20,6 +20,7 @@\n /git-apply\n /git-archimport\n /git-archive\n+/git-backup-log\n /git-bisect\n /git-bisect--helper\n /git-blame\ndiff --git a/Makefile b/Makefile\nindex 1a44c811aa..d1e1a925c3 100644\n--- a/Makefile\n+++ b/Makefile\n@@ -836,6 +836,7 @@ LIB_OBJS += archive-tar.o\n LIB_OBJS += archive-zip.o\n LIB_OBJS += argv-array.o\n LIB_OBJS += attr.o\n+LIB_OBJS += backup-log.o\n LIB_OBJS += base85.o\n LIB_OBJS += bisect.o\n LIB_OBJS += blame.o\n@@ -1027,6 +1028,7 @@ BUILTIN_OBJS += builtin/am.o\n BUILTIN_OBJS += builtin/annotate.o\n BUILTIN_OBJS += builtin/apply.o\n BUILTIN_OBJS += builtin/archive.o\n+BUILTIN_OBJS += builtin/backup-log.o\n BUILTIN_OBJS += builtin/bisect--helper.o\n BUILTIN_OBJS += builtin/blame.o\n BUILTIN_OBJS += builtin/branch.o\ndiff --git a/backup-log.c b/backup-log.c\nnew file mode 100644\nindex 0000000000..c1e805b09e\n--- /dev/null\n+++ b/backup-log.c\n@@ -0,0 +1,46 @@\n+#include \"cache.h\"\n+#include \"backup-log.h\"\n+#include \"lockfile.h\"\n+#include \"strbuf.h\"\n+\n+void bkl_append(struct strbuf *output, const char *path,\n+\t\tconst struct object_id *from,\n+\t\tconst struct object_id *to)\n+{\n+\tif (oideq(from, to))\n+\t\treturn;\n+\n+\tstrbuf_addf(output, \"%s %s %s\\t%s\\n\", oid_to_hex(from),\n+\t\t    oid_to_hex(to), git_committer_info(0), path);\n+}\n+\n+static int bkl_write_unlocked(const char *path, struct strbuf *new_log)\n+{\n+\tint fd = open(path, O_CREAT | O_WRONLY | O_APPEND, 0666);\n+\tif (fd == -1)\n+\t\treturn error_errno(_(\"unable to open %s\"), path);\n+\tif (write_in_full(fd, new_log->buf, new_log->len) < 0) {\n+\t\tclose(fd);\n+\t\treturn error_errno(_(\"unable to update %s\"), path);\n+\t}\n+\tclose(fd);\n+\treturn 0;\n+}\n+\n+int bkl_write(const char *path, struct strbuf *new_log)\n+{\n+\tstruct lock_file lk;\n+\tint ret;\n+\n+\tret = hold_lock_file_for_update(&lk, path, LOCK_REPORT_ON_ERROR);\n+\tif (ret == -1)\n+\t\treturn -1;\n+\tret = bkl_write_unlocked(path, new_log);\n+\t/*\n+\t * We do not write the the .lock file and append to the real one\n+\t * instead to reduce update cost. So we can't commit even in\n+\t * successful case.\n+\t */\n+\trollback_lock_file(&lk);\n+\treturn ret;\n+}\ndiff --git a/backup-log.h b/backup-log.h\nnew file mode 100644\nindex 0000000000..5e475d6f35\n--- /dev/null\n+++ b/backup-log.h\n@@ -0,0 +1,13 @@\n+#ifndef __BACKUP_LOG_H__\n+#define __BACKUP_LOG_H__\n+\n+struct object_id;\n+struct strbuf;\n+\n+void bkl_append(struct strbuf *output, const char *path,\n+\t\tconst struct object_id *from,\n+\t\tconst struct object_id *to);\n+\n+int bkl_write(const char *path, struct strbuf *new_log);\n+\n+#endif\ndiff --git a/builtin.h b/builtin.h\nindex 6538932e99..4e16c52bfa 100644\n--- a/builtin.h\n+++ b/builtin.h\n@@ -132,6 +132,7 @@ extern int cmd_am(int argc, const char **argv, const char *prefix);\n extern int cmd_annotate(int argc, const char **argv, const char *prefix);\n extern int cmd_apply(int argc, const char **argv, const char *prefix);\n extern int cmd_archive(int argc, const char **argv, const char *prefix);\n+extern int cmd_backup_log(int argc, const char **argv, const char *prefix);\n extern int cmd_bisect__helper(int argc, const char **argv, const char *prefix);\n extern int cmd_blame(int argc, const char **argv, const char *prefix);\n extern int cmd_branch(int argc, const char **argv, const char *prefix);\ndiff --git a/builtin/backup-log.c b/builtin/backup-log.c\nnew file mode 100644\nindex 0000000000..75a02c8878\n--- /dev/null\n+++ b/builtin/backup-log.c\n@@ -0,0 +1,81 @@\n+#include \"cache.h\"\n+#include \"builtin.h\"\n+#include \"backup-log.h\"\n+#include \"parse-options.h\"\n+\n+static char const * const backup_log_usage[] = {\n+\tN_(\"git backup-log [--path=<path> | --id=<id>] update <path> <old-hash> <new-hash>\"),\n+\tNULL\n+};\n+\n+static int update(int argc, const char **argv,\n+\t\t  const char *prefix, const char *log_path)\n+{\n+\tstruct strbuf sb = STRBUF_INIT;\n+\tstruct object_id oid_from, oid_to;\n+\tconst char *path;\n+\tint ret;\n+\n+\tif (argc != 4)\n+\t\tusage_with_options(backup_log_usage, NULL);\n+\n+\tpath = argv[1];\n+\n+\tif (get_oid(argv[2], &oid_from))\n+\t\tdie(_(\"not a valid object name: %s\"), argv[2]);\n+\n+\tif (get_oid(argv[3], &oid_to))\n+\t\tdie(_(\"not a valid object name: %s\"), argv[3]);\n+\n+\tbkl_append(&sb, path, &oid_from, &oid_to);\n+\tret = bkl_write(log_path, &sb);\n+\tstrbuf_release(&sb);\n+\n+\treturn ret;\n+}\n+\n+static char *log_id_to_path(const char *id)\n+{\n+\tif (!strcmp(id, \"index\"))\n+\t\treturn git_pathdup(\"index.bkl\");\n+\telse if (!strcmp(id, \"worktree\"))\n+\t\treturn git_pathdup(\"worktree.bkl\");\n+\telse if (!strcmp(id, \"gitdir\"))\n+\t\treturn git_pathdup(\"common/gitdir.bkl\");\n+\telse\n+\t\tdie(_(\"backup log id '%s' is not recognized\"), id);\n+}\n+\n+int cmd_backup_log(int argc, const char **argv, const char *prefix)\n+{\n+\tconst char *log_id = NULL;\n+\tconst char *log_path = NULL;\n+\tchar *to_free = NULL;\n+\tstruct option common_opts[] = {\n+\t\tOPT_STRING(0, \"id\", &log_id, N_(\"id\"), N_(\"backup log file id\")),\n+\t\tOPT_FILENAME(0, \"path\", &log_path, N_(\"backup log file path\")),\n+\t\tOPT_END()\n+\t};\n+\n+\targc = parse_options(argc, argv, prefix, common_opts,\n+\t\t\t     backup_log_usage,\n+\t\t\t     PARSE_OPT_STOP_AT_NON_OPTION);\n+\tif (!argc)\n+\t\tdie(_(\"expected a subcommand\"));\n+\n+\tif (log_id) {\n+\t\tif (log_path)\n+\t\t\tdie(_(\"--id and --path are incompatible\"));\n+\t\tlog_path = to_free = log_id_to_path(log_id);\n+\t}\n+\tif (!log_path)\n+\t\tdie(_(\"either --id or --path is required\"));\n+\n+\tif (!strcmp(argv[0], \"update\"))\n+\t\treturn update(argc, argv, prefix, log_path);\n+\telse\n+\t\tdie(_(\"unknown subcommand: %s\"), argv[0]);\n+\n+\tfree(to_free);\n+\treturn 0;\n+}\ndiff --git a/command-list.txt b/command-list.txt\nindex 3a9af104b5..d4162f73d5 100644\n--- a/command-list.txt\n+++ b/command-list.txt\n@@ -51,6 +51,7 @@ git-annotate                            ancillaryinterrogators\n git-apply                               plumbingmanipulators            complete\n git-archimport                          foreignscminterface\n git-archive                             mainporcelain\n+git-backup-log                          plumbinginterrogators\n git-bisect                              mainporcelain           info\n git-blame                               ancillaryinterrogators          complete\n git-branch                              mainporcelain           history\ndiff --git a/git.c b/git.c\nindex 2f604a41ea..d46d22c8b1 100644\n--- a/git.c\n+++ b/git.c\n@@ -447,6 +447,7 @@ static struct cmd_struct commands[] = {\n \t{ \"annotate\", cmd_annotate, RUN_SETUP | NO_PARSEOPT },\n \t{ \"apply\", cmd_apply, RUN_SETUP_GENTLY },\n \t{ \"archive\", cmd_archive, RUN_SETUP_GENTLY },\n+\t{ \"backup-log\", cmd_backup_log, RUN_SETUP_GENTLY },\n \t{ \"bisect--helper\", cmd_bisect__helper, RUN_SETUP },\n \t{ \"blame\", cmd_blame, RUN_SETUP },\n \t{ \"branch\", cmd_branch, RUN_SETUP | DELAY_PAGER_CONFIG },\ndiff --git a/parse-options.c b/parse-options.c\nindex 3b874a83a0..e1fc080cc3 100644\n--- a/parse-options.c\n+++ b/parse-options.c\n@@ -705,7 +705,7 @@ static int usage_with_options_internal(struct parse_opt_ctx_t *ctx,\n \n \tneed_newline = 1;\n \n-\tfor (; opts->type != OPTION_END; opts++) {\n+\tfor (; opts && opts->type != OPTION_END; opts++) {\n \t\tsize_t pos;\n \t\tint pad;\n \ndiff --git a/t/t2080-backup-log.sh b/t/t2080-backup-log.sh\nnew file mode 100755\nindex 0000000000..267c34bb25\n--- /dev/null\n+++ b/t/t2080-backup-log.sh\n@@ -0,0 +1,20 @@\n+#!/bin/sh\n+\n+test_description='backup-log'\n+\n+. ./test-lib.sh\n+\n+test_expect_success 'setup' '\n+\ttest_commit initial &&\n+\tmkdir sub\n+'\n+\n+test_expect_success 'backup-log add new item' '\n+\tID=$(git rev-parse HEAD:./initial.t) &&\n+\ttest_tick &&\n+\tgit -C sub backup-log --id=index update foo $ZERO_OID $ID &&\n+\techo \"$ZERO_OID $ID $GIT_COMMITTER_NAME <$GIT_COMMITTER_EMAIL> $test_tick -0700\tfoo\" >expected &&\n+\ttest_cmp expected .git/index.bkl\n+'\n+\n+test_done\n-- \n2.20.0.rc2.486.g9832c05c3d\n\n"},{"id":"364846","messageId":"20181209104419.12639-4-pclouds@gmail.com","threadId":"49985","inReplyTo":"20181209104419.12639-1-pclouds@gmail.com","subject":"[PATCH 03/24] read-cache.c: new flag for add_index_entry() to write to backup log","fromName":"Nguyễn Thái Ngọc Duy","fromEmail":"pclouds@gmail.com","sentAt":"2018-12-09T10:43:58Z","receivedAt":"2018-12-09T10:45:09Z","isPatch":true,"sender":{"key":"pclouds@gmail.com","avatar":"https://avatars.githubusercontent.com/u/720?v=4"},"body":"Index update API is updated to write to backup log if requested. The\nentry deletion API is not updated because file removal is not\n\"interesting\" from the undo point of view.\n\nNote, we do double locking when writing $GIT_DIR/index now:\n\n- $GIT_DIR/index.lock is created\n- then $GIT_DIR/index.bkl.lock is created\n\nNobody will lock these in reverse order so no chance for dead lock\n(yet)\n\nSigned-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>\n---\n cache.h      |  2 ++\n read-cache.c | 49 +++++++++++++++++++++++++++++++++++++++++++++++--\n 2 files changed, 49 insertions(+), 2 deletions(-)\n\ndiff --git a/cache.h b/cache.h\nindex ca36b44ee0..51ffae7961 100644\n--- a/cache.h\n+++ b/cache.h\n@@ -336,6 +336,7 @@ struct index_state {\n \tuint64_t fsmonitor_last_update;\n \tstruct ewah_bitmap *fsmonitor_dirty;\n \tstruct mem_pool *ce_mem_pool;\n+\tstruct strbuf *backup_log;\n };\n \n extern struct index_state the_index;\n@@ -745,6 +746,7 @@ extern int index_name_pos(const struct index_state *, const char *name, int name\n #define ADD_CACHE_JUST_APPEND 8\t\t/* Append only; tree.c::read_tree() */\n #define ADD_CACHE_NEW_ONLY 16\t\t/* Do not replace existing ones */\n #define ADD_CACHE_KEEP_CACHE_TREE 32\t/* Do not invalidate cache-tree */\n+#define ADD_CACHE_LOG_UPDATES 64\t/* Log changes in $GIT_DIR/index.bkl */\n extern int add_index_entry(struct index_state *, struct cache_entry *ce, int option);\n extern void rename_index_entry_at(struct index_state *, int pos, const char *new_name);\n \ndiff --git a/read-cache.c b/read-cache.c\nindex bd45dc3e24..a53cfabc2e 100644\n--- a/read-cache.c\n+++ b/read-cache.c\n@@ -25,6 +25,7 @@\n #include \"fsmonitor.h\"\n #include \"thread-utils.h\"\n #include \"progress.h\"\n+#include \"backup-log.h\"\n \n /* Mask for the name length in ce_flags in the on-disk index */\n \n@@ -691,6 +692,21 @@ void set_object_name_for_intent_to_add_entry(struct cache_entry *ce)\n \toidcpy(&ce->oid, &oid);\n }\n \n+static void update_backup_log(struct index_state *istate,\n+\t\t\t      const struct object_id *prev,\n+\t\t\t      const struct cache_entry *ce)\n+{\n+\tstruct strbuf *sb = istate->backup_log;\n+\n+\tif (!sb) {\n+\t\tsb = xmalloc(sizeof(*sb));\n+\t\tstrbuf_init(sb, 0);\n+\t\tistate->backup_log = sb;\n+\t}\n+\n+\tbkl_append(sb, ce->name, prev, &ce->oid);\n+}\n+\n int add_to_index(struct index_state *istate, const char *path, struct stat *st, int flags)\n {\n \tint namelen, was_same;\n@@ -765,6 +781,7 @@ int add_to_index(struct index_state *istate, const char *path, struct stat *st,\n \t\t\tdiscard_cache_entry(ce);\n \t\t\treturn error(\"unable to index file %s\", path);\n \t\t}\n+\t\tadd_option |= flags & ADD_CACHE_LOG_UPDATES;\n \t} else\n \t\tset_object_name_for_intent_to_add_entry(ce);\n \n@@ -1257,6 +1274,7 @@ static int add_index_entry_with_check(struct index_state *istate, struct cache_e\n \tint ok_to_replace = option & ADD_CACHE_OK_TO_REPLACE;\n \tint skip_df_check = option & ADD_CACHE_SKIP_DFCHECK;\n \tint new_only = option & ADD_CACHE_NEW_ONLY;\n+\tstruct object_id backup_prev;\n \n \tif (!(option & ADD_CACHE_KEEP_CACHE_TREE))\n \t\tcache_tree_invalidate_path(istate, ce->name);\n@@ -1273,8 +1291,12 @@ static int add_index_entry_with_check(struct index_state *istate, struct cache_e\n \n \t/* existing match? Just replace it. */\n \tif (pos >= 0) {\n-\t\tif (!new_only)\n-\t\t\treplace_index_entry(istate, pos, ce);\n+\t\tif (new_only)\n+\t\t\treturn 0;\n+\n+\t\tif (option & ADD_CACHE_LOG_UPDATES)\n+\t\t\tupdate_backup_log(istate, &istate->cache[pos]->oid, ce);\n+\t\treplace_index_entry(istate, pos, ce);\n \t\treturn 0;\n \t}\n \tpos = -pos-1;\n@@ -1282,6 +1304,7 @@ static int add_index_entry_with_check(struct index_state *istate, struct cache_e\n \tif (!(option & ADD_CACHE_KEEP_CACHE_TREE))\n \t\tuntracked_cache_add_to_index(istate, ce->name);\n \n+\toidclr(&backup_prev);\n \t/*\n \t * Inserting a merged entry (\"stage 0\") into the index\n \t * will always replace all non-merged entries..\n@@ -1289,6 +1312,8 @@ static int add_index_entry_with_check(struct index_state *istate, struct cache_e\n \tif (pos < istate->cache_nr && ce_stage(ce) == 0) {\n \t\twhile (ce_same_name(istate->cache[pos], ce)) {\n \t\t\tok_to_add = 1;\n+\t\t\tif (ce_stage(ce) == 2)\n+\t\t\t\toidcpy(&backup_prev, &istate->cache[pos]->oid);\n \t\t\tif (!remove_index_entry_at(istate, pos))\n \t\t\t\tbreak;\n \t\t}\n@@ -1307,6 +1332,8 @@ static int add_index_entry_with_check(struct index_state *istate, struct cache_e\n \t\tpos = index_name_stage_pos(istate, ce->name, ce_namelen(ce), ce_stage(ce));\n \t\tpos = -pos-1;\n \t}\n+\tif (option & ADD_CACHE_LOG_UPDATES)\n+\t\tupdate_backup_log(istate, &backup_prev, ce);\n \treturn pos + 1;\n }\n \n@@ -2323,6 +2350,10 @@ int discard_index(struct index_state *istate)\n \tdiscard_split_index(istate);\n \tfree_untracked_cache(istate->untracked);\n \tistate->untracked = NULL;\n+\tif (istate->backup_log) {\n+\t\tstrbuf_release(istate->backup_log);\n+\t\tFREE_AND_NULL(istate->backup_log);\n+\t}\n \n \tif (istate->ce_mem_pool) {\n \t\tmem_pool_discard(istate->ce_mem_pool, should_validate_cache_entries());\n@@ -3157,6 +3188,20 @@ int write_locked_index(struct index_state *istate, struct lock_file *lock,\n \tif (istate->fsmonitor_last_update)\n \t\tfill_fsmonitor_bitmap(istate);\n \n+\tif (istate->backup_log && istate->backup_log->len) {\n+\t\tstruct strbuf sb = STRBUF_INIT;\n+\t\tchar *path = get_locked_file_path(lock);\n+\n+\t\tstrbuf_addf(&sb, \"%s.bkl\", path);\n+\t\tfree(path);\n+\t\tif (bkl_write(sb.buf, istate->backup_log)) {\n+\t\t\tstrbuf_release(&sb);\n+\t\t\treturn -1;\n+\t\t}\n+\t\tstrbuf_reset(istate->backup_log);\n+\t\tstrbuf_release(&sb);\n+\t}\n+\n \tif (!si || alternate_index_output ||\n \t    (istate->cache_changed & ~EXTMASK)) {\n \t\tif (si)\n-- \n2.20.0.rc2.486.g9832c05c3d\n\n"},{"id":"364847","messageId":"20181209104419.12639-5-pclouds@gmail.com","threadId":"49985","inReplyTo":"20181209104419.12639-1-pclouds@gmail.com","subject":"[PATCH 04/24] add: support backup log","fromName":"Nguyễn Thái Ngọc Duy","fromEmail":"pclouds@gmail.com","sentAt":"2018-12-09T10:43:59Z","receivedAt":"2018-12-09T10:45:11Z","isPatch":true,"sender":{"key":"pclouds@gmail.com","avatar":"https://avatars.githubusercontent.com/u/720?v=4"},"body":"There is not much to say about the actual change in this patch, which\nis straightforward. There is something to say about the lack of change\nthough.\n\nThe definition of \"interesting\" changes to keep in backup log\npreviously is \"file modification, except file removal\". It is further\nrefined: only changes coming from worktree (for from the user to be\nmore accurate) are interesting. Changes in the index from object\ndatabase (e.g. merging, switching branches, resetting...) are not\ninteresting because the actual content is already in the object\ndatabase and can be recovered (provided that you still have the\nhistory of commands you used, of course)\n\nSigned-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>\n---\n builtin/add.c         |  5 +++++\n t/t2080-backup-log.sh | 11 +++++++++++\n 2 files changed, 16 insertions(+)\n\ndiff --git a/builtin/add.c b/builtin/add.c\nindex f65c172299..f21d9efdd9 100644\n--- a/builtin/add.c\n+++ b/builtin/add.c\n@@ -393,6 +393,7 @@ int cmd_add(int argc, const char **argv, const char *prefix)\n \tint require_pathspec;\n \tchar *seen = NULL;\n \tstruct lock_file lock_file = LOCK_INIT;\n+\tint core_backup_log = 0;\n \n \tgit_config(add_config, NULL);\n \n@@ -439,6 +440,10 @@ int cmd_add(int argc, const char **argv, const char *prefix)\n \t\t (!(addremove || take_worktree_changes)\n \t\t  ? ADD_CACHE_IGNORE_REMOVAL : 0));\n \n+\trepo_config_get_bool(the_repository, \"core.backuplog\", &core_backup_log);\n+\tif (core_backup_log)\n+\t\tflags |= ADD_CACHE_LOG_UPDATES;\n+\n \tif (require_pathspec && argc == 0) {\n \t\tfprintf(stderr, _(\"Nothing specified, nothing added.\\n\"));\n \t\tfprintf(stderr, _(\"Maybe you wanted to say 'git add .'?\\n\"));\ndiff --git a/t/t2080-backup-log.sh b/t/t2080-backup-log.sh\nindex 267c34bb25..f7bdaaa3f6 100755\n--- a/t/t2080-backup-log.sh\n+++ b/t/t2080-backup-log.sh\n@@ -17,4 +17,15 @@ test_expect_success 'backup-log add new item' '\n \ttest_cmp expected .git/index.bkl\n '\n \n+test_expect_success 'add writes backup log' '\n+\ttest_tick &&\n+\techo update >>initial.t &&\n+\tOLD=$(git rev-parse :./initial.t) &&\n+\tNEW=$(git hash-object initial.t) &&\n+\tgit -c core.backupLog=true add initial.t &&\n+\techo \"$OLD $NEW $GIT_COMMITTER_NAME <$GIT_COMMITTER_EMAIL> $test_tick -0700\tinitial.t\" >expected &&\n+\ttail -n1 .git/index.bkl >actual &&\n+\ttest_cmp expected actual\n+'\n+\n test_done\n-- \n2.20.0.rc2.486.g9832c05c3d\n\n"},{"id":"364848","messageId":"20181209104419.12639-6-pclouds@gmail.com","threadId":"49985","inReplyTo":"20181209104419.12639-1-pclouds@gmail.com","subject":"[PATCH 05/24] update-index: support backup log with --keep-backup","fromName":"Nguyễn Thái Ngọc Duy","fromEmail":"pclouds@gmail.com","sentAt":"2018-12-09T10:44:00Z","receivedAt":"2018-12-09T10:45:12Z","isPatch":true,"sender":{"key":"pclouds@gmail.com","avatar":"https://avatars.githubusercontent.com/u/720?v=4"},"body":"Since this is a plumbing command, backup log support remains off by\ndefault and only active when both --keep-backup and core.backupLog=true\nare specified.\n\nThe check of core.backupLog is mostly for convenient, the calling script\ndoes not have to explicitly check core.backupLog every time it executes\nupdate-index. Truly disabling backup log must be done with something\nlike\n\n    git -c core.backupLog=false update-index ...\n\nSigned-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>\n---\n Documentation/git-update-index.txt |  3 +++\n builtin/update-index.c             |  7 +++++++\n t/t2080-backup-log.sh              | 11 +++++++++++\n 3 files changed, 21 insertions(+)\n\ndiff --git a/Documentation/git-update-index.txt b/Documentation/git-update-index.txt\nindex 1c4d146a41..31fe330c88 100644\n--- a/Documentation/git-update-index.txt\n+++ b/Documentation/git-update-index.txt\n@@ -218,6 +218,9 @@ will remove the intended effect of the option.\n \tthe configured value will take effect next time the index is\n \tread and this will remove the intended effect of the option.\n \n+--keep-backup::\n+\tEnable index backup log.\n+\n \\--::\n \tDo not interpret any more arguments as options.\n \ndiff --git a/builtin/update-index.c b/builtin/update-index.c\nindex 31e7cce301..295b5f5277 100644\n--- a/builtin/update-index.c\n+++ b/builtin/update-index.c\n@@ -28,6 +28,8 @@\n static int allow_add;\n static int allow_remove;\n static int allow_replace;\n+static int update_backup_log;\n+static int core_backup_log;\n static int info_only;\n static int force_remove;\n static int verbose;\n@@ -289,6 +291,7 @@ static int add_one_path(const struct cache_entry *old, const char *path, int len\n \t}\n \toption = allow_add ? ADD_CACHE_OK_TO_ADD : 0;\n \toption |= allow_replace ? ADD_CACHE_OK_TO_REPLACE : 0;\n+\toption |= update_backup_log && core_backup_log ? ADD_CACHE_LOG_UPDATES : 0;\n \tif (add_cache_entry(ce, option)) {\n \t\tdiscard_cache_entry(ce);\n \t\treturn error(\"%s: cannot add to the index - missing --add option?\", path);\n@@ -419,6 +422,7 @@ static int add_cacheinfo(unsigned int mode, const struct object_id *oid,\n \t\tce->ce_flags |= CE_VALID;\n \toption = allow_add ? ADD_CACHE_OK_TO_ADD : 0;\n \toption |= allow_replace ? ADD_CACHE_OK_TO_REPLACE : 0;\n+\toption |= update_backup_log && core_backup_log ? ADD_CACHE_LOG_UPDATES : 0;\n \tif (add_cache_entry(ce, option))\n \t\treturn error(\"%s: cannot add to the index - missing --add option?\",\n \t\t\t     path);\n@@ -969,6 +973,8 @@ int cmd_update_index(int argc, const char **argv, const char *prefix)\n \t\t\tN_(\"let files replace directories and vice-versa\"), 1),\n \t\tOPT_SET_INT(0, \"remove\", &allow_remove,\n \t\t\tN_(\"notice files missing from worktree\"), 1),\n+\t\tOPT_SET_INT(0, \"keep-backup\", &update_backup_log,\n+\t\t\tN_(\"update index backup log if core.backupLog is set\"), 1),\n \t\tOPT_BIT(0, \"unmerged\", &refresh_args.flags,\n \t\t\tN_(\"refresh even if index contains unmerged entries\"),\n \t\t\tREFRESH_UNMERGED),\n@@ -1060,6 +1066,7 @@ int cmd_update_index(int argc, const char **argv, const char *prefix)\n \t\tusage_with_options(update_index_usage, options);\n \n \tgit_config(git_default_config, NULL);\n+\trepo_config_get_bool(the_repository, \"core.backupLog\", &core_backup_log);\n \n \t/* we will diagnose later if it turns out that we need to update it */\n \tnewfd = hold_locked_index(&lock_file, 0);\ndiff --git a/t/t2080-backup-log.sh b/t/t2080-backup-log.sh\nindex f7bdaaa3f6..6b3814c172 100755\n--- a/t/t2080-backup-log.sh\n+++ b/t/t2080-backup-log.sh\n@@ -28,4 +28,15 @@ test_expect_success 'add writes backup log' '\n \ttest_cmp expected actual\n '\n \n+test_expect_success 'update-index --keep-backup writes backup log' '\n+\ttest_tick &&\n+\techo update-index >>initial.t &&\n+\tOLD=$(git rev-parse :./initial.t) &&\n+\tgit -c core.backupLog=true update-index --keep-backup initial.t &&\n+\tNEW=$(git hash-object initial.t) &&\n+\techo \"$OLD $NEW $GIT_COMMITTER_NAME <$GIT_COMMITTER_EMAIL> $test_tick -0700\tinitial.t\" >expected &&\n+\ttail -n1 .git/index.bkl >actual &&\n+\ttest_cmp expected actual\n+'\n+\n test_done\n-- \n2.20.0.rc2.486.g9832c05c3d\n\n"},{"id":"364850","messageId":"20181209104419.12639-9-pclouds@gmail.com","threadId":"49985","inReplyTo":"20181209104419.12639-1-pclouds@gmail.com","subject":"[PATCH 08/24] add--interactive: support backup log","fromName":"Nguyễn Thái Ngọc Duy","fromEmail":"pclouds@gmail.com","sentAt":"2018-12-09T10:44:03Z","receivedAt":"2018-12-09T10:45:15Z","isPatch":true,"sender":{"key":"pclouds@gmail.com","avatar":"https://avatars.githubusercontent.com/u/720?v=4"},"body":"Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>\n---\n git-add--interactive.perl | 14 +++++++-------\n 1 file changed, 7 insertions(+), 7 deletions(-)\n\ndiff --git a/git-add--interactive.perl b/git-add--interactive.perl\nindex 20eb81cc92..730133f57e 100755\n--- a/git-add--interactive.perl\n+++ b/git-add--interactive.perl\n@@ -102,28 +102,28 @@ sub colored {\n my %patch_modes = (\n \t'stage' => {\n \t\tDIFF => 'diff-files -p',\n-\t\tAPPLY => sub { apply_patch 'apply --cached', @_; },\n+\t\tAPPLY => sub { apply_patch 'apply --cached --keep-backup', @_; },\n \t\tAPPLY_CHECK => 'apply --cached',\n \t\tFILTER => 'file-only',\n \t\tIS_REVERSE => 0,\n \t},\n \t'stash' => {\n \t\tDIFF => 'diff-index -p HEAD',\n-\t\tAPPLY => sub { apply_patch 'apply --cached', @_; },\n+\t\tAPPLY => sub { apply_patch 'apply --cached --keep-backup', @_; },\n \t\tAPPLY_CHECK => 'apply --cached',\n \t\tFILTER => undef,\n \t\tIS_REVERSE => 0,\n \t},\n \t'reset_head' => {\n \t\tDIFF => 'diff-index -p --cached',\n-\t\tAPPLY => sub { apply_patch 'apply -R --cached', @_; },\n+\t\tAPPLY => sub { apply_patch 'apply -R --cached --keep-backup', @_; },\n \t\tAPPLY_CHECK => 'apply -R --cached',\n \t\tFILTER => 'index-only',\n \t\tIS_REVERSE => 1,\n \t},\n \t'reset_nothead' => {\n \t\tDIFF => 'diff-index -R -p --cached',\n-\t\tAPPLY => sub { apply_patch 'apply --cached', @_; },\n+\t\tAPPLY => sub { apply_patch 'apply --cached --keep-backup', @_; },\n \t\tAPPLY_CHECK => 'apply --cached',\n \t\tFILTER => 'index-only',\n \t\tIS_REVERSE => 0,\n@@ -628,7 +628,7 @@ sub update_cmd {\n \t\t\t\t       HEADER => $status_head, },\n \t\t\t\t     @mods);\n \tif (@update) {\n-\t\tsystem(qw(git update-index --add --remove --),\n+\t\tsystem(qw(git update-index --add --remove --keep-backup --),\n \t\t       map { $_->{VALUE} } @update);\n \t\tsay_n_paths('updated', @update);\n \t}\n@@ -648,7 +648,7 @@ sub revert_cmd {\n \t\t\tmy @lines = run_cmd_pipe(qw(git ls-tree HEAD --),\n \t\t\t\t\t\t map { $_->{VALUE} } @update);\n \t\t\tmy $fh;\n-\t\t\topen $fh, '| git update-index --index-info'\n+\t\t\topen $fh, '| git update-index --keep-backup --index-info'\n \t\t\t    or die;\n \t\t\tfor (@lines) {\n \t\t\t\tprint $fh $_;\n@@ -673,7 +673,7 @@ sub add_untracked_cmd {\n \tmy @add = list_and_choose({ PROMPT => __('Add untracked') },\n \t\t\t\t  list_untracked());\n \tif (@add) {\n-\t\tsystem(qw(git update-index --add --), @add);\n+\t\tsystem(qw(git update-index --keep-backup --add --), @add);\n \t\tsay_n_paths('added', @add);\n \t} else {\n \t\tprint __(\"No untracked files.\\n\");\n-- \n2.20.0.rc2.486.g9832c05c3d\n\n"},{"id":"364851","messageId":"20181209104419.12639-7-pclouds@gmail.com","threadId":"49985","inReplyTo":"20181209104419.12639-1-pclouds@gmail.com","subject":"[PATCH 06/24] commit: support backup log","fromName":"Nguyễn Thái Ngọc Duy","fromEmail":"pclouds@gmail.com","sentAt":"2018-12-09T10:44:01Z","receivedAt":"2018-12-09T10:45:15Z","isPatch":true,"sender":{"key":"pclouds@gmail.com","avatar":"https://avatars.githubusercontent.com/u/720?v=4"},"body":"Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>\n---\n builtin/commit.c      | 16 +++++++++++-----\n t/t2080-backup-log.sh | 24 ++++++++++++++++++++++++\n 2 files changed, 35 insertions(+), 5 deletions(-)\n\ndiff --git a/builtin/commit.c b/builtin/commit.c\nindex c021b119bb..2bdbeff4a2 100644\n--- a/builtin/commit.c\n+++ b/builtin/commit.c\n@@ -268,7 +268,7 @@ static int list_paths(struct string_list *list, const char *with_tree,\n \treturn ret;\n }\n \n-static void add_remove_files(struct string_list *list)\n+static void add_remove_files(struct string_list *list, int flags)\n {\n \tint i;\n \tfor (i = 0; i < list->nr; i++) {\n@@ -280,7 +280,7 @@ static void add_remove_files(struct string_list *list)\n \t\t\tcontinue;\n \n \t\tif (!lstat(p->string, &st)) {\n-\t\t\tif (add_to_cache(p->string, &st, 0))\n+\t\t\tif (add_to_cache(p->string, &st, flags))\n \t\t\t\tdie(_(\"updating files failed\"));\n \t\t} else\n \t\t\tremove_file_from_cache(p->string);\n@@ -331,8 +331,14 @@ static const char *prepare_index(int argc, const char **argv, const char *prefix\n \tstruct string_list partial = STRING_LIST_INIT_DUP;\n \tstruct pathspec pathspec;\n \tint refresh_flags = REFRESH_QUIET;\n+\tint add_flags = 0;\n+\tint core_backup_log = 0;\n \tconst char *ret;\n \n+\trepo_config_get_bool(the_repository, \"core.backuplog\", &core_backup_log);\n+\tif (core_backup_log)\n+\t\tadd_flags = ADD_CACHE_LOG_UPDATES;\n+\n \tif (is_status)\n \t\trefresh_flags |= REFRESH_UNMERGED;\n \tparse_pathspec(&pathspec, 0,\n@@ -391,7 +397,7 @@ static const char *prepare_index(int argc, const char **argv, const char *prefix\n \t */\n \tif (all || (also && pathspec.nr)) {\n \t\thold_locked_index(&index_lock, LOCK_DIE_ON_ERROR);\n-\t\tadd_files_to_cache(also ? prefix : NULL, &pathspec, 0);\n+\t\tadd_files_to_cache(also ? prefix : NULL, &pathspec, add_flags);\n \t\trefresh_cache_or_die(refresh_flags);\n \t\tupdate_main_cache_tree(WRITE_TREE_SILENT);\n \t\tif (write_locked_index(&the_index, &index_lock, 0))\n@@ -460,7 +466,7 @@ static const char *prepare_index(int argc, const char **argv, const char *prefix\n \t\tdie(_(\"cannot read the index\"));\n \n \thold_locked_index(&index_lock, LOCK_DIE_ON_ERROR);\n-\tadd_remove_files(&partial);\n+\tadd_remove_files(&partial, add_flags);\n \trefresh_cache(REFRESH_QUIET);\n \tupdate_main_cache_tree(WRITE_TREE_SILENT);\n \tif (write_locked_index(&the_index, &index_lock, 0))\n@@ -472,7 +478,7 @@ static const char *prepare_index(int argc, const char **argv, const char *prefix\n \t\t\t\t  LOCK_DIE_ON_ERROR);\n \n \tcreate_base_index(current_head);\n-\tadd_remove_files(&partial);\n+\tadd_remove_files(&partial, 0);\n \trefresh_cache(REFRESH_QUIET);\n \n \tif (write_locked_index(&the_index, &false_lock, 0))\ndiff --git a/t/t2080-backup-log.sh b/t/t2080-backup-log.sh\nindex 6b3814c172..b19e26a807 100755\n--- a/t/t2080-backup-log.sh\n+++ b/t/t2080-backup-log.sh\n@@ -39,4 +39,28 @@ test_expect_success 'update-index --keep-backup writes backup log' '\n \ttest_cmp expected actual\n '\n \n+test_expect_success 'commit -a writes backup log' '\n+\ttest_tick &&\n+\techo update-again >>initial.t &&\n+\tOLD=$(git rev-parse :./initial.t) &&\n+\tNEW=$(git hash-object initial.t) &&\n+\tgit -c core.backupLog=true commit -am update-again &&\n+\techo \"$OLD $NEW $GIT_COMMITTER_NAME <$GIT_COMMITTER_EMAIL> $test_tick -0700\tinitial.t\" >expected &&\n+\ttail -n1 .git/index.bkl >actual &&\n+\ttest_cmp expected actual\n+'\n+\n+test_expect_success 'partial commit writes backup log' '\n+\trm -f .git/index.bkl &&\n+\ttest_tick &&\n+\techo rising >>new-file &&\n+\tgit add -N new-file &&\n+\tOLD=$EMPTY_BLOB &&\n+\tNEW=$(git hash-object new-file) &&\n+\tgit -c core.backupLog=true commit -m update-once-more -- new-file initial.t &&\n+\techo \"$OLD $NEW $GIT_COMMITTER_NAME <$GIT_COMMITTER_EMAIL> $test_tick -0700\tnew-file\" >expected &&\n+\t# expect file deletion to not log anything &&\n+\ttest_cmp expected .git/index.bkl\n+'\n+\n test_done\n-- \n2.20.0.rc2.486.g9832c05c3d\n\n"},{"id":"364849","messageId":"20181209104419.12639-8-pclouds@gmail.com","threadId":"49985","inReplyTo":"20181209104419.12639-1-pclouds@gmail.com","subject":"[PATCH 07/24] apply: support backup log with --keep-backup","fromName":"Nguyễn Thái Ngọc Duy","fromEmail":"pclouds@gmail.com","sentAt":"2018-12-09T10:44:02Z","receivedAt":"2018-12-09T10:45:16Z","isPatch":true,"sender":{"key":"pclouds@gmail.com","avatar":"https://avatars.githubusercontent.com/u/720?v=4"},"body":"Normally changes from git-apply are not worth logging because they\ncome from a file and can be recovered from there. This option will\nmostly be used by add--interactive.pl where patches are generated\nduring an interactive add session and are very much worth logging.\n\nThe logging is a bit more complicated because an update is done in two\nphases. The old version of all patches is removed first then the new\none added. We need to keep track of the old version in order to make a\nbackup.\n\nSigned-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>\n---\n Documentation/git-apply.txt |  3 +++\n apply.c                     | 38 +++++++++++++++++++++++++++++++++++--\n apply.h                     |  1 +\n t/t2080-backup-log.sh       | 25 ++++++++++++++++++++++++\n 4 files changed, 65 insertions(+), 2 deletions(-)\n\ndiff --git a/Documentation/git-apply.txt b/Documentation/git-apply.txt\nindex b9aa39000f..7b0ae790db 100644\n--- a/Documentation/git-apply.txt\n+++ b/Documentation/git-apply.txt\n@@ -250,6 +250,9 @@ When `git apply` is used as a \"better GNU patch\", the user can pass\n the `--unsafe-paths` option to override this safety check.  This option\n has no effect when `--index` or `--cached` is in use.\n \n+--keep-backup::\n+\tEnable index backup log when `--cached` or `--index` is used.\n+\n CONFIGURATION\n -------------\n \ndiff --git a/apply.c b/apply.c\nindex 01793d6126..716c9a658f 100644\n--- a/apply.c\n+++ b/apply.c\n@@ -21,6 +21,7 @@\n #include \"quote.h\"\n #include \"rerere.h\"\n #include \"apply.h\"\n+#include \"backup-log.h\"\n \n static void git_apply_config(void)\n {\n@@ -226,6 +227,7 @@ struct patch {\n \tchar old_oid_prefix[GIT_MAX_HEXSZ + 1];\n \tchar new_oid_prefix[GIT_MAX_HEXSZ + 1];\n \tstruct patch *next;\n+\tstruct object_id old_oid;\n \n \t/* three-way fallback result */\n \tstruct object_id threeway_stage[3];\n@@ -4261,6 +4263,16 @@ static void patch_stats(struct apply_state *state, struct patch *patch)\n static int remove_file(struct apply_state *state, struct patch *patch, int rmdir_empty)\n {\n \tif (state->update_index && !state->ita_only) {\n+\t\tif (state->backup_log) {\n+\t\t\tint pos = index_name_pos(state->repo->index,\n+\t\t\t\t\t\t patch->old_name,\n+\t\t\t\t\t\t strlen(patch->old_name));\n+\t\t\tif (pos >= 0)\n+\t\t\t\toidcpy(&patch->old_oid,\n+\t\t\t\t       &state->repo->index->cache[pos]->oid);\n+\t\t\telse\n+\t\t\t\toidclr(&patch->old_oid);\n+\t\t}\n \t\tif (remove_file_from_index(state->repo->index, patch->old_name) < 0)\n \t\t\treturn error(_(\"unable to remove %s from index\"), patch->old_name);\n \t}\n@@ -4276,7 +4288,8 @@ static int add_index_file(struct apply_state *state,\n \t\t\t  const char *path,\n \t\t\t  unsigned mode,\n \t\t\t  void *buf,\n-\t\t\t  unsigned long size)\n+\t\t\t  unsigned long size,\n+\t\t\t  const struct object_id *old_oid)\n {\n \tstruct stat st;\n \tstruct cache_entry *ce;\n@@ -4314,6 +4327,16 @@ static int add_index_file(struct apply_state *state,\n \t\t\t\t       \"for newly created file %s\"), path);\n \t\t}\n \t}\n+\tif (state->backup_log) {\n+\t\tstruct strbuf *sb = state->repo->index->backup_log;\n+\n+\t\tif (!sb) {\n+\t\t\tsb = xmalloc(sizeof(*sb));\n+\t\t\tstrbuf_init(sb, 0);\n+\t\t\tstate->repo->index->backup_log = sb;\n+\t\t}\n+\t\tbkl_append(sb, ce->name, old_oid, &ce->oid);\n+\t}\n \tif (add_index_entry(state->repo->index, ce, ADD_CACHE_OK_TO_ADD) < 0) {\n \t\tdiscard_cache_entry(ce);\n \t\treturn error(_(\"unable to add cache entry for %s\"), path);\n@@ -4484,7 +4507,8 @@ static int create_file(struct apply_state *state, struct patch *patch)\n \tif (patch->conflicted_threeway)\n \t\treturn add_conflicted_stages_file(state, patch);\n \telse if (state->update_index)\n-\t\treturn add_index_file(state, path, mode, buf, size);\n+\t\treturn add_index_file(state, path, mode, buf, size,\n+\t\t\t\t      &patch->old_oid);\n \treturn 0;\n }\n \n@@ -4659,6 +4683,7 @@ static int apply_patch(struct apply_state *state,\n \tstruct patch *list = NULL, **listp = &list;\n \tint skipped_patch = 0;\n \tint res = 0;\n+\tint core_backup_log = 0;\n \n \tstate->patch_input_file = filename;\n \tif (read_patch_file(&buf, fd) < 0)\n@@ -4721,6 +4746,13 @@ static int apply_patch(struct apply_state *state,\n \t\tgoto end;\n \t}\n \n+\tif (state->backup_log &&\n+\t    (!state->update_index ||\n+\t     repo_config_get_bool(state->repo, \"core.backupLog\",\n+\t\t\t\t  &core_backup_log) ||\n+\t     !core_backup_log))\n+\t\tstate->backup_log = 0;\n+\n \tif (state->check || state->apply) {\n \t\tint r = check_patch_list(state, list);\n \t\tif (r == -128) {\n@@ -4982,6 +5014,8 @@ int apply_parse_options(int argc, const char **argv,\n \t\t\tN_(\"mark new files with `git add --intent-to-add`\")),\n \t\tOPT_BOOL(0, \"cached\", &state->cached,\n \t\t\tN_(\"apply a patch without touching the working tree\")),\n+\t\tOPT_BOOL(0, \"keep-backup\", &state->backup_log,\n+\t\t\t N_(\"log index changes if the feature is enabled\")),\n \t\tOPT_BOOL_F(0, \"unsafe-paths\", &state->unsafe_paths,\n \t\t\t   N_(\"accept a patch that touches outside the working area\"),\n \t\t\t   PARSE_OPT_NOCOMPLETE),\ndiff --git a/apply.h b/apply.h\nindex 5948348133..d4de2ebcb9 100644\n--- a/apply.h\n+++ b/apply.h\n@@ -51,6 +51,7 @@ struct apply_state {\n \tint check_index; /* preimage must match the indexed version */\n \tint update_index; /* check_index && apply */\n \tint ita_only;\t  /* add intent-to-add entries to the index */\n+\tint backup_log;\t  /* enable backup log */\n \n \t/* These control cosmetic aspect of the output */\n \tint diffstat; /* just show a diffstat, and don't actually apply */\ndiff --git a/t/t2080-backup-log.sh b/t/t2080-backup-log.sh\nindex b19e26a807..8d1c8c5935 100755\n--- a/t/t2080-backup-log.sh\n+++ b/t/t2080-backup-log.sh\n@@ -63,4 +63,29 @@ test_expect_success 'partial commit writes backup log' '\n \ttest_cmp expected .git/index.bkl\n '\n \n+test_expect_success 'apply --cached writes backup log' '\n+\trm -f .git/index.bkl &&\n+\tgit reset --hard &&\n+\ttest_tick &&\n+\techo to-be-deleted >to-be-deleted &&\n+\techo to-be-modified >to-be-modified &&\n+\tOLD_M=$(git hash-object to-be-modified) &&\n+\tgit add . &&\n+\tgit commit -m first &&\n+\trm to-be-deleted &&\n+\techo modified >>to-be-modified &&\n+\tNEW_M=$(git hash-object to-be-modified) &&\n+\tOLD_A=$ZERO_OID &&\n+\techo to-be-added >to-be-added &&\n+\tNEW_A=$(git hash-object to-be-added) &&\n+\tgit add . &&\n+\tgit commit -m second &&\n+\tgit diff HEAD^ HEAD >patch &&\n+\tgit reset --hard HEAD^ &&\n+\tgit -c core.backupLog=true apply --cached --keep-backup patch &&\n+\techo \"$OLD_A $NEW_A $GIT_COMMITTER_NAME <$GIT_COMMITTER_EMAIL> $test_tick -0700\tto-be-added\" >expected &&\n+\techo \"$OLD_M $NEW_M $GIT_COMMITTER_NAME <$GIT_COMMITTER_EMAIL> $test_tick -0700\tto-be-modified\" >>expected &&\n+\ttest_cmp expected .git/index.bkl\n+'\n+\n test_done\n-- \n2.20.0.rc2.486.g9832c05c3d\n\n"},{"id":"364852","messageId":"20181209104419.12639-10-pclouds@gmail.com","threadId":"49985","inReplyTo":"20181209104419.12639-1-pclouds@gmail.com","subject":"[PATCH 09/24] backup-log.c: add API for walking backup log","fromName":"Nguyễn Thái Ngọc Duy","fromEmail":"pclouds@gmail.com","sentAt":"2018-12-09T10:44:04Z","receivedAt":"2018-12-09T10:45:17Z","isPatch":true,"sender":{"key":"pclouds@gmail.com","avatar":"https://avatars.githubusercontent.com/u/720?v=4"},"body":"Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>\n---\n backup-log.c | 173 +++++++++++++++++++++++++++++++++++++++++++++++++++\n backup-log.h |  21 ++++++-\n 2 files changed, 193 insertions(+), 1 deletion(-)\n\ndiff --git a/backup-log.c b/backup-log.c\nindex c1e805b09e..49f2ce68fe 100644\n--- a/backup-log.c\n+++ b/backup-log.c\n@@ -44,3 +44,176 @@ int bkl_write(const char *path, struct strbuf *new_log)\n \trollback_lock_file(&lk);\n \treturn ret;\n }\n+\n+int bkl_parse_entry(struct strbuf *sb, struct bkl_entry *re)\n+{\n+\tchar *email_end, *message;\n+\tconst char *p = sb->buf;\n+\n+\t/* old SP new SP name <email> SP time TAB msg LF */\n+\tif (!sb->len || sb->buf[sb->len - 1] != '\\n' ||\n+\t    parse_oid_hex(p, &re->old_oid, &p) || *p++ != ' ' ||\n+\t    parse_oid_hex(p, &re->new_oid, &p) || *p++ != ' ' ||\n+\t    !(email_end = strchr(p, '>')) ||\n+\t    email_end[1] != ' ')\n+\t\treturn -1;\t/* corrupt? */\n+\tre->email = p;\n+\tre->timestamp = parse_timestamp(email_end + 2, &message, 10);\n+\tif (!re->timestamp ||\n+\t    !message || message[0] != ' ' ||\n+\t    (message[1] != '+' && message[1] != '-') ||\n+\t    !isdigit(message[2]) || !isdigit(message[3]) ||\n+\t    !isdigit(message[4]) || !isdigit(message[5]))\n+\t\treturn -1; /* corrupt? */\n+\temail_end[1] = '\\0';\n+\tre->tz = strtol(message + 1, NULL, 10);\n+\tif (message[6] != '\\t')\n+\t\tmessage += 6;\n+\telse\n+\t\tmessage += 7;\n+\tsb->buf[sb->len - 1] = '\\0'; /* no LF */\n+\tre->path = message;\n+\treturn 0;\n+}\n+\n+static char *find_beginning_of_line(char *bob, char *scan)\n+{\n+\twhile (bob < scan && *(--scan) != '\\n')\n+\t\t; /* keep scanning backwards */\n+\t/*\n+\t * Return either beginning of the buffer, or LF at the end of\n+\t * the previous line.\n+\t */\n+\treturn scan;\n+}\n+\n+int bkl_parse_file_reverse(const char *path,\n+\t\t\t   int (*parse)(struct strbuf *line, void *data),\n+\t\t\t   void *data)\n+{\n+\tstruct strbuf sb = STRBUF_INIT;\n+\tFILE *logfp;\n+\tlong pos;\n+\tint ret = 0, at_tail = 1;\n+\n+\tlogfp = fopen(path, \"r\");\n+\tif (!logfp) {\n+\t\tif (errno == ENOENT || errno == ENOTDIR)\n+\t\t\treturn 0;\n+\t\treturn -1;\n+\t}\n+\n+\t/* Jump to the end */\n+\tif (fseek(logfp, 0, SEEK_END) < 0)\n+\t\tret = error_errno(_(\"cannot seek back in %s\"), path);\n+\tpos = ftell(logfp);\n+\twhile (!ret && 0 < pos) {\n+\t\tint cnt;\n+\t\tsize_t nread;\n+\t\tchar buf[BUFSIZ];\n+\t\tchar *endp, *scanp;\n+\n+\t\t/* Fill next block from the end */\n+\t\tcnt = (sizeof(buf) < pos) ? sizeof(buf) : pos;\n+\t\tif (fseek(logfp, pos - cnt, SEEK_SET)) {\n+\t\t\tret = error_errno(_(\"cannot seek back in %s\"), path);\n+\t\t\tbreak;\n+\t\t}\n+\t\tnread = fread(buf, cnt, 1, logfp);\n+\t\tif (nread != 1) {\n+\t\t\tret = error_errno(_(\"cannot read %d bytes from %s\"),\n+\t\t\t\t\t  cnt, path);\n+\t\t\tbreak;\n+\t\t}\n+\t\tpos -= cnt;\n+\n+\t\tscanp = endp = buf + cnt;\n+\t\tif (at_tail && scanp[-1] == '\\n')\n+\t\t\t/* Looking at the final LF at the end of the file */\n+\t\t\tscanp--;\n+\t\tat_tail = 0;\n+\n+\t\twhile (buf < scanp) {\n+\t\t\t/*\n+\t\t\t * terminating LF of the previous line, or the beginning\n+\t\t\t * of the buffer.\n+\t\t\t */\n+\t\t\tchar *bp;\n+\n+\t\t\tbp = find_beginning_of_line(buf, scanp);\n+\n+\t\t\tif (*bp == '\\n') {\n+\t\t\t\t/*\n+\t\t\t\t * The newline is the end of the previous line,\n+\t\t\t\t * so we know we have complete line starting\n+\t\t\t\t * at (bp + 1). Prefix it onto any prior data\n+\t\t\t\t * we collected for the line and process it.\n+\t\t\t\t */\n+\t\t\t\tstrbuf_splice(&sb, 0, 0, bp + 1, endp - (bp + 1));\n+\t\t\t\tscanp = bp;\n+\t\t\t\tendp = bp + 1;\n+\t\t\t\tret = parse(&sb, data);\n+\t\t\t\tstrbuf_reset(&sb);\n+\t\t\t\tif (ret)\n+\t\t\t\t\tbreak;\n+\t\t\t} else if (!pos) {\n+\t\t\t\t/*\n+\t\t\t\t * We are at the start of the buffer, and the\n+\t\t\t\t * start of the file; there is no previous\n+\t\t\t\t * line, and we have everything for this one.\n+\t\t\t\t * Process it, and we can end the loop.\n+\t\t\t\t */\n+\t\t\t\tstrbuf_splice(&sb, 0, 0, buf, endp - buf);\n+\t\t\t\tret = parse(&sb, data);\n+\t\t\t\tstrbuf_reset(&sb);\n+\t\t\t\tbreak;\n+\t\t\t}\n+\n+\t\t\tif (bp == buf) {\n+\t\t\t\t/*\n+\t\t\t\t * We are at the start of the buffer, and there\n+\t\t\t\t * is more file to read backwards. Which means\n+\t\t\t\t * we are in the middle of a line. Note that we\n+\t\t\t\t * may get here even if *bp was a newline; that\n+\t\t\t\t * just means we are at the exact end of the\n+\t\t\t\t * previous line, rather than some spot in the\n+\t\t\t\t * middle.\n+\t\t\t\t *\n+\t\t\t\t * Save away what we have to be combined with\n+\t\t\t\t * the data from the next read.\n+\t\t\t\t */\n+\t\t\t\tstrbuf_splice(&sb, 0, 0, buf, endp - buf);\n+\t\t\t\tbreak;\n+\t\t\t}\n+\t\t}\n+\n+\t}\n+\tif (!ret && sb.len)\n+\t\tBUG(\"reverse reflog parser had leftover data\");\n+\n+\tfclose(logfp);\n+\tstrbuf_release(&sb);\n+\treturn ret;\n+}\n+\n+int bkl_parse_file(const char *path,\n+\t\t   int (*parse)(struct strbuf *line, void *data),\n+\t\t   void *data)\n+{\n+\tstruct strbuf sb = STRBUF_INIT;\n+\tFILE *logfp;\n+\tint ret = 0;\n+\n+\tlogfp = fopen(path, \"r\");\n+\tif (!logfp) {\n+\t\tif (errno == ENOENT || errno == ENOTDIR)\n+\t\t\treturn 0;\n+\t\treturn -1;\n+\t}\n+\n+\twhile (!ret && !strbuf_getwholeline(&sb, logfp, '\\n'))\n+\t\tret = parse(&sb, data);\n+\tfclose(logfp);\n+\tstrbuf_release(&sb);\n+\treturn ret;\n+}\ndiff --git a/backup-log.h b/backup-log.h\nindex 5e475d6f35..c9de9c687c 100644\n--- a/backup-log.h\n+++ b/backup-log.h\n@@ -1,13 +1,32 @@\n #ifndef __BACKUP_LOG_H__\n #define __BACKUP_LOG_H__\n \n-struct object_id;\n+#include \"cache.h\"\n+\n struct strbuf;\n \n+struct bkl_entry\n+{\n+\tstruct object_id old_oid;\n+\tstruct object_id new_oid;\n+\tconst char *email;\n+\ttimestamp_t timestamp;\n+\tint tz;\n+\tconst char *path;\n+};\n+\n void bkl_append(struct strbuf *output, const char *path,\n \t\tconst struct object_id *from,\n \t\tconst struct object_id *to);\n \n int bkl_write(const char *path, struct strbuf *new_log);\n \n+int bkl_parse_entry(struct strbuf *sb, struct bkl_entry *re);\n+int bkl_parse_file_reverse(const char *path,\n+\t\t\t   int (*parse)(struct strbuf *line, void *data),\n+\t\t\t   void *data);\n+int bkl_parse_file(const char *path,\n+\t\t   int (*parse)(struct strbuf *line, void *data),\n+\t\t   void *data);\n+\n #endif\n-- \n2.20.0.rc2.486.g9832c05c3d\n\n"},{"id":"364853","messageId":"20181209104419.12639-12-pclouds@gmail.com","threadId":"49985","inReplyTo":"20181209104419.12639-1-pclouds@gmail.com","subject":"[PATCH 11/24] backup-log: add diff command","fromName":"Nguyễn Thái Ngọc Duy","fromEmail":"pclouds@gmail.com","sentAt":"2018-12-09T10:44:06Z","receivedAt":"2018-12-09T10:45:18Z","isPatch":true,"sender":{"key":"pclouds@gmail.com","avatar":"https://avatars.githubusercontent.com/u/720?v=4"},"body":"This gives you a patch so you can apply if you want.\n\nFIXME pathspec support?\n\nPS. the file list in diff queue is not sorted like a normal \"git\ndiff\". If this is a big deal, we probably can sort it later.\n\nSigned-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>\n---\n builtin/backup-log.c  | 100 ++++++++++++++++++++++++++++++++++++++++++\n t/t2080-backup-log.sh |  25 +++++++++++\n 2 files changed, 125 insertions(+)\n\ndiff --git a/builtin/backup-log.c b/builtin/backup-log.c\nindex b370ff9d27..f8029e0011 100644\n--- a/builtin/backup-log.c\n+++ b/builtin/backup-log.c\n@@ -1,6 +1,8 @@\n #include \"cache.h\"\n #include \"builtin.h\"\n #include \"backup-log.h\"\n+#include \"diff.h\"\n+#include \"diffcore.h\"\n #include \"dir.h\"\n #include \"object-store.h\"\n #include \"parse-options.h\"\n@@ -8,6 +10,7 @@\n static char const * const backup_log_usage[] = {\n \tN_(\"git backup-log [--path=<path> | --id=<id>] update <path> <old-hash> <new-hash>\"),\n \tN_(\"git backup-log [--path=<path> | --id=<id>] cat [<options>] <change-id> <path>\"),\n+\tN_(\"git backup-log [--path=<path> | --id=<id>] diff [<diff-options>] <change-id>\"),\n \tNULL\n };\n \n@@ -115,6 +118,101 @@ static int cat(int argc, const char **argv,\n \treturn ret - 1;\n }\n \n+static void queue_blk_entry_for_diff(const struct bkl_entry *e)\n+{\n+\tunsigned mode = canon_mode(S_IFREG | 0644);\n+\tstruct diff_filespec *one, *two;\n+\tconst struct object_id *old, *new;\n+\n+\tif (is_null_oid(&e->old_oid))\n+\t\told = the_hash_algo->empty_blob;\n+\telse\n+\t\told = &e->old_oid;\n+\n+\tif (is_null_oid(&e->new_oid))\n+\t\tnew = the_hash_algo->empty_blob;\n+\telse\n+\t\tnew = &e->new_oid;\n+\n+\tif (oideq(old, new))\n+\t\treturn;\n+\n+\tone = alloc_filespec(e->path);\n+\ttwo = alloc_filespec(e->path);\n+\n+\tfill_filespec(one, old, 1, mode);\n+\tfill_filespec(two, new, 1, mode);\n+\tdiff_queue(&diff_queued_diff, one, two);\n+}\n+\n+static int diff_parse(struct strbuf *line, void *data)\n+{\n+\ttimestamp_t id = *(timestamp_t *)data;\n+\tstruct bkl_entry entry;\n+\n+\tif (bkl_parse_entry(line, &entry))\n+\t\treturn -1;\n+\n+\tif (entry.timestamp < id)\n+\t\treturn 1;\n+\n+\tif (entry.timestamp > id)\n+\t\treturn 0;\n+\n+\tqueue_blk_entry_for_diff(&entry);\n+\treturn 0;\n+}\n+\n+static int diff(int argc, const char **argv,\n+\t\tconst char *prefix, const char *log_path)\n+{\n+\tchar *end = NULL;\n+\tstruct diff_options diffopt;\n+\ttimestamp_t id = -1;\n+\tint ret, i, found_dash_dash = 0;\n+\n+\trepo_diff_setup(the_repository, &diffopt);\n+\ti = 1;\n+\twhile (i < argc) {\n+\t\tconst char *arg = argv[i];\n+\n+\t\tif (!found_dash_dash && *arg == '-') {\n+\t\t\tif (!strcmp(arg, \"--\")) {\n+\t\t\t\tfound_dash_dash = 1;\n+\t\t\t\ti++;\n+\t\t\t\tcontinue;\n+\t\t\t}\n+\t\t\tret = diff_opt_parse(&diffopt, argv + i, argc - i, prefix);\n+\t\t\tif (ret < 0)\n+\t\t\t\texit(128);\n+\t\t\ti += ret;\n+\t\t\tcontinue;\n+\t\t}\n+\n+\t\tid = strtol(arg, &end, 10);\n+\t\tif (!end || *end)\n+\t\t\tdie(_(\"not a valid change id: %s\"), arg);\n+\t\ti++;\n+\t\tbreak;\n+\t}\n+\tif (!diffopt.output_format)\n+\t\tdiffopt.output_format = DIFF_FORMAT_PATCH;\n+\tdiff_setup_done(&diffopt);\n+\tif (i != argc || id == -1)\n+\t\tusage_with_options(backup_log_usage, NULL);\n+\n+\tret = bkl_parse_file_reverse(log_path, diff_parse, &id);\n+\tif (ret < 0)\n+\t\tdie(_(\"failed to parse '%s'\"), log_path);\n+\n+\tif (ret == 1) {\n+\t\tsetup_pager();\n+\t\tdiffcore_std(&diffopt);\n+\t\tdiff_flush(&diffopt);\n+\t}\n+\treturn ret - 1;\n+}\n+\n static char *log_id_to_path(const char *id)\n {\n \tif (!strcmp(id, \"index\"))\n@@ -156,6 +254,8 @@ int cmd_backup_log(int argc, const char **argv, const char *prefix)\n \t\treturn update(argc, argv, prefix, log_path);\n \telse if (!strcmp(argv[0], \"cat\"))\n \t\treturn cat(argc, argv, prefix, log_path);\n+\telse if (!strcmp(argv[0], \"diff\"))\n+\t\treturn diff(argc, argv, prefix, log_path);\n \telse\n \t\tdie(_(\"unknown subcommand: %s\"), argv[0]);\n \ndiff --git a/t/t2080-backup-log.sh b/t/t2080-backup-log.sh\nindex 9c004c9727..cd4288d386 100755\n--- a/t/t2080-backup-log.sh\n+++ b/t/t2080-backup-log.sh\n@@ -105,4 +105,29 @@ test_expect_success 'backup-log cat' '\n \ttest_cmp expected actual\n '\n \n+test_expect_success 'backup-log diff' '\n+\techo first >initial.t &&\n+\tgit add initial.t &&\n+\techo second >initial.t &&\n+\ttest_tick &&\n+\tgit -c core.backupLog=true add initial.t &&\n+\tgit backup-log --id=index diff $test_tick >actual &&\n+\tcat >expected <<-\\EOF &&\n+\tdiff --git a/initial.t b/initial.t\n+\tindex 9c59e24..e019be0 100644\n+\t--- a/initial.t\n+\t+++ b/initial.t\n+\t@@ -1 +1 @@\n+\t-first\n+\t+second\n+\tEOF\n+\ttest_cmp expected actual &&\n+\tgit backup-log --id=index diff --stat $test_tick >stat.actual &&\n+\tcat >stat.expected <<-\\EOF &&\n+\t initial.t | 2 +-\n+\t 1 file changed, 1 insertion(+), 1 deletion(-)\n+\tEOF\n+\ttest_cmp stat.expected stat.actual\n+'\n+\n test_done\n-- \n2.20.0.rc2.486.g9832c05c3d\n\n"},{"id":"364854","messageId":"20181209104419.12639-11-pclouds@gmail.com","threadId":"49985","inReplyTo":"20181209104419.12639-1-pclouds@gmail.com","subject":"[PATCH 10/24] backup-log: add cat command","fromName":"Nguyễn Thái Ngọc Duy","fromEmail":"pclouds@gmail.com","sentAt":"2018-12-09T10:44:05Z","receivedAt":"2018-12-09T10:45:20Z","isPatch":true,"sender":{"key":"pclouds@gmail.com","avatar":"https://avatars.githubusercontent.com/u/720?v=4"},"body":"This command introduces a new concept, \"change id\". This is similar to\n\"n\" in reflog sha-1 extended syntax @{n}. I'm trying to group changes of\nthe same second together, and this timestamp becomes \"change id\", so\nyou view roughly a snapshot of changes. Of course it's not 100%\naccurate. But it works most of the time and it keeps log update low.\n\nSigned-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>\n---\n builtin/backup-log.c  | 83 +++++++++++++++++++++++++++++++++++++++++++\n t/t2080-backup-log.sh | 17 +++++++++\n 2 files changed, 100 insertions(+)\n\ndiff --git a/builtin/backup-log.c b/builtin/backup-log.c\nindex 75a02c8878..b370ff9d27 100644\n--- a/builtin/backup-log.c\n+++ b/builtin/backup-log.c\n@@ -1,10 +1,13 @@\n #include \"cache.h\"\n #include \"builtin.h\"\n #include \"backup-log.h\"\n+#include \"dir.h\"\n+#include \"object-store.h\"\n #include \"parse-options.h\"\n \n static char const * const backup_log_usage[] = {\n \tN_(\"git backup-log [--path=<path> | --id=<id>] update <path> <old-hash> <new-hash>\"),\n+\tN_(\"git backup-log [--path=<path> | --id=<id>] cat [<options>] <change-id> <path>\"),\n \tNULL\n };\n \n@@ -34,6 +37,84 @@ static int update(int argc, const char **argv,\n \treturn ret;\n }\n \n+struct cat_options {\n+\ttimestamp_t id;\n+\tchar *path;\n+\tint before;\n+\tint show_hash;\n+};\n+\n+static int cat_parse(struct strbuf *line, void *data)\n+{\n+\tstruct cat_options *opts = data;\n+\tstruct bkl_entry entry;\n+\tstruct object_id *oid;\n+\tvoid *buf;\n+\tunsigned long size;\n+\tenum object_type type;\n+\n+\tif (bkl_parse_entry(line, &entry))\n+\t\treturn -1;\n+\n+\tif (entry.timestamp < opts->id)\n+\t\treturn 2;\n+\n+\tif (entry.timestamp != opts->id ||\n+\t    fspathcmp(entry.path, opts->path))\n+\t\treturn 0;\n+\n+\tif (opts->before)\n+\t\toid = &entry.old_oid;\n+\telse\n+\t\toid = &entry.new_oid;\n+\n+\tif (opts->show_hash) {\n+\t\tputs(oid_to_hex(oid));\n+\t\treturn 1;\n+\t}\n+\n+\tif (is_null_oid(oid))\n+\t\treturn 1;\t/* treat null oid like empty blob */\n+\n+\tbuf = read_object_file(oid, &type, &size);\n+\tif (!buf)\n+\t\tdie(_(\"object not found: %s\"), oid_to_hex(oid));\n+\tif (type != OBJ_BLOB)\n+\t\tdie(_(\"not a blob: %s\"), oid_to_hex(oid));\n+\n+\twrite_in_full(1, buf, size);\n+\tfree(buf);\n+\n+\treturn 1;\n+}\n+\n+static int cat(int argc, const char **argv,\n+\t\tconst char *prefix, const char *log_path)\n+{\n+\tstruct cat_options opts;\n+\tstruct option options[] = {\n+\t\tOPT_BOOL(0, \"before\", &opts.before, N_(\"select the version before the update\")),\n+\t\tOPT_BOOL(0, \"hash\", &opts.show_hash, N_(\"show the hash instead of the content\")),\n+\t\tOPT_END()\n+\t};\n+\tchar *end = NULL;\n+\tint ret;\n+\n+\tmemset(&opts, 0, sizeof(opts));\n+\targc = parse_options(argc, argv, prefix, options, backup_log_usage, 0);\n+\tif (argc != 2)\n+\t\tusage_with_options(backup_log_usage, options);\n+\topts.id = strtol(argv[0], &end, 10);\n+\tif (!end || *end)\n+\t\tdie(_(\"not a valid change id: %s\"), argv[0]);\n+\topts.path = prefix_path(prefix, prefix ? strlen(prefix) : 0, argv[1]);\n+\tsetup_pager();\n+\tret = bkl_parse_file_reverse(log_path, cat_parse, &opts);\n+\tif (ret < 0)\n+\t\tdie(_(\"failed to parse '%s'\"), log_path);\n+\treturn ret - 1;\n+}\n+\n static char *log_id_to_path(const char *id)\n {\n \tif (!strcmp(id, \"index\"))\n@@ -73,6 +154,8 @@ int cmd_backup_log(int argc, const char **argv, const char *prefix)\n \n \tif (!strcmp(argv[0], \"update\"))\n \t\treturn update(argc, argv, prefix, log_path);\n+\telse if (!strcmp(argv[0], \"cat\"))\n+\t\treturn cat(argc, argv, prefix, log_path);\n \telse\n \t\tdie(_(\"unknown subcommand: %s\"), argv[0]);\n \ndiff --git a/t/t2080-backup-log.sh b/t/t2080-backup-log.sh\nindex 8d1c8c5935..9c004c9727 100755\n--- a/t/t2080-backup-log.sh\n+++ b/t/t2080-backup-log.sh\n@@ -88,4 +88,21 @@ test_expect_success 'apply --cached writes backup log' '\n \ttest_cmp expected .git/index.bkl\n '\n \n+test_expect_success 'backup-log cat' '\n+\tOLD=$(git rev-parse :./initial.t) &&\n+\techo update >>initial.t &&\n+\ttest_tick &&\n+\tgit -c core.backupLog=true add initial.t &&\n+\tNEW=$(git rev-parse :./initial.t) &&\n+\tgit backup-log --id=index cat --before --hash $test_tick initial.t >actual &&\n+\techo $OLD >expected &&\n+\ttest_cmp expected actual &&\n+\tgit backup-log --id=index cat --hash $test_tick initial.t >actual &&\n+\techo $NEW >expected &&\n+\ttest_cmp expected actual &&\n+\tgit backup-log --id=index cat $test_tick initial.t >actual &&\n+\tgit cat-file blob $NEW >expected &&\n+\ttest_cmp expected actual\n+'\n+\n test_done\n-- \n2.20.0.rc2.486.g9832c05c3d\n\n"},{"id":"364855","messageId":"20181209104419.12639-13-pclouds@gmail.com","threadId":"49985","inReplyTo":"20181209104419.12639-1-pclouds@gmail.com","subject":"[PATCH 12/24] backup-log: add log command","fromName":"Nguyễn Thái Ngọc Duy","fromEmail":"pclouds@gmail.com","sentAt":"2018-12-09T10:44:07Z","receivedAt":"2018-12-09T10:45:22Z","isPatch":true,"sender":{"key":"pclouds@gmail.com","avatar":"https://avatars.githubusercontent.com/u/720?v=4"},"body":"One note about the default relative mode. Since this is mostly used\nafter a panic moment \"oops I did it again!\". The user would most\nlikely look for latest changes of a certain path and relative dates\nmake it easier to spot if the change is within a time frame.\n\nDefault to --patch too because the output is pretty useless otherwise.\n\nSigned-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>\n---\n builtin/backup-log.c  | 90 +++++++++++++++++++++++++++++++++++++++++++\n t/t2080-backup-log.sh | 34 ++++++++++++++++\n 2 files changed, 124 insertions(+)\n\ndiff --git a/builtin/backup-log.c b/builtin/backup-log.c\nindex f8029e0011..2496d73ba5 100644\n--- a/builtin/backup-log.c\n+++ b/builtin/backup-log.c\n@@ -6,11 +6,13 @@\n #include \"dir.h\"\n #include \"object-store.h\"\n #include \"parse-options.h\"\n+#include \"revision.h\"\n \n static char const * const backup_log_usage[] = {\n \tN_(\"git backup-log [--path=<path> | --id=<id>] update <path> <old-hash> <new-hash>\"),\n \tN_(\"git backup-log [--path=<path> | --id=<id>] cat [<options>] <change-id> <path>\"),\n \tN_(\"git backup-log [--path=<path> | --id=<id>] diff [<diff-options>] <change-id>\"),\n+\tN_(\"git backup-log [--path=<path> | --id=<id>] log [<options>] [--] [<path>]\"),\n \tNULL\n };\n \n@@ -213,6 +215,92 @@ static int diff(int argc, const char **argv,\n \treturn ret - 1;\n }\n \n+struct log_options {\n+\tstruct rev_info revs;\n+\ttimestamp_t last_time;\n+\tint last_tz;\n+};\n+\n+static void dump(struct bkl_entry *entry, struct log_options *opts)\n+{\n+\ttimestamp_t last_time = opts->last_time;\n+\tint last_tz = opts->last_tz;\n+\n+\tif (entry) {\n+\t\topts->last_time = entry->timestamp;\n+\t\topts->last_tz = entry->tz;\n+\t}\n+\n+\tif (last_time != -1 &&\n+\t    ((!entry && diff_queued_diff.nr) ||\n+\t     (entry && last_time != entry->timestamp))) {\n+\t\tprintf(\"Change-Id: %\"PRItime\"\\n\", last_time);\n+\t\tprintf(\"Date: %s\\n\", show_date(last_time, last_tz, &opts->revs.date_mode));\n+\t\tprintf(\"\\n--\\n\");\n+\t\tdiffcore_std(&opts->revs.diffopt);\n+\t\tdiff_flush(&opts->revs.diffopt);\n+\t\tprintf(\"\\n\");\n+\t}\n+}\n+\n+static int log_parse(struct strbuf *line, void *data)\n+{\n+\tstruct log_options *opts = data;\n+\tstruct bkl_entry entry;\n+\n+\tif (bkl_parse_entry(line, &entry))\n+\t\treturn -1;\n+\n+\tif (opts->revs.max_age != -1 && entry.timestamp < opts->revs.max_age)\n+\t\treturn 1;\n+\n+\tif (!match_pathspec(the_repository->index, &opts->revs.prune_data,\n+\t\t\t    entry.path, strlen(entry.path),\n+\t\t\t    0, NULL, 0))\n+\t\treturn 0;\n+\n+\tdump(&entry, opts);\n+\n+\tqueue_blk_entry_for_diff(&entry);\n+\n+\treturn 0;\n+}\n+\n+static int log_(int argc, const char **argv,\n+\t\tconst char *prefix, const char *log_path)\n+{\n+\tstruct log_options opts;\n+\tint ret;\n+\n+\tmemset(&opts, 0, sizeof(opts));\n+\topts.last_time = -1;\n+\topts.last_tz = -1;\n+\n+\trepo_init_revisions(the_repository, &opts.revs, prefix);\n+\topts.revs.date_mode.type = DATE_RELATIVE;\n+\targc = setup_revisions(argc, argv, &opts.revs, NULL);\n+\tif (!opts.revs.diffopt.output_format) {\n+\t\topts.revs.diffopt.output_format = DIFF_FORMAT_PATCH;\n+\t\tdiff_setup_done(&opts.revs.diffopt);\n+\t}\n+\n+\tsetup_pager();\n+\tif (opts.revs.reverse) {\n+\t\t/*\n+\t\t * Default order is reading file from the bottom. --reverse\n+\t\t * makes it read from the top instead (i.e. forward)\n+\t\t */\n+\t\tret = bkl_parse_file(log_path, log_parse, &opts);\n+\t} else {\n+\t\tret = bkl_parse_file_reverse(log_path, log_parse, &opts);\n+\t}\n+\tif (ret < 0)\n+\t\tdie(_(\"failed to parse '%s'\"), log_path);\n+\tdump(NULL, &opts);\t/* flush */\n+\n+\treturn ret;\n+}\n+\n static char *log_id_to_path(const char *id)\n {\n \tif (!strcmp(id, \"index\"))\n@@ -256,6 +344,8 @@ int cmd_backup_log(int argc, const char **argv, const char *prefix)\n \t\treturn cat(argc, argv, prefix, log_path);\n \telse if (!strcmp(argv[0], \"diff\"))\n \t\treturn diff(argc, argv, prefix, log_path);\n+\telse if (!strcmp(argv[0], \"log\"))\n+\t\treturn log_(argc, argv, prefix, log_path);\n \telse\n \t\tdie(_(\"unknown subcommand: %s\"), argv[0]);\n \ndiff --git a/t/t2080-backup-log.sh b/t/t2080-backup-log.sh\nindex cd4288d386..3cdabc2ba2 100755\n--- a/t/t2080-backup-log.sh\n+++ b/t/t2080-backup-log.sh\n@@ -130,4 +130,38 @@ test_expect_success 'backup-log diff' '\n \ttest_cmp stat.expected stat.actual\n '\n \n+test_expect_success 'backup-log log' '\n+\trm -f .git/index.bkl &&\n+\techo first >initial.t &&\n+\tgit add initial.t &&\n+\techo second >initial.t &&\n+\ttest_tick &&\n+\tgit -c core.backupLog=true add initial.t &&\n+\techo third >initial.t &&\n+\t# note, same tick!\n+\tgit -c core.backupLog=true add initial.t &&\n+\ttest_tick &&\n+\techo forth >initial.t &&\n+\tgit -c core.backupLog=true add initial.t &&\n+\tgit backup-log --id=index log --stat --date=iso >actual &&\n+\tcat >expected <<-\\EOF &&\n+\tChange-Id: 1112912593\n+\tDate: 2005-04-07 15:23:13 -0700\n+\t\n+\t--\n+\t initial.t | 2 +-\n+\t 1 file changed, 1 insertion(+), 1 deletion(-)\n+\t\n+\tChange-Id: 1112912533\n+\tDate: 2005-04-07 15:22:13 -0700\n+\t\n+\t--\n+\t initial.t | 2 +-\n+\t initial.t | 2 +-\n+\t 2 files changed, 2 insertions(+), 2 deletions(-)\n+\t\n+\tEOF\n+\ttest_cmp expected actual\n+'\n+\n test_done\n-- \n2.20.0.rc2.486.g9832c05c3d\n\n"},{"id":"364857","messageId":"20181209104419.12639-18-pclouds@gmail.com","threadId":"49985","inReplyTo":"20181209104419.12639-1-pclouds@gmail.com","subject":"[PATCH 17/24] config --edit: support backup log","fromName":"Nguyễn Thái Ngọc Duy","fromEmail":"pclouds@gmail.com","sentAt":"2018-12-09T10:44:12Z","receivedAt":"2018-12-09T10:45:24Z","isPatch":true,"sender":{"key":"pclouds@gmail.com","avatar":"https://avatars.githubusercontent.com/u/720?v=4"},"body":"Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>\n---\n builtin/config.c      | 27 ++++++++++++++++++++++++++-\n t/t2080-backup-log.sh | 14 ++++++++++++++\n 2 files changed, 40 insertions(+), 1 deletion(-)\n\ndiff --git a/builtin/config.c b/builtin/config.c\nindex 84385ef165..a42044d03e 100644\n--- a/builtin/config.c\n+++ b/builtin/config.c\n@@ -1,5 +1,6 @@\n #include \"builtin.h\"\n #include \"cache.h\"\n+#include \"backup-log.h\"\n #include \"config.h\"\n #include \"color.h\"\n #include \"parse-options.h\"\n@@ -593,6 +594,15 @@ static char *default_user_config(void)\n \treturn strbuf_detach(&buf, NULL);\n }\n \n+static void hash_one_path(const char *path, struct object_id *oid)\n+{\n+\tstruct stat st;\n+\n+\tif (lstat(path, &st) ||\n+\t    index_path(NULL, oid, path, &st, HASH_WRITE_OBJECT))\n+\t\toidclr(oid);\n+}\n+\n int cmd_config(int argc, const char **argv, const char *prefix)\n {\n \tint nongit = !startup_info->have_repository;\n@@ -735,6 +745,8 @@ int cmd_config(int argc, const char **argv, const char *prefix)\n \t}\n \telse if (actions == ACTION_EDIT) {\n \t\tchar *config_file;\n+\t\tint core_backup_log = 0;\n+\t\tstruct object_id old, new;\n \n \t\tcheck_argc(argc, 0, 0);\n \t\tif (!given_config_source.file && nongit)\n@@ -747,6 +759,9 @@ int cmd_config(int argc, const char **argv, const char *prefix)\n \t\tconfig_file = given_config_source.file ?\n \t\t\t\txstrdup(given_config_source.file) :\n \t\t\t\tgit_pathdup(\"config\");\n+\t\trepo_config_get_bool(the_repository, \"core.backuplog\",\n+\t\t\t\t     &core_backup_log);\n+\t\toidclr(&old);\n \t\tif (use_global_config) {\n \t\t\tint fd = open(config_file, O_CREAT | O_EXCL | O_WRONLY, 0666);\n \t\t\tif (fd >= 0) {\n@@ -757,9 +772,19 @@ int cmd_config(int argc, const char **argv, const char *prefix)\n \t\t\t}\n \t\t\telse if (errno != EEXIST)\n \t\t\t\tdie_errno(_(\"cannot create configuration file %s\"), config_file);\n-\t\t}\n+\t\t} else if (!given_config_source.file && core_backup_log)\n+\t\t\thash_one_path(git_path(\"config\"), &old);\n \t\tlaunch_editor(config_file, NULL, NULL);\n \t\tfree(config_file);\n+\t\tif (!is_null_oid(&old)) {\n+\t\t\tstruct strbuf sb = STRBUF_INIT;\n+\n+\t\t\thash_one_path(git_path(\"config\"), &new);\n+\t\t\tbkl_append(&sb, \"config\", &old, &new);\n+\t\t\tmkdir(git_path(\"common\"), 0777);\n+\t\t\tbkl_write(git_path(\"common/gitdir.bkl\"), &sb);\n+\t\t\tstrbuf_release(&sb);\n+\t\t}\n \t}\n \telse if (actions == ACTION_SET) {\n \t\tint ret;\ndiff --git a/t/t2080-backup-log.sh b/t/t2080-backup-log.sh\nindex 3cdabc2ba2..dbd19db757 100755\n--- a/t/t2080-backup-log.sh\n+++ b/t/t2080-backup-log.sh\n@@ -164,4 +164,18 @@ test_expect_success 'backup-log log' '\n \ttest_cmp expected actual\n '\n \n+test_expect_success 'config --edit makes a backup' '\n+\tcat >edit.sh <<-EOF &&\n+\t#!$SHELL_PATH\n+\techo \";Edited\" >>\"\\$1\"\n+\tEOF\n+\tchmod +x edit.sh &&\n+\tOLD=$(git hash-object .git/config) &&\n+\tEDITOR=./edit.sh git -c core.backupLog=true config --edit &&\n+\tNEW=$(git hash-object .git/config) &&\n+\tgrep config .git/common/gitdir.bkl &&\n+\tgrep ^$OLD .git/common/gitdir.bkl &&\n+\tgrep $NEW .git/common/gitdir.bkl\n+'\n+\n test_done\n-- \n2.20.0.rc2.486.g9832c05c3d\n\n"},{"id":"364856","messageId":"20181209104419.12639-19-pclouds@gmail.com","threadId":"49985","inReplyTo":"20181209104419.12639-1-pclouds@gmail.com","subject":"[PATCH 18/24] refs: keep backup of deleted reflog","fromName":"Nguyễn Thái Ngọc Duy","fromEmail":"pclouds@gmail.com","sentAt":"2018-12-09T10:44:13Z","receivedAt":"2018-12-09T10:45:25Z","isPatch":true,"sender":{"key":"pclouds@gmail.com","avatar":"https://avatars.githubusercontent.com/u/720?v=4"},"body":"As noted in git-backup-log.txt a long time ago, this is mostly meant\nfor recovering a branch immediately after an accidental deletion.\n\nReferences from the deleted reflog will not be included in\nreachability tests and they will be deleted at the next gc. At that\npoint, this deleted reflog becomes useless.\n\nSigned-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>\n---\n refs/files-backend.c  | 32 ++++++++++++++++++++++++++++++++\n t/t2080-backup-log.sh | 12 ++++++++++++\n 2 files changed, 44 insertions(+)\n\ndiff --git a/refs/files-backend.c b/refs/files-backend.c\nindex dd8abe9185..9afb274075 100644\n--- a/refs/files-backend.c\n+++ b/refs/files-backend.c\n@@ -11,6 +11,7 @@\n #include \"../dir.h\"\n #include \"../chdir-notify.h\"\n #include \"worktree.h\"\n+#include \"backup-log.h\"\n \n /*\n  * This backend uses the following flags in `ref_update::flags` for\n@@ -1873,6 +1874,35 @@ static int files_reflog_exists(struct ref_store *ref_store,\n \treturn ret;\n }\n \n+static void backup_reflog(struct files_ref_store *refs,\n+\t\t\t  const char *reflog_path,\n+\t\t\t  const char *refname)\n+{\n+\tint core_backup_log = 0;\n+\tstruct stat st;\n+\tstruct strbuf line = STRBUF_INIT;\n+\tstruct strbuf path = STRBUF_INIT;\n+\tstruct object_id old, new;\n+\n+\trepo_config_get_bool(the_repository, \"core.backuplog\",\n+\t\t\t     &core_backup_log);\n+\tif (!core_backup_log)\n+\t\treturn;\n+\tif (ref_type(refname) != REF_TYPE_NORMAL)\n+\t\treturn;\n+\tif (lstat(reflog_path, &st) ||\n+\t    index_path(NULL, &old, reflog_path, &st, HASH_WRITE_OBJECT))\n+\t\treturn;\n+\n+\tstrbuf_addf(&path, \"logs/%s\", refname);\n+\toidclr(&new);\n+\tbkl_append(&line, path.buf, &old, &new);\n+\tstrbuf_release(&path);\n+\tmkdir_in_gitdir(git_path(\"common\"));\n+\tbkl_write(git_path(\"common/gitdir.bkl\"), &line);\n+\tstrbuf_release(&line);\n+}\n+\n static int files_delete_reflog(struct ref_store *ref_store,\n \t\t\t       const char *refname)\n {\n@@ -1882,6 +1912,7 @@ static int files_delete_reflog(struct ref_store *ref_store,\n \tint ret;\n \n \tfiles_reflog_path(refs, &sb, refname);\n+\tbackup_reflog(refs, sb.buf, refname);\n \tret = remove_path(sb.buf);\n \tstrbuf_release(&sb);\n \treturn ret;\n@@ -2797,6 +2828,7 @@ static int files_transaction_finish(struct ref_store *ref_store,\n \t\t    !(update->flags & REF_IS_PRUNING)) {\n \t\t\tstrbuf_reset(&sb);\n \t\t\tfiles_reflog_path(refs, &sb, update->refname);\n+\t\t\tbackup_reflog(refs, sb.buf, update->refname);\n \t\t\tif (!unlink_or_warn(sb.buf))\n \t\t\t\ttry_remove_empty_parents(refs, update->refname,\n \t\t\t\t\t\t\t REMOVE_EMPTY_PARENTS_REFLOG);\ndiff --git a/t/t2080-backup-log.sh b/t/t2080-backup-log.sh\nindex dbd19db757..710df1ec8b 100755\n--- a/t/t2080-backup-log.sh\n+++ b/t/t2080-backup-log.sh\n@@ -178,4 +178,16 @@ test_expect_success 'config --edit makes a backup' '\n \tgrep $NEW .git/common/gitdir.bkl\n '\n \n+test_expect_success 'deleted reflog is kept' '\n+\tgit checkout -b foo &&\n+\tgit commit -am everything-else &&\n+\ttest_commit two &&\n+\ttest_commit three &&\n+\tgit checkout -f master &&\n+\tOLD=$(git hash-object .git/logs/refs/heads/foo) &&\n+\tgit -c core.backupLog=true branch -D foo &&\n+\tgrep logs/refs/heads/foo .git/common/gitdir.bkl &&\n+\tgrep ^$OLD .git/common/gitdir.bkl\n+'\n+\n test_done\n-- \n2.20.0.rc2.486.g9832c05c3d\n\n"},{"id":"364858","messageId":"20181209104419.12639-16-pclouds@gmail.com","threadId":"49985","inReplyTo":"20181209104419.12639-1-pclouds@gmail.com","subject":"[PATCH 15/24] backup-log: keep all blob references around","fromName":"Nguyễn Thái Ngọc Duy","fromEmail":"pclouds@gmail.com","sentAt":"2018-12-09T10:44:10Z","receivedAt":"2018-12-09T10:45:27Z","isPatch":true,"sender":{"key":"pclouds@gmail.com","avatar":"https://avatars.githubusercontent.com/u/720?v=4"},"body":"The four commands prune, rev-list, pack-objects and repack are updated\nto not consider backup-log's blobs as unreachable and either delete\nthem outright or not repack them.\n\nSigned-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>\n---\n backup-log.c           | 65 ++++++++++++++++++++++++++++++++++++++++++\n backup-log.h           |  2 ++\n builtin/pack-objects.c |  9 +++++-\n builtin/repack.c       |  1 +\n reachable.c            |  3 ++\n revision.c             |  3 ++\n 6 files changed, 82 insertions(+), 1 deletion(-)\n\ndiff --git a/backup-log.c b/backup-log.c\nindex dbb6d5487e..37fa71e4e0 100644\n--- a/backup-log.c\n+++ b/backup-log.c\n@@ -3,6 +3,7 @@\n #include \"blob.h\"\n #include \"lockfile.h\"\n #include \"object-store.h\"\n+#include \"revision.h\"\n #include \"strbuf.h\"\n #include \"worktree.h\"\n \n@@ -321,3 +322,67 @@ void bkl_prune_all_or_die(struct repository *r, timestamp_t expire)\n \t}\n \tfree_worktrees(worktrees);\n }\n+\n+struct pending_cb {\n+\tstruct rev_info *revs;\n+\tunsigned flags;\n+};\n+\n+static void add_blob_to_pending(const struct object_id *oid,\n+\t\t\t\tconst char *path,\n+\t\t\t\tstruct pending_cb *cb)\n+{\n+\tstruct blob *blob;\n+\n+\tif (!good_oid(cb->revs->repo, oid))\n+\t\treturn;\n+\n+\tblob = lookup_blob(cb->revs->repo, oid);\n+\tblob->object.flags |= cb->flags;\n+\tadd_pending_object(cb->revs, &blob->object, path);\n+}\n+\n+static int add_pending(struct strbuf *line, void *cb)\n+{\n+\tstruct bkl_entry entry;\n+\n+\tif (bkl_parse_entry(line, &entry))\n+\t\treturn -1;\n+\n+\tadd_blob_to_pending(&entry.old_oid, entry.path, cb);\n+\tadd_blob_to_pending(&entry.new_oid, entry.path, cb);\n+\treturn 0;\n+}\n+\n+static void add_backup_log_to_pending(const char *path, struct pending_cb *cb)\n+{\n+\tbkl_parse_file(path, add_pending, cb);\n+}\n+\n+void add_backup_logs_to_pending(struct rev_info *revs, unsigned flags)\n+{\n+\tstruct worktree **worktrees, **p;\n+\tchar *path;\n+\tstruct pending_cb cb;\n+\n+\tcb.revs = revs;\n+\tcb.flags = flags;\n+\n+\tworktrees = get_worktrees(0);\n+\tfor (p = worktrees; *p; p++) {\n+\t\tstruct worktree *wt = *p;\n+\n+\t\tpath = xstrdup(worktree_git_path(wt, \"index.bkl\"));\n+\t\tadd_backup_log_to_pending(path, &cb);\n+\t\tfree(path);\n+\n+\t\tpath = xstrdup(worktree_git_path(wt, \"worktree.bkl\"));\n+\t\tadd_backup_log_to_pending(path, &cb);\n+\t\tfree(path);\n+\t}\n+\tfree_worktrees(worktrees);\n+\n+\tpath = git_pathdup(\"common/gitdir.bkl\");\n+\tadd_backup_log_to_pending(path, &cb);\n+\tfree(path);\n+}\ndiff --git a/backup-log.h b/backup-log.h\nindex 6572ce9c93..aaa76b7fe2 100644\n--- a/backup-log.h\n+++ b/backup-log.h\n@@ -4,6 +4,7 @@\n #include \"cache.h\"\n \n struct repository;\n+struct rev_info;\n struct strbuf;\n \n struct bkl_entry\n@@ -30,6 +31,7 @@ int bkl_parse_file(const char *path,\n \t\t   int (*parse)(struct strbuf *line, void *data),\n \t\t   void *data);\n \n+void add_backup_logs_to_pending(struct rev_info *revs, unsigned flags);\n int bkl_prune(struct repository *r, const char *id, timestamp_t expire);\n void bkl_prune_all_or_die(struct repository *r, timestamp_t expire);\n \ndiff --git a/builtin/pack-objects.c b/builtin/pack-objects.c\nindex 411aefd687..940eb0c768 100644\n--- a/builtin/pack-objects.c\n+++ b/builtin/pack-objects.c\n@@ -3230,7 +3230,7 @@ int cmd_pack_objects(int argc, const char **argv, const char *prefix)\n \tint all_progress_implied = 0;\n \tstruct argv_array rp = ARGV_ARRAY_INIT;\n \tint rev_list_unpacked = 0, rev_list_all = 0, rev_list_reflog = 0;\n-\tint rev_list_index = 0;\n+\tint rev_list_index = 0, rev_list_backuplog = 0;\n \tstruct string_list keep_pack_list = STRING_LIST_INIT_NODUP;\n \tstruct option pack_objects_options[] = {\n \t\tOPT_SET_INT('q', \"quiet\", &progress,\n@@ -3278,6 +3278,9 @@ int cmd_pack_objects(int argc, const char **argv, const char *prefix)\n \t\tOPT_SET_INT_F(0, \"reflog\", &rev_list_reflog,\n \t\t\t      N_(\"include objects referred by reflog entries\"),\n \t\t\t      1, PARSE_OPT_NONEG),\n+\t\tOPT_SET_INT_F(0, \"backup-log\", &rev_list_backuplog,\n+\t\t\t      N_(\"include objects referred by backup-log entries\"),\n+\t\t\t      1, PARSE_OPT_NONEG),\n \t\tOPT_SET_INT_F(0, \"indexed-objects\", &rev_list_index,\n \t\t\t      N_(\"include objects referred to by the index\"),\n \t\t\t      1, PARSE_OPT_NONEG),\n@@ -3366,6 +3369,10 @@ int cmd_pack_objects(int argc, const char **argv, const char *prefix)\n \t\tuse_internal_rev_list = 1;\n \t\targv_array_push(&rp, \"--reflog\");\n \t}\n+\tif (rev_list_backuplog) {\n+\t\tuse_internal_rev_list = 1;\n+\t\targv_array_push(&rp, \"--backup-log\");\n+\t}\n \tif (rev_list_index) {\n \t\tuse_internal_rev_list = 1;\n \t\targv_array_push(&rp, \"--indexed-objects\");\ndiff --git a/builtin/repack.c b/builtin/repack.c\nindex 45583683ee..a8a9fad9c6 100644\n--- a/builtin/repack.c\n+++ b/builtin/repack.c\n@@ -365,6 +365,7 @@ int cmd_repack(int argc, const char **argv, const char *prefix)\n \targv_array_push(&cmd.args, \"--non-empty\");\n \targv_array_push(&cmd.args, \"--all\");\n \targv_array_push(&cmd.args, \"--reflog\");\n+\targv_array_push(&cmd.args, \"--backup-log\");\n \targv_array_push(&cmd.args, \"--indexed-objects\");\n \tif (repository_format_partial_clone)\n \t\targv_array_push(&cmd.args, \"--exclude-promisor-objects\");\ndiff --git a/reachable.c b/reachable.c\nindex 6e9b810d2a..61f6560b54 100644\n--- a/reachable.c\n+++ b/reachable.c\n@@ -12,6 +12,7 @@\n #include \"packfile.h\"\n #include \"worktree.h\"\n #include \"object-store.h\"\n+#include \"backup-log.h\"\n \n struct connectivity_progress {\n \tstruct progress *progress;\n@@ -185,6 +186,8 @@ void mark_reachable_objects(struct rev_info *revs, int mark_reflog,\n \tif (mark_reflog)\n \t\tadd_reflogs_to_pending(revs, 0);\n \n+\tadd_backup_logs_to_pending(revs, 0);\n+\n \tcp.progress = progress;\n \tcp.count = 0;\n \ndiff --git a/revision.c b/revision.c\nindex 13e0519c02..755edea61e 100644\n--- a/revision.c\n+++ b/revision.c\n@@ -27,6 +27,7 @@\n #include \"commit-reach.h\"\n #include \"commit-graph.h\"\n #include \"prio-queue.h\"\n+#include \"backup-log.h\"\n \n volatile show_early_output_fn_t show_early_output;\n \n@@ -2286,6 +2287,8 @@ static int handle_revision_pseudo_opt(const char *submodule,\n \t\tclear_ref_exclusion(&revs->ref_excludes);\n \t} else if (!strcmp(arg, \"--reflog\")) {\n \t\tadd_reflogs_to_pending(revs, *flags);\n+\t} else if (!strcmp(arg, \"--backup-log\")) {\n+\t\tadd_backup_logs_to_pending(revs, *flags);\n \t} else if (!strcmp(arg, \"--indexed-objects\")) {\n \t\tadd_index_objects_to_pending(revs, *flags);\n \t} else if (!strcmp(arg, \"--not\")) {\n-- \n2.20.0.rc2.486.g9832c05c3d\n\n"},{"id":"364859","messageId":"20181209104419.12639-20-pclouds@gmail.com","threadId":"49985","inReplyTo":"20181209104419.12639-1-pclouds@gmail.com","subject":"[PATCH 19/24] unpack-trees.c: keep backup of ignored files being overwritten","fromName":"Nguyễn Thái Ngọc Duy","fromEmail":"pclouds@gmail.com","sentAt":"2018-12-09T10:44:14Z","receivedAt":"2018-12-09T10:45:28Z","isPatch":true,"sender":{"key":"pclouds@gmail.com","avatar":"https://avatars.githubusercontent.com/u/720?v=4"},"body":"Ignored files are usually machine generated (e.g. *.o) and not worth\nkeeping, so when a merge or branch switch happens and need to\noverwrite them, we just go ahead and do it. Occasionally though\nignored files _can_ have valuable content.\n\nWe will likely have a separate mechanism to protect these \"precious\"\nignored files, but a surprise is still a surprise. Keep a backup of\nignored files when they are overwritten (until we are explicitly told\n\"rubbish\" by said mechanism). This lets the user recover the content\nand start telling git that the file is precious so it won't happen\nagain.\n\nSigned-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>\n---\n builtin/checkout.c    |  2 ++\n merge.c               |  2 ++\n t/t2080-backup-log.sh | 21 ++++++++++++\n unpack-trees.c        | 77 +++++++++++++++++++++++++++++++++++--------\n unpack-trees.h        |  3 ++\n 5 files changed, 92 insertions(+), 13 deletions(-)\n\ndiff --git a/builtin/checkout.c b/builtin/checkout.c\nindex acdafc6e4c..b5e27a5f6d 100644\n--- a/builtin/checkout.c\n+++ b/builtin/checkout.c\n@@ -620,6 +620,8 @@ static int merge_working_tree(const struct checkout_opts *opts,\n \t\t\ttopts.dir = xcalloc(1, sizeof(*topts.dir));\n \t\t\ttopts.dir->flags |= DIR_SHOW_IGNORED;\n \t\t\tsetup_standard_excludes(topts.dir);\n+\t\t\trepo_config_get_bool(the_repository, \"core.backupLog\",\n+\t\t\t\t\t     &topts.keep_backup);\n \t\t}\n \t\ttree = parse_tree_indirect(old_branch_info->commit ?\n \t\t\t\t\t   &old_branch_info->commit->object.oid :\ndiff --git a/merge.c b/merge.c\nindex 91008f7602..9f20305d7a 100644\n--- a/merge.c\n+++ b/merge.c\n@@ -1,4 +1,5 @@\n #include \"cache.h\"\n+#include \"config.h\"\n #include \"diff.h\"\n #include \"diffcore.h\"\n #include \"lockfile.h\"\n@@ -85,6 +86,7 @@ int checkout_fast_forward(struct repository *r,\n \t\tdir.flags |= DIR_SHOW_IGNORED;\n \t\tsetup_standard_excludes(&dir);\n \t\topts.dir = &dir;\n+\t\trepo_config_get_bool(r, \"core.backupLog\", &opts.keep_backup);\n \t}\n \n \topts.head_idx = 1;\ndiff --git a/t/t2080-backup-log.sh b/t/t2080-backup-log.sh\nindex 710df1ec8b..a283528912 100755\n--- a/t/t2080-backup-log.sh\n+++ b/t/t2080-backup-log.sh\n@@ -190,4 +190,25 @@ test_expect_success 'deleted reflog is kept' '\n \tgrep ^$OLD .git/common/gitdir.bkl\n '\n \n+test_expect_success 'overwritten ignored file is backed up' '\n+\tgit init overwrite-ignore &&\n+\t(\n+\t\tcd overwrite-ignore &&\n+\t\techo ignored-overwritten >ignored &&\n+\t\tNEW=$(git hash-object ignored) &&\n+\t\tgit add ignored &&\n+\t\tgit commit -m ignored &&\n+\t\tgit rm --cached ignored &&\n+\t\techo /ignored >.gitignore &&\n+\t\tgit add .gitignore &&\n+\t\tgit commit -m first-commit-no-ignored &&\n+\t\techo precious >ignored &&\n+\t\tOLD=$(git hash-object ignored) &&\n+\t\ttest_tick &&\n+\t\tgit -c core.backupLog=true checkout --detach HEAD^ &&\n+\t\techo \"$OLD $NEW $GIT_COMMITTER_NAME <$GIT_COMMITTER_EMAIL> $test_tick -0700\tignored\" >expected &&\n+\t\ttest_cmp expected .git/worktree.bkl\n+\t)\n+'\n+\n test_done\ndiff --git a/unpack-trees.c b/unpack-trees.c\nindex 7570df481b..8d7273af2b 100644\n--- a/unpack-trees.c\n+++ b/unpack-trees.c\n@@ -1,6 +1,7 @@\n #define NO_THE_INDEX_COMPATIBILITY_MACROS\n #include \"cache.h\"\n #include \"argv-array.h\"\n+#include \"backup-log.h\"\n #include \"repository.h\"\n #include \"config.h\"\n #include \"dir.h\"\n@@ -191,6 +192,23 @@ void clear_unpack_trees_porcelain(struct unpack_trees_options *opts)\n \tmemset(opts->msgs, 0, sizeof(opts->msgs));\n }\n \n+static void make_backup(const struct cache_entry *ce,\n+\t\t\tconst struct object_id *old_hash,\n+\t\t\tconst struct object_id *new_hash,\n+\t\t\tstruct unpack_trees_options *o)\n+{\n+\tstruct object_id null_hash;\n+\n+\tif (!o->keep_backup || is_null_oid(old_hash))\n+\t\treturn;\n+\n+\tif (!new_hash) {\n+\t\toidclr(&null_hash);\n+\t\tnew_hash = &null_hash;\n+\t}\n+\tbkl_append(&o->backup_log, ce->name, old_hash, new_hash);\n+}\n+\n static int do_add_entry(struct unpack_trees_options *o, struct cache_entry *ce,\n \t\t\t unsigned int set, unsigned int clear)\n {\n@@ -462,6 +480,9 @@ static int check_updates(struct unpack_trees_options *o)\n \tif (o->clone)\n \t\treport_collided_checkout(index);\n \n+\tif (o->backup_log.len)\n+\t\tbkl_write(git_path(\"worktree.bkl\"), &o->backup_log);\n+\n \ttrace_performance_leave(\"check_updates\");\n \treturn errs != 0;\n }\n@@ -1460,7 +1481,8 @@ static void mark_new_skip_worktree(struct exclude_list *el,\n \n static int verify_absent(const struct cache_entry *,\n \t\t\t enum unpack_trees_error_types,\n-\t\t\t struct unpack_trees_options *);\n+\t\t\t struct unpack_trees_options *,\n+\t\t\t struct object_id *);\n /*\n  * N-way merge \"len\" trees.  Returns 0 on success, -1 on failure to manipulate the\n  * resulting index, -2 on failure to reflect the changes to the work tree.\n@@ -1489,6 +1511,10 @@ int unpack_trees(unsigned len, struct tree_desc *t, struct unpack_trees_options\n \t\tfree(sparse);\n \t}\n \n+\tstrbuf_init(&o->backup_log, 0);\n+\tif (!o->update)\n+\t\to->keep_backup = 0;\n+\n \tmemset(&o->result, 0, sizeof(o->result));\n \to->result.initialized = 1;\n \to->result.timestamp.sec = o->src_index->timestamp.sec;\n@@ -1596,7 +1622,7 @@ int unpack_trees(unsigned len, struct tree_desc *t, struct unpack_trees_options\n \t\t\t * correct CE_NEW_SKIP_WORKTREE\n \t\t\t */\n \t\t\tif (ce->ce_flags & CE_ADDED &&\n-\t\t\t    verify_absent(ce, ERROR_WOULD_LOSE_UNTRACKED_OVERWRITTEN, o)) {\n+\t\t\t    verify_absent(ce, ERROR_WOULD_LOSE_UNTRACKED_OVERWRITTEN, o, NULL)) {\n \t\t\t\tif (!o->show_all_errors)\n \t\t\t\t\tgoto return_failed;\n \t\t\t\tret = -1;\n@@ -1646,6 +1672,7 @@ int unpack_trees(unsigned len, struct tree_desc *t, struct unpack_trees_options\n \to->src_index = NULL;\n \n done:\n+\tstrbuf_release(&o->backup_log);\n \ttrace_performance_leave(\"unpack_trees\");\n \tclear_exclude_list(&el);\n \treturn ret;\n@@ -1880,7 +1907,8 @@ static int icase_exists(struct unpack_trees_options *o, const char *name, int le\n static int check_ok_to_remove(const char *name, int len, int dtype,\n \t\t\t      const struct cache_entry *ce, struct stat *st,\n \t\t\t      enum unpack_trees_error_types error_type,\n-\t\t\t      struct unpack_trees_options *o)\n+\t\t\t      struct unpack_trees_options *o,\n+\t\t\t      struct object_id *old_hash)\n {\n \tconst struct cache_entry *result;\n \n@@ -1895,12 +1923,16 @@ static int check_ok_to_remove(const char *name, int len, int dtype,\n \t\treturn 0;\n \n \tif (o->dir &&\n-\t    is_excluded(o->dir, o->src_index, name, &dtype))\n+\t    is_excluded(o->dir, o->src_index, name, &dtype)) {\n+\t\tif (o->keep_backup && old_hash)\n+\t\t\tindex_path(NULL, old_hash, name, st,\n+\t\t\t\t   HASH_WRITE_OBJECT);\n \t\t/*\n \t\t * ce->name is explicitly excluded, so it is Ok to\n \t\t * overwrite it.\n \t\t */\n \t\treturn 0;\n+\t}\n \tif (S_ISDIR(st->st_mode)) {\n \t\t/*\n \t\t * We are checking out path \"foo\" and\n@@ -1935,7 +1967,8 @@ static int check_ok_to_remove(const char *name, int len, int dtype,\n  */\n static int verify_absent_1(const struct cache_entry *ce,\n \t\t\t   enum unpack_trees_error_types error_type,\n-\t\t\t   struct unpack_trees_options *o)\n+\t\t\t   struct unpack_trees_options *o,\n+\t\t\t   struct object_id *old_hash)\n {\n \tint len;\n \tstruct stat st;\n@@ -1960,7 +1993,7 @@ static int verify_absent_1(const struct cache_entry *ce,\n \t\t\t\t\t\t\t\tNULL, o);\n \t\t\telse\n \t\t\t\tret = check_ok_to_remove(path, len, DT_UNKNOWN, NULL,\n-\t\t\t\t\t\t\t &st, error_type, o);\n+\t\t\t\t\t\t\t &st, error_type, o, old_hash);\n \t\t}\n \t\tfree(path);\n \t\treturn ret;\n@@ -1975,17 +2008,20 @@ static int verify_absent_1(const struct cache_entry *ce,\n \n \t\treturn check_ok_to_remove(ce->name, ce_namelen(ce),\n \t\t\t\t\t  ce_to_dtype(ce), ce, &st,\n-\t\t\t\t\t  error_type, o);\n+\t\t\t\t\t  error_type, o, old_hash);\n \t}\n }\n \n static int verify_absent(const struct cache_entry *ce,\n \t\t\t enum unpack_trees_error_types error_type,\n-\t\t\t struct unpack_trees_options *o)\n+\t\t\t struct unpack_trees_options *o,\n+\t\t\t struct object_id *old_hash)\n {\n+\tif (o->keep_backup && old_hash)\n+\t\toidclr(old_hash);\n \tif (!o->skip_sparse_checkout && (ce->ce_flags & CE_NEW_SKIP_WORKTREE))\n \t\treturn 0;\n-\treturn verify_absent_1(ce, error_type, o);\n+\treturn verify_absent_1(ce, error_type, o, old_hash);\n }\n \n static int verify_absent_sparse(const struct cache_entry *ce,\n@@ -1996,7 +2032,7 @@ static int verify_absent_sparse(const struct cache_entry *ce,\n \tif (orphaned_error == ERROR_WOULD_LOSE_UNTRACKED_OVERWRITTEN)\n \t\torphaned_error = ERROR_WOULD_LOSE_ORPHANED_OVERWRITTEN;\n \n-\treturn verify_absent_1(ce, orphaned_error, o);\n+\treturn verify_absent_1(ce, orphaned_error, o, NULL);\n }\n \n static int merged_entry(const struct cache_entry *ce,\n@@ -2007,6 +2043,8 @@ static int merged_entry(const struct cache_entry *ce,\n \tstruct cache_entry *merge = dup_cache_entry(ce, &o->result);\n \n \tif (!old) {\n+\t\tstruct object_id old_hash;\n+\n \t\t/*\n \t\t * New index entries. In sparse checkout, the following\n \t\t * verify_absent() will be delayed until after\n@@ -2023,10 +2061,15 @@ static int merged_entry(const struct cache_entry *ce,\n \t\tmerge->ce_flags |= CE_NEW_SKIP_WORKTREE;\n \n \t\tif (verify_absent(merge,\n-\t\t\t\t  ERROR_WOULD_LOSE_UNTRACKED_OVERWRITTEN, o)) {\n+\t\t\t\t  ERROR_WOULD_LOSE_UNTRACKED_OVERWRITTEN,\n+\t\t\t\t  o, &old_hash)) {\n \t\t\tdiscard_cache_entry(merge);\n \t\t\treturn -1;\n \t\t}\n+\t\tif (o->keep_backup)\n+\t\t\tbkl_append(&o->backup_log, merge->name,\n+\t\t\t\t   &old_hash, &merge->oid);\n+\n \t\tinvalidate_ce_path(merge, o);\n \n \t\tif (submodule_from_ce(ce)) {\n@@ -2083,8 +2126,12 @@ static int deleted_entry(const struct cache_entry *ce,\n {\n \t/* Did it exist in the index? */\n \tif (!old) {\n-\t\tif (verify_absent(ce, ERROR_WOULD_LOSE_UNTRACKED_REMOVED, o))\n+\t\tstruct object_id old_hash;\n+\n+\t\tif (verify_absent(ce, ERROR_WOULD_LOSE_UNTRACKED_REMOVED,\n+\t\t\t\t  o, &old_hash))\n \t\t\treturn -1;\n+\t\tmake_backup(ce, &old_hash, NULL, o);\n \t\treturn 0;\n \t}\n \tif (!(old->ce_flags & CE_CONFLICTED) && verify_uptodate(old, o))\n@@ -2236,8 +2283,12 @@ int threeway_merge(const struct cache_entry * const *stages,\n \t\t\tif (index)\n \t\t\t\treturn deleted_entry(index, index, o);\n \t\t\tif (ce && !head_deleted) {\n-\t\t\t\tif (verify_absent(ce, ERROR_WOULD_LOSE_UNTRACKED_REMOVED, o))\n+\t\t\t\tstruct object_id old_hash;\n+\n+\t\t\t\tif (verify_absent(ce, ERROR_WOULD_LOSE_UNTRACKED_REMOVED,\n+\t\t\t\t\t\t  o, &old_hash))\n \t\t\t\t\treturn -1;\n+\t\t\t\tmake_backup(ce, &old_hash, NULL, o);\n \t\t\t}\n \t\t\treturn 0;\n \t\t}\ndiff --git a/unpack-trees.h b/unpack-trees.h\nindex 0135080a7b..e2a64e2401 100644\n--- a/unpack-trees.h\n+++ b/unpack-trees.h\n@@ -60,6 +60,7 @@ struct unpack_trees_options {\n \t\t     exiting_early,\n \t\t     show_all_errors,\n \t\t     dry_run;\n+\tint keep_backup;\n \tconst char *prefix;\n \tint cache_bottom;\n \tstruct dir_struct *dir;\n@@ -83,6 +84,8 @@ struct unpack_trees_options {\n \tstruct index_state *src_index;\n \tstruct index_state result;\n \n+\tstruct strbuf backup_log;\n+\n \tstruct exclude_list *el; /* for internal use */\n };\n \n-- \n2.20.0.rc2.486.g9832c05c3d\n\n"},{"id":"364860","messageId":"20181209104419.12639-23-pclouds@gmail.com","threadId":"49985","inReplyTo":"20181209104419.12639-1-pclouds@gmail.com","subject":"[PATCH 22/24] am: keep backup of overwritten files on --skip or --abort","fromName":"Nguyễn Thái Ngọc Duy","fromEmail":"pclouds@gmail.com","sentAt":"2018-12-09T10:44:17Z","receivedAt":"2018-12-09T10:45:30Z","isPatch":true,"sender":{"key":"pclouds@gmail.com","avatar":"https://avatars.githubusercontent.com/u/720?v=4"},"body":"Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>\n---\n builtin/am.c | 3 +++\n 1 file changed, 3 insertions(+)\n\ndiff --git a/builtin/am.c b/builtin/am.c\nindex 8f27f3375b..098bbaab39 100644\n--- a/builtin/am.c\n+++ b/builtin/am.c\n@@ -1886,6 +1886,9 @@ static int fast_forward_to(struct tree *head, struct tree *remote, int reset)\n \topts.update = 1;\n \topts.merge = 1;\n \topts.reset = reset;\n+\tif (opts.reset)\n+\t\trepo_config_get_bool(the_repository, \"core.backupLog\",\n+\t\t\t\t     &opts.keep_backup);\n \topts.fn = twoway_merge;\n \tinit_tree_desc(&t[0], head->buffer, head->size);\n \tinit_tree_desc(&t[1], remote->buffer, remote->size);\n-- \n2.20.0.rc2.486.g9832c05c3d\n\n"},{"id":"364862","messageId":"20181209104419.12639-22-pclouds@gmail.com","threadId":"49985","inReplyTo":"20181209104419.12639-1-pclouds@gmail.com","subject":"[PATCH 21/24] checkout -f: keep backup of overwritten files","fromName":"Nguyễn Thái Ngọc Duy","fromEmail":"pclouds@gmail.com","sentAt":"2018-12-09T10:44:16Z","receivedAt":"2018-12-09T10:45:31Z","isPatch":true,"sender":{"key":"pclouds@gmail.com","avatar":"https://avatars.githubusercontent.com/u/720?v=4"},"body":"Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>\n---\n builtin/checkout.c | 2 ++\n 1 file changed, 2 insertions(+)\n\ndiff --git a/builtin/checkout.c b/builtin/checkout.c\nindex b5e27a5f6d..3ae001ae35 100644\n--- a/builtin/checkout.c\n+++ b/builtin/checkout.c\n@@ -438,6 +438,8 @@ static int reset_tree(struct tree *tree, const struct checkout_opts *o,\n \topts.verbose_update = o->show_progress;\n \topts.src_index = &the_index;\n \topts.dst_index = &the_index;\n+\trepo_config_get_bool(the_repository, \"core.backupLog\",\n+\t\t\t     &opts.keep_backup);\n \tparse_tree(tree);\n \tinit_tree_desc(&tree_desc, tree->buffer, tree->size);\n \tswitch (unpack_trees(1, &tree_desc, &opts)) {\n-- \n2.20.0.rc2.486.g9832c05c3d\n\n"},{"id":"364861","messageId":"20181209104419.12639-15-pclouds@gmail.com","threadId":"49985","inReplyTo":"20181209104419.12639-1-pclouds@gmail.com","subject":"[PATCH 14/24] gc: prune backup logs","fromName":"Nguyễn Thái Ngọc Duy","fromEmail":"pclouds@gmail.com","sentAt":"2018-12-09T10:44:09Z","receivedAt":"2018-12-09T10:45:32Z","isPatch":true,"sender":{"key":"pclouds@gmail.com","avatar":"https://avatars.githubusercontent.com/u/720?v=4"},"body":"Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>\n---\n backup-log.c | 33 +++++++++++++++++++++++++++++++++\n backup-log.h |  1 +\n builtin/gc.c |  3 +++\n 3 files changed, 37 insertions(+)\n\ndiff --git a/backup-log.c b/backup-log.c\nindex 5e38725981..dbb6d5487e 100644\n--- a/backup-log.c\n+++ b/backup-log.c\n@@ -4,6 +4,7 @@\n #include \"lockfile.h\"\n #include \"object-store.h\"\n #include \"strbuf.h\"\n+#include \"worktree.h\"\n \n void bkl_append(struct strbuf *output, const char *path,\n \t\tconst struct object_id *from,\n@@ -288,3 +289,35 @@ int bkl_prune(struct repository *r, const char *path, timestamp_t expire)\n \tstrbuf_release(&opts.copy);\n \treturn ret;\n }\n+\n+void bkl_prune_all_or_die(struct repository *r, timestamp_t expire)\n+{\n+\tstruct worktree **worktrees, **p;\n+\tchar *bkl_path;\n+\n+\tbkl_path = repo_git_path(r, \"common/gitdir.bkl\");\n+\tif (bkl_prune(r, bkl_path, expire))\n+\t\tdie(_(\"failed to prune %s\"), \"gitdir.bkl\");\n+\tfree(bkl_path);\n+\n+\tworktrees = get_worktrees(0);\n+\tfor (p = worktrees; *p; p++) {\n+\t\tstruct worktree *wt = *p;\n+\n+\t\tif (bkl_prune(r, worktree_git_path(wt, \"index.bkl\"), expire)) {\n+\t\t\tif (wt->id)\n+\t\t\t\tdie(_(\"failed to prune %s on working tree '%s'\"),\n+\t\t\t\t    \"index.bkl\", wt->id);\n+\t\t\telse\n+\t\t\t\tdie(_(\"failed to prune %s\"), \"index.bkl\");\n+\t\t}\n+\t\tif (bkl_prune(r, worktree_git_path(wt, \"worktree.bkl\"), expire)) {\n+\t\t\tif (wt->id)\n+\t\t\t\tdie(_(\"failed to prune %s on working tree '%s'\"),\n+\t\t\t\t    \"worktree.bkl\", wt->id);\n+\t\t\telse\n+\t\t\t\tdie(_(\"failed to prune %s\"), \"worktree.bkl\");\n+\t\t}\n+\t}\n+\tfree_worktrees(worktrees);\n+}\ndiff --git a/backup-log.h b/backup-log.h\nindex 06fe706f81..6572ce9c93 100644\n--- a/backup-log.h\n+++ b/backup-log.h\n@@ -31,5 +31,6 @@ int bkl_parse_file(const char *path,\n \t\t   void *data);\n \n int bkl_prune(struct repository *r, const char *id, timestamp_t expire);\n+void bkl_prune_all_or_die(struct repository *r, timestamp_t expire);\n \n #endif\ndiff --git a/builtin/gc.c b/builtin/gc.c\nindex 871a56f1c5..50a5d46abb 100644\n--- a/builtin/gc.c\n+++ b/builtin/gc.c\n@@ -27,6 +27,7 @@\n #include \"pack-objects.h\"\n #include \"blob.h\"\n #include \"tree.h\"\n+#include \"backup-log.h\"\n \n #define FAILED_RUN \"failed to run %s\"\n \n@@ -657,6 +658,8 @@ int cmd_gc(int argc, const char **argv, const char *prefix)\n \tif (run_command_v_opt(rerere.argv, RUN_GIT_CMD))\n \t\tdie(FAILED_RUN, rerere.argv[0]);\n \n+\tbkl_prune_all_or_die(the_repository, time(NULL) - 90 * 24 * 3600);\n+\n \treport_garbage = report_pack_garbage;\n \treprepare_packed_git(the_repository);\n \tif (pack_garbage.nr > 0)\n-- \n2.20.0.rc2.486.g9832c05c3d\n\n"},{"id":"364863","messageId":"20181209104419.12639-24-pclouds@gmail.com","threadId":"49985","inReplyTo":"20181209104419.12639-1-pclouds@gmail.com","subject":"[PATCH 23/24] rebase: keep backup of overwritten files on --skip or --abort","fromName":"Nguyễn Thái Ngọc Duy","fromEmail":"pclouds@gmail.com","sentAt":"2018-12-09T10:44:18Z","receivedAt":"2018-12-09T10:45:33Z","isPatch":true,"sender":{"key":"pclouds@gmail.com","avatar":"https://avatars.githubusercontent.com/u/720?v=4"},"body":"Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>\n---\n builtin/rebase.c | 6 +++++-\n 1 file changed, 5 insertions(+), 1 deletion(-)\n\ndiff --git a/builtin/rebase.c b/builtin/rebase.c\nindex b5c99ec10c..5c7b223843 100644\n--- a/builtin/rebase.c\n+++ b/builtin/rebase.c\n@@ -573,8 +573,12 @@ static int reset_head(struct object_id *oid, const char *action,\n \tunpack_tree_opts.fn = reset_hard ? oneway_merge : twoway_merge;\n \tunpack_tree_opts.update = 1;\n \tunpack_tree_opts.merge = 1;\n-\tif (!detach_head)\n+\tif (!detach_head) {\n \t\tunpack_tree_opts.reset = 1;\n+\t\trepo_config_get_bool(the_repository, \"core.backupLog\",\n+\t\t\t\t     &unpack_tree_opts.keep_backup);\n+\t}\n+\n \n \tif (read_index_unmerged(the_repository->index) < 0) {\n \t\tret = error(_(\"could not read index\"));\n-- \n2.20.0.rc2.486.g9832c05c3d\n\n"},{"id":"364864","messageId":"20181209104419.12639-25-pclouds@gmail.com","threadId":"49985","inReplyTo":"20181209104419.12639-1-pclouds@gmail.com","subject":"[PATCH 24/24] FIXME","fromName":"Nguyễn Thái Ngọc Duy","fromEmail":"pclouds@gmail.com","sentAt":"2018-12-09T10:44:19Z","receivedAt":"2018-12-09T10:45:34Z","isPatch":true,"sender":{"key":"pclouds@gmail.com","avatar":"https://avatars.githubusercontent.com/u/720?v=4"},"body":"Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>\n---\n Documentation/git-backup-log.txt | 2 ++\n 1 file changed, 2 insertions(+)\n\ndiff --git a/Documentation/git-backup-log.txt b/Documentation/git-backup-log.txt\nindex 98998156c1..fe03726337 100644\n--- a/Documentation/git-backup-log.txt\n+++ b/Documentation/git-backup-log.txt\n@@ -41,6 +41,8 @@ following commands will save backups:\n   `--hard` or linkgit:git-am[1] and linkgit:git-rebase[1] with\n   `--skip` or `--abort` will make a backup before overwriting non\n   up-to-date files.\n+- FIXME perhaps `git checkout <paths>` only makes backups on\n+  \"precious\" paths only?\n \n Backups are split in three groups, changes related in the index, in\n working directory or in $GIT_DIR. These can be selected with `--id`\n-- \n2.20.0.rc2.486.g9832c05c3d\n\n"},{"id":"364865","messageId":"20181209104419.12639-21-pclouds@gmail.com","threadId":"49985","inReplyTo":"20181209104419.12639-1-pclouds@gmail.com","subject":"[PATCH 20/24] reset --hard: keep backup of overwritten files","fromName":"Nguyễn Thái Ngọc Duy","fromEmail":"pclouds@gmail.com","sentAt":"2018-12-09T10:44:15Z","receivedAt":"2018-12-09T10:45:35Z","isPatch":true,"sender":{"key":"pclouds@gmail.com","avatar":"https://avatars.githubusercontent.com/u/720?v=4"},"body":"Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>\n---\n builtin/reset.c       |  2 ++\n merge-recursive.c     |  2 +-\n t/t2080-backup-log.sh | 14 +++++++++\n unpack-trees.c        | 70 +++++++++++++++++++++++++++++++++----------\n unpack-trees.h        |  3 +-\n 5 files changed, 74 insertions(+), 17 deletions(-)\n\ndiff --git a/builtin/reset.c b/builtin/reset.c\nindex 58166964f8..517a27dce5 100644\n--- a/builtin/reset.c\n+++ b/builtin/reset.c\n@@ -67,6 +67,8 @@ static int reset_index(const struct object_id *oid, int reset_type, int quiet)\n \t\tbreak;\n \tcase HARD:\n \t\topts.update = 1;\n+\t\trepo_config_get_bool(the_repository, \"core.backupLog\",\n+\t\t\t\t     &opts.keep_backup);\n \t\t/* fallthrough */\n \tdefault:\n \t\topts.reset = 1;\ndiff --git a/merge-recursive.c b/merge-recursive.c\nindex acc2f64a4e..10a9d3180a 100644\n--- a/merge-recursive.c\n+++ b/merge-recursive.c\n@@ -896,7 +896,7 @@ static int was_dirty(struct merge_options *o, const char *path)\n \n \tce = index_file_exists(o->unpack_opts.src_index,\n \t\t\t       path, strlen(path), ignore_case);\n-\tdirty = verify_uptodate(ce, &o->unpack_opts) != 0;\n+\tdirty = verify_uptodate(ce, &o->unpack_opts, NULL) != 0;\n \treturn dirty;\n }\n \ndiff --git a/t/t2080-backup-log.sh b/t/t2080-backup-log.sh\nindex a283528912..901755ce93 100755\n--- a/t/t2080-backup-log.sh\n+++ b/t/t2080-backup-log.sh\n@@ -211,4 +211,18 @@ test_expect_success 'overwritten ignored file is backed up' '\n \t)\n '\n \n+test_expect_success 'overwritten out-of-date file is backed up' '\n+\tgit init overwrite-outofdate &&\n+\t(\n+\t\tcd overwrite-outofdate &&\n+\t\ttest_commit haha &&\n+\t\tNEW=$(git hash-object haha.t) &&\n+\t\techo bad >>haha.t &&\n+\t\tOLD=$(git hash-object haha.t) &&\n+\t\tgit -c core.backupLog reset --hard &&\n+\t\techo \"$OLD $NEW $GIT_COMMITTER_NAME <$GIT_COMMITTER_EMAIL> $test_tick -0700\thaha.t\" >expected &&\n+\t\ttest_cmp expected .git/worktree.bkl\n+\t)\n+'\n+\n test_done\ndiff --git a/unpack-trees.c b/unpack-trees.c\nindex 8d7273af2b..221869b47c 100644\n--- a/unpack-trees.c\n+++ b/unpack-trees.c\n@@ -1715,7 +1715,8 @@ static int same(const struct cache_entry *a, const struct cache_entry *b)\n  */\n static int verify_uptodate_1(const struct cache_entry *ce,\n \t\t\t     struct unpack_trees_options *o,\n-\t\t\t     enum unpack_trees_error_types error_type)\n+\t\t\t     enum unpack_trees_error_types error_type,\n+\t\t\t     struct object_id *old_hash)\n {\n \tstruct stat st;\n \n@@ -1727,10 +1728,16 @@ static int verify_uptodate_1(const struct cache_entry *ce,\n \t * if this entry is truly up-to-date because this file may be\n \t * overwritten.\n \t */\n-\tif ((ce->ce_flags & CE_VALID) || ce_skip_worktree(ce))\n+\tif ((ce->ce_flags & CE_VALID) || ce_skip_worktree(ce)) {\n \t\t; /* keep checking */\n-\telse if (o->reset || ce_uptodate(ce))\n+\t} else if (o->reset) {\n+\t\tif (o->keep_backup && old_hash && !lstat(ce->name, &st))\n+\t\t\tindex_path(NULL, old_hash, ce->name, &st,\n+\t\t\t\t   HASH_WRITE_OBJECT);\n+\t\treturn 0;\n+\t} else if (ce_uptodate(ce)) {\n \t\treturn 0;\n+\t}\n \n \tif (!lstat(ce->name, &st)) {\n \t\tint flags = CE_MATCH_IGNORE_VALID|CE_MATCH_IGNORE_SKIP_WORKTREE;\n@@ -1764,17 +1771,20 @@ static int verify_uptodate_1(const struct cache_entry *ce,\n }\n \n int verify_uptodate(const struct cache_entry *ce,\n-\t\t    struct unpack_trees_options *o)\n+\t\t    struct unpack_trees_options *o,\n+\t\t    struct object_id *old_hash)\n {\n+\tif (o->keep_backup && old_hash)\n+\t\toidclr(old_hash);\n \tif (!o->skip_sparse_checkout && (ce->ce_flags & CE_NEW_SKIP_WORKTREE))\n \t\treturn 0;\n-\treturn verify_uptodate_1(ce, o, ERROR_NOT_UPTODATE_FILE);\n+\treturn verify_uptodate_1(ce, o, ERROR_NOT_UPTODATE_FILE, old_hash);\n }\n \n static int verify_uptodate_sparse(const struct cache_entry *ce,\n \t\t\t\t  struct unpack_trees_options *o)\n {\n-\treturn verify_uptodate_1(ce, o, ERROR_SPARSE_NOT_UPTODATE_FILE);\n+\treturn verify_uptodate_1(ce, o, ERROR_SPARSE_NOT_UPTODATE_FILE, NULL);\n }\n \n /*\n@@ -1862,8 +1872,11 @@ static int verify_clean_subdirectory(const struct cache_entry *ce,\n \t\t * removed.\n \t\t */\n \t\tif (!ce_stage(ce2)) {\n-\t\t\tif (verify_uptodate(ce2, o))\n+\t\t\tstruct object_id old_hash;\n+\n+\t\t\tif (verify_uptodate(ce2, o, &old_hash))\n \t\t\t\treturn -1;\n+\t\t\tmake_backup(ce2, &old_hash, NULL, o);\n \t\t\tadd_entry(o, ce2, CE_REMOVE, 0);\n \t\t\tinvalidate_ce_path(ce, o);\n \t\t\tmark_ce_used(ce2, o);\n@@ -1973,8 +1986,13 @@ static int verify_absent_1(const struct cache_entry *ce,\n \tint len;\n \tstruct stat st;\n \n-\tif (o->index_only || o->reset || !o->update)\n+\tif (o->index_only || o->reset || !o->update) {\n+\t\tif (o->reset && o->keep_backup &&\n+\t\t    old_hash && !lstat(ce->name, &st))\n+\t\t\tindex_path(NULL, old_hash, ce->name, &st,\n+\t\t\t\t   HASH_WRITE_OBJECT);\n \t\treturn 0;\n+\t}\n \n \tlen = check_leading_path(ce->name, ce_namelen(ce));\n \tif (!len)\n@@ -2092,10 +2110,12 @@ static int merged_entry(const struct cache_entry *ce,\n \t\t\tcopy_cache_entry(merge, old);\n \t\t\tupdate = 0;\n \t\t} else {\n-\t\t\tif (verify_uptodate(old, o)) {\n+\t\t\tstruct object_id old_hash;\n+\t\t\tif (verify_uptodate(old, o, &old_hash)) {\n \t\t\t\tdiscard_cache_entry(merge);\n \t\t\t\treturn -1;\n \t\t\t}\n+\t\t\tmake_backup(old, &old_hash, &merge->oid, o);\n \t\t\t/* Migrate old flags over */\n \t\t\tupdate |= old->ce_flags & (CE_SKIP_WORKTREE | CE_NEW_SKIP_WORKTREE);\n \t\t\tinvalidate_ce_path(old, o);\n@@ -2124,18 +2144,20 @@ static int deleted_entry(const struct cache_entry *ce,\n \t\t\t const struct cache_entry *old,\n \t\t\t struct unpack_trees_options *o)\n {\n+\tstruct object_id old_hash;\n+\n \t/* Did it exist in the index? */\n \tif (!old) {\n-\t\tstruct object_id old_hash;\n-\n \t\tif (verify_absent(ce, ERROR_WOULD_LOSE_UNTRACKED_REMOVED,\n \t\t\t\t  o, &old_hash))\n \t\t\treturn -1;\n \t\tmake_backup(ce, &old_hash, NULL, o);\n \t\treturn 0;\n \t}\n-\tif (!(old->ce_flags & CE_CONFLICTED) && verify_uptodate(old, o))\n+\tif (!(old->ce_flags & CE_CONFLICTED) &&\n+\t    verify_uptodate(old, o, &old_hash))\n \t\treturn -1;\n+\tmake_backup(ce, &old_hash, NULL, o);\n \tadd_entry(o, ce, CE_REMOVE, 0);\n \tinvalidate_ce_path(ce, o);\n \treturn 1;\n@@ -2305,8 +2327,16 @@ int threeway_merge(const struct cache_entry * const *stages,\n \t * conflict resolution files.\n \t */\n \tif (index) {\n-\t\tif (verify_uptodate(index, o))\n+\t\tstruct object_id old_hash;\n+\n+\t\tif (verify_uptodate(index, o, &old_hash))\n \t\t\treturn -1;\n+\t\t/*\n+\t\t * A new conflict appears. We could make a backup from\n+\t\t * worktree version to stage 2 or 3. But neither makes much\n+\t\t * sense. Make a deletion backup instead.\n+\t\t */\n+\t\tmake_backup(index, &old_hash, NULL, o);\n \t}\n \n \to->nontrivial_merge = 1;\n@@ -2447,16 +2477,26 @@ int oneway_merge(const struct cache_entry * const *src,\n \t\treturn deleted_entry(old, old, o);\n \n \tif (old && same(old, a)) {\n+\t\tstruct object_id old_hash;\n \t\tint update = 0;\n+\n+\t\toidclr(&old_hash);\n \t\tif (o->reset && o->update && !ce_uptodate(old) && !ce_skip_worktree(old)) {\n \t\t\tstruct stat st;\n+\n \t\t\tif (lstat(old->name, &st) ||\n-\t\t\t    ie_match_stat(o->src_index, old, &st, CE_MATCH_IGNORE_VALID|CE_MATCH_IGNORE_SKIP_WORKTREE))\n+\t\t\t    ie_match_stat(o->src_index, old, &st,\n+\t\t\t\t\t  CE_MATCH_IGNORE_VALID|CE_MATCH_IGNORE_SKIP_WORKTREE))\n \t\t\t\tupdate |= CE_UPDATE;\n+\n+\t\t\tif (update & CE_UPDATE && o->keep_backup)\n+\t\t\t\tindex_path(NULL, &old_hash, old->name, &st,\n+\t\t\t\t\t   HASH_WRITE_OBJECT);\n \t\t}\n \t\tif (o->update && S_ISGITLINK(old->ce_mode) &&\n-\t\t    should_update_submodules() && !verify_uptodate(old, o))\n+\t\t    should_update_submodules() && !verify_uptodate(old, o, NULL))\n \t\t\tupdate |= CE_UPDATE;\n+\t\tmake_backup(old, &old_hash, &old->oid, o);\n \t\tadd_entry(o, old, update, 0);\n \t\treturn 0;\n \t}\ndiff --git a/unpack-trees.h b/unpack-trees.h\nindex e2a64e2401..a453def564 100644\n--- a/unpack-trees.h\n+++ b/unpack-trees.h\n@@ -93,7 +93,8 @@ int unpack_trees(unsigned n, struct tree_desc *t,\n \t\t struct unpack_trees_options *options);\n \n int verify_uptodate(const struct cache_entry *ce,\n-\t\t    struct unpack_trees_options *o);\n+\t\t    struct unpack_trees_options *o,\n+\t\t    struct object_id *old_hash);\n \n int threeway_merge(const struct cache_entry * const *stages,\n \t\t   struct unpack_trees_options *o);\n-- \n2.20.0.rc2.486.g9832c05c3d\n\n"},{"id":"364866","messageId":"20181209104419.12639-17-pclouds@gmail.com","threadId":"49985","inReplyTo":"20181209104419.12639-1-pclouds@gmail.com","subject":"[PATCH 16/24] sha1-file.c: let index_path() accept NULL istate","fromName":"Nguyễn Thái Ngọc Duy","fromEmail":"pclouds@gmail.com","sentAt":"2018-12-09T10:44:11Z","receivedAt":"2018-12-09T10:45:37Z","isPatch":true,"sender":{"key":"pclouds@gmail.com","avatar":"https://avatars.githubusercontent.com/u/720?v=4"},"body":"istate is needed for content conversion. Allow to pass NULL istate\n(which implies that the caller does not care at all about file content\nconversion).\n\nSigned-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>\n---\n sha1-file.c | 8 +++++---\n 1 file changed, 5 insertions(+), 3 deletions(-)\n\ndiff --git a/sha1-file.c b/sha1-file.c\nindex 5bd11c85bc..214d62f3b5 100644\n--- a/sha1-file.c\n+++ b/sha1-file.c\n@@ -1829,7 +1829,8 @@ static int index_mem(struct index_state *istate,\n \t */\n \tif ((type == OBJ_BLOB) && path) {\n \t\tstruct strbuf nbuf = STRBUF_INIT;\n-\t\tif (convert_to_git(istate, path, buf, size, &nbuf,\n+\t\tif (istate &&\n+\t\t    convert_to_git(istate, path, buf, size, &nbuf,\n \t\t\t\t   get_conv_flags(flags))) {\n \t\t\tbuf = strbuf_detach(&nbuf, &size);\n \t\t\tre_allocated = 1;\n@@ -1957,12 +1958,13 @@ int index_fd(struct index_state *istate, struct object_id *oid,\n \t * Call xsize_t() only when needed to avoid potentially unnecessary\n \t * die() for large files.\n \t */\n-\tif (type == OBJ_BLOB && path && would_convert_to_git_filter_fd(istate, path))\n+\tif (type == OBJ_BLOB && path && istate &&\n+\t    would_convert_to_git_filter_fd(istate, path))\n \t\tret = index_stream_convert_blob(istate, oid, fd, path, flags);\n \telse if (!S_ISREG(st->st_mode))\n \t\tret = index_pipe(istate, oid, fd, type, path, flags);\n \telse if (st->st_size <= big_file_threshold || type != OBJ_BLOB ||\n-\t\t (path && would_convert_to_git(istate, path)))\n+\t\t (path && istate && would_convert_to_git(istate, path)))\n \t\tret = index_core(istate, oid, fd, xsize_t(st->st_size),\n \t\t\t\t type, path, flags);\n \telse\n-- \n2.20.0.rc2.486.g9832c05c3d\n\n"},{"id":"364867","messageId":"20181209104419.12639-14-pclouds@gmail.com","threadId":"49985","inReplyTo":"20181209104419.12639-1-pclouds@gmail.com","subject":"[PATCH 13/24] backup-log: add prune command","fromName":"Nguyễn Thái Ngọc Duy","fromEmail":"pclouds@gmail.com","sentAt":"2018-12-09T10:44:08Z","receivedAt":"2018-12-09T10:45:38Z","isPatch":true,"sender":{"key":"pclouds@gmail.com","avatar":"https://avatars.githubusercontent.com/u/720?v=4"},"body":"Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>\n---\n backup-log.c         | 71 ++++++++++++++++++++++++++++++++++++++++++++\n backup-log.h         |  3 ++\n builtin/backup-log.c | 17 +++++++++++\n 3 files changed, 91 insertions(+)\n\ndiff --git a/backup-log.c b/backup-log.c\nindex 49f2ce68fe..5e38725981 100644\n--- a/backup-log.c\n+++ b/backup-log.c\n@@ -1,6 +1,8 @@\n #include \"cache.h\"\n #include \"backup-log.h\"\n+#include \"blob.h\"\n #include \"lockfile.h\"\n+#include \"object-store.h\"\n #include \"strbuf.h\"\n \n void bkl_append(struct strbuf *output, const char *path,\n@@ -217,3 +219,72 @@ int bkl_parse_file(const char *path,\n \tstrbuf_release(&sb);\n \treturn ret;\n }\n+\n+struct prune_options {\n+\tstruct repository *repo;\n+\tFILE *fp;\n+\ttimestamp_t expire;\n+\tstruct strbuf copy;\n+};\n+\n+static int good_oid(struct repository *r, const struct object_id *oid)\n+{\n+\tif (is_null_oid(oid))\n+\t\treturn 1;\n+\n+\treturn oid_object_info(r, oid, NULL) == OBJ_BLOB;\n+}\n+\n+static int prune_parse(struct strbuf *line, void *data)\n+{\n+\tstruct prune_options *opts = data;\n+\tstruct bkl_entry entry;\n+\n+\tstrbuf_reset(&opts->copy);\n+\tstrbuf_addbuf(&opts->copy, line);\n+\n+\tif (bkl_parse_entry(line, &entry))\n+\t\treturn -1;\n+\n+\tif (entry.timestamp < opts->expire)\n+\t\treturn 0;\n+\n+\tif (oideq(&entry.old_oid, &entry.new_oid))\n+\t\treturn 0;\n+\n+\tif (!good_oid(opts->repo, &entry.old_oid) ||\n+\t    !good_oid(opts->repo, &entry.new_oid))\n+\t\treturn 0;\n+\n+\tif (!opts->fp)\n+\t\treturn -1;\n+\n+\tfputs(opts->copy.buf, opts->fp);\n+\treturn 0;\n+}\n+\n+int bkl_prune(struct repository *r, const char *path, timestamp_t expire)\n+{\n+\tstruct lock_file lk;\n+\tstruct prune_options opts;\n+\tint ret;\n+\n+\tret = hold_lock_file_for_update(&lk, path, 0);\n+\tif (ret == -1) {\n+\t\tif (errno == ENOTDIR || errno == ENOENT)\n+\t\t\treturn 0;\n+\t\treturn error(_(\"failed to lock '%s'\"), path);\n+\t}\n+\topts.repo = r;\n+\topts.expire = expire;\n+\topts.fp = fdopen_lock_file(&lk, \"w\");\n+\tstrbuf_init(&opts.copy, 0);\n+\n+\tret = bkl_parse_file(path, prune_parse, &opts);\n+\tif (ret < 0)\n+\t\trollback_lock_file(&lk);\n+\telse\n+\t\tret = commit_lock_file(&lk);\n+\tstrbuf_release(&opts.copy);\n+\treturn ret;\n+}\ndiff --git a/backup-log.h b/backup-log.h\nindex c9de9c687c..06fe706f81 100644\n--- a/backup-log.h\n+++ b/backup-log.h\n@@ -3,6 +3,7 @@\n \n #include \"cache.h\"\n \n+struct repository;\n struct strbuf;\n \n struct bkl_entry\n@@ -29,4 +30,6 @@ int bkl_parse_file(const char *path,\n \t\t   int (*parse)(struct strbuf *line, void *data),\n \t\t   void *data);\n \n+int bkl_prune(struct repository *r, const char *id, timestamp_t expire);\n+\n #endif\ndiff --git a/builtin/backup-log.c b/builtin/backup-log.c\nindex 2496d73ba5..2291124c38 100644\n--- a/builtin/backup-log.c\n+++ b/builtin/backup-log.c\n@@ -301,6 +301,21 @@ static int log_(int argc, const char **argv,\n \treturn ret;\n }\n \n+static int prune(int argc, const char **argv,\n+\t\t const char *prefix, const char *log_path)\n+{\n+\ttimestamp_t expire = time(NULL) - 90 * 24 * 3600;\n+\tstruct option options[] = {\n+\t\tOPT_EXPIRY_DATE(0, \"expire\", &expire,\n+\t\t\t\tN_(\"expire objects older than <time>\")),\n+\t\tOPT_END()\n+\t};\n+\n+\targc = parse_options(argc, argv, prefix, options, backup_log_usage, 0);\n+\n+\treturn bkl_prune(the_repository, log_path, expire);\n+}\n+\n static char *log_id_to_path(const char *id)\n {\n \tif (!strcmp(id, \"index\"))\n@@ -346,6 +361,8 @@ int cmd_backup_log(int argc, const char **argv, const char *prefix)\n \t\treturn diff(argc, argv, prefix, log_path);\n \telse if (!strcmp(argv[0], \"log\"))\n \t\treturn log_(argc, argv, prefix, log_path);\n+\telse if (!strcmp(argv[0], \"prune\"))\n+\t\treturn prune(argc, argv, prefix, log_path);\n \telse\n \t\tdie(_(\"unknown subcommand: %s\"), argv[0]);\n \n-- \n2.20.0.rc2.486.g9832c05c3d\n\n"}]}